Language Models and Externalism: Reply to Mandelkern and Linzen

Abstract

Do texts generated by language models (LMs) refer? Mandelkern and Linzen (2024) argue that externalist principles point to an affirmative conclusion. What grounds reference, according to their externalism, is a term’s “natural history”. For example, ‘water’ refers to H₂O among English speakers, and not to the phenomenally indistinguishable chemical XYZ, because H₂O, and not XYZ, is implicated in the natural history of ‘water’. Appealing to the literature on contrastive explanation, I show that a term’s natural history does not generally ground its referential properties. Thus, Mandelkern and Linzen’s quick route to the referentiality of LM-generated texts fails.

Do texts generated by language models (LMs) refer? There seems ample reason to be skeptical. Since their inputs “are only strings of symbols,” it would seem that LMs “cannot produce referential words: reference cannot be derived from form.” While the quotation is from Mandelkern and Linzen (2024), they go on to suggest that this pessimistic attitude is mistaken. In their view, the externalist tradition emanating from Kripke (1980) and Putnam (1975) provides the resources to challenge the “very tempting argument” just described. As they argue, externalism, with its emphasis on the natural history of a symbol, opens up the possibility that an LM-generated text can indeed refer, since it may lay claim to the same sort of natural history. Moreover, if LM-generated texts can be shown to refer, then we can be reasonably optimistic concerning the question whether such texts are meaningful.¹ (If they cannot, then a large chunk of generated discourse—indeed almost all of what we care about—will be meaningless.) As I argue, however, the natural history of a term, in the austere sense assumed by Mandelkern and Linzen, cannot ground the reference relation. Since they give clear expression to a growing consensus among philosophers of language engaging in these questions, it is worth considering Mandelkern and Linzen’s central argument in detail, especially given that the discussion has come to the attention of researchers in AI and natural-language processing.