By Gordon Hull
In previous posts (one, two, three), I’ve been exploring the issue of what I’m calling the implicit normativity in language models, especially those that have been trained with RLHF (reinforcement learning with human feedback). In the most recent one, I argued that LLMs are dependent on what Derrida called iterability in language, which most generally means that any given unit of language, to be language, has to be repeatable and intelligible as language, in indefinitely many other contexts. Here I want to pursue that thought a little further, in the context of Derrida’s (in)famous exchange with John Searle’s speech act theory.
Searle begins his “What is a Speech Act” essay innocently enough, with “a typical speech situation involving a speaker, a hearer, and an utterance by the speaker.”
That is enough for Derrida! In Limited, Inc., he responds by accusing Searle over and over of falling for intentionality, on the one hand, and for illicitly assuming that a given speech situation is “typical,” on the other.
Let’s look at intentionality first. In responding to Searle, Derrida explains that he finds himself “to be in many respects quite close to Austin, both interested in and indebted to his problematic” and that “when I do raise questions or objections, it is always at points where I recognize in Austin’s theory presuppositions which are the most tenacious and the most central presuppositions of the continental metaphysical tradition” (38). Derrida means by this a reliance on things like subjectivity and representation – the sorts of things that Foucault is getting at when he complains in the 1960s about philosophies of “the subject” (think: Sartre and phenomenology). Derrida is involved in the same general effort against phenomenology, though he adds a page later that he thinks the archaeological Foucault falls into this tendency to treat speech acts or discursive events in a “fundamentally moralistic” way (39). No doubt Searle is relieved to know that he belongs in the same camp as Foucault. In any case, Derrida explicitly says a few pages later that “the entire substratum of Sarl’s discourse, is phenomenological in character” (56) in that it is over-reliant on intentionality.
It seems to me that what I characterized last time as the “intentionality gap” in LLMs shows why this is relevant here. To recall, Derrida argues that the intentionality behind an utterance can never be made fully present to itself or its content, because whatever that intention is has to be mediated through words (and really this is probably to concede too much to phenomenology; on Derridean grounds, it’s problematic to say that the intention somehow precedes its realization in iterable words) that by definition escape (and have preceded) their current context. Language models radicalize the point because of course there is no intention behind the utterances of an LLM, only statistical prediction. Any intention we assign to the LLM is retrospective. Even causal descriptions of what the model does are imposed post facto, like when we explain the outputs of a model with a linear regression in the same way we’d explain other empirical phenomena.
In other words, “intention” is a model of the output, but it is not the output. There is a model of language that assumes that a speaking subject is behind utterances and that we can understand what that speaking subject “intends” to say. But this turns out to be wrong on two counts – LLMs show us that there is not necessarily a speaking subject behind an utterance, and that intentionality is at least partly a post hoc endeavor – it says something about us as interpreters.
The normative question, then, is what the implications are of applying the model of intentionality to the outputs of language models. If the intentionality model is problematic in the sorts of contexts Derrida is talking about, it is radically more so when applied to language models. The relevant question is what the political meaning of pushing intentionality here is. I argued in a recent paper against this sort of “Cartesian” approach to language AI and in favor of a political one (which I called “Hobbesian”). But I think something more fundamental is going in on the case of language models.
At one level, we are simply in the habit of using intentionality to model discourse. LLMs show us why that habit is an accident of our historical inability to make artifacts that produce language or credible language. We are also seem to be wired psychologically that way, as evidence dating all the way back to ELIZA suggests. These are familiar points. More interesting, perhaps is that a great deal of effort is being spent to get us to treat bots as having intentional states, as Dylan Wittkower has documented. The user experience is carefully designed to get us to forget that we are dealing with machines. One can debate whether or not this is a good thing, but it’s a thing.
At another level, the intentionality model tends to build in a model of origin – the model as the “origin” of speech – in a way that Derrida (and others) have criticized as belonging to a metaphysical tradition that needs to be overcome. For example, Foucault, in “Nietzsche, Genealogy, History,” famously distinguished between different understandings of forays into the past. A search for “origin” is “an attempt to capture the exact essence of things … because this search assumes the existence of immobile forms that precede the external worlds of accident and succession” (78; pagination to the Foucault Reader). Foucault, channeling Nietzsche, responds that “what is found at the historical beginning of things is not the inviolable identity of their origin; it is the dissension of other things. It is also disparity” (79). This is of course what motivates the “genealogical” project.
What diversity does the move to intentionality occlude? All of the various things I’ve been suggested are buried into the history of language models – the way training data is sourced and curated, for example. The extractive nature of the AI industry, in the way it exploits both human and natural resources, externalizing the costs. And of course, as Matteo Pasquinelli has shown, the material history of AI as a process of automating social intelligence.
In his history of the reception of Heidegger in France (recommended!), Ethan Kleinberg emphasizes the way that, from Heidegger’s “Letter on Humanism” forward, both Heidegger and some of his sharpest critics understood language to block philosophical projects, like Sartre’s in Being and Nothingness, that affirmed a radical sense of subjectivity. Heidegger of course buried his head in the sand of “Being,” but people like Blanchot and Levinas took the presence of language to suggest a reorientation to alterity. I’m not a scholar of either Blanchot or Levinas, but as Kleinberg notes, the Blanchot/Levinas reading was directly behind the sorts of arguments that a Foucault or Derrida would make. All of which is to underscore why LLMs are so odd: at one level, they suggest a truth about language, that it’s iterability all the way down. At another, they disturb some deeply held assumptions we tend to make about what the presence of language signals. Next time I’ll try to talk about something a little more subtle that emeges in Derrida’s critique of Searle.
Recent Comments