New APPS

By Gordon Hull

As part of thinking through the implications of Lydia Liu’s papers (here and here) demonstrating a Wittgensteinian influence on the development of large language models, I’ve made a detour into Derrida’s critique of writing (my earlier parts: one, two, three).  My initial suggestion last time was that Derrida’s discussion is designed to show that “Platonism” is a political move (not a metaphysical one).  For Derrida the Platonic priority of voice over writing disguises the fact that both are (in his own terms) repetitions of the eidos, and so the claim that writing is bad is the claim that it’s the wrong kind of repetition.  I suggested that for Platonism as read by Derrida, one could easily imagine a hierarchy of writing systems, based on their proximity to voice/speech.  Chinese ideography – which Liu argues is central to Masterman’s breakthroughs in computer language modeling – would be at the very bottom of a Platonic hierarchy.  But because this is Derrida, we can’t either proceed quickly or proceed without talking about Hegel.  So I closed last time with a long passage from Hegel in which he denigrates Chinese for being insufficiently spiritual and too hard to learn.  Today I’ll start with why it’s relevant.

(1) First, Derrida takes up Hegel’s understanding of language in “The Pit and the Pyramid,” first delivered in 1968 and thus almost exactly contemporaneous with “Plato’s Pharmacy.”  There, working from the other end of metaphysics (Hegel, not Plato), Derrida describes such a hierarchy:

“The heart of the thesis is quickly stated: the privilege or excellence of the linguistic system – that is, the phonic system – as concerns any other semiotic system. Therefore, the privilege of speech over writing and of phonetic writing over every other system of inscription, particularly over hieroglyphic or ideographic writing, but equally over mathematical writing, over all formal symbols, algebras, pasigraphies and other projects of the Leibnizian sort – phonetic writing’s privilege over everything which has no need, as Leibniz said, ‘to refer to the voice’ or to the word (vox)” (Margins of Philosophy, 88; references to this edition)

Not surprisingly, given that Derrida attaches this basic reading to the entire history of philosophy, some of that is at least implicit in the discussion of Plato.  In both cases, the mythology of hierarchy makes it impossible to see that there is only ever a grammar of resemblances, not an originary anchoring of meaning.

(2) Second, as Derrida argues in “The Pit and the Pyramid,” Chinese is not actually the worst deviation from spoken language for Hegel – that distinction belongs to Egyptian hieroglyphics, which he calls a primitive form of symbolization (Margins, 97ff).  Like Hegel always says about Egyptian culture, it’s half above ground as spirit and half below ground as matter.  The problem with Chinese, on the Hegelian reading, is rather that it is too abstract.  As Derrida writes, “the Chinese model … nevertheless marks a progress over the Egyptian hieroglyph. A progress in formalizing abstraction, a detachment as concerns the sensuous and the natural symbol.  But this progress, which corresponds to the moment of abstract understanding, never recovers what it loses” (Margins, 102).  As a result, “Chinese culture and writing are reproached simultaneously for their empiricism (naturalism, historicism) and their formalism (mathematizing abstraction)” (ibid). On the one hand, Derrida points out that this characterization of Chinese is fundamentally incoherent, and he accuses Hegel of lapsing into symptomatic self-contradiction in several points.

(3) On the other hand, in setting up this point, Derrida says some things about Hegel that are directly relevant to Liu’s recovery of Masterman’s use of Chinese:

(3)(a) First, Hegel establishes the priority of words as units of vocalization and thus of meaning, such that “the linguistics implied by all these propositions [of Hegel’s] is a linguistics of the word, and singularly of the name.” (Margins, 96, emphasis original).  Chinese and “other projects for a universal writing of a nonphonetic type seem to be marked by the abusive pretensions and insufficiencies of all the formalisms denounced by Hegel. The indictment is directed precisely against the risks of dislocating the word and the name” (Margins, 96-7).  This is the linguistics that Liu sees Masterman pursuing, since Masterman’s use of the Chinese zi is precisely to challenge the primacy of the word.  

Interestingly, Derrida indicates his awareness of this kind of project: “today we know that the word no longer has the linguistic rank that had almost always been accorded to it. It is a relative unity, made to stand out between larger or smaller unities” (96, emphasis mine).

Derrida footnotes a paper, “Le Mot” (“The Word;” translated here, references to this version) by the linguist André Martinet.  Martinet explores a tension within linguistics over the priority given to words as conveyances of meaning.  Thus: “why is dans le château [in the house] three words while donnerons [we will give] one word only?” (43). It’s true that they appear as one word, but the declension of donnerons, he has already shown, makes matters more complicated: “three successive units can be identified: a root donn-, an element -(e)r- that denotes the future, and an ending -ons, which denotes the first person plural” (40).  Semantically, the verbal declension conveys the same number of units of meaning as does the three words.

The analysis initially seems quite promising, from the point of view of the Wittgenstein-Masterman line, since Masterman herself (as Liu cites her) writes a paper deconstructing the idea of a “word.”  Here is Martinet:

“The notion of word, completed by that of enclitic, has permitted us to account for the sentence structure of inflected languages such as Greek or Latin. In other languages it permits useful groupings of certain facts, but its extension to all utterance of all languages complicates grammatical description more often than it simplifies it” (53).

That said, it’s not clear to me how close Martinet comes to the Wittgenstein-Masterman point.  First, although there is a periodic gesture to “all languages,” Martinet never cites a non-European language.  There’s a brief example from Russian, a couple of Norwegian examples and something from Romanian, but nothing from anything like Chinese.  It’s still an account based on meaning in the sense that Liu says Wittgenstein-Masterman are deconstructing. 

The reference to Russian I remember from Wittgenstein goes in the same direction but goes a step further in that Wittgenstein also tries to shift “meaning” onto “use.”  Wittgenstein is talking about a sentence as a unit, and notes that the way we construct sentences is anchored to “a particular paradigm of our grammar” (PI 20).  He then asks this rhetorical question about Russian: “In Russian one says ‘stone red’ instead of ‘the stone is red;’ do they feel the copula to be missing in the sense, or attach it in thought?” (PI 20).  Obviously neither happens, since the expressions function identically (Russian does not generally use present-tense be verbs, so “where table?” “stone red,” and so forth).  His point in this part of the PI is to and to push for use as a unit of analysis.  That is not what Martinet is doing, although I think that Martinet’s analysis could be pushed in that direction.

Second, and I think this follows from the first, Martinet’s analysis is about the transition from spoken to written language and the ways that words do or do not map the semantics of the spoken language.  For example:

“The separability of successive elements of speech is, fundamentally, what normally brings about the writing of separate words. The three words dans le château affirm their independence in an expression such as dans tout le grand château where the insertion of tout and of grand justifies the traditional transcription and interpretation. In donnerons, on the other hand, only a single insertion may be envisaged, that is -i- after the -r-, so that we have donnerions [we would give] but we have here a satellite, the same one we opposed to the element -(e)r- above and which here turns out to be combinable with it. There is only one unit in the series of satellites which may be combined with a verb root like donn(e), each occupying the place which is reserved for it by tradition” (45).

As a result, Martinet argues that we should stop talking as much about words, but instead talk about “monemes,” which are basically the minimal units of conveying meaning.  He puts it this way:

“There is a moneme every time that the speaker is forced to give to his utterance a particular turn in order to convey exactly the message that he has in mind and not some other message that the language might permit him to transmit. There are as many monemes as there are choices. The French phrase Au fur et à mesure [gradually; step-by-step] which seems so easy to analyse, represents without question five traditional words. But it is nothing more than a moneme because once the speaker has chosen to use fur, he cannot refrain from uttering the remainder of the phrase.”

In sum, the idea here seems to be to track how meaning in spoken language is represented in written language, given the fact that meaning is sometimes conveyed through inflection, sometimes by adding other words, and sometimes by combining small words into big ones.  But the idea that the meaning is fully associative, as Liu says happens in ideographic Chinese, isn’t really a part of this analysis. 

(3)(b) Second, Hegel associates the problems of Chinese with the problems of arithmetic.  The problem with arithmetic – and here the model sounds like algebra – is that it manipulates abstract symbols without regard to their content, and thus without regard to the truth (ha! arithmetic is bullshit!).  As Derrida says, the exteriority of arithmetic abstraction remains sensory for Hegel. It has certainly shed all empirical, sensory diversity, is pure of all determined sensory content; but, as “this thought of externality … it has retained nothing but the abstract determination of externality itself’” (Margins, 106, citing Hegel’s Logic).  The attachment to truth, the voice, spirit, logos, etc. disappear:

“Now calculation, the machine, and mute writing belong to the same system of equivalences, and their work poses the same problem: at the moment when meaning is lost, when thought is opposed to its other, when spirit is absent from itself, is the result of the operation certain” (107)?

[aside: The lack of attachment to something anchoring is also what Hobbes sees as wrong with algebra, but he’s going in the opposite direction, wanting to favor the referentiality of counting.  Also, with regard to LLMs, now is the place to insert a mention of referentiality and the vector grounding problem.  The point here is that all of these are political issues, because the metaphysics is politics in disguise].

Derrida suggests that “what Hegel … could never think is a machine that would work. That would work without, to this extent, being governed by an order of reappropriation [une machine qui fonctionnerait. Qui fonctionnerait sans être en cela réglée par un ordre de reappropriation]. Such a functioning would be unthinkable in that it inscribes within itself an effect of pure loss” (Margins, 107; Marges 126)).

We’re in a post-Hegelian moment: LLMs, insofar as they follow the Wittenstein-Masterman line, present the possibility of machines that function without an order of reappropriation, since they work entirely by resemblances.  Except that they also don’t: to get them to work this way, or at least in a way that we find satisfactory, requires both pre- and post- training of the models.

A final comment will let me close things here and return briefly to Wittgenstein.  It seems to me that the Derridean critique of the distinction between voice and writing and the Wittgensteinian critique of words are analogous insofar as both track what Derrida calls Platonism and what Wittgenstein calls the urge to look for the mouse under the gray rags and dust – the urge to find some sort of anchoring story for language. 

I’ll continue with Derrida next time.