New APPS: Art, Politics, Philosophy, Science

A Least-Bad Solution for Language Model Defamation?

By Gordon Hull

Large Language Models (LLMs) are well-known to “hallucinate,” which is to say that they generate text that is plausible-sounding but completely made-up.  These difficulties are persistent, well-documented, and well-publicized.  The basic issue is that the model is indifferent to the relation between its output and any sort of referential truth.  In other words, as Carl Bergstrom and C. Brandon Ogbunu point out, the issue isn’t so much hallucination in the drug sense, but “bullshitting” in Harry Frankfurt’s sense. One of the reasons this matters is defamation: saying false and bad things about someone can be grounds to get sued.  Last April, ChatGPT made the news (twice!) for defamatory content.  In one case, it fabricated a sexual harassment story and then accused a law professor.  In another, it accused a local politician in Australia of corruption.

Can LLMs defame?  According to a recent and thorough analysis by Eugene Volokh, the answer is almost certainly yes.  Volokh looks at two kinds of situation.  One is when the LLM defames public figures, which is covered by the “actual malice” standard.  Per NYT v. Sullivan, “The constitutional guarantees require … a federal rule that prohibits a public official from recovering damages for a defamatory falsehood relating to his official conduct unless he proves that the statement was made with ‘actual malice’ – that is, with knowledge that it was false or with reckless disregard of whether it was false or not” (279-80).

Deepfake Research: ChatGPT can produce fake data

By Gordon Hull

There’s been a lot of concern about the role of language models in research.  I had some initial thoughts on some of that based around Foucault and authorial responsibility (part 1, part 2, part 3).  A lot of those concerns have to do with the role of ChatGPT or other LLM-based product and how to process that.  The consensus of the journal editorial policies that are emerging is that AI cannot be an author, and my posts largely agreed with that.

Now there’s news of a whole other angle on these questions: a research letter in JAMA Ophthalmology reports that the authors were able to use ChatGPT-4’s Advance Data Analysis capabilities to produce a fake dataset validating their preferred research results.  Specifically:

“The LLM was asked to fabricate data for 300 eyes belonging to 250 patients with keratoconus who underwent deep anterior lamellar keratoplasty (DALK) or penetrating keratoplasty (PK). For categorical variables, target percentages were predetermined for the distribution of each category. For continuous variables, target mean and range were defined. Additionally, ADA was instructed to fabricate data that would result in a statistically significant difference between preoperative and postoperative values of best spectacle-corrected visual acuity (BSCVA) and topographic cylinder. ADA was programmed to yield significantly better visual and topographic results for DALK compared with PK”

This is a very technical request!  It took a bit of tweaking, but soon “the LLM created a seemingly authentic database, showing better results for DALK than PK,” P < .001.

The authors suggest some possible strategies to manage this but suffice it to say it is terrifying.  There is already a longstanding, huge problem with fabricated, doctored or otherwise bogus scientific research out there.  One report suggests that 70,000 “paper mill” (= almost completely faked) papers were published in the last year alone.  In real papers, references are often inaccurate.  Publishers already are having to grapple with lots of problematic doctored images, and Pharma has long tilted the entire scientific enterprise to produce results favorable to its products.  At the end of last year, Stanford’s president was forced out over research misconduct in his labs.  In an initial report into the Stanford investigation, STAT News reported data from Retraction Watch to the effect that a paper is retracted, on average, every other day for image manipulation.  Retraction Watch had, at that time (Dec. 2022) 37,000 papers in its database.  The top 5 most-retracted authors have at least 100 retracted papers each.

Into that mess, enter the ability to generate bespoke data on demand.

RLHF and Curation Transparency

By Gordon Hull

Last time, I followed a reading of Kathleen Creel’s recent “Transparency in Complex Computational Systems” to think about the ways that RLHF (Reinforcement Learning with Human Feedback) in Large Language Models (LLMs) like ChatGPT necessarily involves an opaque, implicit normativity.  To recap: RLHF improves the models by involving actual humans (usually gig workers) in their training: the model presents two possible answers to a prompt, and the human tells it which one is better.  As I suggested, and will pursue in a later post, this introduces all sorts of weird and difficult-to-measure normative aspects into the model performance, above and beyond those that are lurking in the training data.  Here I want to pause to consider this as a question of opacity and transparency. I’m going to end up by proposing that there’s a fourth kind of transparency that we should care about, for both epistemic and moral reasons, which I’ll call “curation transparency.”

Implicit Normativity in Reinforcement Learning with Human Feedback in Large Language Models

By Gordon Hull

This is somewhat circuitous – but I want to approach the question of Reinforcement Learning with Human Feedback (RLHF) by way of recent work on algorithmic transparency.  So bear with me… RLHF is currently all the rage in improving large language models (LLMs).  Basically, it’s a way to try to deal with the problem that LLMs aren’t referentially grounded, which means that their output is not in any direct way connected to the world outside the model.

LLMs train on large corpora of internet text – typically sources like Wikipedia, Reddit, patent applications and so forth.  They learn to predict what kinds of text are likely to come next, given a specific input text.  The results, as anybody who has sat down with ChatGPT for long knows, can be spectacular.  Those results also evidence that the models function, in one paper’s memorable phrasing, as “stochastic parrots.”  What they say is about what their training data says is most likely, not about what’s, say, contextually appropriate.  But appropriate human speech is context-dependent, and answers that sound right (in the statistical sense: these words, in general, are likely to come after those words) in one context may be wrong in another (because language does not get used “in general”).  RLHF is designed to get at that problem, as a blogpost at HuggingFace explains:

More on AI and Copyright

Another case percolating through the system, this one about Westlaw headnotes.  The judge basically ruled against a series of motions for summary judgment, which means that the case is going to a jury.  Discussion here (link via Copyhype)

Cars are Privacy Disasters

This article from Gizmodo reports on research done over at Mozilla.  Newer cars – the ones that connect to the internet and have lots of cameras – are privacy disasters.  Here’s a paragraph to give you a sense of the epic scope of the disaster:

“The worst offender was Nissan, Mozilla said. The carmaker’s privacy policy suggests the manufacturer collects information including sexual activity, health diagnosis data, and genetic data, though there’s no details about how exactly that data is gathered. Nissan reserves the right to share and sell “preferences, characteristics, psychological trends, predispositions, behavior, attitudes, intelligence, abilities, and aptitudes” to data brokers, law enforcement, and other third parties.”

Nissan’s response tells you everything that’s wrong with current privacy legislation:

““When we do collect or share personal data, we comply with all applicable laws and provide the utmost transparency,” said Lloryn Love-Carter, a Nissan spokesperson. “Nissan’s Privacy Policy incorporates a broad definition of Personal Information and Sensitive Personal Information, as expressly listed in the growing patchwork of evolving state privacy laws, and is inclusive of types of data it may receive through incidental means.””

Let’s translate.  Nissan is probably compliant.  Also, privacy compliance is a joke.  Also, compliance apparently only requires that you receive NOTICE that they take your data AND CONSENT to that policy, probably merely by driving the vehicle.  Also, they probably reserve the right to change their privacy policies unilaterally, at will.  Also, they almost certainly do not let you opt-out of any of it while CONSENTING by driving the car.  It’s a very special kind of “contract” and “consent.”  Also, how do they know about your sex life?  Also, even if you have sex in the car, there is basically no answer to that question that is not beyond creepy!

As you may have guessed, NOTICE AND CONSENT is an utter sham and has been for a while.  The gizmodo article spells out some of the particular absurdities here – for example, you may not want to ride in one of these cars either, as passengers are “users” deemed to have CONSENTED to the privacy policy.  Your driver should probably provide you NOTICE beforehand!  “A number of car brands say it’s the driver’s responsibility to let passengers know about their car’s privacy policies—as if the privacy policies are comprehensible to drivers in the first place.”  No wonder folks are cynical and resigned about corporate privacy – they’re manipulated into it by corporations.  Also, they’re confused, frustrated and angry about the fact they don’t actually get to consent.

This is the best example I’ve seen of all that in a while, and a crystal-clear indicator of why we need not just new privacy legislation (we do!) but a new direction (more real regulation, less soft compliance and "notice and consent" fig-leaves).

PS – not picking only on Nissan:

“Other brands didn’t fare much better. Volkswagen, for example, collects your driving behaviors such as your seatbelt and braking habits and pairs that with details such as age and gender for targeted advertising. Kia’s privacy policy reserves the right to monitor your “sex life,” and Mercedes-Benz ships cars with TikTok pre-installed on the infotainment system, an app that has its own thicket of privacy problems.”

On Foucault on “Chardino-Marxism,” part 5: Lunacharsky

By Gordon Hull

I’ve been developing (first, second, third, fourth) some reflections on what Foucault means by a reference to “Chardino-Marxism,” a disturbing trend that he credits Althusser with “courageously fighting.”  The real opposition point seems to be Roger Garaudy, a PCF intellectual who is a leader in the effort to establish a post-Stalinist humanist Marxism, and who had a real sympathy for religion.  Last time, I traced some of Garaudy’s sources on religion to Engels.  Some of what Garaudy says also sounds like it’s coming straight from the Russian Marxist Anatoly Lunacharsky’s Religion and Socialism (1908).  The claim here is categorically not that Garaudy read Lunacharsky – as will become evident in a minute, I think that’s highly unlikely.  What I do want to underscore is that there is a coherent line of thought behind Garaudy’s religious impulse.  As I’ll note when I get back to Garaudy and Althusser, there is a very specific political context to Althusser’s attacks on Garaudy having to do with the latter’s role in the PCF and his effort to use a humanist Marxism as a (from Althusser’s point of view, failed) alternative to Stalinism. 

I know very little about Lunacharsky (Wikipedia here), but apparently he was tolerated by Lenin (despite being criticised heavily), and fell out of favor under Stalin.  He died before he could be repressed, but in 1936-8, his memoirs were banned and he was erased from the official histories of communism.  He enjoyed somewhat of a revival after Stalin’s death.  Religion and Socialism is very obscure now: Google books reports a Yiddish translation (!) as well as a Spanish one from the 1970s.  It’s not been translated into English or French.  Marxists.org refers only to his later works in English and in French, and he doesn’t even show up on the German part. 

Most of the work available on Lunacharsky now seems to be attributable to patient work by Roland Boer (upon whom I am completely dependent here).  Religion and Socialism fell out of favor due to Lenin’s denunciation after it was published, was left out of Lunacharsky’s collected works, and was reduced to a few copies.  Here is Boer in his paper on the text:

On Foucault on “Chardino-Marxism,” part 4

By Gordon Hull

Over the course of a few posts (first, second, third), I’ve been exploring the question of what Foucault means when he refers disparagingly to “Chardino-Marxism” in a mid-1960s interview, comparing it unfavorably to what Althusser and his circle are doing.  Although the “Chardino” part refers to Teilhard de Chardin, it’s fairly clear that the real target is humanist Marxism, of which Roger Garaudy is taken to be a leading example, probably due to his role in the PCF.  Here I want to take an initial look at the chapter “Marxism and Religion” in his Marxism in the Twentieth Century and situate it with reference to Engels and Marx.

Garaudy’s chapter is long, and his general concern is to vindicate the idea that a non-institutional version of religion (or of faith, not religion) has its place in Marxist discourse.  This is a carefully-defined position: he is at pains to distinguish the sense of faith he is talking about from most of what passes under the name.  In the present context, a few features should be noted.  First, Teilhard de Chardin is one of his principal reference points for a contemporary person who is going in the direction he wants (the other is Dietrich Bonhoeffer).  Second, this is a recuperative project of humanism: it is “man” that is of concern the entire time.  Third, Garaudy gets his Marx references almost entirely from the early writings.  There are very few references to Capital in the chapter, and relatively few to Engels: most of the work is done in the 1845 and earlier writings.  This is significant not least because of Althusser’s well-known denunciation of early Marx as ideological and pre-scientific.  In other words, the version of Marx being used to support the humanist reading is the same one Althusser wants to get rid of in the name of anti-humanism (I will complicate this point below).  Finally, Garaudy puts a lot of emphasis on the possibility of “love,” which he thinks can be rescued from a Platonic version that goes via God (I love the other insofar as I see God in them) to the direct love of alterity and other people.

AI Is not a (Copyright) Author (at least not today)

By Gordon Hull

This one has been percolating a while… Steven Thaler’s AI created a picture (below the fold), and Thaler has been using it to push for the copyrightability of AI-generated material.  That endeavor has been getting nowhere, and a DC District Court just ruled on the question of “whether a work generated autonomously by a computer falls under the protection of copyright law upon its creation,” in the same way as a work generated by a person.  Copyright attaches to human work very generously – this blog post is copyrighted automatically when I write it, and so are doodles you make on napkins.  You get lots of extra protections and litigation benefits if you register, but registration is not a requirement for copyright in itself.  Per 17 U.S.C. Sec. 102, copyright subsists in “original works of authorship fixed in any tangible medium of expression, now known or later developed, from which they can be perceived, reproduced, or otherwise communicated, either directly or with the aid of a machine or device.”  Given this, it’s not hard to see why someone would want to know whether AI could be an “author” in the relevant sense.

The Court ruled that “United States copyright law protects only works of human creation.”  This is not a surprise.  The central argument is that “Copyright is designed to adapt with the times. Underlying that adaptability, however, has been a consistent understanding that human creativity is the sine qua non at the core of copyrightability, even as that human creativity is channeled through new tools or into new media.”  Indeed, “human authorship is a bedrock requirement of copyright.”  The Court both cites historical precedent and grounds it in the purpose of Copyright, which is constitutionally to incentivize the creation of new works:

On Foucault on “Chardino-Marxism,” Part 3

By Gordon Hull

The last couple of times (here then here), I’ve started trying to work through a disparaging reference in the mid-1960s Foucault to “Chardino-Marxism.”  Foucault is associating it with Marxist humanism, and comparing it unfavorably to the Althusserian alternative.  As I noted, the name Foucault uses is Teilhard de Chardin, but the consistent target of the Foucault-aligned theorists appears to be Roger Garaudy.

So why, exactly, might Teilhard appeal to Marxism?  More precisely, in what sense would Teilhard appeal to Garaudy?  In a 1969 paper, Ladis KD Kristof offers some context (for Kristof’s remarkable life, see the memorial notice here).  The “Phenomenon Teilhard” was widely discussed within the Soviet bloc countries, and within the USSR as early as 1962; a Russian translation of Teilhard’s Phenomenon of Man appeared in 1965.  Kristof suggests that the initial Marxist attraction to Teilhard lies simply in that he has a world view – something they can respect, as opposed to (for example) American positivism or empiricism.  More specifically, Teilhard: (a) has a scientific worldview, in that he has a Baconian belief that science can solve all problems; (b) has an evolutionary worldview, arguably even more so than Marx. On Kristof’s account, the difference is first in scope: Teilhard’s evolution is cosmic and Marx’s human.

This leads to a second fundamental difference; following Engels in Anti-Dühring, Marxists think that when man [I am following 1960s usage here – this is the generic “man”] starts taking control of nature (= making history), that is the final qualitative change, and that that future changes are quantitative.  Teilhard, on the other hand, thinks that the end of the process of what he calls “hominization” will involve a qualitative leap.  However, both camps are fundamentally anthropocentric in that “man” is the focus throughout.  Finally, (c) Marxism involves a movement of faith: if one is struggling for the revolution, this requires a prior faith that one can effect progress and so forth; in this, there is a convergence with Teilhard’s optimism.  Something of the sense of all this is conveyed in the following (long) passage from Teilhard’s Future of Mankind (I’m getting it from Kristof, who quotes part of it):

What is Chardino-Marxism and Why does Foucault Care? (part 2)

By Gordon Hull

Last time, I noted that mid-late 1960s Foucault aligned himself in favor of Althusser’s work on Marx, and against what he called “Chardino-Marxism,” which turns out to be a shorthand for humanist Marxism, in particular any efforts to synthesize Marx and Teilhard de Chardin, as well as (or rather, as exemplified by) the work of PCF intellectual Roger Garaudy.  Foucault’s opposition to “humanism” is well-known, but his differentiation of Marxism into less-desirable humanist varieties and more-desirable Althusserian less so, and so I want to pursue the Chardino-Marxism critique further, because it helps us understand the context in which the humanist critique appears, as well as Foucault’s subsequent efforts to position himself relative to Marxism in the 1970s (obligatory self-promotion: my foray into that is here)

In the 1966 interview, “L’homme est-il mort? [Is man dead?]”, Foucault gives as clear a position statement as I’ve seen on all of this.  The interview is roughly contemporaneous with Order of Things, and certainly the more detailed exposition of humanism and Marxism’s place in it there needs to be taken into account in any full discussion.  The interviewer had asked if Foucault differentiated among different kinds of humanism, naming Sartre.  Foucault responds that “if you set aside the facile humanism that Teilhard and Camus represent, the problem of Sartre appears completely different.”  Foucault then stops talking about Sartre and offers a general characterization: “humanism, anthropology and dialectical thought are related.  What ignores man, is contemporary analytic reason which we saw born with Russell, [and] which appears in Levi-Strauss and the linguists.”  On the other hand, dialectics, Foucault says, promotes the idea that the human being “will become an authentic and true man.”  That is, it “promotes man to man and, to this extent, it is indissociable from humanist morality.  In this sense, the great officials of contemporary humanism are evidently Hegel and Marx” (D&E I, 569).  So we are back to the Lindung interview, where Foucault accuses Garaudy of having indiscriminately “picked up everything from Hegel to Teilhard de Chardin” (discussed last time), though with perhaps an emerging sense of what that lineage looks like.

What is “Chardino-Marxism” and why is Foucault talking about it? (part 1)

By Gordon hull

In a 1966 interview with Madeline Chapsal, Foucault proposes that “our task currently is to definitively liberate ourselves from humanism” and offers the following example:

“Our task is to free ourselves definitively from humanism, and it is in this sense that our work is political work, insofar as all the regimes of the East or West pass out their bad goods under the flag of humanism. We have to denounce all these mystifications, like today, inside the Communist Party, where Althusser and his courageous companions are struggling against “Chardino-Marxism.”” (33)

Other than the fusion of Teilhard de Chardin and Marx, what is “Chardino-Marxism” and why does Foucault care?

Teilhard was a Jesuit priest and scientist who tried to reconcile his work in paleontology with Christianity.  I’m dependent on Wikipedia for this, so I’ll just let you read the desription there of his main work, the Phenomenon of Man:

“His posthumously published book, The Phenomenon of Man, set forth a sweeping account of the unfolding of the cosmos and the evolution of matter to humanity, to ultimately a reunion with Christ. In the book, Teilhard abandoned literal interpretations of creation in the Book of Genesis in favor of allegorical and theological interpretations. The unfolding of the material cosmos is described from primordial particles to the development of life, human beings and the noosphere, and finally to his vision of the Omega Point in the future, which is "pulling" all creation towards it. He was a leading proponent of orthogenesis, the idea that evolution occurs in a directional, goal-driven way. Teilhard argued in Darwinian terms with respect to biology, and supported the synthetic model of evolution, but argued in Lamarckian terms for the development of culture, primarily through the vehicle of education.”

Teilhard is also known for developing the concept of “noosphere” to refer to the space of human reason and a transcendence of biology; this concept had some uptake among early Internet theorists.  Apparently Teilhard was very trendy in the 1950s and 1960s.  Foucault mentions him occasionally, and always to associate him with humanism.  Later in the interview quoted above, and in response to question about whether the direction he was taking philosophy didn’t appear “cold and rather abstract,” Foucault exclaims: “it is humanism that is abstract!” and adds:

“What makes me angry about humanism is that it is now this screen behind which the most reactionary thought takes refuge, where monstrous and unthinkable alliances are formed: they want to combine Sartre and Teilhard, for example. In the name of what? Of man! Who would dare speak ill of man? And yet, the effort currently being made by people of our generation is not done in order to claim man [a]s against knowledge and against technology, but is precisely to show that our thought, our life, our manner of being, even our most everyday manner of being, are part of the same systematic organisation and therefore depend on the same categories as the scientific and technical world.” (34-5, translation slightly revised. Emphases original).

In a 1968 interview with Yngve Lindung that appeared in Stockholm (D&E #54; I can’t find a translation), Foucault elaborates a more on the topic. Asked if structuralism was opposed to Marxism, he acknowledges that “it is true that there are certain Marxists who are declared anti-structuralists,” but adds that “at the same time, we need to say that there are a large number of Marxists, among the youngest and let’s say the most dynamic, who on the contrary feel very close to structuralist research.”  He explains:

“In general, one can say that we have to deal today with a soft, bland, humanist Marxism which tries to pick up everything that traditional philosophy has been able to say from Hegel up to Teilhard de Chardin [qui essaie de ramasser tout ce que la philosophie traditionnelle a pu dire depuis Hegel jusqu’à Teilhard de Chardin].  This Marxism is anti-structuralist insofar as it is opposed to structuralism’s having put in question the old values of bourgeois liberalism.  Then we have an opposing group of Marxists that we could call anti-revisionists and for whom the future of Marxist thought and of the communist movement indeed requires that one reject all this eclecticism, all this interior revisionism, all this peaceful coexistence concerning the plan of ideas, and these Marxists are instead structuralists” (D&E I, 682-3 (2 vol version)). 

Later in the interview he names the PCF intellectual Roger Garaudy in this regard.  Now, Garaudy claims to be inspired by structuralism.  But he also claims to be a humanist.  Foucault responds that “I don’t believe that one could reasonably pretend that Garaudy is a Marxist.”  He then adds that “it doesn’t in any way surprise me that Garaudy wants to gather [recueillir] what he is able to call a concrete structuralism and humanism.  He has picked up everything from Hegel to Teilhard de Chardin.  He will pick me up too [Il a tout ramassé depuis Hegel jusqu’à Teilhard de Chardin. Il me ramassera aussi]” (684).

The objection is to the eclecticism, and one can imagine Garaudy as having produced what one might call a scavenger Marxism.  There is definitely this sense to Garaudy.  I’ll say more in a later post, but in Marxism in the Twentieth Century (1966), for example, he writes that “structuralism can, like cybernetics [!], be one of the ways of comprehending the world and of conceiving man and his action, which corresponds the best to the spirit of our time, to the development of a new humanism: this will be precisely the humanism of which Marx was the pioneer, integrating all that was one by Graeco-Roman humanism and Judeo-Christian humanism, and going beyond both in a new synthesis of nature and man, of the external world and subjectivity, of necessary law and liberty” (75).

That’s a lot!  Foucault wants nothing of it.  Later in the Lindung interview he says the following, which is worth quoting at length:

“The situation of the French left is still dominated by the presence of the communist party.  The current problematic at its interior is essentially the following: should the party, theoretically and politically, make itself the agent of peaceful coexistence, which politically drives [entraîne] a sort of neutralization of the conflict with the U.S., and which comports, from an ideological point of view, with an attempt at ecumenicism thanks to which all the important ideological currents in Europe and in the world are found [retrouveraient] more or less reconciled?  It is clear that people like Sartre and Garaudy work toward this peaceful coexistence between diverse intellectual currents, and they precisely say: but we don’t have to abandon humanism, but we don’t have to abandon Teilhard de Chardin, but existentialism is also a little bit right , but structuralism too, if only it wasn’t doctrinaire, but concrete and open to the world.  Opposed to this current, which puts coexistence at the first rank, you have a current that the ‘right wing people’ call doctrinaire, neostalinist and Chinese.  This tendency inside the PCF is an attempt to reestablish a Marxist theory of politics, of science and of philosophy which is a consequential theory, ideologically acceptable, in accord with the doctrine of Marx.  It is this attempt which at this moment effected [opérée] by the intellectuals of the left wing of the party, and they are more or less regrouped around Althusser.  This structuralist wing is the left.  You understand now what this maneuver of Sartre and Garaudy consists in, knowing how to pretend that structuralism is a typical ideology of the right.  It lets them designate as accomplices of the right those who are in reality to their left.  It lets them consequently present themselves as the only true representatives of the French left and communists.  But this is only a maneuver” (686)

In this sense, Teilhard seems to be less of interest to Foucault than what Garaudy in particular is doing with Teilhard within the PCF.  Teilhard is a stalking horse for Marxist humanism, which (like other humanisms) is to be combatted. 

Next time, I’ll pick up from here…

Signal or Noise? Foucault and Communication Theory (part 2): Noise as Resistance

By Gordon Hull

Last time, I followed up on a reference in Bernard Dionysius Geoghegan’s Code to Foucault’s short text “Message ou bruit” (1966). Here I want to trace out some of the political implications of that text, or at least to suggest a path from it to some of his later work in the 1970s and current forms of political resistance.

One of Foucault’s emergent interests in philosophy of language is in pragmatics and speech act theory. I don’t know enough about his reading in the relevant time period to know if this exactly tracks his growing interest in politics, but by a 1978 lecture in Japan (which I discussed here) he is able to say that:

“Perhaps one could see that there is still a certain possibility for philosophy to play a role in relation to power, which would be a role neither of foundation nor of renewal of power. Perhaps philosophy can still play a role on the side of counter-power, on the condition that this role does not consist in exercising, in the face of power, the very law of philosophy, on the condition that philosophy stops thinking of itself as prophesy, on the condition that philosophy stops thinking of itself either as pedagogy, or as legislation, and that it gives itself the task to analyse, clarify, and make visible, and thus intensify the struggles that develop around power, the strategies of the antagonists within relations of power, the tactics employed, the foyers of resistance, on the condition in sum that philosophy stops posing the question of power in terms of good and bad, but rather poses it in terms of existence. The question is not: is power good or bad, legitimate or illegitimate, a question of right or morality? Rather, one should simply try to relieve the question of power of all the moral and juridical overloads that one has placed on it, and ask the following naïve question, which has not been posed so often, even if a certain number of people have actually posed it for a long while: what do power relations fundamentally consist in?” (192)

Foucault’s emphases here – from the emphasis on local struggles to the rejection of prophetic thinking – should be familiar. He then immediately suggests that “We have known for a long time that the role of philosophy is not to discover what is concealed, but rather to make visible what precisely is visible, which is to say to make appear what is so close, so immediate, so intimately connected with ourselves that we cannot perceive it.” This is recognizable as a reference to Wittgenstein – specifically, the remark in Philosophical Investigations 128 that “the work of the philosopher consists in assembling reminders for a particular purpose.”

Signal or Noise? Foucault and Communication Theory (Part 1)

By Gordon Hull

Last time, I offered a quick synopsis of Bernard Dionysius Geoghegan’s excellent new book Code.  Here, I’d like to track one specific Foucault reference in it.  Geoghegan takes Lévi-Strauss’s Savage Mind as a central text in the ambivalence French theorists came to feel about American communication theories, and he notes that the book “occasioned a broader reassessment of the human sciences marked by a new ascent of ‘coding’ as a key concept poised to dislocate and perhaps dissolve, existing scientific hierarchies” (152).  He adds:

“Learning to code – that is, to cast cultural objects in terms of codes, relays, patterns, and systems – did more than reframe existing knowledge in cybernetic jargon. It also reflected a growing cynicism toward existing cultural and scientific nodes. From the 1960s onward, the semiotic task of deciphering obscure ‘codes’ in culture, politics, and science overtook the structuralist project. This crypto-structuralism shifted emphasis from the neutral connotations of ‘communication’ to antagonistic notions of code …. If these terms furthered the technocratic project of US foundations, they also set in motion a radical critique of scientific neutrality. Beneath the neutral science, something ‘savage’ lurked.” (152-3).

Geoghegan cites Lacan, Barthes and the Tel Quel group (on which see Danielle Marx-Scouras’s excellent study).  He also quietly footnotes Foucault’s “Message ou bruit [message or noise]” “for a critical discussion of these same terms by Foucault” (215n81).

Reading List: Bernard Dionysius Geoghegan, Code

By Gordon Hull

I made myself wait until I was settled into the summer to read Bernard Dionysius Geoghegan’s Code: From Information Theory to French Theory.  It was absolutely worth the wait. Code offers a look into the role of cybernetic theory in the development of postwar French theory, especially structuralism and what Geoghegan calls “crypto-structuralism.”  The story starts in the progressive era U.S., with the emergence of technocratic forms of government and expertise “against perceived threats of anarchy and communism” and the “progressive hopes to submit divisive political issues for neutral technical analysis” (25).  This governance as depoliticization then generates the postwar emphasis on cybernetics and information theory.  Along the way, it picks up and reorganizes psychology and anthropology in figures like Margaret Mead and Gregory Bateson, as the emerging information theory disciplines are given extensive funding by “Robber Baron philanthropies” (and later, covertly of course, by the CIA).  This then sets the stage for postwar cybernetic theory and the careful cultivation (again, substantially by philanthropies and the CIA) of intellectuals like Roman Jakobson and Lévi-Strauss.

This is not a story I’d heard before – and I get the impression that almost no one has, at least not in philosophy, which is why this book is so important – and the details are fascinating.  It makes a compelling case for the need for those of us who work on the post-war French to get a handle on cybernetic theory in particular, especially because of the link to structuralism (more on that in a moment).  It calls to mind some of Katherine Hayles’ work – I’m thinking of How we Became Posthuman and My Mother Was a Computer – that probably needs rereading in this context.

The Supreme Court’s Disappearing Public

In the face of the general disaster of the Republican majority on the Supreme Court’s ongoing power grab in the student loan case, I worry that the damage of the LGBTQ Wedding Website decision, Creative LLC v. Elenis, will get overlooked.  It seems to me, based mainly on a reading of Justice Sotomayor’s dissent, that the real forerunner of Creative LLC is a case mentioned nowhere in the decision or dissent: Burwell v. Hobby Lobby (2014).  Recall that in Burwell, the Court ruled that the Hobby Lobby Corporation could not be compelled by the Affordable Care Act to provide contraceptive coverage as part of its employees’ healthcare coverage, on account of the corporation’s religious beliefs.  At the time, I noted that Hobby Lobby seemed very happy to avail itself of things like police and fire protection.  I don’t usually quote myself in blog posts, but here’s what I said at the time:

“Hobby Lobby is a large, big-box retail chain that employs over 13,000 people.  If those people (or others like them) didn’t exist or refused to work for Hobby Lobby, the corporation would go out of business immediately and the owners would have to find something else to do.  Hobby Lobby, Inc. takes advantage of the publicly-provided roads that its employees, managers, and customers take to get to its stores and that its owners use to get to their corporate offices.  Those offices were erected with the protection of enforceable building codes that make sure they don’t fall down, and that try to make sure that everyone can evacuate them in the event of a fire.  Hobby Lobby, Inc. also takes advantage of municipally provided services, including the installation of stormwater systems that deal with the massive runoff caused by big-box stores’ parking lots.  Hobby Lobby, Inc. also takes advantage of local police and fire services that protect their investment in their stores.  All of these things are provided substantially by property taxes paid by everyone living in the municipalities where the owners exercise their freedom to open a store.  Hobby Lobby, Inc. also freely avails itself of services provided by state and federal taxes, such as the Interstate highways on which it can transport its goods (highways which have to be widened at great public expense when suburbanization creates new local markets for its stores).  Hobby Lobby, Inc. also has no moral objections to taking advantage of the national defense system that keeps its stores safe from foreign intervention, or the publicly funded legal system that allowed them to challenge the ACA and that enables them to recover money from those who owe them.  No, in general, it seems that Hobby Lobby, Inc. depends quite a lot on the society in which it does business, even as its owners seek to excuse themselves from its rules.  In the meantime, Hobby Lobby’s owners also take advantage of the legal structure governing corporations (Hobby Lobby, Inc. isn’t a sole proprietorship!), such as the fact that they aren’t personally liable for any bad things that their corporation might do.  In other words, Hobby Lobby’s owners get to identify with the corporation when it’s a matter of religious belief, but not when doing so is inconvenient.”

It was this line of thought that I most remembered when reading Justice Sotomayor’s dissent in Creative LLC.  She notes that:

Bing also hallucinates, even with footnotes

Large Language Models (LLMs) like ChatGPT are well-known to hallucinate – to make up answers that sound pretty plausible, but have no relation to reality.  That of course is because they’re designed to produce text that sounds about right given a prompt.  What sounds kind of right may or may not be right, however.   ChatGPT-3 made up a hilariously bad answer to a Kierkagaard prompt I gave it and put a bunch of words into Sartre’s mouth.  It also fabricated a medical journal article to support a fabricated risk to oral contraception. ChatGPT-4 kept right on making up cites for me.  It has also defamed an Australian mayor and an American law professor.  Let’s call this a known problem.  You might even suggest, following Harry Frankfurt, that it’s not so much hallucinating as it is bullshitting.

Microsoft’s Bing chatbot-assisted search puts footnotes in its answers.  So it makes sense to wonder if it also hallucinates, or if it does better.  I started with ChatGPT today and asked it to name some articles by “Gordon Hull the philosopher.”  I’ll spare you the details, but suffice it to say it produced a list of six things that I did not write.  When I asked it where I might read one of them, it gave me a reference to an issue of TCS that included neither an article by me nor an article of that title.

So Bing doesn’t have to be spectacular to do better!  I asked Bing the same question and got the following:

ChatGPT Reread Kierkegaard but still makes up cites

Recall that ChatGPT a couple of months ago did a total face plant on the topic of Kierkegaard's knight of faith from the knight of infinite resignation.  Well, with the fullness of time and an upgrade, it's a lot better now: (screen grabs below the fold)

Some Reasons to be Skeptical of AI Authorship, Part 3: Capitalism and Social Justice

By Gordon Hull

In the previous two posts (here and here) I’ve developed a political account of authorship (according to which whether we should treat an AI as an author for journal articles and the like is a political question, not one about what the AI is, or whether its output resembles human output), and argued that AIs can’t be property held accountable.  Here I want to argue that AI authorship risks social justice concerns.

That is, there are social justice reasons to expand human authorship that are not present in AI.  As I mentioned in the original post, researchers like Liboiron are trying to make sure that the humans who put effort into papers, in the sense that they make it possible, get credit.  In a comment to that post, Michael Muller underlines that authorship interacts with precarity in complex ways.  For example, “some academic papers have been written by collectives. Some academic papers have been written by anonymous authors, who fear retribution for what they have said.”  Many authors have precarious employment or political circumstances, and sometimes works are sufficiently communal that entire communities are listed as authors. There are thus very good reasons to use authorship strategically when there are minoritized individuals or people in question.  My reference to Liboiron is meant only to indicate the sort of issue in the strategic use of authorship to protect minoritized or precarious individuals, and to gesture to the more complex versions of the problem that Muller points to.  The claim I want to make here is that , as a general matter, AI authorship isn’t going to help those minoritized people, and might well make matters worse.

If anything, therre’s a plausible case that elevating an AI to author status will make social justice issues worse.  There’s at least two ways to get to that result, one specific to AI and one more generally applicable to cognitive labor.

ChatGPT Putting Words in Sartre's Mouth

As if Sartre didn't produce enough words all by himself!

ChatGPT's response to the following prompt is instructive for those of us who are concerned about ChatGPT being used to cheat.  Read past the content of the answer to notice the made-up citations.  The "consciousness is a question..." line is in fact in the Barnes translation of Being and Nothingness, but is actually a term in the glossary provided by the translator (so it's not on p. 60 - it's on p. 629).  Where did the AI find this?  I'm guessing on the Wikipedia page for the book, which has a "special terms" section that includes the quote (and attributes it to Barnes.  I should add as an aside that Barnes puts it in quote marks, but doesn't reference any source).  The "separation" quote is, as far as I can tell, made up whole cloth.  It does sound vaguely Sartrean, but it doesn't appear to be in the Barnes translation, and I can't find it on Google.  It's also worth pointing out that neither quote is from the section about the cafe - both page numbers are from the bad faith discussion. 

I don't doubt that LLMs will get better (etc etc etc) but for now, bogus citations are a well-known hallmark of ChatGPT.  Watch it make-up quotes from Foucault (and generally cause him to turn over in his grave) here.