New APPS: Art, Politics, Philosophy, Science

Gatecrashers, Blue Buses and Speeding Drivers: Philosophy of Law as a way into AI Accountability (part 1)

By Gordon Hull

AI systems are notoriously opaque black boxes.  In a now standard paper, Jenna Burrell dissects this notion of opacity into three versions.  The first is when companies deliberately hide information about their algorithms, to avoid competition, maintain trade secrets, and to guard against gaming their algorithms, as happens with Search Engine Optimization techniques.  The second is when reading and understanding code is an esoteric skill, so the systems will remain opaque to all but a very small number of specially-trained individuals.  The third form is unique to ML systems, and boils down to the argument that ML systems generate internal networks of connections that don’t reason like people.  Looking into the mechanics of a system for recognizing handwritten numbers or even a spam detection filter wouldn’t produce anything that a human could understand.  This form of opacity is also the least tractable, and there is a lot of work trying to establish how ML decisions could be made either more transparent or at least more explicable.

Joshua Kroll argues instead that the quest for potentially impossible transparency distracts from what we might more plausibly expect from our ML systems: accountability.  After all, they are designed to do something, and we could begin to assess them according to the internal processes by which they are developed to achieve their design goals, as well as by empirical evidence of what happens when they are employed.  In other words, we don’t need to know exactly how the system can tell a ‘2’ from a ‘3’ as long as we can assess whether it does, and whether that objective is serving nefarious purposes.

I’ve thought for a while that there’s potential help for understanding what accountability means in the philosophy of law literature.  For example, a famous thought experiment features a traffic accident caused by a bus.  We have two sources of information about this accident.  One is an eyewitness who is 70% reliable and says that the bus was blue.  The other is the knowledge that 70% of the buses that were in the area at the time were blue.  Epistemically, these ought to be equal – in both cases, you can say with 70% confidence that the blue bus company is liable for the accident.  But we don’t treat them as the same: as David Enoch and Talia Fisher elaborate, most people prefer the witness to the statistical number.  This is presumably because when the witness is wrong, we can inquire what went wrong.  When the statistic is wrong, it’s not clear that anything like a mistake even happened: the statistics operate at a population level; when applied to individuals, the use of statistical probability will be wrong 30% of the time, and so we have to expect that.  It seems to me that our desire for what amounts to an auditable result is the sort of thing that Kroll is pointing to.

Foucault and Epistemic Injustice (Part 3)

In the previous two posts (first, second), I took up the invitation provided by a recent paper by Daniele Lorenzini to develop some thoughts on the relationship between Foucault’s thought and theorizing around epistemic injustice.  In particular, Miranda Fricker’s account both draws heavily from Foucault and pushes back against his historicism to advocate for a more a-historical normative ground for the theory: testimonial injustice “distorts” who someone is.  Last time, I looked at some of Foucault’s own work in the lectures leading up to Discipline and Punish to develop a sense of how both “truth” and “power” are relevant – and distinguishable – in that work, even as they both are historical constructs. In particular, following Lorenzini, we can distinguish between “x is your diagnosis” and “therefore, you ought to do y.” Here I begin with the complexity introduced in Foucault’s work by his addition of an embryonic genealogy of truth practices. 

Let’s begin with the Psychiatric Power lectures, where Foucault had been talking about the strange role of science (and its personification in the doctor) in the governance of asylums.  There, when speaking of the historical contingency of the modern scientific enterprise, Foucault writes:

New Paper: How Foucault got rid of (Bossy) Marxism

Now published in Critical Review.  Here's the abstract:

Foucault distanced himself from Marxism even though he worked in an environment—left French theory of the 1960s and 1970s—where Marxism was the dominant frame of reference. By viewing Foucault in the context of French Marxist theoretical debates of his day, we can connect his criticisms of Marxism to his discussions of the status of intellectuals. Foucault viewed standard Marxist approaches to the role of intellectuals as a problem of power and knowledge applicable to the Communist party. Marxist party intellectuals, in his view, had developed rigid and universal theories and had used them to prescribe action, which prevented work on the sorts of problems that he uncovered—even though these problems were central to the development of capitalism.

The paper is an attempt to cut a path through some (mostly 1970s) texts to get a handle on what Foucault is doing with his inconsistent references to Marx and Marxism.  There's a complex tangle of issues here, many related to the vicissitudes of the reception of Marx, and I hope that others will be able to add to our understanding of them and the period.

A huge thanks to Shterna Friedman, whose editorial work resulted in a much better article.  Also, my paper is going to be part of a special issue of Critical Review on Foucault - the other papers should be appearing relatively soon.

Foucault and Epistemic Injustice (Part 2)

By Gordon Hull

Last time, I took the opportunity provided by a recent paper by Daniele Lorenzini to develop some thought on the relationship between Foucault’s thought and theorizing around epistemic injustice.  Lorenzini’s initial point, with which I agree fully, is that Fricker’s development of epistemic injustice is, on her own terms, incompatible with Foucault because she wants to maintain a less historicized normative standpoint than Foucauldian genealogy allows.  Epistemic injustice, on Fricker’s reading, involves a distortion of someone’s true identity.  Lorenzini also suggests that Foucault’s late work, which distinguishes between an epistemic “game of truth” and a normative/political “regime of truth” offers the distinction Fricker’s theory needs, by allowing one to critique the regime of truth dependent on a game of truth.  In terms of Foucault’s earlier writings, he does not fully reduce knowledge to power, in the sense that is can be useful to analytically separate them. Here I want to look at a couple of examples of how that plays out in the context of disciplinary power.

Consider the case of delinquency, and what Foucault calls the double mode of disciplinary power (Discipline and Punish, 199): a binary division into two categories (sane/mad, etc.) and then the coercive assignment of individuals into one group or the other.  The core modern division is between normal and abnormal, and we have a whole “set of techniques and institutions for measuring, supervising and correcting the abnormal” (199).  The delinquent, then, is defined epistemically or juridically (in other words, as a matter of science or law; as I will suggest below, Foucault thinks that one of the ways that psychology instituted itself as a science was by successfully blurring the science/law distinction), and then things are done to her.  This is the sort of gap that epistemic injustice theory, at least in its testimonial version, needs: in Fricker’s trial example, there is the epistemic apparatus of “scientific” racism, and then there is the set of techniques that work during the trial.  Both of these can be targets of critique, but testimonial injustice most obviously works within the second of the two.