New APPS: Art, Politics, Philosophy, Science

New Paper: Epistemic Injustice in Machine Learning Systems

From the Department of Shameless Self-Promotion, here is the abstract for my new paper, "Dirty Data Labeled Dirt Cheap: Epistemic Injustice in Machine Learning Systems:"

"Artificial Intelligence (AI) and Machine Learning (ML) systems increasingly purport to deliver knowledge about people and the world or to assist people in doing so.  Unfortunately, they also seem to frequently present results that repeat or magnify biased treatment of racial and other vulnerable minorities, suggesting that they are “unfair” to members of those groups.  However, critique based on formal concepts of fairness seems increasingly unable to account for these problems, partly because it may well be impossible to simultaneously satisfy intuitively plausible operationalizations of the concept and partly because fairness fails to capture structural power asymmetries underlying the data AI systems learn from.  This paper proposes that at least some of the problems with AI’s treatment of minorities can be captured by the concept of epistemic injustice.  I argue that (1) pretrial detention systems and physiognomic AI systems commit testimonial injustice because their target variables reflect inaccurate and unjust proxies for what they claim to measure; (2) classification systems, such as facial recognition, commit hermeneutic injustice because their classification taxonomies, almost no matter how they are derived, reflect and perpetuate racial and other stereotypes; and (3) epistemic injustice better explains what is going wrong in these types of situations than does (un)fairness."

The path from idea to paper here was slow, but I hope the paper is convincing on the point that the literature on epistemic injustice can offer some needed resources for understanding harms caused by (some kinds of ) AI/algorithmic systems.

Just because the computer talks to you about its feelings doesn’t make it sentient

By Gordon Hull

UPDATE: 6/14: Here's a nice takedown ("Nonsense on Stilts") of the idea that AI can be sentient.

I don’t remember where I read about an early text-based chatbot named JULIA, but it was likely about 20 years ago. JULIA played a flirt, and managed to keep a college student in Florida flirting back for something like three days.  The comment in whatever I read was that it wasn’t clear if JULIA had passed a Turing test, or if the student had failed one.  I suppose this was inevitable, but it appears now that Google engineer Blake Lemoine is failing a Turing test, having convinced himself that the natural language processing (NLP) system LaMDA is “sentient.”

The WaPo article linked above includes discussion with Emily Bender and Margaret Mitchell, which his exactly correct, as they’re two of the lead authors (along with Timnit Gebru) on a paper (recall here) that reminds everyone that NLP is basically a string prediction task: it scrapes a ton of text from the Internet and whatever other sources are readily available, and gets good at predicting what is likely to come next, given a particular input text.  This is why there’s such concern about bias being built into NLP systems: if you get your text from Reddit, then for any given bit of text, what’s likely to come next is racist or sexist (or both).  The system may sound real, but it’s basically a stochastic parrot, as Bender, Gebru and Mitchell put it.

So point one: LaMDA is not sentient, any more than ELIZA and JULIA were sentient, but chatbots are getting pretty good at convincing people they are.  Still, it’s disturbing that the belief is spreading to people like Lemoine who really, really ought to know better.