New APPS: Art, Politics, Philosophy, Science

New Paper: "The Death of the Data Subject"

Shameless self-promotion dept... here's the preprint for my new paper, "The Death of the Data Subject," now forthcoming in Law, Culture and the Humanities.  And here's the abstract:

This paper situates the data privacy debate in the context of what I call the death of the data subject.  My central claim is that concept of a rights-bearing data subject is being pulled in two contradictory directions at once, and that simultaneous attention to these is necessary to understand and resist the extractive practices of the data industry. Specifically, it is necessary to treat the problems facing the data subject structurally, rather than by narrowly attempting to vindicate its rights.  On the one hand, the data industry argues that subjects of biometric identification lack legal standing to pursue claims in court, and Facebook recently denied that that its facial recognition software recognizes faces.  On the other hand, industry takes consent to terms of service and arbitration clauses to create enforceable legal subject positions, while using promises of personalization to create a phenomenological subject that is unaware of the extent to which it is being manipulated.  Data subjects thus have no legal existence when it is a matter of corporate liability, but legal accountability when it is a matter of their own liability.  Successful reform should address the power asymmetries between individuals and data companies that enable this structural disempowerment.

 

 

When does the Supreme Court lose its legitimacy?

And what does that mean?  Now is a good time to ask.  The Court has let stand a 5^th Circuit decision upholding a Texas law that is plainly unconstitutional under current SCOTUS jurisprudence (it bans abortion at 6 weeks) and involves an enforcement mechanism that comes straight from Stalin’s playbook (it allows individuals to sue people they suspect of assisting a woman of obtaining an abortion).  This piece on Vox runs through how deeply perverse the Texas law in question in, how thoroughly Trump has corrupted the 5^th Circuit, and how alarming SCOTUS inaction is.

In Planned Parenthood v. Casey – which, along with Roe is apparently being overruled in Texas without a hearing and without a reasoned opinion – the Court favorably cites earlier opinion to the effect that the Court is supposed to give reasons when it overturns its precedent:

"A basic change in the law upon a ground no firmer than a change in our membership invites the popular misconception that this institution is little different from the two political branches of the Government. No misconception could do more lasting injury to this Court and to the system of law which it is our abiding mission to serve"

Justice O’Connor adds:

“The root of American governmental power is revealed most clearly in the instance of the power conferred by the Constitution upon the Judiciary of the United States and specifically upon this Court. As Americans of each succeeding generation are rightly told, the Court cannot buy support for its decisions by spending money and, except to a minor degree, it cannot independently coerce obedience to its decrees. The Court's power lies, rather, in its legitimacy, a product of substance and perception that shows itself in the people's acceptance of the Judiciary as fit to determine what the Nation's law means and to declare what it demands.  The underlying substance of this legitimacy is of course the warrant for the Court's decisions in the Constitution and the lesser sources of legal principle on which the Court draws. That substance is expressed in the Court's opinions, and our contemporary understanding is such that a decision without principled justification would be no judicial act at all. But even when justification is furnished by apposite legal principle, something more is required. Because not every conscientious claim of principled justification will be accepted as such, the justification claimed must be beyond dispute. The Court must take care to speak and act in ways that allow people to accept its decisions on the terms the Court claims for them, as grounded truly in principle, not as compromises with social and political pressures having, as such, no bearing on the principled choices that the Court is obliged to make. Thus, the Court's legitimacy depends on making legally principled decisions under circumstances in which their principled character is sufficiently plausible to be accepted by the Nation” (emphasis added)

O’Connor adds that if “the Court decides a case in such a way as to resolve the sort of intensely divisive controversy reflected in Roe” that “only the most convincing justification under accepted standards of precedent could suffice to demonstrate that a later decision overruling the first was anything but a surrender to political pressure, and an unjustified repudiation of the principle on which the Court staked its authority in the first instance.”

The Court’s so-called “shadow docket” – consequential decisions rendered in orders for cases not heard, as for example prioritizing religious claims over public health – has been on the rise over the last couple of years, and commentators have worried about the damage this does to the rule of law as an institution.  In the case of abortion, the Court has a case on the docket that would give it the opportunity to overturn Roe; it could have waited until then (and avoided the need to validate Texas’ enforcement mechanism).  You could argue that the Court didn’t “decide” anything last night, but it was faced with a clearly erroneous 5^th Circuit decision that it let stand.  I don’t see how last night’s failure to enjoin the Texas law doesn’t utterly gut its legitimacy in the sense articulated in Casey.  Texas just banned abortion, and SCOTUS offered no justification at all in letting the law stand.

Again, Casey: “to overrule under fire in the absence of the most compelling reason to reexamine a watershed decision would subvert the Court's legitimacy beyond any serious question.”  Court legitimacy is important:

“It is true that diminished legitimacy may be restored, but only slowly. Unlike the political branches, a Court thus weakened could not seek to regain its position with a new mandate from the voters, and even if the Court could somehow go to the polls, the loss of its principled character could not be retrieved by the casting of so many votes. Like the character of an individual, the legitimacy of the Court must be earned over time. So, indeed, must be the character of a Nation of people who aspire to live according to the rule of law. Their belief in themselves as such a people is not readily separable from their understanding of the Court invested with the authority to decide their constitutional cases and speak before all others for their constitutional ideals. If the Court's legitimacy should be undermined, then, so would the country be in its very ability to see itself through its constitutional ideals. The Court's concern with legitimacy is not for the sake of the Court, but for the sake of the Nation to which it is responsible.”

So much for Justice Roberts’ efforts to preserve the Court as an institution.

No Standing for You! (part 2)

By Gordon Hull

Last time, I started to look at the details of the Supreme Court’s recent TransUnion decision, which ruled that a credit agency that wrongly labeled someone as a match for a terrorist watch list (using only first and last names, with no effort at verification.  Sorry “John Smith”…) could only be sued if that person could show that the credit agency also distributed this information to others.  So basically you have to lose a job opportunity or a mortgage approval, and then you can sue.  There is nothing you can do to pre-emptively force them to correct the problem.  Worse, this ruling is in defiance of explicit statutory language in the FCRA.  The opinion directly says that SCOTUS can override Congressional determination of when something is legally actionable, and (just to be contrary, I guess), it does so in the name of separation of powers.

So what are we to make of this? As the various critical theories will remind you, law is a form of power. One thing that’s happening here is the protection of the powerful from the weak. With some high profile exceptions (like Justice Gorsuch’s endorsement of LGBTQ rights), the Roberts court has fairly consistently sided with the powerful, like corrupt politicians and conservative dark money donors by whittling away at laws designed to rein them in. One limitation to that thesis is to argue, as Erwin Chemerinsky has, that “Throughout history the court has overwhelmingly favored corporate power over employees, consumers, and the public, and has favored government power over individuals’ rights.” While it’s tempting to leave it at that here, I think there’s more nuance to be had. So two points.

No Standing for You! (part 1)

By Gordon Hull

In one of the Seinfeld episodes, the proprietor of a popular lunch stop would deny service to customers who offended his arbitrary sensibilities with a loud “No Soup for You!” This is basically the outcome of the Supreme Court’s June decision on standing, TransUnion v. Ramirez. “Standing” in this sense refers to access to the federal legal system; to simplify, to be able to sue somebody, you need to show that you’ve suffered a “concrete” injury. A number of recent standing cases have been about the outer limits of “concrete.” In principle, privacy harms can be concrete, but SCOTUS has been making standing harder, particularly in data breach cases.

And so it goes here. Per a 5-4 opinion authored by Justice Kavanaugh, you don’t have standing to sue a credit agency for flagging you as a terrorist, unless you can show that they also disseminated that information to somebody else. This opinion explicitly overrides a Congressional determination that you do, in fact, have standing in cases like that, arrogating to the Court the right to decide when Congress is and is not allowed to establish when statutory rights are actionable. This was too much for Justice Thomas, who, joined by the court’s liberals, points out in dissent that this both violates separation of powers in the name of preserving it, and turns historical standing doctrine on its head. As Dan Solove and Danielle Citron put it in a sharp critique that builds on Thomas’ dissent, TransUnion is “an activist decision that nullifies Congress’s power to protect consumers and that enables courts to rewrite privacy laws to alter how they are enforced.”

Foucault, Marx and Prophecy Part 3: On Bureaucrats

By Gordon Hull

In a previous post, I noted that Foucault strongly implies in a 1978 interview that his communist detractors are bureaucrats, and tied that to an earlier interview with Maoists in which he suggests that structuring populist tribunals on the model of bourgeois courts would fail to break with the power structure of the bourgeois court system: the model of impartiality is intrinsically bourgeois, and so importing that model into communist popular tribunals would iterate the very power structures that were to be replaced.  Here I want to flesh out a little more some of the resonances of an accusation of bureaucracy in the context of 1970s Marx debates in France.   I should say in advance that these are notes more than a complete assessment, designed to pick out highlights.  The back and forth polemics of Marxists are byzantine, and we should all be grateful that Foucault sets it as a rule not to engage in them.  Here I will mainly draw from Trotsky’s critique of Stalin and Lenin, with a closing gesture to a representative text of the French far-left that emerged in the aftermath of the 1968 student uprisings.

Patently Harmful

My further thoughts on the background through which we should interpret Covid patent waivers, at Real Life.

Foucault, Marx and Prophecy, Part 2: The Role of the Intellectual

By Gordon Hull

In a previous post, I made the case for reading Foucault’s 1978 comments on Marxism (especially in the Yoshimoto interview) in the context of theory/practice questions raised by the re-evaluation of Marx’s 11^th “Thesis on Feuerbach.”  Here I want to flesh out that position a little more, starting with reference to another 1978 piece.  Shortly after the Japan interviews, Foucault writes a response to the Italian communist Massimo Cacciari’s “Rationality and Irrationality of Politics in Delezue and Foucault” (the piece appeared in Italian; as far as I can tell, there is no translation of it).  Much of Foucault’s response is dedicated to accusing Cacciari of using the tactics of Stalinism: “always having a unique adversary” (created in this case by amalgamating Foucault, Deleuze and the New Philosophers), of engaging in a trial-like procedure, of assimilating ‘the enemy’ and danger, of reducing thought immediately to a system, and so forth.

He then defends the argument of Discipline and Punish, in particular with reference to the understanding of power; Foucault says he “would like precisely to show this heterogeneity of power, that is to say how it is always born of something other than itself” (D&E #238, II, 651 (2 vol. ed, 2001]).  He underscores that there is no metaphysics of power here, no “power with a capital P.”  Rather, “it is necessary to put the relations of power back into the interior of struggles and not suppose that there is, on the one hand, power, and on the other, that on which it is exercised, and that the struggle unfolds between power and non-power” (632).  In other words, Foucault’s entire effort is to resist the sort of abstraction that the Theses on Feuerbach warn against reifying.  The goal is rather to write a text that does something, that constitutes an intervention or a tool.  In language similar to that of the Japan interviews, and against the accusation that he is merely telling a story, Foucault proposes:

“In reality, what I am trying to do, and therein lies the difficulty of the attempt, is to work out an interpretation, a reading of a certain reality, which is such that, on the one hand, this interpretation is able to produce effects of truth and that, on the other hand, these effects of truth can become instruments at the heart of possible struggles.  To speak the truth so that it can be attacked [dire la vérité pour qu’elle soit attaquable].  It is a reality of possible struggles that I seek to make appear.” (633)

A few sentences later, he argues that “the effect of the truth that I seek to produce resides in this manner of showing that the real is polemical” (633).  However, “I do not speak of the current situation.  I effect an interpretation of history, and the problem – but I do not resolve it – is to know what the possible utilization of these analyses in the current situation is” (633-4).

This then leads to a very clear statement of the role of the intellectual.  “It is absolutely true that, when I write a book, I refuse to take a prophetic position which consists in saying to people: here is what you ought to do, or again, this is good, that is not.”  Instead, the goal is to show in a general way “it seems to me that these things have happened, but I describe them in such a manner that views of possible lines of attack are traced.  But in this I do not force or constrain anyone to attack” (634).  What to actually do – and he refers to his own decisions about throwing himself into certain political actions - “is a problem of groups, of physical and personal engagement,” and not of books.  Indeed, bossy Marxists are not radical: “one is not radical because one has pronounced some formulas; no, radicality is physical, radicality concerns existence” (634).

The Necropolitics of Intellectual Property

By Gordon Hull

Update (4/30).  Essential piece by Amy Kapczynski (who is one of my sources below).  Also see this Twitter thread by Dennis Crouch of PatentlyO, characterizing the issue as one of technology transfer more generally than patents specifically.  Trade secrets are an important issue - like patents, they allow companies to control access to innovation.  Companies also will use a strategy of combining patents and trade secrets in order to maximize their revenue (so if they think something can't be patented, they might claim it as a trade secret).  The theory behind trade secrets is based more in contracts than property, which means that they are arguably even worse if you're worried about neoliberalism, since they are an even further privatization of law and policy.

Update (4/29).  Here's more on IP and other barriers to vaccine distribution in India (and elsewhere):

• The Modi government is incompetent at every level, and India's base capacity was lower than it estimated.
• Most of the world wants IP waivers.  The U.S. and Europe and a few other countries do not. 
• This WaPo piece argues that waivers are bad but licensing (preferably voluntary, but with compulsory as a stick) is good.  Note that licensing is not what Bill Gates is talking about, as far as I can tell, which is buying vaccines and donating.

In my Biopolitics of Intellectual Property, I argue that IP policy has shifted from what I call a “public biopolitics” model to a neoliberal version.  In its briefest form: the public version treats IP as a necessary but limited monopoly to promote public goods, and the neoliberal version focuses on private wealth gain through proprietization (I summarize the argument here).

Something that I don’t particularly talk about in the book, but that one knows from Foucault, is that biopolitics comes with its inverse, necropolitics: if biopolitics is about promoting life and health for the “population,” it is also about who is allowed to die.  As Foucault puts it, “the ancient right to take life or let live was replaced by a power to foster life or disallow it to the point of death [au vieux droit de faire mourir ou de laisser vivre s'est substitué un pouvoir de faire vivre ou de rejeter dans la mort]” (History of Sexuality I, 138).  Governmental power goes from the right to kill to the power to cause people to live; death becomes something into which one is literally “thrown back.”  The leading examples of necropolitics are political, as for example Foucault’s discussion of state racism (of which Nazism is the apotheosis) in Society must be Defended.  Achille Mbembe’s “Necropolitics” article spends time on how post-colonial African states have dismantled populations, which are “disaggregated into rebels, child soldiers, victims or refugees, or civilians incapacitated by mutilation or simply massacred on the model of ancient sacrifices” (34).  Building on these, Ege Selin Islekel’s brilliant treatment of the disappeared in Turkey notes that in necropolitical spaces, “the entire content and the fact of living, constituted by the ethical, political, and epistemological conditions of life, are subsumed under death.”

However, as Ute Tellman has recently demonstrated, the political treatment of biopolitics needs to take seriously how it is co-configured with the economy.  On Tellman’s account, the notion of economic scarcity first appears in Malthus (it was missing in Smith!) as a way to police the behavior of the poor (and “savages” in the colonies) by training them to think in terms of futurity.  For Malthus, the poor have to be trained not to eat and procreate their way into oblivion by forcing them to think in terms of economic rationality.  This brings us to the neoliberal justification of IP, which is partly underpinned by the Schumpeterian thesis that innovation is to be pursued at all costs, because the gains of future innovation (“dynamic efficiency”) are more important than whatever short-term distribution problems (“static inefficiencies”) they entail.  Thus, more or less, is Harold Demsetz’s reply to Kenneth Arrow. 

It also subtends the argument being given for why IP rights around Covid vaccines shouldn’t be licensed to the poor in India (side note: Malthus served as professor at the East India Company College).  Developing countries have proposed a waiver of related IP rights to ensure the rapid production of generic Covid vaccines, and Pharma has responded with an army of lobbyists to explain that no, IP can’t possibly be the problem with Covid vaccine distribution, and it would be much better for philanthropies to purchase lots of drugs and then distribute them.  Other unrelated IP industries have followed with their own lobbyists.  Whatever other difficulties exist in getting vaccinations to people in developing countries, it seems hard to deny that insisting on IP rights and thereby limiting production of the drugs isn’t one of them.  Allowing generics – especially in India and Brazil – increases capacity.

Foucault, Marx and Prophecy: the 11th “Thesis on Feuerbach”

By Gordon Hull

Foucault thinks Marxism is bossy.  In Society must be Defended, he lays down the gauntlet clearly enough: totalizing theories get in the way of useful things at the local level.  As he notes, one should beware of:

“the inhibiting effect specific to totalitarian theories, or at least – what I mean is – all-encompassing and global theories.  Not that all-encompassing and global theories haven’t, in fairly constant fashion, provided – and don’t continue to provide – tools that can be used at the local level; Marxism and psychoanalysis are living proof that they can. But they have, I think, provided tools that can be used at the  local level only when, and this  is the real point ,the theoretical unity of their discourse is, so to speak, suspended, or at least cut up, ripped up, torn to shreds, turned inside out, displaced, caricatured, dramatized, theatricalized, and so on. Or at least that the totalizing approach always has the effect of putting the brakes on” (SMD 6).

That is, when you insist on your theoretical unities, you get in the way of actually doing anything.  What we need are to unearth “subjugated knowledges” and specific histories, and such activity requires the “removal of the tyranny of overall discourses” (SMD 8).  In his 1978 interviews with the Italian Communist Duccio Trombadori, Foucault underlines that “I absolutely will not play the part of one who prescribes solutions.  I hold that the notion of the intellectual today is not that of establishing laws or proposing solutions or prophesying, since by doing that one can only contribute to the functioning of a determinate situation of power that to my mind must be criticized” (Remarks on Marx, 157).

He similarly says at the end of an earlier 1978 interview in Japan that “I think that the role of intellectuals, in reality, absolutely does not consist in playing [the role of] prophets or legislators” (D&E #236; Vol II, 264-5 (2 vol, 2001 ed.)).  Marxism is theological, as he says in his 1979-80 Collège de France Lectures:

“With Marxism, it’s the same thing.  You have the model of the fall, alienation and dis-alienation.  You have the model of the two ways: Mao Zedong.  And you have, of course, the problem of the stain of those who are originally soiled and must be purified: Stalinism.  Marx, Mao, Stalin; the three models of the two ways, the fall, and the stain” (Government of the Living,  108)

At one level this is clear enough.   But Foucault also of course is an advocate of social change, and he wants his works to be picked up and used in local struggles, as he also says repeatedly.  Here I want to add a little specificity to the question about Foucault’s relation to Marxism (at least as he understands it around 1978) by picking up on his remarks in the Japan interview. Immediately after saying that intellectuals should not be prophets or legislators, he adds that: “for two thousand years, philosophers have always spoken of what we should have done [de ce qu’on devait faire].  But this always led to a tragic end.  What is important is that philosophers speak of what is currently happening, but not of what could happen” (624).  The first thing to note is the implicit reference to Lenin, at least in the context of a discussion of Marxism: Lenin’s What is to be Done is translated into French as Que Faire (the French rendering is correct: the book is Что Дълать, literally “What to Do”).  More significantly, the connection to prophetic discourse is something Foucault repeats in his slightly earlier interview with Yoshimoto, where he says that:

Paul Rabinow (1944-2021)

Via Foucault News:

“Paul Rabinow, UC Berkeley professor emeritus of anthropology and world-renowned anthropologist, died April 6 at the age of 76 in his Berkeley home.

Rabinow spent about 41 years at UC Berkeley between 1978 to 2019, serving as the director of anthropology for the Contemporary Research Collaboratory and as the former director of human practices for the Synthetic Biology Engineering Research Center.”

Rabinow was of course a vital conduit for Foucault’s work into English; my introduction to Foucault in grad school was by way of the Foucault Reader he edited.

His work was also important to me as I worked to understand biopolitics beyond Foucault’s own texts, extending the inquiry into topics like the human genome project and medical risk, and offering cautions against appropriations of the concept by Negri and Agamben. 

He writes at the end of his Anthropos Today that “my diagnosis is that worldviews concerned with progress and decadence as essential elements of a totalizing figure should be allowed to retire into the past, to take their place as historical memories.  By relinquishing them we will enable reason to better confront contemporary problems” (133-4), and concludes:

“Jean Starobinski advocates a criticism that seeks neither “the totality (as with the gaze from above), nor [. . .] intimacy (as does a self-identificatory intuition).” The critical practice is one that finds the means to navigate these relations of distance and closeness. There is no “quasi-divinity” present here, only a disciplined human curiosity. Let us agree with Starobinski that method re-quires motion. A movement that goes on “inlassablement,” tirelessly, steadfastly, persistently. I advocate pursuing in our thought and writing something like the motion, through different scales and different subject positions, that Starobinski proposes in the quote and exemplifies in his criticism. Such movement is easy to initiate and hard to master. Yet I firmly believe that in the actual conjuncture of things, it is a paramount challenge for philosophy and the human sciences to experiment with forms that will be, if not fully adequate to, at least cognizant of, the need for such movement through scale and subjectivity. Such motion might help us to leave notions like progress behind and even to help us to take better care of things, ourselves, and others” (136)

On Covid data, part n

I've been commenting off and on about the vagaries of Covid data - for example, in knowing what "covid cases" refers to (and here); states' early conflation  of PCR and antibody tests;  the vagaries of different testing technologies, or the ways that even death certificates can mislead about mortality.  This reflective piece by the co-founders of the Covid Tracking Project at the Atlantic - which became the de facto national source for Covid information as the federal government fell flat on its face - is absolutely worth the read.  Here's one short paragraph that says everything you need to know about data in general:

"Data are just a bunch of qualitative conclusions arranged in a countable way. Data-driven thinking isn’t necessarily more accurate than other forms of reasoning, and if you do not understand how data are made, their seams and scars, they might even be more likely to mislead you"

Marx explains how AI systems exploit workers (Part 1)

By Gordon Hull

“Factory work exhausts the nervous system to the uttermost; at the same time, it does away with the many-sided play of the muscles, and confiscates every atom of freedom, both in bodily and in intellectual activity” (Marx, Capital I [Penguin Ed.], 548).

A recent piece by Josh Dzieza in the Verge about the working conditions of those subject to discipline by AI in the workplace is wrenching.  It tells the story of Amazon workers and call center employees and others whose work lives have become an exhausting monotony of ever-intensifying robot-like demands.  The “AI boss” becomes ever more demanding, filling every moment at work with the requirement to be more productive and with absurd metrics for deciding when its demands are met.  Amazon workers report that they are basically ground down by the unsustainability of the pace of productivity; they are either fired or forced to quit, often with injuries.  Call center employees are subject to Orwellian monitoring by a machine that purports (often laughably inaccurately) to monitor and tweak their affect when speaking to customers.  Folks working from home find it impossible to go to the bathroom, because they are forced to sit in front of their computer all the time, which takes frequent pictures of them and reports on their productivity, as measured by lines of code or keystrokes per unit time.

All of this optimized misery reminds us of something important: AI systems are being deployed under capitalism.  A worker subject to an “AI Boss” is not some sort of new-fangled social form; they are fundamentally a worker subject to capitalist exploitation.  What AI has done is enable capitalism to be more intensively exploitative.  The stories in the article echo the stories Marx reports in Capital in his discussion of machinery, as does a lot of the logic.  This suggests that two popular narratives about AI need to be displaced.  First, it may be that AI will someday swoop in and take away everyone’s jobs.  As Dzieza notes, this narrative deflects attention from the pressing reality of deteriorating workplace conditions under AI supervision.  Second, Marx’s work on machines and technology is frequently read (as by the autonomists) through the Grundrisse “Fragment on Machines,” and in a way that emphasizes affective labor and cognitive capital.  The reality of AI today serves as an important reminder that the treatment in Capital is still very important.  Let me explain.

The Elephant in the Vaccine Room

You might have heard that minorities are hesitant about getting a Covid vaccine?  Well, about that.  According to polling reported by Axios, the group least likely to want a vaccine is White Republicans... to the point that "White Americans are now less likely than Black and Latino Americans to say they plan to get the vaccine."  I

Google fires another AI ethicist

This time Margaret Mitchell, one of the other authors on the fabulous "Stochastic Parrots" paper (that's my post on it.  The paper is here) on natural language processing.  This was obviously coming, since they'd suspended her email account weeks ago.  In case you haven't read the paper (and you really should!), it's worth mentioning that the paper does not mention Google, and the goal is to point to strategies to make AI better, i.e., less likely to entrench racism, sexism and so forth while destroying the planet in climate catastrophe.  It turns out that those are related, because bigger and bigger datasets are both carbon intensive and likely to pick up on and magnify hegemonic and toxic speech.  Attention to efficiency and curation moves the needle on both.

No commitment to even looking like they care about ethical AI.

Reading List: Bender and Gebru, “Stochastic Parrots”

By Gordon Hull

Not long ago, Google summarily dumped Timnit Gebru, one of its lead AI researchers and one of the few Black women working in AI.  Her coauthor Emily Bender has now posted the paper (to be presented this spring) that apparently caused all the trouble.  It should be required reading for anybody who cares about the details of how AI and data systems can perpetuate racism, or who cares more generally about the social implications of brute-force approaches to AI.  Bender and Gebru take up a common approach to natural-language processing (NLP), which involves an AI system learning how to anticipate what speech is likely to follow a given unit of speech.  If, for example, I say “Hello, how are,” the system learns by studying a dataset of existing phrases and text snippets that the next word is likely to be “you,” but almost certainly will not be “ice cream.”  How good the computer gets at this game is going to be substantially determined by the quantity and quality of its training data, i.e., the text that it examines.

Bender and Gebru outline the social costs of one approach to this problem, which is basically brute force.  As processing power increases, it’s possible to train computers with larger and larger datasets, and the use of larger datasets reliably improves system performance.  But should we be doing that?  Bender and Gebru detail several kinds of problems.  The first is environmental justice: all that processing power uses a lot of energy.  Although some of it may come from carbon-neutral sources, the net climate cost is significant.  Worse, the NLP systems being produced don’t benefit the people that are going to suffer the most from climate change.  As they memorably put it:

“Is it fair or just to ask, for example, that the residents of the Maldives (likely to be underwater by 2100) or the 800,000 people in Sudan affected by drastic floods, pay the environmental price of training and deploying ever larger English LMs, when similar large-scale models aren’t being produced for Dhivehi or Sudanese Arabic?”

Covid Deaths and the Politics of Obituaries

By Gordon Hull

As of this writing, approximately 421,000 people in the United States have officially died of Covid-19.  We also know that this number is fewer than the number that have actually died of Covid for a variety of reasons.  For example, early in the pandemic, there was nowhere near enough testing, and so many people who died of Covid-19 never received an official diagnosis.  For that sort of reason, measuring “excess deaths” during a given period is one way to try to get a handle on how many people actually died of Covid-19.  If n more people die during a given period this year than last year and/or typically, that number can be a useful indicator for how many deaths can be attributed to the pandemic.

Some of these will of course be indirect: people who didn’t go to the ER over a heart attack because they were afraid of contracting Covid, for example.  The isolation of Covid may be driving an increase in opioid overdose deaths.  Others will be direct, deaths where Covid was the underlying cause. STAT News is now reporting on the results of a recent study that takes a dive into this question, concluding that the overall direct Covid death toll is 31% higher than official figures.  That’s the top line, and it means that the national number of Covid deaths is approaching 550,000.  What should surprise no one (but is surprising, because deaths are so much more reliable for understanding infection rates than reported cases) is that the mortality data is deeply shaped by politics: excess deaths are a window into the sociology of the pandemic.

New Paper: "Infrastructure, Modulation, Portal: Thinking with Foucault about how Internet Architecture Shapes Subjects"

Now up on SSRN.  This paper uses Foucault's works on disciplinary power to develop a typology for understanding different models of Internet governance.  Here is the abstract:

Following Foucault’s remarks on the importance of architecture to disciplinary power, this paper offers a typology of power relations expressed in different models of Internet governance. Infrastructure governance understands the Internet as a common pool or public resource, on the model of traditional infrastructures like roads and bridges. Modulation, which I study by way of Net Neutrality debates in the U.S., understands Internet governance as traffic shaping. Portal governance, which I study by way of data collection policies of dominant platform companies, understands the Internet as creating a user experience that facilitates data mining. The latter two are forms of architectural disciplinary power that undermine the first. I then argue that the rise of portal and modulation governance primarily serves to remake parts of civil society by fostering market norms of consumption and entrepreneurialism. In that sense, efforts to shape Internet architecture need to be understood as techniques of subjectification.

 

Epistemic Justice with AI: indicators, diverse training data and knee pain

Knee pain is common and debilitating, and it’s often caused by osteoarthritis in the knee.  Treatment options range from analgesics (including opioids) to knee-replacement surgery.  If you go to the doctor with arthritic knee pain, you can get an x-ray which can then be interpreted using standard rubrics like the Kellgren–Lawrence Grade (KLG) to quantify damage to your knee and then guide treatment options. The KLG isn’t perfect in that the correlation between pain and objective scores of damage to the knee isn’t perfect.  Some people’s knees are a wreck and they report no pain; others have pain beyond what their KLG score indicate. But here’s the thing: Black patients consistently report more knee pain than white patients.  They also tend to have more knee damage on the KLG – but even when you factor that in, Black patients report much more knee pain than white patients with comparable KLG scores.  What’s going on?

One possibility is that factors external to the knee – stress, for example – explain the higher pain.  If that’s the case, then patients need less knee treatment.  But what if their knees were in worse shape?  To answer that question, you’d have to ask yourself what in an x-ray indicated poor knee condition.

Disease is often measured through indicators, and we know that these indicators can lead to all sorts of complexity.  In the context of Covid, for example, there are all sorts of questions about testing and sensitivity that I’ve talked about before.  Along the way, I referred to a fantastic paper on malaria testing in sub-Saharan Africa – suffice it to say that “cases of malaria” reported to donor organizations is a difficult number to parse for reasons having to do with vagaries in testing and diagnosis.

In a new paper in Nature Medicine, a team led by Emma Pierson makes ingenious use of artificial intelligence to tackle the problem of racial disparities in knee pain.  Since algorithms and data are so often implicated in increasing or magnifying racial disparities (see, for example, Safiya Noble on Google, or Timnit Gebru on facial recognition, or Margaret Hu’s chilling “Algorithmic Jim Crow”), it’s encouraging to learn about machine learning working to undermine racial disparities.  Ordinarily, you train an algorithm to perform like an excellent clinician.  In this case, that would mean training it to look at radiography and determine the correct KLG score.  The trick here was to instead train it to look at pain: to determine what features of the x-ray predicted that the patient would report pain.  It turns out that the algorithm’s diagnoses reduced racial disparities in diagnosis by a jaw-dropping 47%.

BIPA Continues to Gain Teeth: Data Retention Policy & Standing

By Gordon Hull

I’ve written about the importance of Illinois’ Biometric Information Privacy Act (BIPA) before (see also here). Briefly, BIPA is the most important and powerful of the (relatively few) state laws designed to protect biometric privacy. The statute establishes a notice-and-consent regime (sigh. better than nothing, though N&C doesn’t work well, and is disturbing as a norm) for private parties that collect biometric information like face scans, establishes the need for data retention policies, establishes a private right of action (individuals can sue; other states make you go through the state attorney general) and establishes a statutory harm – as underlined by the Illinois Supreme Court, violating the statute is enough to collect damages.

Companies like Facebook have been fighting BIPA hard, because it’s bad for their business model, and some of the main litigation has been around Facebook’s phototagging feature. A lot of the issue has been about standing – whether aggrieved parties have the right to sue. Standing sounds simple: in order to have standing under Article 3 of the Constitution, three conditions need to be met. Going through them, however, will indicate why this is harder than it looks.

• First, the plaintiff must have suffered an “injury in fact,” an invasion of a legally protected interest which is (a) concrete and particularized, and (b) actual or imminent, not “conjectural” or “hypothetical.”
• Second, there must be a causal connection between the injury and the conduct complained of, the injury has to be fairly traceable to the challenged action of the defendant, and not the result of the independent action of some third party not before the court.
• Third, it must be "likely," as opposed to merely "speculative," that the injury will be "redressed by a favorable decision."

The Strange Story of the Oxford Covid-19 Vaccine Dosing

To review the issue: The Oxford/AstraZeneca vaccine uses a modified adenovirus, as do several other vaccines in development, most notably the Russian Gamaleya Institute one.  Early, puzzling results suggested that the Oxford vaccine was about 70% effective overall, but that the overall number obscured a disparity between two groups: a two-dose group of all ages that showed efficacy in the 62% range but 90% in a group (including no elderly people) that had received a half-strength dose first.  Eh?  Even more strangely, the half-dose group seemed to be have given that dose... by mistake?  What happened?

Reuters does a deep dive.  There's a lot of threads, but it appears that the problem began when the Oxford team didn't trust measurements of the strength of a batch of vaccine from an Italian manufacturer.  The Oxford team then measured it using a different technique, concluded it was more potent than the manufacturer said, and trusted its own measurement.  Guess who was right? [Time to scream at the void: WHY on earth, given discrepancies between two measurements, both supposedly reliable but using different techniques, WOULD YOU NOT MEASURE AGAIN UNTIL YOU FELT VERY GOOD ABOUT THE DISCREPANCY AND HOW IT HAPPENED?]

"Oxford’s measurement showed that the batch was more potent than the Italian manufacturer had found, the documents show. Oxford trusted its own result and wanted to remain consistent with a measuring tool it had used throughout an earlier trial phase. So it asked Britain’s drugs regulator for permission to reduce the volume of vaccine injected into trial participants from the K.0011 batch. Permission was granted .... The documents published in The Lancet confirm that the error lay with the Oxford researchers. A common emulsifier, polysorbate 80, used in vaccines to facilitate mixing, had interfered with the ultraviolet-light meter that measures the quantity of viral material, according to the documents. As a result, the vaccine’s viral concentration was overstated and Oxford ended up administering half doses of vaccine, believing they were full doses."

This vaccine matters because it is one that developing countries are depending on: it is cheap, and it is stable in refrigerator temperatures, which makes the logistics a lot easier.  There is a hypothesis that explains the half-dose/full-dose discrepancy: it's possible that the lower first does primes the immune system better than the higher one; adenovirus variants also circulate in some human populations, and so it's possible that some combination of prior infection and the first dose generated an immune response sufficiently robust that participant's own immune systems destroyed the second dose's virus before it could generate further immune response.   The Russians report a 90% efficacy, and that vaccine uses two different adenovirus vectors across the two doses to avoid precisely this risk.  These questions need more data, and we're still waiting for the full results of ongoing trials.  Adenovirus vaccines matter too: in addition to Oxford, Gamaleya and a Chinese vaccine, the single-dose Johnson & Johnson vaccine, with trial results expected in January, is also adenovirus.  In the meantime, Gamaleya is sharing their adenovirus vector, in order to enable a combined trial: one does of the Russian vaccine, and one of the Oxford/Astrazeneca one.