New APPS: Art, Politics, Philosophy, Science

The Necropolitics of Intellectual Property

By Gordon Hull

Update (4/30).  Essential piece by Amy Kapczynski (who is one of my sources below).  Also see this Twitter thread by Dennis Crouch of PatentlyO, characterizing the issue as one of technology transfer more generally than patents specifically.  Trade secrets are an important issue - like patents, they allow companies to control access to innovation.  Companies also will use a strategy of combining patents and trade secrets in order to maximize their revenue (so if they think something can't be patented, they might claim it as a trade secret).  The theory behind trade secrets is based more in contracts than property, which means that they are arguably even worse if you're worried about neoliberalism, since they are an even further privatization of law and policy.

Update (4/29).  Here's more on IP and other barriers to vaccine distribution in India (and elsewhere):

• The Modi government is incompetent at every level, and India's base capacity was lower than it estimated.
• Most of the world wants IP waivers.  The U.S. and Europe and a few other countries do not. 
• This WaPo piece argues that waivers are bad but licensing (preferably voluntary, but with compulsory as a stick) is good.  Note that licensing is not what Bill Gates is talking about, as far as I can tell, which is buying vaccines and donating.

In my Biopolitics of Intellectual Property, I argue that IP policy has shifted from what I call a “public biopolitics” model to a neoliberal version.  In its briefest form: the public version treats IP as a necessary but limited monopoly to promote public goods, and the neoliberal version focuses on private wealth gain through proprietization (I summarize the argument here).

Something that I don’t particularly talk about in the book, but that one knows from Foucault, is that biopolitics comes with its inverse, necropolitics: if biopolitics is about promoting life and health for the “population,” it is also about who is allowed to die.  As Foucault puts it, “the ancient right to take life or let live was replaced by a power to foster life or disallow it to the point of death [au vieux droit de faire mourir ou de laisser vivre s'est substitué un pouvoir de faire vivre ou de rejeter dans la mort]” (History of Sexuality I, 138).  Governmental power goes from the right to kill to the power to cause people to live; death becomes something into which one is literally “thrown back.”  The leading examples of necropolitics are political, as for example Foucault’s discussion of state racism (of which Nazism is the apotheosis) in Society must be Defended.  Achille Mbembe’s “Necropolitics” article spends time on how post-colonial African states have dismantled populations, which are “disaggregated into rebels, child soldiers, victims or refugees, or civilians incapacitated by mutilation or simply massacred on the model of ancient sacrifices” (34).  Building on these, Ege Selin Islekel’s brilliant treatment of the disappeared in Turkey notes that in necropolitical spaces, “the entire content and the fact of living, constituted by the ethical, political, and epistemological conditions of life, are subsumed under death.”

However, as Ute Tellman has recently demonstrated, the political treatment of biopolitics needs to take seriously how it is co-configured with the economy.  On Tellman’s account, the notion of economic scarcity first appears in Malthus (it was missing in Smith!) as a way to police the behavior of the poor (and “savages” in the colonies) by training them to think in terms of futurity.  For Malthus, the poor have to be trained not to eat and procreate their way into oblivion by forcing them to think in terms of economic rationality.  This brings us to the neoliberal justification of IP, which is partly underpinned by the Schumpeterian thesis that innovation is to be pursued at all costs, because the gains of future innovation (“dynamic efficiency”) are more important than whatever short-term distribution problems (“static inefficiencies”) they entail.  Thus, more or less, is Harold Demsetz’s reply to Kenneth Arrow. 

It also subtends the argument being given for why IP rights around Covid vaccines shouldn’t be licensed to the poor in India (side note: Malthus served as professor at the East India Company College).  Developing countries have proposed a waiver of related IP rights to ensure the rapid production of generic Covid vaccines, and Pharma has responded with an army of lobbyists to explain that no, IP can’t possibly be the problem with Covid vaccine distribution, and it would be much better for philanthropies to purchase lots of drugs and then distribute them.  Other unrelated IP industries have followed with their own lobbyists.  Whatever other difficulties exist in getting vaccinations to people in developing countries, it seems hard to deny that insisting on IP rights and thereby limiting production of the drugs isn’t one of them.  Allowing generics – especially in India and Brazil – increases capacity.

Foucault, Marx and Prophecy: the 11th “Thesis on Feuerbach”

By Gordon Hull

Foucault thinks Marxism is bossy.  In Society must be Defended, he lays down the gauntlet clearly enough: totalizing theories get in the way of useful things at the local level.  As he notes, one should beware of:

“the inhibiting effect specific to totalitarian theories, or at least – what I mean is – all-encompassing and global theories.  Not that all-encompassing and global theories haven’t, in fairly constant fashion, provided – and don’t continue to provide – tools that can be used at the local level; Marxism and psychoanalysis are living proof that they can. But they have, I think, provided tools that can be used at the  local level only when, and this  is the real point ,the theoretical unity of their discourse is, so to speak, suspended, or at least cut up, ripped up, torn to shreds, turned inside out, displaced, caricatured, dramatized, theatricalized, and so on. Or at least that the totalizing approach always has the effect of putting the brakes on” (SMD 6).

That is, when you insist on your theoretical unities, you get in the way of actually doing anything.  What we need are to unearth “subjugated knowledges” and specific histories, and such activity requires the “removal of the tyranny of overall discourses” (SMD 8).  In his 1978 interviews with the Italian Communist Duccio Trombadori, Foucault underlines that “I absolutely will not play the part of one who prescribes solutions.  I hold that the notion of the intellectual today is not that of establishing laws or proposing solutions or prophesying, since by doing that one can only contribute to the functioning of a determinate situation of power that to my mind must be criticized” (Remarks on Marx, 157).

He similarly says at the end of an earlier 1978 interview in Japan that “I think that the role of intellectuals, in reality, absolutely does not consist in playing [the role of] prophets or legislators” (D&E #236; Vol II, 264-5 (2 vol, 2001 ed.)).  Marxism is theological, as he says in his 1979-80 Collège de France Lectures:

“With Marxism, it’s the same thing.  You have the model of the fall, alienation and dis-alienation.  You have the model of the two ways: Mao Zedong.  And you have, of course, the problem of the stain of those who are originally soiled and must be purified: Stalinism.  Marx, Mao, Stalin; the three models of the two ways, the fall, and the stain” (Government of the Living,  108)

At one level this is clear enough.   But Foucault also of course is an advocate of social change, and he wants his works to be picked up and used in local struggles, as he also says repeatedly.  Here I want to add a little specificity to the question about Foucault’s relation to Marxism (at least as he understands it around 1978) by picking up on his remarks in the Japan interview. Immediately after saying that intellectuals should not be prophets or legislators, he adds that: “for two thousand years, philosophers have always spoken of what we should have done [de ce qu’on devait faire].  But this always led to a tragic end.  What is important is that philosophers speak of what is currently happening, but not of what could happen” (624).  The first thing to note is the implicit reference to Lenin, at least in the context of a discussion of Marxism: Lenin’s What is to be Done is translated into French as Que Faire (the French rendering is correct: the book is Что Дълать, literally “What to Do”).  More significantly, the connection to prophetic discourse is something Foucault repeats in his slightly earlier interview with Yoshimoto, where he says that:

Paul Rabinow (1944-2021)

Via Foucault News:

“Paul Rabinow, UC Berkeley professor emeritus of anthropology and world-renowned anthropologist, died April 6 at the age of 76 in his Berkeley home.

Rabinow spent about 41 years at UC Berkeley between 1978 to 2019, serving as the director of anthropology for the Contemporary Research Collaboratory and as the former director of human practices for the Synthetic Biology Engineering Research Center.”

Rabinow was of course a vital conduit for Foucault’s work into English; my introduction to Foucault in grad school was by way of the Foucault Reader he edited.

His work was also important to me as I worked to understand biopolitics beyond Foucault’s own texts, extending the inquiry into topics like the human genome project and medical risk, and offering cautions against appropriations of the concept by Negri and Agamben. 

He writes at the end of his Anthropos Today that “my diagnosis is that worldviews concerned with progress and decadence as essential elements of a totalizing figure should be allowed to retire into the past, to take their place as historical memories.  By relinquishing them we will enable reason to better confront contemporary problems” (133-4), and concludes:

“Jean Starobinski advocates a criticism that seeks neither “the totality (as with the gaze from above), nor [. . .] intimacy (as does a self-identificatory intuition).” The critical practice is one that finds the means to navigate these relations of distance and closeness. There is no “quasi-divinity” present here, only a disciplined human curiosity. Let us agree with Starobinski that method re-quires motion. A movement that goes on “inlassablement,” tirelessly, steadfastly, persistently. I advocate pursuing in our thought and writing something like the motion, through different scales and different subject positions, that Starobinski proposes in the quote and exemplifies in his criticism. Such movement is easy to initiate and hard to master. Yet I firmly believe that in the actual conjuncture of things, it is a paramount challenge for philosophy and the human sciences to experiment with forms that will be, if not fully adequate to, at least cognizant of, the need for such movement through scale and subjectivity. Such motion might help us to leave notions like progress behind and even to help us to take better care of things, ourselves, and others” (136)

On Covid data, part n

I've been commenting off and on about the vagaries of Covid data - for example, in knowing what "covid cases" refers to (and here); states' early conflation  of PCR and antibody tests;  the vagaries of different testing technologies, or the ways that even death certificates can mislead about mortality.  This reflective piece by the co-founders of the Covid Tracking Project at the Atlantic - which became the de facto national source for Covid information as the federal government fell flat on its face - is absolutely worth the read.  Here's one short paragraph that says everything you need to know about data in general:

"Data are just a bunch of qualitative conclusions arranged in a countable way. Data-driven thinking isn’t necessarily more accurate than other forms of reasoning, and if you do not understand how data are made, their seams and scars, they might even be more likely to mislead you"

Marx explains how AI systems exploit workers (Part 1)

By Gordon Hull

“Factory work exhausts the nervous system to the uttermost; at the same time, it does away with the many-sided play of the muscles, and confiscates every atom of freedom, both in bodily and in intellectual activity” (Marx, Capital I [Penguin Ed.], 548).

A recent piece by Josh Dzieza in the Verge about the working conditions of those subject to discipline by AI in the workplace is wrenching.  It tells the story of Amazon workers and call center employees and others whose work lives have become an exhausting monotony of ever-intensifying robot-like demands.  The “AI boss” becomes ever more demanding, filling every moment at work with the requirement to be more productive and with absurd metrics for deciding when its demands are met.  Amazon workers report that they are basically ground down by the unsustainability of the pace of productivity; they are either fired or forced to quit, often with injuries.  Call center employees are subject to Orwellian monitoring by a machine that purports (often laughably inaccurately) to monitor and tweak their affect when speaking to customers.  Folks working from home find it impossible to go to the bathroom, because they are forced to sit in front of their computer all the time, which takes frequent pictures of them and reports on their productivity, as measured by lines of code or keystrokes per unit time.

All of this optimized misery reminds us of something important: AI systems are being deployed under capitalism.  A worker subject to an “AI Boss” is not some sort of new-fangled social form; they are fundamentally a worker subject to capitalist exploitation.  What AI has done is enable capitalism to be more intensively exploitative.  The stories in the article echo the stories Marx reports in Capital in his discussion of machinery, as does a lot of the logic.  This suggests that two popular narratives about AI need to be displaced.  First, it may be that AI will someday swoop in and take away everyone’s jobs.  As Dzieza notes, this narrative deflects attention from the pressing reality of deteriorating workplace conditions under AI supervision.  Second, Marx’s work on machines and technology is frequently read (as by the autonomists) through the Grundrisse “Fragment on Machines,” and in a way that emphasizes affective labor and cognitive capital.  The reality of AI today serves as an important reminder that the treatment in Capital is still very important.  Let me explain.

The Elephant in the Vaccine Room

You might have heard that minorities are hesitant about getting a Covid vaccine?  Well, about that.  According to polling reported by Axios, the group least likely to want a vaccine is White Republicans... to the point that "White Americans are now less likely than Black and Latino Americans to say they plan to get the vaccine."  I

Google fires another AI ethicist

This time Margaret Mitchell, one of the other authors on the fabulous "Stochastic Parrots" paper (that's my post on it.  The paper is here) on natural language processing.  This was obviously coming, since they'd suspended her email account weeks ago.  In case you haven't read the paper (and you really should!), it's worth mentioning that the paper does not mention Google, and the goal is to point to strategies to make AI better, i.e., less likely to entrench racism, sexism and so forth while destroying the planet in climate catastrophe.  It turns out that those are related, because bigger and bigger datasets are both carbon intensive and likely to pick up on and magnify hegemonic and toxic speech.  Attention to efficiency and curation moves the needle on both.

No commitment to even looking like they care about ethical AI.

Reading List: Bender and Gebru, “Stochastic Parrots”

By Gordon Hull

Not long ago, Google summarily dumped Timnit Gebru, one of its lead AI researchers and one of the few Black women working in AI.  Her coauthor Emily Bender has now posted the paper (to be presented this spring) that apparently caused all the trouble.  It should be required reading for anybody who cares about the details of how AI and data systems can perpetuate racism, or who cares more generally about the social implications of brute-force approaches to AI.  Bender and Gebru take up a common approach to natural-language processing (NLP), which involves an AI system learning how to anticipate what speech is likely to follow a given unit of speech.  If, for example, I say “Hello, how are,” the system learns by studying a dataset of existing phrases and text snippets that the next word is likely to be “you,” but almost certainly will not be “ice cream.”  How good the computer gets at this game is going to be substantially determined by the quantity and quality of its training data, i.e., the text that it examines.

Bender and Gebru outline the social costs of one approach to this problem, which is basically brute force.  As processing power increases, it’s possible to train computers with larger and larger datasets, and the use of larger datasets reliably improves system performance.  But should we be doing that?  Bender and Gebru detail several kinds of problems.  The first is environmental justice: all that processing power uses a lot of energy.  Although some of it may come from carbon-neutral sources, the net climate cost is significant.  Worse, the NLP systems being produced don’t benefit the people that are going to suffer the most from climate change.  As they memorably put it:

“Is it fair or just to ask, for example, that the residents of the Maldives (likely to be underwater by 2100) or the 800,000 people in Sudan affected by drastic floods, pay the environmental price of training and deploying ever larger English LMs, when similar large-scale models aren’t being produced for Dhivehi or Sudanese Arabic?”

Covid Deaths and the Politics of Obituaries

By Gordon Hull

As of this writing, approximately 421,000 people in the United States have officially died of Covid-19.  We also know that this number is fewer than the number that have actually died of Covid for a variety of reasons.  For example, early in the pandemic, there was nowhere near enough testing, and so many people who died of Covid-19 never received an official diagnosis.  For that sort of reason, measuring “excess deaths” during a given period is one way to try to get a handle on how many people actually died of Covid-19.  If n more people die during a given period this year than last year and/or typically, that number can be a useful indicator for how many deaths can be attributed to the pandemic.

Some of these will of course be indirect: people who didn’t go to the ER over a heart attack because they were afraid of contracting Covid, for example.  The isolation of Covid may be driving an increase in opioid overdose deaths.  Others will be direct, deaths where Covid was the underlying cause. STAT News is now reporting on the results of a recent study that takes a dive into this question, concluding that the overall direct Covid death toll is 31% higher than official figures.  That’s the top line, and it means that the national number of Covid deaths is approaching 550,000.  What should surprise no one (but is surprising, because deaths are so much more reliable for understanding infection rates than reported cases) is that the mortality data is deeply shaped by politics: excess deaths are a window into the sociology of the pandemic.

New Paper: "Infrastructure, Modulation, Portal: Thinking with Foucault about how Internet Architecture Shapes Subjects"

Now up on SSRN.  This paper uses Foucault's works on disciplinary power to develop a typology for understanding different models of Internet governance.  Here is the abstract:

Following Foucault’s remarks on the importance of architecture to disciplinary power, this paper offers a typology of power relations expressed in different models of Internet governance. Infrastructure governance understands the Internet as a common pool or public resource, on the model of traditional infrastructures like roads and bridges. Modulation, which I study by way of Net Neutrality debates in the U.S., understands Internet governance as traffic shaping. Portal governance, which I study by way of data collection policies of dominant platform companies, understands the Internet as creating a user experience that facilitates data mining. The latter two are forms of architectural disciplinary power that undermine the first. I then argue that the rise of portal and modulation governance primarily serves to remake parts of civil society by fostering market norms of consumption and entrepreneurialism. In that sense, efforts to shape Internet architecture need to be understood as techniques of subjectification.

 

Epistemic Justice with AI: indicators, diverse training data and knee pain

Knee pain is common and debilitating, and it’s often caused by osteoarthritis in the knee.  Treatment options range from analgesics (including opioids) to knee-replacement surgery.  If you go to the doctor with arthritic knee pain, you can get an x-ray which can then be interpreted using standard rubrics like the Kellgren–Lawrence Grade (KLG) to quantify damage to your knee and then guide treatment options. The KLG isn’t perfect in that the correlation between pain and objective scores of damage to the knee isn’t perfect.  Some people’s knees are a wreck and they report no pain; others have pain beyond what their KLG score indicate. But here’s the thing: Black patients consistently report more knee pain than white patients.  They also tend to have more knee damage on the KLG – but even when you factor that in, Black patients report much more knee pain than white patients with comparable KLG scores.  What’s going on?

One possibility is that factors external to the knee – stress, for example – explain the higher pain.  If that’s the case, then patients need less knee treatment.  But what if their knees were in worse shape?  To answer that question, you’d have to ask yourself what in an x-ray indicated poor knee condition.

Disease is often measured through indicators, and we know that these indicators can lead to all sorts of complexity.  In the context of Covid, for example, there are all sorts of questions about testing and sensitivity that I’ve talked about before.  Along the way, I referred to a fantastic paper on malaria testing in sub-Saharan Africa – suffice it to say that “cases of malaria” reported to donor organizations is a difficult number to parse for reasons having to do with vagaries in testing and diagnosis.

In a new paper in Nature Medicine, a team led by Emma Pierson makes ingenious use of artificial intelligence to tackle the problem of racial disparities in knee pain.  Since algorithms and data are so often implicated in increasing or magnifying racial disparities (see, for example, Safiya Noble on Google, or Timnit Gebru on facial recognition, or Margaret Hu’s chilling “Algorithmic Jim Crow”), it’s encouraging to learn about machine learning working to undermine racial disparities.  Ordinarily, you train an algorithm to perform like an excellent clinician.  In this case, that would mean training it to look at radiography and determine the correct KLG score.  The trick here was to instead train it to look at pain: to determine what features of the x-ray predicted that the patient would report pain.  It turns out that the algorithm’s diagnoses reduced racial disparities in diagnosis by a jaw-dropping 47%.

BIPA Continues to Gain Teeth: Data Retention Policy & Standing

By Gordon Hull

I’ve written about the importance of Illinois’ Biometric Information Privacy Act (BIPA) before (see also here). Briefly, BIPA is the most important and powerful of the (relatively few) state laws designed to protect biometric privacy. The statute establishes a notice-and-consent regime (sigh. better than nothing, though N&C doesn’t work well, and is disturbing as a norm) for private parties that collect biometric information like face scans, establishes the need for data retention policies, establishes a private right of action (individuals can sue; other states make you go through the state attorney general) and establishes a statutory harm – as underlined by the Illinois Supreme Court, violating the statute is enough to collect damages.

Companies like Facebook have been fighting BIPA hard, because it’s bad for their business model, and some of the main litigation has been around Facebook’s phototagging feature. A lot of the issue has been about standing – whether aggrieved parties have the right to sue. Standing sounds simple: in order to have standing under Article 3 of the Constitution, three conditions need to be met. Going through them, however, will indicate why this is harder than it looks.

• First, the plaintiff must have suffered an “injury in fact,” an invasion of a legally protected interest which is (a) concrete and particularized, and (b) actual or imminent, not “conjectural” or “hypothetical.”
• Second, there must be a causal connection between the injury and the conduct complained of, the injury has to be fairly traceable to the challenged action of the defendant, and not the result of the independent action of some third party not before the court.
• Third, it must be "likely," as opposed to merely "speculative," that the injury will be "redressed by a favorable decision."

The Strange Story of the Oxford Covid-19 Vaccine Dosing

To review the issue: The Oxford/AstraZeneca vaccine uses a modified adenovirus, as do several other vaccines in development, most notably the Russian Gamaleya Institute one.  Early, puzzling results suggested that the Oxford vaccine was about 70% effective overall, but that the overall number obscured a disparity between two groups: a two-dose group of all ages that showed efficacy in the 62% range but 90% in a group (including no elderly people) that had received a half-strength dose first.  Eh?  Even more strangely, the half-dose group seemed to be have given that dose... by mistake?  What happened?

Reuters does a deep dive.  There's a lot of threads, but it appears that the problem began when the Oxford team didn't trust measurements of the strength of a batch of vaccine from an Italian manufacturer.  The Oxford team then measured it using a different technique, concluded it was more potent than the manufacturer said, and trusted its own measurement.  Guess who was right? [Time to scream at the void: WHY on earth, given discrepancies between two measurements, both supposedly reliable but using different techniques, WOULD YOU NOT MEASURE AGAIN UNTIL YOU FELT VERY GOOD ABOUT THE DISCREPANCY AND HOW IT HAPPENED?]

"Oxford’s measurement showed that the batch was more potent than the Italian manufacturer had found, the documents show. Oxford trusted its own result and wanted to remain consistent with a measuring tool it had used throughout an earlier trial phase. So it asked Britain’s drugs regulator for permission to reduce the volume of vaccine injected into trial participants from the K.0011 batch. Permission was granted .... The documents published in The Lancet confirm that the error lay with the Oxford researchers. A common emulsifier, polysorbate 80, used in vaccines to facilitate mixing, had interfered with the ultraviolet-light meter that measures the quantity of viral material, according to the documents. As a result, the vaccine’s viral concentration was overstated and Oxford ended up administering half doses of vaccine, believing they were full doses."

This vaccine matters because it is one that developing countries are depending on: it is cheap, and it is stable in refrigerator temperatures, which makes the logistics a lot easier.  There is a hypothesis that explains the half-dose/full-dose discrepancy: it's possible that the lower first does primes the immune system better than the higher one; adenovirus variants also circulate in some human populations, and so it's possible that some combination of prior infection and the first dose generated an immune response sufficiently robust that participant's own immune systems destroyed the second dose's virus before it could generate further immune response.   The Russians report a 90% efficacy, and that vaccine uses two different adenovirus vectors across the two doses to avoid precisely this risk.  These questions need more data, and we're still waiting for the full results of ongoing trials.  Adenovirus vaccines matter too: in addition to Oxford, Gamaleya and a Chinese vaccine, the single-dose Johnson & Johnson vaccine, with trial results expected in January, is also adenovirus.  In the meantime, Gamaleya is sharing their adenovirus vector, in order to enable a combined trial: one does of the Russian vaccine, and one of the Oxford/Astrazeneca one.

 

 

Boilerplate, or Contract’s Pharmakon

By Gordon Hull

In an important recent article, Robin Kar and Margaret Radin propose a way to interpret the volumes of boilerplate that accompany pretty much any electronically-mediated consumer transaction.  Rather, they propose a way to interpret the phenomenon of the deluge of such boilerplate.  We all know the scenario: you decide to buy a song for $0.99 on a site called SketchyFiles.com, and at some point in the process, you click to indicate your acceptance of the “terms and conditions.”  Did you read those terms and conditions?  Of course you did not: if you were to print them, they’d probably run in excess of 30 pages, most of them using language that you don’t understand.  It’s not rational to slog through and try to understand all of that for a 99-cent purchase!

But the flip side is that SketchyFiles is very much going to interpret this as a contract when it suits them.  Say, for example, that you think that your purchase included a virus that destroyed a couple of files on your computer. You spend a few minutes online, and discover that there are 50,000 people to whom this exact thing happened!  You lawyer up and file suit against SketchyFiles, including a request for class certification, since your files aren’t individually worth a lot, but the aggregate of them are, and you think SketchyFiles ought to have to own the problem and scan their songs for viruses.  You will promptly discover in court that the “contract you signed” included a “mandatory arbitration clause,” which says that you agree that any and all disputes involving your purchase are to be settled out-of-court, using an arbitration procedure and a venue chosen by SketchyFiles.  Not only that, you’ve probably also contractually agreed that no class certifications are possible: all users’ claims must be adjudicated one at a time.

Ethical AI at Google? Not so much.

At least not from the appearance of things.  Google summarily fired Timnit Gebru, one of its lead AI Ethics researchers and one of the few Black women in a leadership position at the company.  Her sin?  Producing academic research critical of biases in AI:

“The email and the firing were the culmination of about a week of wrangling over the company’s request that Gebru retract an AI ethics paper she had co-written with six others, including four Google employees, that was submitted for consideration for an industry conference next year, Gebru said in an interview Thursday. If she wouldn’t retract the paper, Google at least wanted the names of the Google employees removed.  Gebru asked Google Research vice president Megan Kacholia for an explanation and told her that without more discussion on the paper and the way it was handled she would plan to resign after a transition period. She also wanted to make sure she was clear on what would happen with future, similar research projects her team might undertake. …. The paper called out the dangers of using large language models to train algorithms that could, for example, write tweets, answer trivia and translate poetry, according to a copy of the document. The models are essentially trained by analyzing language from the internet, which doesn’t reflect large swaths of the global population not yet online, according to the paper. Gebru highlights the risk that the models will only reflect the worldview of people who have been privileged enough to be a part of the training data”

The company responded by “accepting” her “resignation,” effective immediately (while she was on vacation!).

Let’s be clear: this sort of research is what Gebru does, and she’s very good at it.  Before she got to Google, she co-authored a paper that proved that facial recognition software misrecognized people of color (especially women) at much higher rates than white men.  The paper is influential, and supported efforts like the one that eventually got Amazon to stop selling its facial recognition software to police. Why is Google surprised that she would pursue this topic now?  Also, the link between biased training data and crappy outcomes is a thing.  A thing that a company like Google ought to care about.

Gebru’s unceremonious firing suggests that Google isn’t seriously committed either to actual critical thinking about AI and ethics, or to dealing with Silicon Valley’s glaring diversity problems.  I suppose file under disappointing, but not surprising.

On going to Churches, Schools and Bars

UPDATE 12/6: For more on the mess in Rhode Island, see here.

This week’s SCOTUS opinion overturning New York’s restrictions on religious gatherings is disappointing in many ways. Most obviously, it hamstrings the ability of governors to respond with science to Covid and is part of a conservative backlash to that effort; Justice Breyer makes some of the important points in dissent, followed by more of them in Justice Sotomayor’s dissent. More ominously, it signals the accelerating arrival of the era of having to pander to right wing religious people who think that their religious freedom outweighs the harm principle (see, Thanksgiving is not just about spreading disease. It’s about imposing religion!). This has been coming for a while – recall the Court’s endorsement of Hobby Lobby’s theory that it’s ok to freeload on public services and goods, but that it does not have to extend contraception rights to citizen-employees. But Amy Coney Barrett makes it faster. That is, part of the acceleration is that the Court is illegitimate, by Mitch McConnell’s own standard: the rule is either that you seat a justice before an election or not. Those can’t both be true, except in a shameless right-wing power grab. We’ll get back to Moscow Mitch in a minute.

Also unfortunate here is Governor Cuomo, who seems to have gone out of his way to antagonize the Hassidic community in particular. The order in question also apparently did not index capacity to the size of the building or social distancing plans. The antagonism plus the restrictions lets the Court make disingenuous comparisons between the number of people in a grocery store and a church. It also gives people like Gorsuch the opportunity to say “the Governor is remarkably frank about this: In his judgment laundry and liquor, travel and tools, are all “essential” while traditional religious exercises are not. That is exactly the kind of discrimination the First Amendment forbids” (2). No one should view this as anything other than opportunism; after all, Gorsuch signed onto the Court’s opinion ignoring Trump’s many, many anti-Muslim tweets on the way to upholding his Muslim travel ban.  Politicians’ chatter is only part of the law when you don’t like the politician, it seems.

These are important, but what I haven’t seen discussed is that this opinion intersects with the other news percolating up around Thanksgiving, which is the absolute incoherence of public health restrictions. Folks like Ellie Murray and Julia Marcus have been trying to sound the alarm on Twitter, and Amanda Mull wrote about it in the Atlantic. The public health message is basically: “Thanksgiving is dangerous! It is dangerous to sit around a table with people from outside your household for a long meal! If you want to do that safely, go to Cheesecake Factory!” Ummm… in other words, it is bonkers incoherent that in most of the country, indoor dining at restaurants is still possible when pretty much anything else is closed. Indoor dining is on the same epidemiological list as church for likely spread!

In this particular context, is it any surprise that a group of people who are predisposed to prioritize religious rights noticed something amiss? This is from the per curiam:

“In a red zone, while a synagogue or church may not admit more than 10 persons, businesses categorized as “essential” may admit as many people as they wish. And the list of “essential” businesses includes things such as acupuncture facilities, camp grounds, garages, as well as many whose services are not limited to those that can be regarded as essential, such as all plants manufacturing chemicals and microelectronics and all transportation facilities.” (2)

The invitation for activist judges to ignore science is too much for them to pass up. Here is Kavanaugh: “in a red zone, for example, a church or synagogue must adhere to a 10-person attendance cap, while a grocery store, pet store, or big-box store down the street does not face the same restriction.” (2). Gorsuch piles on: “Nor is the problem an isolated one. In recent months, certain other Governors have issued similar edicts. At the flick of a pen, they have asserted the right to privilege restaurants, marijuana dispensaries, and casinos over churches, mosques, and temples” (3). And he concludes: “It is time—past time—to make plain that, while the pandemic poses many grave challenges, there is no world in which the Constitution tolerates color-coded executive edicts that reopen liquor stores and bike shops but shutter churches, synagogues, and mosques.” (7).  Kavanaugh again: “once a State creates a favored class of businesses, as New York has done in this case, the State must justify why houses of worship are excluded from that favored class” (3).

Nevermind that you do different things in all of those facilities; as Justice Sotomayor acidly notes, “bike repair shops and liquor stores generally do not feature customers gathering inside to sing and speak together for an hour or more at a time” (3). She adds:

“Justices of this Court play a deadly game in second guessing the expert judgment of health officials about the environments in which a contagious virus, now infecting a million Ameri-cans each week, spreads most easily” (3).

So the conservatives are simply wrong on the facts, even though those facts are articulated quite clearly in the decision they overturn.  And that decision will probably cost people their lives.  Freedom!

But: bars and restaurants? Perhaps I am particularly sensitive to this point because schools are closed too (as one meme wondered – should NYC parents send their kids to the bars that are still open, now that schools are closed?), but it’s impossible not to notice that moneymaking endeavors get a different set of rules from civic ones. As Mull puts it:

“With people out of work and small businesses set up to fail en masse, America has landed on its current contradiction: Tell people it’s safe to return to bars and restaurants and spend money inside while following some often useless restrictions, but also tell them it’s unsafe to gather in their home, where nothing is for sale. It’s a woefully inadequate stimulus plan, funded by money extracted little by little from the pockets of people who are mostly just confused about what they’re being compelled to do.”

She cites the example of Rhode Island, whose restrictions are completely absurd:

“Residents are prohibited from gathering with even one person outside their household, even in the open air of a public park. But inside a restaurant? Well, 25 people is fine. Hire a caterer? You’re legally cleared to have up to 75 outdoors. The governor’s executive order merely notes: “The lower attendance at such events, the lower the risk.” (The Rhode Island governor’s office did not respond to a request for comment.).”

And how did we get here? Well, as Mull underlines, closing bars and restaurants (etc) will cause a lot of unemployment. Closing schools and churches won’t.

In other words, it’s not that the plaintiffs in the New York case necessarily raised a compelling First Amendment argument, but they are a canary in the coal mine for a deeply problematic public health policy, one that systematically favors commercial activities over civic ones.  Governors’ hands are tied, even the ones who want to do the right thing, because the proper public health measures will cost money in the form of income guarantees to the people affected by them. And guess who has – perhaps singlehandedly – stood most in the way of that financial assistance? That’s right – Moscow Mitch and his repeated bad-faith refusals to negotiate a second stimulus package. Turns out he was so busy ramming right-wing court justices through the Senate that he made it more dangerous to do stuff that is more important, like school, worship and family Thanksgiving.

ToS and the Subject Position of Website Users

By Gordon Hull

As one knows, online privacy policies (and access to the Internet in general) are generally conditioned on a user’s acceptance of some sort of boilerplate terms of service.  Lots of people (myself included) have complained about this state of affairs as attempting to get users to consent to all sorts of practices by way of reams of unintelligible legalese.  In addition to allowing unlimited data collections, these contracts often include mandatory arbitration provisions, which means that they compel users to take disputes with the sites to arbitrators selected by the company, rather than taking them to courts.

In other words, the idea is that using the product or service constitutes consent to a series of wildly one-sided terms that the user probably could not understand if she tried.  There is certainly no chance of negotiation or bargaining involved.  As Julie Cohen put it, “the conception of consent emerging from that default condition is unprecedented in the law of contracts or any other body of law,” and more closely resembles older status contracts than the contracts of modern liberalism (58-9).  The result is “a form of Kabuki theater that distracts both users and regulators from what is really going on” (59).

In a recent paper, Woodrow Hartzog and Neil Richards detail this state of affairs in terms of contract theory and its imaginary of two autonomous individuals negotiating and agreeing to terms.  They argue that:

“Consent is most valid when we are asked to choose infrequently, when the potential harms that result from the consent are easy to imagine, and when we have the correct incentives to consent consciously and seriously. The further we fall from this gold standard, the more a particular consent is pathological and thus suspect” (1465).

The design of notice-and-consent privacy regimes is deeply pathological from this point of view.  Consent is often unwitting – consumers don’t understand either the terms or the technology behind them.  They also don’t understand the consequences and risks (which is entirely expected, given that they are futural and difficult to visualize).   If not fully coerced, consent can be induced by both the fact that it is difficult now not to use the internet (so no opt-out) and both ISPs and networks are de facto monopolies (so no market competition).  Various website design techniques make opting out difficult: pop-ups that cover up the screen, browser tabs with no ‘back’ button, and various techniques of shaming (“Yes! I want this product.  No, I choose to die a lonely death”).  And of course there is the question of children: COPPA only protects children up to age 13, even though contractual consent typically starts at age 18.  And kids can easily get around age checks anyway.

If we step back from the normative arguments about why this situation is abusive or otherwise bad, I think there’s an interesting point to be made about subjectivity here.  Basically, the sites are trying to construct a legally cognizable subject position that users then occupy.  Both steps are important.  The contractual language presented in the ToS defines the subject position, and it seems to me that it’s important to distinguish this subject position from a few alternatives. First, this is not the traditional subject of law.  That position is foreclosed by the lack of availability of the courts.  Disputes are to be settled by a privatized procedure.  Second, it’s not the subject of administrative procedure either, such as recently upheld by SCOTUS in patent arbitration.  That procedure also allows an appeal to courts.  Third, it’s not the position of the disciplinary (?) subject that consumes mass media: there is no pretext to contract when you open a book, as Hartzog notes in an earlier paper on these contracts.  In short, I think this is a quite specific subject position constructed by the technology companies for their own advantage.

There is also a split between the structural subject position and the phenomenological experience of users.  Users are encouraged by the sites in question to view themselves as free and emancipated by the technology, and as empowered by its affordances.  They are (for obvious reasons) not encouraged to view themselves as signing away their rights.

The gap between user experience and described subject position leads to the second question: whether users can be said to occupy this subject position, i.e., whether these contracts are enforceable.  Can technology companies rely on the coercive power of the state to define the subjectivity of users on their terms?

It is the position of the technology companies that use of their products constitutes acceptance of the terms of service; sometimes this requires clicking a button.  Courts have been generally unwilling to say that the contracts are objectively unenforceable as unconscionable or otherwise inconsentable (though Hartzog and Evan Selinger more recently make the case that facial recognition ought to be the sort of thing that someone cannot consent to – the harms are potentially too great and difficult to understand).  This means that the question legally has come down to whether the notice and consent procedures are adequate.  Hartzog’s earlier paper suggested that, as of roughly 2010, courts had been most likely to refuse to enforce ToS on casual users – those who just read a website but do not otherwise particularly interact with it.  Still, the law was inconsistent and this emerging de facto carve-out was far from either settled or explicit.

Here, a recent survey of the case law by Nancy Kim is encouraging.  She reports that the case law is still very inconsistent, and users often lose.  But: there’s a movement toward demanding more of websites:

“The standards of “notice” and “manifestation of assent” remain the same, but how those standards are applied varies depending upon the facts of the case and the jurisdiction. Berkson v. Gogo LLC and Nguyen v. Barnes & Noble, Inc. have been particularly influential in helping inform the application of the standards. Reflecting the influence of these two cases, courts have started to assess the reasonableness of notice by evaluating the website experience from the perspective of the offeree and by considering how the drafter chose to present terms. Thus, instead of asking why the offeree failed to read visible terms, more and more courts are asking why the offeror failed to draft terms in a more conspicuous manner. As the court in Rushing v. Viacom Inc. emphasized, the burden is on website owners to “put users on notice of the terms to which they wish to bind consumers.” Accordingly, courts are increasingly asking for two types of evidence from drafters. The first is in the form of screenshots or webflows that demonstrate how the terms were presented to plaintiffs. The second is in the form of the drafter’s internal records that document that the plaintiff actually accessed the website at the time the notice was allegedly presented.” (1693)

There’s a lot to be done here, of course, but it’s nice to see some legal pushback, and to see how courts understand evidence and other traditional legal concepts in this context.  Kim cites several cases where courts determined that reasonably prudent users would not realize that they were entering into a contractual relation – for example, there was a hyperlinked “terms of service” but it was buried several pages into a website and well below the prominent “click here for the next step of your purchase.”  Nearly all of the cases Kim cites are about the enforceability of mandatory arbitration clauses, which suggests both the specificity of the subject position the websites are constructing, and the extent to which its enforceability is an artifact of their power over not just users but the state.

Encouraging case on Qualified Immunity

Given the role of qualified immunity in absolving police officers of murdering unarmed black men (and doing all sorts of other nefarious things), it’s encouraging to see that the Supreme Court said in a per curiam opinion today that there is an outer limit to how far that doctrine can be extended.  Recall the problem: courts will rule that yes, some behavior or another would be unconstitutional, but the law on the subject is not “clearly established,” so the officer is shielded from liability.  The trick is that “clearly established” morphs into “an officer in an identical situation has already been found liable for doing exactly what is alleged here.”  Then of course liability becomes almost impossible to establish.

Consider now the case of Trent Taylor:

“Petitioner Trent Taylor is an inmate in the custody of the Texas Department of Criminal Justice. Taylor alleges that, for six full days in September 2013, correctional officers confined him in a pair of shockingly unsanitary cells. The first cell was covered, nearly floor to ceiling, in “massive amounts’ of feces”: all over the floor, the ceiling, the window, the walls, and even “packed inside the water faucet.” Fearing that his food and water would be contaminated, Taylor did not eat or drink for nearly four days. Correctional officers then moved Taylor to a second, frigidly cold cell, which was equipped with only a clogged drain in the floor to dispose of bodily wastes. Taylor held his bladder for over 24 hours, but he eventually (and involuntarily) relieved himself, causing the drain to overflow and raw sewage to spill across the floor. Because the cell lacked a bunk, and because Taylor was confined without clothing, he was left to sleep naked in sewage”

Like clockwork, the extra-conservative 5^th Circuit concluded that the treatment was unconstitutional, but that there was no liability (I’m quoting SCOTUS) because “based on its [the 5^th Circuit’s] assessment that “[t]he law wasn’t clearly established” that “prisoners couldn’t be housed in  cells teeming with human waste” “for only six days,” the court concluded that the prison officials responsible for Taylor’s confinement did not have “ ‘fair warning’ that their specific acts were unconstitutional.” SCOTUS points out the obvious: “no reasonable correctional officer could have concluded that, under the extreme circumstances of this case, it was constitutionally permissible to house Taylor in such deplorably unsanitary conditions for such an extended period of time.”  They add: “Confronted with the particularly egregious facts of this case, any reasonable officer should have realized that Taylor’s conditions of confinement offended the Constitution.”  So back to litigation – no summary judgment in favor of the officers.

We still have a long way to go before qualified immunity is reined in adequately, but this shows there is at least a floor.  In the meantime, Justice Alito writes a concurrence in which he first says that the SCOTUS shouldn’t have reviewed the case, but given that it has, he concurs in the judgment.  Oh, and Justice Thomas dissented.  But you could have guessed that.  More importantly, at least seven justices realized that there could be limits to qualified immunity.  These days, it’s important to recognize even small first steps.

Foucault on Public Opinion

If you’re like me, you spend too much time – way too much time – these days looking at polling data.  I ran across some interesting remarks by Foucault on opinion yesterday, which I’ll share here as a technique of distraction.  He makes them in the context of a 1976 conversation with J. P. Barou and Michelle Perrot (whose work on resistance to disciplinary power he favorably cites near the end of the conversation) that was published as the preface to an edition of Bentham’s Panopticon writings.  It appears as “L’oeil du pouvoir” (D&E #195, pp. 190-207 in my 2 volume edition) and is translated in Foucault Live (=FL).   For context, then, the conversation appears in the year after Discipline and Punish.  It covers a range of topics, including Foucault’s own path to discovering the panopticon (initially via hospital architecture, which had the dual need to see patients and keep them physically separated to avoid the spread of disease).

Foucault suggests that Bentham’s original work enjoyed considerable uptake in revolutionary France.  Barou sets Foucault up: “isn’t it surprising to realize that the French Revolution, in the persons of people such as Lafayette, favorably received the project of the panopticon?” (D&E 195).  Foucault answers that Bentham “is the complement of Rousseau” because the “Rousseauian dream” that “animated the revolutionaries” was of a “transparent society, both visible and legible in each of its parts, which had no more obscure zones, no zones established by the privileges of royal power or by the prerogatives of such and such body, or indeed by disorder; that each, from the point that he occupies, is able to see the entirety of society; that their hearts communicate with one another, that the gazes encounter no more obstacles, that opinion rules, that of each on each [celle de chacun sur chacun]” (D&E 195/FL 230).  Opinion thus functions similar to what Foucault will later identify as a function of pastoral power (omnes et singulatim), simultaneously totalizing and individuating.  “Opinion” is an object of “police” power in Discipline and Punish (213), that power that is concerned with understanding and managing the minutiae of daily life.

In the interview, Foucault suggests that Bentham is “both this [Rousseauian dream] and its complete opposite” because he understands the problem of visibility “while thinking of a visibility organized entirely around a dominant and surveillant gaze” (D&E 195/FL 230).  Thus you have joined the “lyricism of Rousseau and the obsession of Bentham.”  Foucault then suggests:

“A fear haunted the second half of the 18^th century: there is a dark space, a screen of obscurity which provides an obstacle to the complete visibility of things, of people, of truths. [One needs] to dissolve the fragments of night which were opposed to the light, to make it so that there is no more dark space in society; to demolish the black rooms where arbitrary political power, the caprices of the monarch, religious superstitions, conspiracies of tyrants and priests, illusions of ignorance and epidemics [all] fomented” (D&E 196/FL 231).

In this context, the then proposes that “the rule of ‘opinion’ that we invoke so often in our age is a mode of functioning where power will be able to be exercised on the sole condition that things are known and that people are seen by a type of immediate gaze [par une sorte de regard immédiat] that is collective and anonymous” (D&E 197/FL 232).

Hold on to the word "immediate" – it will be important in a moment.  But for now, notice the extent to which this is a pretty good description of nights spent gazing at opinion polls, both the obsession with knowing, and the fear that there are (say) hidden Trump voters like there were last time.  More light!  This sort of epistemology is driven by a fundamentally paranoid logic, as Wendy Chun emphasized a while ago: once I use some sort of optical device to see beyond what my naked eye tells me, that opens the fundamental possibility that a better device would see more.  See a molecule with the microscope?  Is that really the smallest particle?  After all, there were molecules lurking below the surface that you hadn’t seen before.  Here the paranoia is inevitable given a sampling process.  The poll represents underlying opinions, and the process of sampling invariably “sees” only part of the data, even when it’s done exactly right.  And that’s before you factor in questions about whether people who don’t have landlines are proportionally represented, or whether young people who say they will vote will actually do so, and so on.  That’s also before you get to issue polling, which I complained about before the last election, arguing that it worked to create populations and truths about them that were fully the product of the polling questions, making it (for example) impossible to express views that didn’t fit the categories the polling firm wanted to hear.

In any case, as anybody who works in data will tell you, our access to data is fundamentally mediated by the process of its collection and description, which means that the immediacy we want – to just see clearly whether Trump has enough hidden voters somewhere in Pennsylvania to pull off the upset.  We have to content ourselves with something like the fivethirtyeight statistical runs: crunch all the polling data (including weighting it by quality of poll), simulate the election 40,000 times (there’s a vision of hell: 40,000 occurrences of this election!) and see what happens (as of this minute, Biden wins 88% of the time).  But the human brain just doesn’t like that: a pair of papers, one in Philosophy and Public Affairs,  and one in the Stanford Law Review, by David Enoch, Levi Spectre and Talia Fisher make the case that we prefer juridical reasoning in the sense that we can explain why somebody is wrong, rather than living with the statistical truth (in this case, that Trump could win and there’s nothing wrong with the model at all.  Enoch Spectre and Fisher use the example of liability for a bus accident).

All of which leads to the following remarkable paragraph.  In response to Perrot’s suggestion that Bentham seems to overstate the power of his panopticon, Foucault says:

“This is the illusion of almost all of the reformers of the 18^th century who lent to opinion a considerable power.  Opinion is only able to be good when it is the immediate consciousness of the complete social body; they believed that people would become virtuous by the fact that they were observed [regardés].  For them, opinion was a spontaneous reactualization of the social contract.  They misunderstood the real conditions of opinion, of the media, a materiality which is caught in the mechanisms of the economy and power” (D&E 204/FL 238).

And:

“They also failed to understand that the media would necessarily be controlled by economic and political interests. They did not perceive the material and economic components of public opinion. They thought that public opinion would be just by its very nature, that it would spread on its own accord, and provide a kind of democratic surveillance. It was essentially journalism—a crucial innovation of the 19th century—that manifested the utopian characteristics of this entire politics of the gaze” (D&E 204/FL 238)

Of course as Foucault notes, journalism comes with its own politics, and different forms of it come with different politics.  I’ll close with two tidbits.  First, one large study found that false news traveled further, faster and deeper on Twitter than true news, almost no matter how one measured diffusion.  It’s the materiality of the news infrastructure and the business models of the platforms through which opinion is constituted.  Second, there appears to be a horrible nexus of Instagram, Facebook, QAnon and multi-level marketing schemes that’s contributing to the diffusion of that conspiracy: it turns out that QAnon and multi-level marketing share a lot in common, and social media tends to facilitate that.  I can’t describe it better, but the article is worth a read.  The paranoid logic is baked in!

Trump's Corruption of the CDC

Among the indefinitely many evil things the Trump administration has done that will take years and years to fix (and in this case has undoubtedly cost many lives already).  Propublica has the story.