New APPS: Art, Politics, Philosophy, Science

AI for Social Good: Calling out Amazon’s Lies to Congress

(story via Julia Angwin) You might remember that Amazon solemnly swore to Congress that they did not artificially elevate their own products in search results.  Except that they do.  Adrianne Jeffries and Leon Yin of The Markup used a machine learning algorithm to predict product placement in search results:

“We found that knowing only whether a product was an Amazon brand or exclusive could predict in seven out of every 10 cases whether Amazon would place it first in search results. These listings are not visibly marked as “sponsored” and they are part of a grid that Amazon identifies as “search results” in the site’s source code.”

Nothing else was anywhere close to as predictive.  Amazon products routinely get rated higher than better rated and better-selling competitors.  And it isn’t just for obvious Amazon products like “Amazon Basics.”  It turns out that a lot of products turn out to be Amazon:

“Using public records from the U.S. Patent and Trademark Office and Amazon’s own statements, we identified more than 150 brands registered by or owned by Amazon. These include both brands with an obvious connection, such as Amazon Basics and Amazon Commercial, and those that are generally known to be owned by the company, including Kindle and Zappos. But they also include dozens more, such as Happy Belly, Daily Ritual, and Society New York, where the connection to the company is not obvious. Those are in addition to the estimated hundreds of third-party brands that are exclusive to the site.”

Consumers are (understandably) totally confused about who makes the products they see.  Actual independent sellers are terrified to complain, because they’re worried about retaliation from Amazon.  Those who want to elevate their products can of course pay (one of the companies listed in the article was paying $10,000 a month, another $40,000 – not small change!), though even that might not get them listed above the Amazon products.  All of this is, not surprisingly, a potential antitrust problem:

“Bill Baer, a former assistant attorney general in charge of the antitrust division of the U.S. Department of Justice and former director of the Bureau of Competition at the FTC, said if consumers expect Amazon’s product search results to be neutral, but they are not, and the site is essentially a monopoly, that could be a violation of the FTC Act of 1914, which prohibits unfair competition and unfair or deceptive practices in commerce, or the U.S. Sherman Antirust Act, which prohibits monopolies from using their market power to harm competition.  If basically you’ve got somebody with market power that is restraining competition both in terms of site access or where things appear on the site,” he said, “that is potentially problematic.”

It’s also a problem because it suggests that Amazon executives lied to Congress.  Congress, you’ll recall, is completely dysfunctional and polarized on all topics except one: they all hate big tech.  Hence the “social good” part of the AI: the House Judiciary committee just sent a very threatening letter, which opens as follows:

“We write in response to recent, credible reporting that directly contradicts the sworn testimony and representations of Amazon’s top executives—including former CEO Jeffrey Bezos—to the Committee about their company’s business practices during our investigation last Congress. At best, this reporting confirms that Amazon’s representatives misled the Committee. At worst, it demonstrates that they may have lied to Congress in possible violation of federal criminal law.  In light of the serious nature of this matter, we are providing you with a final opportunity to provide exculpatory evidence to corroborate the prior testimony and statements on behalf of Amazon to the Committee. We strongly encourage you to make use of this opportunity to correct the record and provide the Committee with sworn, truthful, and accurate responses to this request as we consider whether a referral of this matter to the Department of Justice for criminal investigation is appropriate.”

Facebook as a Hostile Foreign Power

This piece, on Facebook's behaving more like an autocratic, hostile state than a large company, is worth the read.  Here's an excerpt:

"Perhaps Americans have become so cynical that they have given up on defending their freedom from surveillance, manipulation, and exploitation. But if Russia or China were taking the exact same actions to undermine democracy [that FB is], Americans would surely feel differently. Seeing Facebook as a hostile foreign power could force people to acknowledge what they’re participating in, and what they’re giving up, when they log in. In the end it doesn’t really matter what Facebook is; it matters what Facebook is doing."

Charles Mills (1951-2021)

I was both saddened and stunned this morning to read of the passing of Charles Mills.  I first met him at a SPEP years ago; I was having lunch at some random sandwich shop with friends.  He knew one of us, and asked if he could join.  Nevermind that we were all junior.  I managed to find him at a few later conferences, to join a meal or even just to say hi.  He was generous, warm and wickedly funny when he wanted to be.  And the stunning clarity with which he could call out ideal philosophy and other systems of domination is like nothing else I’ve read.

Daily Nous has more, and links to a really touching remembrance by Liam Kofi Bright.

New Paper: "The Death of the Data Subject"

Shameless self-promotion dept... here's the preprint for my new paper, "The Death of the Data Subject," now forthcoming in Law, Culture and the Humanities.  And here's the abstract:

This paper situates the data privacy debate in the context of what I call the death of the data subject.  My central claim is that concept of a rights-bearing data subject is being pulled in two contradictory directions at once, and that simultaneous attention to these is necessary to understand and resist the extractive practices of the data industry. Specifically, it is necessary to treat the problems facing the data subject structurally, rather than by narrowly attempting to vindicate its rights.  On the one hand, the data industry argues that subjects of biometric identification lack legal standing to pursue claims in court, and Facebook recently denied that that its facial recognition software recognizes faces.  On the other hand, industry takes consent to terms of service and arbitration clauses to create enforceable legal subject positions, while using promises of personalization to create a phenomenological subject that is unaware of the extent to which it is being manipulated.  Data subjects thus have no legal existence when it is a matter of corporate liability, but legal accountability when it is a matter of their own liability.  Successful reform should address the power asymmetries between individuals and data companies that enable this structural disempowerment.

 

 

When does the Supreme Court lose its legitimacy?

And what does that mean?  Now is a good time to ask.  The Court has let stand a 5^th Circuit decision upholding a Texas law that is plainly unconstitutional under current SCOTUS jurisprudence (it bans abortion at 6 weeks) and involves an enforcement mechanism that comes straight from Stalin’s playbook (it allows individuals to sue people they suspect of assisting a woman of obtaining an abortion).  This piece on Vox runs through how deeply perverse the Texas law in question in, how thoroughly Trump has corrupted the 5^th Circuit, and how alarming SCOTUS inaction is.

In Planned Parenthood v. Casey – which, along with Roe is apparently being overruled in Texas without a hearing and without a reasoned opinion – the Court favorably cites earlier opinion to the effect that the Court is supposed to give reasons when it overturns its precedent:

"A basic change in the law upon a ground no firmer than a change in our membership invites the popular misconception that this institution is little different from the two political branches of the Government. No misconception could do more lasting injury to this Court and to the system of law which it is our abiding mission to serve"

Justice O’Connor adds:

“The root of American governmental power is revealed most clearly in the instance of the power conferred by the Constitution upon the Judiciary of the United States and specifically upon this Court. As Americans of each succeeding generation are rightly told, the Court cannot buy support for its decisions by spending money and, except to a minor degree, it cannot independently coerce obedience to its decrees. The Court's power lies, rather, in its legitimacy, a product of substance and perception that shows itself in the people's acceptance of the Judiciary as fit to determine what the Nation's law means and to declare what it demands.  The underlying substance of this legitimacy is of course the warrant for the Court's decisions in the Constitution and the lesser sources of legal principle on which the Court draws. That substance is expressed in the Court's opinions, and our contemporary understanding is such that a decision without principled justification would be no judicial act at all. But even when justification is furnished by apposite legal principle, something more is required. Because not every conscientious claim of principled justification will be accepted as such, the justification claimed must be beyond dispute. The Court must take care to speak and act in ways that allow people to accept its decisions on the terms the Court claims for them, as grounded truly in principle, not as compromises with social and political pressures having, as such, no bearing on the principled choices that the Court is obliged to make. Thus, the Court's legitimacy depends on making legally principled decisions under circumstances in which their principled character is sufficiently plausible to be accepted by the Nation” (emphasis added)

O’Connor adds that if “the Court decides a case in such a way as to resolve the sort of intensely divisive controversy reflected in Roe” that “only the most convincing justification under accepted standards of precedent could suffice to demonstrate that a later decision overruling the first was anything but a surrender to political pressure, and an unjustified repudiation of the principle on which the Court staked its authority in the first instance.”

The Court’s so-called “shadow docket” – consequential decisions rendered in orders for cases not heard, as for example prioritizing religious claims over public health – has been on the rise over the last couple of years, and commentators have worried about the damage this does to the rule of law as an institution.  In the case of abortion, the Court has a case on the docket that would give it the opportunity to overturn Roe; it could have waited until then (and avoided the need to validate Texas’ enforcement mechanism).  You could argue that the Court didn’t “decide” anything last night, but it was faced with a clearly erroneous 5^th Circuit decision that it let stand.  I don’t see how last night’s failure to enjoin the Texas law doesn’t utterly gut its legitimacy in the sense articulated in Casey.  Texas just banned abortion, and SCOTUS offered no justification at all in letting the law stand.

Again, Casey: “to overrule under fire in the absence of the most compelling reason to reexamine a watershed decision would subvert the Court's legitimacy beyond any serious question.”  Court legitimacy is important:

“It is true that diminished legitimacy may be restored, but only slowly. Unlike the political branches, a Court thus weakened could not seek to regain its position with a new mandate from the voters, and even if the Court could somehow go to the polls, the loss of its principled character could not be retrieved by the casting of so many votes. Like the character of an individual, the legitimacy of the Court must be earned over time. So, indeed, must be the character of a Nation of people who aspire to live according to the rule of law. Their belief in themselves as such a people is not readily separable from their understanding of the Court invested with the authority to decide their constitutional cases and speak before all others for their constitutional ideals. If the Court's legitimacy should be undermined, then, so would the country be in its very ability to see itself through its constitutional ideals. The Court's concern with legitimacy is not for the sake of the Court, but for the sake of the Nation to which it is responsible.”

So much for Justice Roberts’ efforts to preserve the Court as an institution.

No Standing for You! (part 2)

By Gordon Hull

Last time, I started to look at the details of the Supreme Court’s recent TransUnion decision, which ruled that a credit agency that wrongly labeled someone as a match for a terrorist watch list (using only first and last names, with no effort at verification.  Sorry “John Smith”…) could only be sued if that person could show that the credit agency also distributed this information to others.  So basically you have to lose a job opportunity or a mortgage approval, and then you can sue.  There is nothing you can do to pre-emptively force them to correct the problem.  Worse, this ruling is in defiance of explicit statutory language in the FCRA.  The opinion directly says that SCOTUS can override Congressional determination of when something is legally actionable, and (just to be contrary, I guess), it does so in the name of separation of powers.

So what are we to make of this? As the various critical theories will remind you, law is a form of power. One thing that’s happening here is the protection of the powerful from the weak. With some high profile exceptions (like Justice Gorsuch’s endorsement of LGBTQ rights), the Roberts court has fairly consistently sided with the powerful, like corrupt politicians and conservative dark money donors by whittling away at laws designed to rein them in. One limitation to that thesis is to argue, as Erwin Chemerinsky has, that “Throughout history the court has overwhelmingly favored corporate power over employees, consumers, and the public, and has favored government power over individuals’ rights.” While it’s tempting to leave it at that here, I think there’s more nuance to be had. So two points.

No Standing for You! (part 1)

By Gordon Hull

In one of the Seinfeld episodes, the proprietor of a popular lunch stop would deny service to customers who offended his arbitrary sensibilities with a loud “No Soup for You!” This is basically the outcome of the Supreme Court’s June decision on standing, TransUnion v. Ramirez. “Standing” in this sense refers to access to the federal legal system; to simplify, to be able to sue somebody, you need to show that you’ve suffered a “concrete” injury. A number of recent standing cases have been about the outer limits of “concrete.” In principle, privacy harms can be concrete, but SCOTUS has been making standing harder, particularly in data breach cases.

And so it goes here. Per a 5-4 opinion authored by Justice Kavanaugh, you don’t have standing to sue a credit agency for flagging you as a terrorist, unless you can show that they also disseminated that information to somebody else. This opinion explicitly overrides a Congressional determination that you do, in fact, have standing in cases like that, arrogating to the Court the right to decide when Congress is and is not allowed to establish when statutory rights are actionable. This was too much for Justice Thomas, who, joined by the court’s liberals, points out in dissent that this both violates separation of powers in the name of preserving it, and turns historical standing doctrine on its head. As Dan Solove and Danielle Citron put it in a sharp critique that builds on Thomas’ dissent, TransUnion is “an activist decision that nullifies Congress’s power to protect consumers and that enables courts to rewrite privacy laws to alter how they are enforced.”

Foucault, Marx and Prophecy Part 3: On Bureaucrats

By Gordon Hull

In a previous post, I noted that Foucault strongly implies in a 1978 interview that his communist detractors are bureaucrats, and tied that to an earlier interview with Maoists in which he suggests that structuring populist tribunals on the model of bourgeois courts would fail to break with the power structure of the bourgeois court system: the model of impartiality is intrinsically bourgeois, and so importing that model into communist popular tribunals would iterate the very power structures that were to be replaced.  Here I want to flesh out a little more some of the resonances of an accusation of bureaucracy in the context of 1970s Marx debates in France.   I should say in advance that these are notes more than a complete assessment, designed to pick out highlights.  The back and forth polemics of Marxists are byzantine, and we should all be grateful that Foucault sets it as a rule not to engage in them.  Here I will mainly draw from Trotsky’s critique of Stalin and Lenin, with a closing gesture to a representative text of the French far-left that emerged in the aftermath of the 1968 student uprisings.

Patently Harmful

My further thoughts on the background through which we should interpret Covid patent waivers, at Real Life.

Foucault, Marx and Prophecy, Part 2: The Role of the Intellectual

By Gordon Hull

In a previous post, I made the case for reading Foucault’s 1978 comments on Marxism (especially in the Yoshimoto interview) in the context of theory/practice questions raised by the re-evaluation of Marx’s 11^th “Thesis on Feuerbach.”  Here I want to flesh out that position a little more, starting with reference to another 1978 piece.  Shortly after the Japan interviews, Foucault writes a response to the Italian communist Massimo Cacciari’s “Rationality and Irrationality of Politics in Delezue and Foucault” (the piece appeared in Italian; as far as I can tell, there is no translation of it).  Much of Foucault’s response is dedicated to accusing Cacciari of using the tactics of Stalinism: “always having a unique adversary” (created in this case by amalgamating Foucault, Deleuze and the New Philosophers), of engaging in a trial-like procedure, of assimilating ‘the enemy’ and danger, of reducing thought immediately to a system, and so forth.

He then defends the argument of Discipline and Punish, in particular with reference to the understanding of power; Foucault says he “would like precisely to show this heterogeneity of power, that is to say how it is always born of something other than itself” (D&E #238, II, 651 (2 vol. ed, 2001]).  He underscores that there is no metaphysics of power here, no “power with a capital P.”  Rather, “it is necessary to put the relations of power back into the interior of struggles and not suppose that there is, on the one hand, power, and on the other, that on which it is exercised, and that the struggle unfolds between power and non-power” (632).  In other words, Foucault’s entire effort is to resist the sort of abstraction that the Theses on Feuerbach warn against reifying.  The goal is rather to write a text that does something, that constitutes an intervention or a tool.  In language similar to that of the Japan interviews, and against the accusation that he is merely telling a story, Foucault proposes:

“In reality, what I am trying to do, and therein lies the difficulty of the attempt, is to work out an interpretation, a reading of a certain reality, which is such that, on the one hand, this interpretation is able to produce effects of truth and that, on the other hand, these effects of truth can become instruments at the heart of possible struggles.  To speak the truth so that it can be attacked [dire la vérité pour qu’elle soit attaquable].  It is a reality of possible struggles that I seek to make appear.” (633)

A few sentences later, he argues that “the effect of the truth that I seek to produce resides in this manner of showing that the real is polemical” (633).  However, “I do not speak of the current situation.  I effect an interpretation of history, and the problem – but I do not resolve it – is to know what the possible utilization of these analyses in the current situation is” (633-4).

This then leads to a very clear statement of the role of the intellectual.  “It is absolutely true that, when I write a book, I refuse to take a prophetic position which consists in saying to people: here is what you ought to do, or again, this is good, that is not.”  Instead, the goal is to show in a general way “it seems to me that these things have happened, but I describe them in such a manner that views of possible lines of attack are traced.  But in this I do not force or constrain anyone to attack” (634).  What to actually do – and he refers to his own decisions about throwing himself into certain political actions - “is a problem of groups, of physical and personal engagement,” and not of books.  Indeed, bossy Marxists are not radical: “one is not radical because one has pronounced some formulas; no, radicality is physical, radicality concerns existence” (634).

The Necropolitics of Intellectual Property

By Gordon Hull

Update (4/30).  Essential piece by Amy Kapczynski (who is one of my sources below).  Also see this Twitter thread by Dennis Crouch of PatentlyO, characterizing the issue as one of technology transfer more generally than patents specifically.  Trade secrets are an important issue - like patents, they allow companies to control access to innovation.  Companies also will use a strategy of combining patents and trade secrets in order to maximize their revenue (so if they think something can't be patented, they might claim it as a trade secret).  The theory behind trade secrets is based more in contracts than property, which means that they are arguably even worse if you're worried about neoliberalism, since they are an even further privatization of law and policy.

Update (4/29).  Here's more on IP and other barriers to vaccine distribution in India (and elsewhere):

• The Modi government is incompetent at every level, and India's base capacity was lower than it estimated.
• Most of the world wants IP waivers.  The U.S. and Europe and a few other countries do not. 
• This WaPo piece argues that waivers are bad but licensing (preferably voluntary, but with compulsory as a stick) is good.  Note that licensing is not what Bill Gates is talking about, as far as I can tell, which is buying vaccines and donating.

In my Biopolitics of Intellectual Property, I argue that IP policy has shifted from what I call a “public biopolitics” model to a neoliberal version.  In its briefest form: the public version treats IP as a necessary but limited monopoly to promote public goods, and the neoliberal version focuses on private wealth gain through proprietization (I summarize the argument here).

Something that I don’t particularly talk about in the book, but that one knows from Foucault, is that biopolitics comes with its inverse, necropolitics: if biopolitics is about promoting life and health for the “population,” it is also about who is allowed to die.  As Foucault puts it, “the ancient right to take life or let live was replaced by a power to foster life or disallow it to the point of death [au vieux droit de faire mourir ou de laisser vivre s'est substitué un pouvoir de faire vivre ou de rejeter dans la mort]” (History of Sexuality I, 138).  Governmental power goes from the right to kill to the power to cause people to live; death becomes something into which one is literally “thrown back.”  The leading examples of necropolitics are political, as for example Foucault’s discussion of state racism (of which Nazism is the apotheosis) in Society must be Defended.  Achille Mbembe’s “Necropolitics” article spends time on how post-colonial African states have dismantled populations, which are “disaggregated into rebels, child soldiers, victims or refugees, or civilians incapacitated by mutilation or simply massacred on the model of ancient sacrifices” (34).  Building on these, Ege Selin Islekel’s brilliant treatment of the disappeared in Turkey notes that in necropolitical spaces, “the entire content and the fact of living, constituted by the ethical, political, and epistemological conditions of life, are subsumed under death.”

However, as Ute Tellman has recently demonstrated, the political treatment of biopolitics needs to take seriously how it is co-configured with the economy.  On Tellman’s account, the notion of economic scarcity first appears in Malthus (it was missing in Smith!) as a way to police the behavior of the poor (and “savages” in the colonies) by training them to think in terms of futurity.  For Malthus, the poor have to be trained not to eat and procreate their way into oblivion by forcing them to think in terms of economic rationality.  This brings us to the neoliberal justification of IP, which is partly underpinned by the Schumpeterian thesis that innovation is to be pursued at all costs, because the gains of future innovation (“dynamic efficiency”) are more important than whatever short-term distribution problems (“static inefficiencies”) they entail.  Thus, more or less, is Harold Demsetz’s reply to Kenneth Arrow. 

It also subtends the argument being given for why IP rights around Covid vaccines shouldn’t be licensed to the poor in India (side note: Malthus served as professor at the East India Company College).  Developing countries have proposed a waiver of related IP rights to ensure the rapid production of generic Covid vaccines, and Pharma has responded with an army of lobbyists to explain that no, IP can’t possibly be the problem with Covid vaccine distribution, and it would be much better for philanthropies to purchase lots of drugs and then distribute them.  Other unrelated IP industries have followed with their own lobbyists.  Whatever other difficulties exist in getting vaccinations to people in developing countries, it seems hard to deny that insisting on IP rights and thereby limiting production of the drugs isn’t one of them.  Allowing generics – especially in India and Brazil – increases capacity.

Foucault, Marx and Prophecy: the 11th “Thesis on Feuerbach”

By Gordon Hull

Foucault thinks Marxism is bossy.  In Society must be Defended, he lays down the gauntlet clearly enough: totalizing theories get in the way of useful things at the local level.  As he notes, one should beware of:

“the inhibiting effect specific to totalitarian theories, or at least – what I mean is – all-encompassing and global theories.  Not that all-encompassing and global theories haven’t, in fairly constant fashion, provided – and don’t continue to provide – tools that can be used at the local level; Marxism and psychoanalysis are living proof that they can. But they have, I think, provided tools that can be used at the  local level only when, and this  is the real point ,the theoretical unity of their discourse is, so to speak, suspended, or at least cut up, ripped up, torn to shreds, turned inside out, displaced, caricatured, dramatized, theatricalized, and so on. Or at least that the totalizing approach always has the effect of putting the brakes on” (SMD 6).

That is, when you insist on your theoretical unities, you get in the way of actually doing anything.  What we need are to unearth “subjugated knowledges” and specific histories, and such activity requires the “removal of the tyranny of overall discourses” (SMD 8).  In his 1978 interviews with the Italian Communist Duccio Trombadori, Foucault underlines that “I absolutely will not play the part of one who prescribes solutions.  I hold that the notion of the intellectual today is not that of establishing laws or proposing solutions or prophesying, since by doing that one can only contribute to the functioning of a determinate situation of power that to my mind must be criticized” (Remarks on Marx, 157).

He similarly says at the end of an earlier 1978 interview in Japan that “I think that the role of intellectuals, in reality, absolutely does not consist in playing [the role of] prophets or legislators” (D&E #236; Vol II, 264-5 (2 vol, 2001 ed.)).  Marxism is theological, as he says in his 1979-80 Collège de France Lectures:

“With Marxism, it’s the same thing.  You have the model of the fall, alienation and dis-alienation.  You have the model of the two ways: Mao Zedong.  And you have, of course, the problem of the stain of those who are originally soiled and must be purified: Stalinism.  Marx, Mao, Stalin; the three models of the two ways, the fall, and the stain” (Government of the Living,  108)

At one level this is clear enough.   But Foucault also of course is an advocate of social change, and he wants his works to be picked up and used in local struggles, as he also says repeatedly.  Here I want to add a little specificity to the question about Foucault’s relation to Marxism (at least as he understands it around 1978) by picking up on his remarks in the Japan interview. Immediately after saying that intellectuals should not be prophets or legislators, he adds that: “for two thousand years, philosophers have always spoken of what we should have done [de ce qu’on devait faire].  But this always led to a tragic end.  What is important is that philosophers speak of what is currently happening, but not of what could happen” (624).  The first thing to note is the implicit reference to Lenin, at least in the context of a discussion of Marxism: Lenin’s What is to be Done is translated into French as Que Faire (the French rendering is correct: the book is Что Дълать, literally “What to Do”).  More significantly, the connection to prophetic discourse is something Foucault repeats in his slightly earlier interview with Yoshimoto, where he says that:

On Covid data, part n

I've been commenting off and on about the vagaries of Covid data - for example, in knowing what "covid cases" refers to (and here); states' early conflation  of PCR and antibody tests;  the vagaries of different testing technologies, or the ways that even death certificates can mislead about mortality.  This reflective piece by the co-founders of the Covid Tracking Project at the Atlantic - which became the de facto national source for Covid information as the federal government fell flat on its face - is absolutely worth the read.  Here's one short paragraph that says everything you need to know about data in general:

"Data are just a bunch of qualitative conclusions arranged in a countable way. Data-driven thinking isn’t necessarily more accurate than other forms of reasoning, and if you do not understand how data are made, their seams and scars, they might even be more likely to mislead you"

Marx explains how AI systems exploit workers (Part 1)

By Gordon Hull

“Factory work exhausts the nervous system to the uttermost; at the same time, it does away with the many-sided play of the muscles, and confiscates every atom of freedom, both in bodily and in intellectual activity” (Marx, Capital I [Penguin Ed.], 548).

A recent piece by Josh Dzieza in the Verge about the working conditions of those subject to discipline by AI in the workplace is wrenching.  It tells the story of Amazon workers and call center employees and others whose work lives have become an exhausting monotony of ever-intensifying robot-like demands.  The “AI boss” becomes ever more demanding, filling every moment at work with the requirement to be more productive and with absurd metrics for deciding when its demands are met.  Amazon workers report that they are basically ground down by the unsustainability of the pace of productivity; they are either fired or forced to quit, often with injuries.  Call center employees are subject to Orwellian monitoring by a machine that purports (often laughably inaccurately) to monitor and tweak their affect when speaking to customers.  Folks working from home find it impossible to go to the bathroom, because they are forced to sit in front of their computer all the time, which takes frequent pictures of them and reports on their productivity, as measured by lines of code or keystrokes per unit time.

All of this optimized misery reminds us of something important: AI systems are being deployed under capitalism.  A worker subject to an “AI Boss” is not some sort of new-fangled social form; they are fundamentally a worker subject to capitalist exploitation.  What AI has done is enable capitalism to be more intensively exploitative.  The stories in the article echo the stories Marx reports in Capital in his discussion of machinery, as does a lot of the logic.  This suggests that two popular narratives about AI need to be displaced.  First, it may be that AI will someday swoop in and take away everyone’s jobs.  As Dzieza notes, this narrative deflects attention from the pressing reality of deteriorating workplace conditions under AI supervision.  Second, Marx’s work on machines and technology is frequently read (as by the autonomists) through the Grundrisse “Fragment on Machines,” and in a way that emphasizes affective labor and cognitive capital.  The reality of AI today serves as an important reminder that the treatment in Capital is still very important.  Let me explain.

The Elephant in the Vaccine Room

You might have heard that minorities are hesitant about getting a Covid vaccine?  Well, about that.  According to polling reported by Axios, the group least likely to want a vaccine is White Republicans... to the point that "White Americans are now less likely than Black and Latino Americans to say they plan to get the vaccine."  I

Google fires another AI ethicist

This time Margaret Mitchell, one of the other authors on the fabulous "Stochastic Parrots" paper (that's my post on it.  The paper is here) on natural language processing.  This was obviously coming, since they'd suspended her email account weeks ago.  In case you haven't read the paper (and you really should!), it's worth mentioning that the paper does not mention Google, and the goal is to point to strategies to make AI better, i.e., less likely to entrench racism, sexism and so forth while destroying the planet in climate catastrophe.  It turns out that those are related, because bigger and bigger datasets are both carbon intensive and likely to pick up on and magnify hegemonic and toxic speech.  Attention to efficiency and curation moves the needle on both.

No commitment to even looking like they care about ethical AI.

Reading List: Bender and Gebru, “Stochastic Parrots”

By Gordon Hull

Not long ago, Google summarily dumped Timnit Gebru, one of its lead AI researchers and one of the few Black women working in AI.  Her coauthor Emily Bender has now posted the paper (to be presented this spring) that apparently caused all the trouble.  It should be required reading for anybody who cares about the details of how AI and data systems can perpetuate racism, or who cares more generally about the social implications of brute-force approaches to AI.  Bender and Gebru take up a common approach to natural-language processing (NLP), which involves an AI system learning how to anticipate what speech is likely to follow a given unit of speech.  If, for example, I say “Hello, how are,” the system learns by studying a dataset of existing phrases and text snippets that the next word is likely to be “you,” but almost certainly will not be “ice cream.”  How good the computer gets at this game is going to be substantially determined by the quantity and quality of its training data, i.e., the text that it examines.

Bender and Gebru outline the social costs of one approach to this problem, which is basically brute force.  As processing power increases, it’s possible to train computers with larger and larger datasets, and the use of larger datasets reliably improves system performance.  But should we be doing that?  Bender and Gebru detail several kinds of problems.  The first is environmental justice: all that processing power uses a lot of energy.  Although some of it may come from carbon-neutral sources, the net climate cost is significant.  Worse, the NLP systems being produced don’t benefit the people that are going to suffer the most from climate change.  As they memorably put it:

“Is it fair or just to ask, for example, that the residents of the Maldives (likely to be underwater by 2100) or the 800,000 people in Sudan affected by drastic floods, pay the environmental price of training and deploying ever larger English LMs, when similar large-scale models aren’t being produced for Dhivehi or Sudanese Arabic?”

Covid Deaths and the Politics of Obituaries

By Gordon Hull

As of this writing, approximately 421,000 people in the United States have officially died of Covid-19.  We also know that this number is fewer than the number that have actually died of Covid for a variety of reasons.  For example, early in the pandemic, there was nowhere near enough testing, and so many people who died of Covid-19 never received an official diagnosis.  For that sort of reason, measuring “excess deaths” during a given period is one way to try to get a handle on how many people actually died of Covid-19.  If n more people die during a given period this year than last year and/or typically, that number can be a useful indicator for how many deaths can be attributed to the pandemic.

Some of these will of course be indirect: people who didn’t go to the ER over a heart attack because they were afraid of contracting Covid, for example.  The isolation of Covid may be driving an increase in opioid overdose deaths.  Others will be direct, deaths where Covid was the underlying cause. STAT News is now reporting on the results of a recent study that takes a dive into this question, concluding that the overall direct Covid death toll is 31% higher than official figures.  That’s the top line, and it means that the national number of Covid deaths is approaching 550,000.  What should surprise no one (but is surprising, because deaths are so much more reliable for understanding infection rates than reported cases) is that the mortality data is deeply shaped by politics: excess deaths are a window into the sociology of the pandemic.

New Paper: "Infrastructure, Modulation, Portal: Thinking with Foucault about how Internet Architecture Shapes Subjects"

Now up on SSRN.  This paper uses Foucault's works on disciplinary power to develop a typology for understanding different models of Internet governance.  Here is the abstract:

Following Foucault’s remarks on the importance of architecture to disciplinary power, this paper offers a typology of power relations expressed in different models of Internet governance. Infrastructure governance understands the Internet as a common pool or public resource, on the model of traditional infrastructures like roads and bridges. Modulation, which I study by way of Net Neutrality debates in the U.S., understands Internet governance as traffic shaping. Portal governance, which I study by way of data collection policies of dominant platform companies, understands the Internet as creating a user experience that facilitates data mining. The latter two are forms of architectural disciplinary power that undermine the first. I then argue that the rise of portal and modulation governance primarily serves to remake parts of civil society by fostering market norms of consumption and entrepreneurialism. In that sense, efforts to shape Internet architecture need to be understood as techniques of subjectification.

 

Epistemic Justice with AI: indicators, diverse training data and knee pain

Knee pain is common and debilitating, and it’s often caused by osteoarthritis in the knee.  Treatment options range from analgesics (including opioids) to knee-replacement surgery.  If you go to the doctor with arthritic knee pain, you can get an x-ray which can then be interpreted using standard rubrics like the Kellgren–Lawrence Grade (KLG) to quantify damage to your knee and then guide treatment options. The KLG isn’t perfect in that the correlation between pain and objective scores of damage to the knee isn’t perfect.  Some people’s knees are a wreck and they report no pain; others have pain beyond what their KLG score indicate. But here’s the thing: Black patients consistently report more knee pain than white patients.  They also tend to have more knee damage on the KLG – but even when you factor that in, Black patients report much more knee pain than white patients with comparable KLG scores.  What’s going on?

One possibility is that factors external to the knee – stress, for example – explain the higher pain.  If that’s the case, then patients need less knee treatment.  But what if their knees were in worse shape?  To answer that question, you’d have to ask yourself what in an x-ray indicated poor knee condition.

Disease is often measured through indicators, and we know that these indicators can lead to all sorts of complexity.  In the context of Covid, for example, there are all sorts of questions about testing and sensitivity that I’ve talked about before.  Along the way, I referred to a fantastic paper on malaria testing in sub-Saharan Africa – suffice it to say that “cases of malaria” reported to donor organizations is a difficult number to parse for reasons having to do with vagaries in testing and diagnosis.

In a new paper in Nature Medicine, a team led by Emma Pierson makes ingenious use of artificial intelligence to tackle the problem of racial disparities in knee pain.  Since algorithms and data are so often implicated in increasing or magnifying racial disparities (see, for example, Safiya Noble on Google, or Timnit Gebru on facial recognition, or Margaret Hu’s chilling “Algorithmic Jim Crow”), it’s encouraging to learn about machine learning working to undermine racial disparities.  Ordinarily, you train an algorithm to perform like an excellent clinician.  In this case, that would mean training it to look at radiography and determine the correct KLG score.  The trick here was to instead train it to look at pain: to determine what features of the x-ray predicted that the patient would report pain.  It turns out that the algorithm’s diagnoses reduced racial disparities in diagnosis by a jaw-dropping 47%.

BIPA Continues to Gain Teeth: Data Retention Policy & Standing

By Gordon Hull

I’ve written about the importance of Illinois’ Biometric Information Privacy Act (BIPA) before (see also here). Briefly, BIPA is the most important and powerful of the (relatively few) state laws designed to protect biometric privacy. The statute establishes a notice-and-consent regime (sigh. better than nothing, though N&C doesn’t work well, and is disturbing as a norm) for private parties that collect biometric information like face scans, establishes the need for data retention policies, establishes a private right of action (individuals can sue; other states make you go through the state attorney general) and establishes a statutory harm – as underlined by the Illinois Supreme Court, violating the statute is enough to collect damages.

Companies like Facebook have been fighting BIPA hard, because it’s bad for their business model, and some of the main litigation has been around Facebook’s phototagging feature. A lot of the issue has been about standing – whether aggrieved parties have the right to sue. Standing sounds simple: in order to have standing under Article 3 of the Constitution, three conditions need to be met. Going through them, however, will indicate why this is harder than it looks.

• First, the plaintiff must have suffered an “injury in fact,” an invasion of a legally protected interest which is (a) concrete and particularized, and (b) actual or imminent, not “conjectural” or “hypothetical.”
• Second, there must be a causal connection between the injury and the conduct complained of, the injury has to be fairly traceable to the challenged action of the defendant, and not the result of the independent action of some third party not before the court.
• Third, it must be "likely," as opposed to merely "speculative," that the injury will be "redressed by a favorable decision."

The Strange Story of the Oxford Covid-19 Vaccine Dosing

To review the issue: The Oxford/AstraZeneca vaccine uses a modified adenovirus, as do several other vaccines in development, most notably the Russian Gamaleya Institute one.  Early, puzzling results suggested that the Oxford vaccine was about 70% effective overall, but that the overall number obscured a disparity between two groups: a two-dose group of all ages that showed efficacy in the 62% range but 90% in a group (including no elderly people) that had received a half-strength dose first.  Eh?  Even more strangely, the half-dose group seemed to be have given that dose... by mistake?  What happened?

Reuters does a deep dive.  There's a lot of threads, but it appears that the problem began when the Oxford team didn't trust measurements of the strength of a batch of vaccine from an Italian manufacturer.  The Oxford team then measured it using a different technique, concluded it was more potent than the manufacturer said, and trusted its own measurement.  Guess who was right? [Time to scream at the void: WHY on earth, given discrepancies between two measurements, both supposedly reliable but using different techniques, WOULD YOU NOT MEASURE AGAIN UNTIL YOU FELT VERY GOOD ABOUT THE DISCREPANCY AND HOW IT HAPPENED?]

"Oxford’s measurement showed that the batch was more potent than the Italian manufacturer had found, the documents show. Oxford trusted its own result and wanted to remain consistent with a measuring tool it had used throughout an earlier trial phase. So it asked Britain’s drugs regulator for permission to reduce the volume of vaccine injected into trial participants from the K.0011 batch. Permission was granted .... The documents published in The Lancet confirm that the error lay with the Oxford researchers. A common emulsifier, polysorbate 80, used in vaccines to facilitate mixing, had interfered with the ultraviolet-light meter that measures the quantity of viral material, according to the documents. As a result, the vaccine’s viral concentration was overstated and Oxford ended up administering half doses of vaccine, believing they were full doses."

This vaccine matters because it is one that developing countries are depending on: it is cheap, and it is stable in refrigerator temperatures, which makes the logistics a lot easier.  There is a hypothesis that explains the half-dose/full-dose discrepancy: it's possible that the lower first does primes the immune system better than the higher one; adenovirus variants also circulate in some human populations, and so it's possible that some combination of prior infection and the first dose generated an immune response sufficiently robust that participant's own immune systems destroyed the second dose's virus before it could generate further immune response.   The Russians report a 90% efficacy, and that vaccine uses two different adenovirus vectors across the two doses to avoid precisely this risk.  These questions need more data, and we're still waiting for the full results of ongoing trials.  Adenovirus vaccines matter too: in addition to Oxford, Gamaleya and a Chinese vaccine, the single-dose Johnson & Johnson vaccine, with trial results expected in January, is also adenovirus.  In the meantime, Gamaleya is sharing their adenovirus vector, in order to enable a combined trial: one does of the Russian vaccine, and one of the Oxford/Astrazeneca one.

 

 

Boilerplate, or Contract’s Pharmakon

By Gordon Hull

In an important recent article, Robin Kar and Margaret Radin propose a way to interpret the volumes of boilerplate that accompany pretty much any electronically-mediated consumer transaction.  Rather, they propose a way to interpret the phenomenon of the deluge of such boilerplate.  We all know the scenario: you decide to buy a song for $0.99 on a site called SketchyFiles.com, and at some point in the process, you click to indicate your acceptance of the “terms and conditions.”  Did you read those terms and conditions?  Of course you did not: if you were to print them, they’d probably run in excess of 30 pages, most of them using language that you don’t understand.  It’s not rational to slog through and try to understand all of that for a 99-cent purchase!

But the flip side is that SketchyFiles is very much going to interpret this as a contract when it suits them.  Say, for example, that you think that your purchase included a virus that destroyed a couple of files on your computer. You spend a few minutes online, and discover that there are 50,000 people to whom this exact thing happened!  You lawyer up and file suit against SketchyFiles, including a request for class certification, since your files aren’t individually worth a lot, but the aggregate of them are, and you think SketchyFiles ought to have to own the problem and scan their songs for viruses.  You will promptly discover in court that the “contract you signed” included a “mandatory arbitration clause,” which says that you agree that any and all disputes involving your purchase are to be settled out-of-court, using an arbitration procedure and a venue chosen by SketchyFiles.  Not only that, you’ve probably also contractually agreed that no class certifications are possible: all users’ claims must be adjudicated one at a time.

Ethical AI at Google? Not so much.

At least not from the appearance of things.  Google summarily fired Timnit Gebru, one of its lead AI Ethics researchers and one of the few Black women in a leadership position at the company.  Her sin?  Producing academic research critical of biases in AI:

“The email and the firing were the culmination of about a week of wrangling over the company’s request that Gebru retract an AI ethics paper she had co-written with six others, including four Google employees, that was submitted for consideration for an industry conference next year, Gebru said in an interview Thursday. If she wouldn’t retract the paper, Google at least wanted the names of the Google employees removed.  Gebru asked Google Research vice president Megan Kacholia for an explanation and told her that without more discussion on the paper and the way it was handled she would plan to resign after a transition period. She also wanted to make sure she was clear on what would happen with future, similar research projects her team might undertake. …. The paper called out the dangers of using large language models to train algorithms that could, for example, write tweets, answer trivia and translate poetry, according to a copy of the document. The models are essentially trained by analyzing language from the internet, which doesn’t reflect large swaths of the global population not yet online, according to the paper. Gebru highlights the risk that the models will only reflect the worldview of people who have been privileged enough to be a part of the training data”

The company responded by “accepting” her “resignation,” effective immediately (while she was on vacation!).

Let’s be clear: this sort of research is what Gebru does, and she’s very good at it.  Before she got to Google, she co-authored a paper that proved that facial recognition software misrecognized people of color (especially women) at much higher rates than white men.  The paper is influential, and supported efforts like the one that eventually got Amazon to stop selling its facial recognition software to police. Why is Google surprised that she would pursue this topic now?  Also, the link between biased training data and crappy outcomes is a thing.  A thing that a company like Google ought to care about.

Gebru’s unceremonious firing suggests that Google isn’t seriously committed either to actual critical thinking about AI and ethics, or to dealing with Silicon Valley’s glaring diversity problems.  I suppose file under disappointing, but not surprising.

On going to Churches, Schools and Bars

UPDATE 12/6: For more on the mess in Rhode Island, see here.

This week’s SCOTUS opinion overturning New York’s restrictions on religious gatherings is disappointing in many ways. Most obviously, it hamstrings the ability of governors to respond with science to Covid and is part of a conservative backlash to that effort; Justice Breyer makes some of the important points in dissent, followed by more of them in Justice Sotomayor’s dissent. More ominously, it signals the accelerating arrival of the era of having to pander to right wing religious people who think that their religious freedom outweighs the harm principle (see, Thanksgiving is not just about spreading disease. It’s about imposing religion!). This has been coming for a while – recall the Court’s endorsement of Hobby Lobby’s theory that it’s ok to freeload on public services and goods, but that it does not have to extend contraception rights to citizen-employees. But Amy Coney Barrett makes it faster. That is, part of the acceleration is that the Court is illegitimate, by Mitch McConnell’s own standard: the rule is either that you seat a justice before an election or not. Those can’t both be true, except in a shameless right-wing power grab. We’ll get back to Moscow Mitch in a minute.

Also unfortunate here is Governor Cuomo, who seems to have gone out of his way to antagonize the Hassidic community in particular. The order in question also apparently did not index capacity to the size of the building or social distancing plans. The antagonism plus the restrictions lets the Court make disingenuous comparisons between the number of people in a grocery store and a church. It also gives people like Gorsuch the opportunity to say “the Governor is remarkably frank about this: In his judgment laundry and liquor, travel and tools, are all “essential” while traditional religious exercises are not. That is exactly the kind of discrimination the First Amendment forbids” (2). No one should view this as anything other than opportunism; after all, Gorsuch signed onto the Court’s opinion ignoring Trump’s many, many anti-Muslim tweets on the way to upholding his Muslim travel ban.  Politicians’ chatter is only part of the law when you don’t like the politician, it seems.

These are important, but what I haven’t seen discussed is that this opinion intersects with the other news percolating up around Thanksgiving, which is the absolute incoherence of public health restrictions. Folks like Ellie Murray and Julia Marcus have been trying to sound the alarm on Twitter, and Amanda Mull wrote about it in the Atlantic. The public health message is basically: “Thanksgiving is dangerous! It is dangerous to sit around a table with people from outside your household for a long meal! If you want to do that safely, go to Cheesecake Factory!” Ummm… in other words, it is bonkers incoherent that in most of the country, indoor dining at restaurants is still possible when pretty much anything else is closed. Indoor dining is on the same epidemiological list as church for likely spread!

In this particular context, is it any surprise that a group of people who are predisposed to prioritize religious rights noticed something amiss? This is from the per curiam:

“In a red zone, while a synagogue or church may not admit more than 10 persons, businesses categorized as “essential” may admit as many people as they wish. And the list of “essential” businesses includes things such as acupuncture facilities, camp grounds, garages, as well as many whose services are not limited to those that can be regarded as essential, such as all plants manufacturing chemicals and microelectronics and all transportation facilities.” (2)

The invitation for activist judges to ignore science is too much for them to pass up. Here is Kavanaugh: “in a red zone, for example, a church or synagogue must adhere to a 10-person attendance cap, while a grocery store, pet store, or big-box store down the street does not face the same restriction.” (2). Gorsuch piles on: “Nor is the problem an isolated one. In recent months, certain other Governors have issued similar edicts. At the flick of a pen, they have asserted the right to privilege restaurants, marijuana dispensaries, and casinos over churches, mosques, and temples” (3). And he concludes: “It is time—past time—to make plain that, while the pandemic poses many grave challenges, there is no world in which the Constitution tolerates color-coded executive edicts that reopen liquor stores and bike shops but shutter churches, synagogues, and mosques.” (7).  Kavanaugh again: “once a State creates a favored class of businesses, as New York has done in this case, the State must justify why houses of worship are excluded from that favored class” (3).

Nevermind that you do different things in all of those facilities; as Justice Sotomayor acidly notes, “bike repair shops and liquor stores generally do not feature customers gathering inside to sing and speak together for an hour or more at a time” (3). She adds:

“Justices of this Court play a deadly game in second guessing the expert judgment of health officials about the environments in which a contagious virus, now infecting a million Ameri-cans each week, spreads most easily” (3).

So the conservatives are simply wrong on the facts, even though those facts are articulated quite clearly in the decision they overturn.  And that decision will probably cost people their lives.  Freedom!

But: bars and restaurants? Perhaps I am particularly sensitive to this point because schools are closed too (as one meme wondered – should NYC parents send their kids to the bars that are still open, now that schools are closed?), but it’s impossible not to notice that moneymaking endeavors get a different set of rules from civic ones. As Mull puts it:

“With people out of work and small businesses set up to fail en masse, America has landed on its current contradiction: Tell people it’s safe to return to bars and restaurants and spend money inside while following some often useless restrictions, but also tell them it’s unsafe to gather in their home, where nothing is for sale. It’s a woefully inadequate stimulus plan, funded by money extracted little by little from the pockets of people who are mostly just confused about what they’re being compelled to do.”

She cites the example of Rhode Island, whose restrictions are completely absurd:

“Residents are prohibited from gathering with even one person outside their household, even in the open air of a public park. But inside a restaurant? Well, 25 people is fine. Hire a caterer? You’re legally cleared to have up to 75 outdoors. The governor’s executive order merely notes: “The lower attendance at such events, the lower the risk.” (The Rhode Island governor’s office did not respond to a request for comment.).”

And how did we get here? Well, as Mull underlines, closing bars and restaurants (etc) will cause a lot of unemployment. Closing schools and churches won’t.

In other words, it’s not that the plaintiffs in the New York case necessarily raised a compelling First Amendment argument, but they are a canary in the coal mine for a deeply problematic public health policy, one that systematically favors commercial activities over civic ones.  Governors’ hands are tied, even the ones who want to do the right thing, because the proper public health measures will cost money in the form of income guarantees to the people affected by them. And guess who has – perhaps singlehandedly – stood most in the way of that financial assistance? That’s right – Moscow Mitch and his repeated bad-faith refusals to negotiate a second stimulus package. Turns out he was so busy ramming right-wing court justices through the Senate that he made it more dangerous to do stuff that is more important, like school, worship and family Thanksgiving.

ToS and the Subject Position of Website Users

By Gordon Hull

As one knows, online privacy policies (and access to the Internet in general) are generally conditioned on a user’s acceptance of some sort of boilerplate terms of service.  Lots of people (myself included) have complained about this state of affairs as attempting to get users to consent to all sorts of practices by way of reams of unintelligible legalese.  In addition to allowing unlimited data collections, these contracts often include mandatory arbitration provisions, which means that they compel users to take disputes with the sites to arbitrators selected by the company, rather than taking them to courts.

In other words, the idea is that using the product or service constitutes consent to a series of wildly one-sided terms that the user probably could not understand if she tried.  There is certainly no chance of negotiation or bargaining involved.  As Julie Cohen put it, “the conception of consent emerging from that default condition is unprecedented in the law of contracts or any other body of law,” and more closely resembles older status contracts than the contracts of modern liberalism (58-9).  The result is “a form of Kabuki theater that distracts both users and regulators from what is really going on” (59).

In a recent paper, Woodrow Hartzog and Neil Richards detail this state of affairs in terms of contract theory and its imaginary of two autonomous individuals negotiating and agreeing to terms.  They argue that:

“Consent is most valid when we are asked to choose infrequently, when the potential harms that result from the consent are easy to imagine, and when we have the correct incentives to consent consciously and seriously. The further we fall from this gold standard, the more a particular consent is pathological and thus suspect” (1465).

The design of notice-and-consent privacy regimes is deeply pathological from this point of view.  Consent is often unwitting – consumers don’t understand either the terms or the technology behind them.  They also don’t understand the consequences and risks (which is entirely expected, given that they are futural and difficult to visualize).   If not fully coerced, consent can be induced by both the fact that it is difficult now not to use the internet (so no opt-out) and both ISPs and networks are de facto monopolies (so no market competition).  Various website design techniques make opting out difficult: pop-ups that cover up the screen, browser tabs with no ‘back’ button, and various techniques of shaming (“Yes! I want this product.  No, I choose to die a lonely death”).  And of course there is the question of children: COPPA only protects children up to age 13, even though contractual consent typically starts at age 18.  And kids can easily get around age checks anyway.

If we step back from the normative arguments about why this situation is abusive or otherwise bad, I think there’s an interesting point to be made about subjectivity here.  Basically, the sites are trying to construct a legally cognizable subject position that users then occupy.  Both steps are important.  The contractual language presented in the ToS defines the subject position, and it seems to me that it’s important to distinguish this subject position from a few alternatives. First, this is not the traditional subject of law.  That position is foreclosed by the lack of availability of the courts.  Disputes are to be settled by a privatized procedure.  Second, it’s not the subject of administrative procedure either, such as recently upheld by SCOTUS in patent arbitration.  That procedure also allows an appeal to courts.  Third, it’s not the position of the disciplinary (?) subject that consumes mass media: there is no pretext to contract when you open a book, as Hartzog notes in an earlier paper on these contracts.  In short, I think this is a quite specific subject position constructed by the technology companies for their own advantage.

There is also a split between the structural subject position and the phenomenological experience of users.  Users are encouraged by the sites in question to view themselves as free and emancipated by the technology, and as empowered by its affordances.  They are (for obvious reasons) not encouraged to view themselves as signing away their rights.

The gap between user experience and described subject position leads to the second question: whether users can be said to occupy this subject position, i.e., whether these contracts are enforceable.  Can technology companies rely on the coercive power of the state to define the subjectivity of users on their terms?

It is the position of the technology companies that use of their products constitutes acceptance of the terms of service; sometimes this requires clicking a button.  Courts have been generally unwilling to say that the contracts are objectively unenforceable as unconscionable or otherwise inconsentable (though Hartzog and Evan Selinger more recently make the case that facial recognition ought to be the sort of thing that someone cannot consent to – the harms are potentially too great and difficult to understand).  This means that the question legally has come down to whether the notice and consent procedures are adequate.  Hartzog’s earlier paper suggested that, as of roughly 2010, courts had been most likely to refuse to enforce ToS on casual users – those who just read a website but do not otherwise particularly interact with it.  Still, the law was inconsistent and this emerging de facto carve-out was far from either settled or explicit.

Here, a recent survey of the case law by Nancy Kim is encouraging.  She reports that the case law is still very inconsistent, and users often lose.  But: there’s a movement toward demanding more of websites:

“The standards of “notice” and “manifestation of assent” remain the same, but how those standards are applied varies depending upon the facts of the case and the jurisdiction. Berkson v. Gogo LLC and Nguyen v. Barnes & Noble, Inc. have been particularly influential in helping inform the application of the standards. Reflecting the influence of these two cases, courts have started to assess the reasonableness of notice by evaluating the website experience from the perspective of the offeree and by considering how the drafter chose to present terms. Thus, instead of asking why the offeree failed to read visible terms, more and more courts are asking why the offeror failed to draft terms in a more conspicuous manner. As the court in Rushing v. Viacom Inc. emphasized, the burden is on website owners to “put users on notice of the terms to which they wish to bind consumers.” Accordingly, courts are increasingly asking for two types of evidence from drafters. The first is in the form of screenshots or webflows that demonstrate how the terms were presented to plaintiffs. The second is in the form of the drafter’s internal records that document that the plaintiff actually accessed the website at the time the notice was allegedly presented.” (1693)

There’s a lot to be done here, of course, but it’s nice to see some legal pushback, and to see how courts understand evidence and other traditional legal concepts in this context.  Kim cites several cases where courts determined that reasonably prudent users would not realize that they were entering into a contractual relation – for example, there was a hyperlinked “terms of service” but it was buried several pages into a website and well below the prominent “click here for the next step of your purchase.”  Nearly all of the cases Kim cites are about the enforceability of mandatory arbitration clauses, which suggests both the specificity of the subject position the websites are constructing, and the extent to which its enforceability is an artifact of their power over not just users but the state.

Encouraging case on Qualified Immunity

Given the role of qualified immunity in absolving police officers of murdering unarmed black men (and doing all sorts of other nefarious things), it’s encouraging to see that the Supreme Court said in a per curiam opinion today that there is an outer limit to how far that doctrine can be extended.  Recall the problem: courts will rule that yes, some behavior or another would be unconstitutional, but the law on the subject is not “clearly established,” so the officer is shielded from liability.  The trick is that “clearly established” morphs into “an officer in an identical situation has already been found liable for doing exactly what is alleged here.”  Then of course liability becomes almost impossible to establish.

Consider now the case of Trent Taylor:

“Petitioner Trent Taylor is an inmate in the custody of the Texas Department of Criminal Justice. Taylor alleges that, for six full days in September 2013, correctional officers confined him in a pair of shockingly unsanitary cells. The first cell was covered, nearly floor to ceiling, in “massive amounts’ of feces”: all over the floor, the ceiling, the window, the walls, and even “packed inside the water faucet.” Fearing that his food and water would be contaminated, Taylor did not eat or drink for nearly four days. Correctional officers then moved Taylor to a second, frigidly cold cell, which was equipped with only a clogged drain in the floor to dispose of bodily wastes. Taylor held his bladder for over 24 hours, but he eventually (and involuntarily) relieved himself, causing the drain to overflow and raw sewage to spill across the floor. Because the cell lacked a bunk, and because Taylor was confined without clothing, he was left to sleep naked in sewage”

Like clockwork, the extra-conservative 5^th Circuit concluded that the treatment was unconstitutional, but that there was no liability (I’m quoting SCOTUS) because “based on its [the 5^th Circuit’s] assessment that “[t]he law wasn’t clearly established” that “prisoners couldn’t be housed in  cells teeming with human waste” “for only six days,” the court concluded that the prison officials responsible for Taylor’s confinement did not have “ ‘fair warning’ that their specific acts were unconstitutional.” SCOTUS points out the obvious: “no reasonable correctional officer could have concluded that, under the extreme circumstances of this case, it was constitutionally permissible to house Taylor in such deplorably unsanitary conditions for such an extended period of time.”  They add: “Confronted with the particularly egregious facts of this case, any reasonable officer should have realized that Taylor’s conditions of confinement offended the Constitution.”  So back to litigation – no summary judgment in favor of the officers.

We still have a long way to go before qualified immunity is reined in adequately, but this shows there is at least a floor.  In the meantime, Justice Alito writes a concurrence in which he first says that the SCOTUS shouldn’t have reviewed the case, but given that it has, he concurs in the judgment.  Oh, and Justice Thomas dissented.  But you could have guessed that.  More importantly, at least seven justices realized that there could be limits to qualified immunity.  These days, it’s important to recognize even small first steps.

Foucault on Public Opinion

If you’re like me, you spend too much time – way too much time – these days looking at polling data.  I ran across some interesting remarks by Foucault on opinion yesterday, which I’ll share here as a technique of distraction.  He makes them in the context of a 1976 conversation with J. P. Barou and Michelle Perrot (whose work on resistance to disciplinary power he favorably cites near the end of the conversation) that was published as the preface to an edition of Bentham’s Panopticon writings.  It appears as “L’oeil du pouvoir” (D&E #195, pp. 190-207 in my 2 volume edition) and is translated in Foucault Live (=FL).   For context, then, the conversation appears in the year after Discipline and Punish.  It covers a range of topics, including Foucault’s own path to discovering the panopticon (initially via hospital architecture, which had the dual need to see patients and keep them physically separated to avoid the spread of disease).

Foucault suggests that Bentham’s original work enjoyed considerable uptake in revolutionary France.  Barou sets Foucault up: “isn’t it surprising to realize that the French Revolution, in the persons of people such as Lafayette, favorably received the project of the panopticon?” (D&E 195).  Foucault answers that Bentham “is the complement of Rousseau” because the “Rousseauian dream” that “animated the revolutionaries” was of a “transparent society, both visible and legible in each of its parts, which had no more obscure zones, no zones established by the privileges of royal power or by the prerogatives of such and such body, or indeed by disorder; that each, from the point that he occupies, is able to see the entirety of society; that their hearts communicate with one another, that the gazes encounter no more obstacles, that opinion rules, that of each on each [celle de chacun sur chacun]” (D&E 195/FL 230).  Opinion thus functions similar to what Foucault will later identify as a function of pastoral power (omnes et singulatim), simultaneously totalizing and individuating.  “Opinion” is an object of “police” power in Discipline and Punish (213), that power that is concerned with understanding and managing the minutiae of daily life.

In the interview, Foucault suggests that Bentham is “both this [Rousseauian dream] and its complete opposite” because he understands the problem of visibility “while thinking of a visibility organized entirely around a dominant and surveillant gaze” (D&E 195/FL 230).  Thus you have joined the “lyricism of Rousseau and the obsession of Bentham.”  Foucault then suggests:

“A fear haunted the second half of the 18^th century: there is a dark space, a screen of obscurity which provides an obstacle to the complete visibility of things, of people, of truths. [One needs] to dissolve the fragments of night which were opposed to the light, to make it so that there is no more dark space in society; to demolish the black rooms where arbitrary political power, the caprices of the monarch, religious superstitions, conspiracies of tyrants and priests, illusions of ignorance and epidemics [all] fomented” (D&E 196/FL 231).

In this context, the then proposes that “the rule of ‘opinion’ that we invoke so often in our age is a mode of functioning where power will be able to be exercised on the sole condition that things are known and that people are seen by a type of immediate gaze [par une sorte de regard immédiat] that is collective and anonymous” (D&E 197/FL 232).

Hold on to the word "immediate" – it will be important in a moment.  But for now, notice the extent to which this is a pretty good description of nights spent gazing at opinion polls, both the obsession with knowing, and the fear that there are (say) hidden Trump voters like there were last time.  More light!  This sort of epistemology is driven by a fundamentally paranoid logic, as Wendy Chun emphasized a while ago: once I use some sort of optical device to see beyond what my naked eye tells me, that opens the fundamental possibility that a better device would see more.  See a molecule with the microscope?  Is that really the smallest particle?  After all, there were molecules lurking below the surface that you hadn’t seen before.  Here the paranoia is inevitable given a sampling process.  The poll represents underlying opinions, and the process of sampling invariably “sees” only part of the data, even when it’s done exactly right.  And that’s before you factor in questions about whether people who don’t have landlines are proportionally represented, or whether young people who say they will vote will actually do so, and so on.  That’s also before you get to issue polling, which I complained about before the last election, arguing that it worked to create populations and truths about them that were fully the product of the polling questions, making it (for example) impossible to express views that didn’t fit the categories the polling firm wanted to hear.

In any case, as anybody who works in data will tell you, our access to data is fundamentally mediated by the process of its collection and description, which means that the immediacy we want – to just see clearly whether Trump has enough hidden voters somewhere in Pennsylvania to pull off the upset.  We have to content ourselves with something like the fivethirtyeight statistical runs: crunch all the polling data (including weighting it by quality of poll), simulate the election 40,000 times (there’s a vision of hell: 40,000 occurrences of this election!) and see what happens (as of this minute, Biden wins 88% of the time).  But the human brain just doesn’t like that: a pair of papers, one in Philosophy and Public Affairs,  and one in the Stanford Law Review, by David Enoch, Levi Spectre and Talia Fisher make the case that we prefer juridical reasoning in the sense that we can explain why somebody is wrong, rather than living with the statistical truth (in this case, that Trump could win and there’s nothing wrong with the model at all.  Enoch Spectre and Fisher use the example of liability for a bus accident).

All of which leads to the following remarkable paragraph.  In response to Perrot’s suggestion that Bentham seems to overstate the power of his panopticon, Foucault says:

“This is the illusion of almost all of the reformers of the 18^th century who lent to opinion a considerable power.  Opinion is only able to be good when it is the immediate consciousness of the complete social body; they believed that people would become virtuous by the fact that they were observed [regardés].  For them, opinion was a spontaneous reactualization of the social contract.  They misunderstood the real conditions of opinion, of the media, a materiality which is caught in the mechanisms of the economy and power” (D&E 204/FL 238).

And:

“They also failed to understand that the media would necessarily be controlled by economic and political interests. They did not perceive the material and economic components of public opinion. They thought that public opinion would be just by its very nature, that it would spread on its own accord, and provide a kind of democratic surveillance. It was essentially journalism—a crucial innovation of the 19th century—that manifested the utopian characteristics of this entire politics of the gaze” (D&E 204/FL 238)

Of course as Foucault notes, journalism comes with its own politics, and different forms of it come with different politics.  I’ll close with two tidbits.  First, one large study found that false news traveled further, faster and deeper on Twitter than true news, almost no matter how one measured diffusion.  It’s the materiality of the news infrastructure and the business models of the platforms through which opinion is constituted.  Second, there appears to be a horrible nexus of Instagram, Facebook, QAnon and multi-level marketing schemes that’s contributing to the diffusion of that conspiracy: it turns out that QAnon and multi-level marketing share a lot in common, and social media tends to facilitate that.  I can’t describe it better, but the article is worth a read.  The paranoid logic is baked in!

Hardin vs. Demsetz, or The Tragedy of the Commons versus the Economists

By Gordon Hull

I’m going to be teaching Harold Demsetz’s “Toward a Theory of Property Rights” (1967) tomorrow, and noticed a couple of things that I hadn’t before. I suspect they’re related, and say something about the moment the article appeared.  At least, that’s what I want to propose here.  To set it up: Demsetz is known for the theory that property rights emerge when it is efficient to internalize externalities.  His example is land rights among Native Americans; he says that property in land increases with the arrival of the fur trade.  Absent land rights, there was always a risk in overly intensive hunting occurring: without private property, it is in nobody’s interest to invest in the growth and maintenance of hunting stock.  Before the arrival of Europeans, this was fine because there was no particular incentive to hunt too intensively.  When the European fur trade arrived however, the incentives to intensive hunting grew, and property rights emerged as a management strategy.  If I have to live with the consequences of depletion of the hunting stock on my land, I’ll have the incentive to manage it correctly.  That, at least, is the story.  It’s quasi descriptive in the sense that Demsetz thinks this is an explanation for why we have property regimes, and somewhat normative in that it’s clear between the lines that he thinks private property is a good way to go.  Here’s the observations: first, the piece is heavily reliant on Ronald Coase, and second, although the narrative sounds like a tragedy of the commons narrative, it actually flies in the face of what Hardin was trying to do in that article.

A Deleuzian Undercurrent to “What is an Author” (part 6: Instaurateur as Event)

By Gordon Hull

I want here to tie together the preceding several posts (one, two, three, four, five) and finish the case for a Deleuzian undercurrent (perhaps better to say, Deleuzian and Althusserian undercurrents) to Foucault’s 1969 “What is an Author” seminar.  Recall the specific point of interest: in a somewhat odd moment near the end of the lecture, Foucault distinguishes between authors and “instaurateurs.”  The first move is to distinguish Marx and Freud from Ann Radcliff, the founder of Gothic novels.  Radcliff “opened the way for a certain number of resemblances and analogies which have their model or princiuple in her work” (114).  Marx and Freud, on the other hand, “have created a possibility for something other than their discourse, yet something belonging to what they founded” (115).  He then disginguishes Marx/Freud from science as established by Galileo; for Marx/Freud, “the initiation of a discursive practice is heterogenous to its subsequent transformations” (115) rather than a development of it.

The most direct Deleuzian way to express this distinction is to say that instaurateurs are repetitions (as I explored earlier) or “events.”  Before doing that, though, it’s worth noting that Foucault did not have to mention Marx in this context.  Indeed, given his dismissals of Marxism and his hostility to the PCF, it is perhaps striking that he does so.  After all, while it is certainly the case that Foucault sees himself as offering a different critical approach from that in Marxism, he could achieve that simply by distinguishing his approach from Marxism.  Why the reference to Marx?  As I have suggested, the Foucault-Marx relation is one that’s under-explored in the literature; here, it’s only necessary to note that Foucault takes appreciative notice of Althusser’s circle and its efforts at rereading Marx.  In a decisive passage (that I noted last time), Althusser  writes: