New APPS: Art, Politics, Philosophy, Science

Foucault and Epistemic Injustice (part 1)

By Gordon Hull

Those of us who have both made extensive use of Foucault and made a foray into questions of epistemic injustice have tended to sweep the question of the relation between the two theoretical approaches under the rug.  Miranda Fricker’s book, which has basically set the agenda for work on epistemic injustice, acknowledges a substantial debt to Foucault, but in later work she backs away from the ultimate implications of his account of power on the grounds that his historicism undermines the ability to make normative claims.  In this her argument makes a fairly standard criticism of Foucault (whose “refusal to separate power and truth” she aligns with Lyotard’s critique of metanarratives (Routledge Handbook of Epistemic Injustice, 55).  As she describes her own project:

“What I hoped for from the concept of epistemic injustice and its cognates was to mark out a delimited space in which to observe some key intersections of knowledge and power at one remove from the long shadows of both Marx and Foucault, by forging an on-the-ground tool of critical understanding that was called for in everyday lived experience of injustice … and which would rely neither on any metaphysically burdened theoretical narrative of an epistemically well-placed sex-class, nor on any risky flirtation with a reduction of truth or knowledge to de facto social power” (Routledge Handbook, 56).

On this reading, then, Marxism relies too much on ideology-critique, on the one hand, and on privileging the position of women/the proletariat (or other, singular subject position).  Foucault goes too far and reduces the normative dimension altogether. 

In a new paper, Daniele Lorenzini addresses the Foucault/Fricker question head-on, centrally focusing on the critique of Foucault’s supposed excessive historicism.  Lorenzini’s contribution, to which I will return later, is to suggest that Foucault’s later writings (1980 and forward) distinguish between “games” of truth and “regimes” of truth. The distinction is basically illustrated in the following sentence: “I accept that x and y are true, therefore I ought to do z.”  The game of truth is the epistemic first half of the sentence, and the “regime” of truth – the part that governs human behavior – is the second half, the “therefore I ought…”  On this reading, genealogy is about unpacking and bringing to light the tendency of the “therefore” to disappear as we are governed by its regime, and to unpacking the power structures that make it operate.  In other words genealogy doesn’t collapse questions of truth and power; rather, it allows us to separate them by showing that a given game of truth does not entail the regime of truth that goes with it.

New Paper: Epistemic Injustice in Machine Learning Systems

From the Department of Shameless Self-Promotion, here is the abstract for my new paper, "Dirty Data Labeled Dirt Cheap: Epistemic Injustice in Machine Learning Systems:"

"Artificial Intelligence (AI) and Machine Learning (ML) systems increasingly purport to deliver knowledge about people and the world or to assist people in doing so.  Unfortunately, they also seem to frequently present results that repeat or magnify biased treatment of racial and other vulnerable minorities, suggesting that they are “unfair” to members of those groups.  However, critique based on formal concepts of fairness seems increasingly unable to account for these problems, partly because it may well be impossible to simultaneously satisfy intuitively plausible operationalizations of the concept and partly because fairness fails to capture structural power asymmetries underlying the data AI systems learn from.  This paper proposes that at least some of the problems with AI’s treatment of minorities can be captured by the concept of epistemic injustice.  I argue that (1) pretrial detention systems and physiognomic AI systems commit testimonial injustice because their target variables reflect inaccurate and unjust proxies for what they claim to measure; (2) classification systems, such as facial recognition, commit hermeneutic injustice because their classification taxonomies, almost no matter how they are derived, reflect and perpetuate racial and other stereotypes; and (3) epistemic injustice better explains what is going wrong in these types of situations than does (un)fairness."

The path from idea to paper here was slow, but I hope the paper is convincing on the point that the literature on epistemic injustice can offer some needed resources for understanding harms caused by (some kinds of ) AI/algorithmic systems.

Just because the computer talks to you about its feelings doesn’t make it sentient

By Gordon Hull

UPDATE: 6/14: Here's a nice takedown ("Nonsense on Stilts") of the idea that AI can be sentient.

I don’t remember where I read about an early text-based chatbot named JULIA, but it was likely about 20 years ago. JULIA played a flirt, and managed to keep a college student in Florida flirting back for something like three days.  The comment in whatever I read was that it wasn’t clear if JULIA had passed a Turing test, or if the student had failed one.  I suppose this was inevitable, but it appears now that Google engineer Blake Lemoine is failing a Turing test, having convinced himself that the natural language processing (NLP) system LaMDA is “sentient.”

The WaPo article linked above includes discussion with Emily Bender and Margaret Mitchell, which his exactly correct, as they’re two of the lead authors (along with Timnit Gebru) on a paper (recall here) that reminds everyone that NLP is basically a string prediction task: it scrapes a ton of text from the Internet and whatever other sources are readily available, and gets good at predicting what is likely to come next, given a particular input text.  This is why there’s such concern about bias being built into NLP systems: if you get your text from Reddit, then for any given bit of text, what’s likely to come next is racist or sexist (or both).  The system may sound real, but it’s basically a stochastic parrot, as Bender, Gebru and Mitchell put it.

So point one: LaMDA is not sentient, any more than ELIZA and JULIA were sentient, but chatbots are getting pretty good at convincing people they are.  Still, it’s disturbing that the belief is spreading to people like Lemoine who really, really ought to know better.

Base Rate Tracking often can’t fix algorithmic fairness

By Gordon Hull

As a criterion for algorithmic assessment, “fairness” has encountered numerous problems.  Many of these emerged in the wake of ProPublica’s argument that Broward County’s pretrial detention system, COMPAS, was unfair to black suspects.  To recall: In 2016, ProPublica published an investigation piece criticizing Broward County, Florida’s use of a software program called COMPAS in its pretrial detention system.  COMPAS produced a recidivism risk score for each suspect, which could then be used in deciding whether someone should be detained prior to their trial.  ProPublica’s investigation found that, among suspects that did not have a rearrest prior to their trial, black suspects were much more likely to have been rated as “high risk” for rearrest than white suspects.  Conversely, among suspects who were arrested a second time, white suspects were more likely to have been labeled “low risk” than black ones.  The system thus appeared to be discriminating against black suspects.  The story led to an extensive debate (for an accessible summary with cites, see Ben Green’s discussion here) over how fairness should be understood in a machine learning context.

The debate basically showed that ProPublica focused on outcomes and demonstrated that the system failed to achieve separation fairness, which is met when all groups subject to the algorithm’s decisions receive the same false negative/positive rate.  The system failed because “high-risk” black suspects were much more likely than white to be false positives.  In response, the software vendor argued that the system made fair predictions because among those classified in the same way (high or low risk), both racial groups exhibited the predicted outcome at the same rate.  In other words, among those classified as “high risk,” there was no racial difference in how likely they were to actually be rearrested.  The algorithm thus satisfied the criterion of sufficiency fairness.  In the ensuing debate, computer scientists arrived at a proof that, except in very limited cases, it was impossible to simultaneously satisfy both separation and sufficiency fairness criteria.

In the meantime, on the philosophy side, Brian Hedden has argued that a provably fair algorithm could nonetheless be shown to potentially violate 11 of 12 possible fairness conditions.  In a response piece, Benjamin Eva showed the limits of the twelfth with a different test and proposed a new criterion:

Clearview settles Facial Recognition Case

Luke Stark argues that Facial recognition should be treated as the “plutonium of AI” – something so dangerous that it’s use should be carefully controlled and limited.  If you follow  the news, you’ll know that we’re currently treating it as the carbon dioxide of AI, a byproduct of profit-making that doesn’t look too awful on its own until you realize its buildup could very well cause something catastrophic to happen.  Activists have worried about this pending catastrophe for a while, but lots of big money supports facial recognition, so they have thrown up a smokescreen of distractions – in one case, Facebook denied that its phototagging software in fact recognized faces (!) – in order to lull everyone into accepting it.

One of the worst offenders is a secretive company called Clearview, whose business model is to scrape the web of all the pictures it can find and then sell the technology to law enforcement.  The company even has an international presence: in one disturbing instance, the Washington Post documents the use of its technology by Ukrainians to identify dead Russian soldiers by way of their Instagram and other social media accounts, and then sometimes to contact their families.  More generally, the Post revealed internal documents showing that the company' database is nearing 100 billion images and that "almost everyone in the world will be identifiable."  They're going all-in; the Post reports that "the company wants to expand beyond scanning faces for the police, saying in the presentation [obtained by the WP] that it could monitor 'gig economy' workers and is researching a number of new technologies that could identify someone based on how they walk, detect their location from a photo or scan their fingerprints from afar."

Clearview is also one of a cohort of companies that has been sued for violating Illinois’ Biometric Information Privacy Act (BIPA).  BIPA, uniquely among American laws, requires opt-in assent for companies to use people’s biometric information (the Facebook case is central to my argument in this paper (preprint here); for some blog-level discussion see here and here).  Of course, BIPA is a state-level law, so its protections do not automatically extend to anyone who lives outside of Illinois.  That’s why yesterday’s news of a settlement with the ACLU is really good news.  The Guardian reports:

Facial recognition startup Clearview AI has agreed to restrict the use of its massive collection of face images to settle allegations that it collected people’s photos without their consent.  The company in a legal filing Monday agreed to permanently stop selling access to its face database to private businesses or individuals around the US, putting a limit on what it can do with its ever-growing trove of billions of images pulled from social media and elsewhere on the internet. The settlement, which must be approved by a federal judge in Chicago, will end a lawsuit brought by the American Civil Liberties Union and other groups in 2020 over alleged violations of an Illinois digital privacy law. Clearview is also agreeing to stop making its database available to Illinois state government and local police departments for five years. The New York-based company will continue offering its services to federal agencies, such as US Immigration and Customs Enforcement, and to other law enforcement agencies and government contractors outside Illinois.

Of course, the company denies the allegations in the lawsuit, and insists that it was just in the process of rolling out a “consent-based” product.  Ok, sure!  This is still a win for privacy and for one of the very few pieces of legislation in the U.S. that has any chance of limiting the use of biometric data.

Yet another terrible AI idea

People make snap judgments about those they see the first time – mentally categorizing someone as friendly, threatening, trustworthy, etc.  Most of us know that those impressions are idiosyncratic, and suffused with cultural biases along race, gender and other lines.  So obviously I know what you’re thinking… we need an AI that do that, right?  At least that’s what this new PNAS paper seems to think (h/t Nico Osaka for the link).  The authors start right in with the significance:

“We quickly and irresistibly form impressions of what other people are like based solely on how their faces look. These impressions have real-life consequences ranging from hiring decisions to sentencing decisions. We model and visualize the perceptual bases of facial impressions in the most comprehensive fashion to date, producing photorealistic models of 34 perceived social and physical attributes (e.g., trustworthiness and age). These models leverage and demonstrate the utility of deep learning in face evaluation, allowing for 1) generation of an infinite number of faces that vary along these perceived attribute dimensions, 2) manipulation of any face photograph along these dimensions, and 3) prediction of the impressions any face image may evoke in the general (mostly White, North American) population”

Let’s maybe think for a minute, yes?  Because we know that people make these impressions on unsound bases!

First, adversarial networks are already able to produce fake faces that are indistinguishable from real ones.  Those fake faces can now be manipulated to appear more or less trustworthy, hostile, friendly, etc.  When you make fake political ads, for example, that’s going to be useful.  Already 6 years ago, one “Melvin Redick of Harrisburg, Pa., a friendly-looking American with a backward baseball cap and a young daughter, posted on Facebook a link to a brand-new website,” saying on June 8, 2016 that “these guys show hidden truth about Hillary Clinton, George Soros and other leaders of the US. Visit #DCLeaks website. It’s really interesting!” Of course, both Melvin Redick and the site he pointed to were complete fabrications by the Russians.  Now we can make Melvin look trustworthy, and Clinton less so.

Second, the ability to manipulate existing face photos is a disaster-in-waiting.  Again, we saw crude efforts with this before – making Obama appear darker than he is, for example.  But here news photos could be altered to make Vladimir Putin appear trustworthy, or Mr. Rogers untrustworthy.  This goes nowhere good, especially when combined with deepfake technology that already takes people out of their contexts and puts them in other ones (disproportionately, so far, women are pasted into porn videos, but the Russians recently tried to produce a deepfake of Zelensky surrendering.  Fortunately that one was done sloppily).

Third, and I think this one is possibly the scariest, what about scanning images to see whether someone will be assessed as trustworthy?  AI-based hiring is already under-regulated!  Now employers will run your photo through the software and make customer-service hiring decisions based on who customers will perceive as trustworthy.  What could go wrong?

All of this of course assumes that this sort of software actually works.  The history of physiognomic AI, which uses all sorts of supposedly objective cues to determine personality and which is basically complete (usually racist) bunk suggests that the science is probably not as good as the article acts like.  So maybe we’re lucky and this algorithm does not actually work as advertised.  Of course, the fact that AI software is garbage doesn't preclude its being used to make people's lives miserable.  Just consider the bizarre case of VibraImage.

But don’t worry.  The PNAS authors are aware of ethics, noting that “the framework developed here adds significantly to the ethical concerns that already enshroud image manipulation software:”

“Our model can induce (perceived) changes within the individual’s face itself and may be difficult to detect when applied subtly enough. We argue that such methods (as well as their implementations and supporting data) should be made transparent from the start, such that the community can develop robust detection and defense protocols to accompany the technology, as they have done, for example, in developing highly accurate image forensics techniques to detect synthetic faces generated by SG2. More generally, to the extent that improper use of the image manipulation techniques described here is not covered by existing defamation law, it is appropriate to consider ways to limit use of these technologies through regulatory frameworks proposed in the broader context of face-recognition technologies.” 

Yes, the very effective American regulation of privacy does inspire confidence!  Also, “There is also potential for our data and models to perpetuate the biases they measure, which are first impressions of the population under study and have no necessary correspondence to the actual identities, attitudes, or competencies of people whom the images resemble or depict.”

Do you think?  As Luke Stark put it, facial recognition is the “plutonium of AI:” very dangerous and with very few legitimate uses.  This algorithm belongs in the same category, and should similarly be regulated like nuclear waste.  For example, as Ari Waldman and Mary Anne Franks have written, one of the problems with deepfakes is that the fake version gets out there on the internet, and it is nearly impossible to make it go away (if you even know about it).  Forensic software gets there too late, and those without resources aren’t going to be able to deploy it anyway.  Lawsuits are even less useful, since they're time-consuming and expensive to pursue, and lots of defendants won't be jurisdictionally available or have pockets deep enough to make the chase worth it.  In other words, not everybody is going to be able to defend themselves like Zelensky, who both warned about deepfakes and was able to produce video of himself not surrendering.  In the meantime, faked and shocking things generally get diffused faster and further than real news.  After all, “engagement” is the business model of social media.  Further, to the extent that people stay inside filter bubbles (like Fox News), they may never see the forensic corrections, and they probably won’t believe the real one is real, even if they do.

And as for reinforcing existing biases, Safiya Noble already wrote a whole book on how algorithms that guess what you’re probably thinking about someone can do just that.

Copyright Office gets this right...

in refusing to grant copyright registration to an AI creation.  I suspect this one to be litigated for a while, since the person who has been trying to get protection for the picture has declared limiting copyright to human authors as something that would be unconstitutional (I also think it would be pretty entertaining to watch somebody try to float that argument in front of the current Supreme Court).  A good article on why this sort of thing is going to be a problem, and an interesting way of parsing law's traditional 'mental state' requirement, is here.

IRS to start using Facial Recogntion Technology

If you want to use their website; WaPo has the story here.  But it's one of those public/private partnerships where data leaks and hacks and thefts happen.  To their credit, the Post went to Joy Buolamwini, whose work proved that facial recognition systems work best on white men and worst on Black women.  But even a perfectly functioning system is frightening.  First, it would unquestionably worsen the divide between those who have good Internet and those who don't, making convenient access to tax records contingent on having a sufficient income and computing skills.  Also, of course, facial recognition is bad - the potential for misuse is so great, and the record is so permanent, that Woody Hartzog and Evan Selinger argue it ought to be legally impossible to consent to. (for an overview of the debate, see Selinger and Leong here).

One of the biggest problems with data is that it gets leaked and hacked, of course, but another big problem is that companies sell it to pretty much whoever arrives with cash.  The company handling IRS facial recognition claims they'll turn it over to law enforcement, but the Post says there's no federal law proscribing what they can do with it.  And they're switching companies and authentication strategies because of a massive data breach at Equifax a few years ago. So its not like nobody has ever heard of a data breach.

Oh, and ID.me, the company getting the contract, totally wants to sell you stuff:

"But advertising is a key part of ID.me’s operation, too. People who sign up on ID.me’s website are asked if they want to subscribe to “offers and discounts” from the company’s online storefront, which links to special deals for veterans, students and first responders. Consumer marketing accounts for 10 percent of the company’s revenue."

What could possibly go wrong? Well, if you look up the ID.me privacy policy, you discover that most of the usual things can go wrong.  For example, they don't police 3rd party use of the data, which they encourage you to opt-in to:

"To avoid any confusion, Users should understand that, while we own and operate the Service and Website, we do not own or operate websites owned and operated by third parties who may avail themselves of the ID.me Service (collectively referred to hereafter as the “Third-Party Websites”). This Privacy Policy is intended to inform Users about our collection, use, storage, and disclosure, destruction and disposal of information that we collect or record in the course of providing the Website and the ID.me Service. Please note, we are not responsible for the privacy practices of Third-Party Websites and they are under no obligation to comply with this Privacy Policy. Before visiting Third-Party Websites, and before providing the User’s ID.me or any other information to any party that operates or advertises on Third-Party Websites, Users should review the privacy policy and practices of that website to determine how information collected from Users will be handled. Please further note, depending on a User’s particular interaction with us (e.g., Users who solely navigate the Website versus Users who create an account and use the ID.me Service at Third-Party Websites), different portions of this policy may apply to Users at different times."

Also, they reserve the right to change their privacy policy at any time, and it's your job to read it frequently to see:

"If we decide to change this Privacy Policy, we will post those changes to this page so that you are aware of what information we collect, how we use it, and under what circumstances, if any, we disclose it. We reserve the right to modify this Privacy Policy at any time, so please review it frequently. If we make material changes to this policy, we will notify you here, by email, or by means of notice on our home page."

That's item 1 on the policy.  Nothing else matters.  This is typical corporate privacy boilerplate that lets them do whatever they want with your facial biometric information.  Good job IRS!

 

 

SCOTUS and Social Murder

The SCOTUS decision yesterday striking down OSHA’s vaccine mandate is based on some of the most sophomoric reasoning the Court has issued in a long time.  And I am aware of what Court I’m talking about.  The gist of the argument is that OSHA is only authorized to enact safety rules that protect someone’s at their place of occupation.  But this is a public health rule because Covid also occurs outside the workplace, ergo etc.

But of course work is one of the main places that you can get Covid, as Justin Feldman documents (he also shows that the predominance of workplace transmission helps to explain the disproportionate impact on non-white folks).  The fact that vaccination also protects you outside of work is nice but not the point.  I have a ladder at home.  I don’t know the OSHA rules, but I bet there’s some covering the construction and use of ladders at work.  If those rules cause ladder manufacturers to make a safer product, that also protects me at home.  But it’s a little hard to explain how that standard doesn’t meet the statutory mandate of protecting people who use ladders in their occupation (the dissent cites several more such examples).   What’s wrong with positive externalities?

The Court opines:

“It is telling that OSHA, in its half century of existence, has never before adopted a broad public health regulation of this kind—addressing a threat that is untethered, in any causal sense, from the workplace.”

Well, duh.  We haven’t had a global pandemic like Covid during the existence of OSHA!  In the meantime, if you read court opinions very often, you learn to expect documentation of bold factual assertions like that one.  But there is no footnote explaining how there is no causal relation between the threat of Covid and the workplace.  That’s because a credible such footnote cannot be written.  As the dissent points out, “because the disease spreads in shared indoor spaces, it presents heightened dangers in most workplaces,” citing OSHA’s documentation of the risks and reminding that majority that Courts are supposed to be deferential in cases like this.  Congress even allocated money to OSHA  to address workplace hazards (dissent, p. 8).  In short,

“The agency backed up its conclusions with hundreds of reports of workplace COVID–19 outbreaks—not just in cheek-by-jowl settings like factory assembly lines, but in retail stores, restaurants, medical facilities, construction areas, and standard offices.” (dissent, p. 9)

We also know that SCOTUS doesn’t even believe its own rhetoric about workplace risk: the justices are all vaccinated, all but Gorsuch wore masks to oral arguments on this case (prompting Sotomayor to participate from her chambers), and court policy is that arguing attorneys have to take a Covid test the day before, and argue remotely if positive.  Attorneys are also supposed to wear KN95 masks when in the Courtroom except when actually speaking.  One of the attorneys arguing against the mandate even had to appear remotely because he had Covid!  So workplace safety is apparently a thing that SCOTUS has heard of – it’s just not one they deem fit to extend to workers who have less control over their environment.

In the meantime, Gorsuch took the time to write a concurrence tediously saying that states might have authority for public health, and that the nondelegation doctrine “ensures democratic accountability by preventing Congress from intentionally delegating its legislative powers to unelected officials.”  Perhaps now is the time to remember that SCOTUS is unelected, and seems to enjoy its own antidemocratic powers quite a bit: this the Court that ordered the Biden administration to reinstate the Remain in Mexico policy, even though that’s foreign policy, traditionally the province of the democratically elected executive (remember, the Court kept trying to greenlight Trump’s border wall with the fake border Caravan emergency, even though Congress specifically withheld funding for it).  This is also the same Justice Gorsuch who was appointed by the minoritarian Senate at the invitation of Donald Trump because Mitch McConnell refused to consider the nomination of the person who was democratically-elected president at the time of the vacancy. (Gorsuch also pontificates about the “major questions doctrine,” which is supposed intervene when an “agency may seek to exploit some gap, ambiguity, or doubtful expression in Congress’s statutes to assume responsibilities far beyond its initial assignment.”  But since the Court made no effort to prove that a vaccination mandate would not improve workplace safety and instead tries to show that the mandate improved safety everywhere, this rhetoric should be filed under the ‘I’m going to cite myself in anti-regulatory rulings in the future” dept).

There is one bit of hope in the opinion, in this paragraph:

“That is not to say OSHA lacks authority to regulate occupation-specific risks related to COVID–19. Where the virus poses a special danger because of the particular features of an employee’s job or workplace, targeted regulations are plainly permissible. We do not doubt, for example, that OSHA could regulate researchers who work with the COVID–19 virus. So too could OSHA regulate risks associated with working in particularly crowded or cramped environments. But the danger present in such workplaces differs in both degree and kind from the everyday risk of contracting COVID–19 that all face. OSHA’s indiscriminate approach fails to account for this crucial distinction—between occupational risk and risk more generally—and accordingly the mandate takes on the character of a general public health measure, rather than an “occupational safety or health standard.” 29 U. S. C. §655(b) (emphasis added).” (slip op, p. 7)

The Biden administration should immediately institute revised standards mandating vaccination in places with disproportionately high Covid rates.  There’s been research on that; as CNBC reports of the study:

“The top five occupations that had higher than a 50% mortality rate increase during the pandemic include cooks, line workers in warehouses, agricultural workers, bakers and construction laborers.”

Feldman links to some other high risk groups.  But the Biden administration needs to immediately call the Court’s bluff.   Will SCOTUS reverse itself here and go full-on Lochner and declare that the baking profession is unregulable?

Marx had lots of words for how the capitalist class treated the lives of workers as disposable.  Engels had the better expression: “social murder.”  How many workers did the right-wing majority in SCOTUS kill yesterday?  “OSHA estimated that in six months the emergency standard would save over 6,500 lives and prevent over 250,000 hospitalizations” (dissent, p. 11), and that number was derived before Omicron emerged. As the dissent sums it up:

“Underlying everything else in this dispute is a single, simple question: Who decides how much protection, and of what kind, American workers need from COVID–19? An agency with expertise in workplace health and safety, acting as Congress and the President authorized? Or a court, lacking any knowledge of how to safeguard workplaces, and insulated from responsibility for any damage it causes?”

There’s definitely a separation of powers problem emerging, but it’s not the one the Court’s conservatives want you to think about.

Can Datasets be Governed Like Public Utilities?

By Gordon Hull

Machine Learning (ML) applications learn by repetition.  That is, they come to recognize what, say, a chair looks like, by seeing lots of images of chairs that have been correctly labeled as such.  Since the machine is trying to figure out a pattern or set of characteristics that distinguish chairs from other objects, the more chairs it sees, the better it will perform at its task.  This collection of labeled chairs is the algorithm’s “training data.”   There are notorious problems with bias in training data due to underrepresentation of certain groups.  For example, one study found that datasets designed to train ML to recognize objects performed poorly in developing countries, most likely due to underrepresentation of images from those places; when combined with pictures labeled in English, and the fact that standard Western commodity forms of household objects might look very different in the developing world, the ML was stumped.  Most famously in this country, facial recognition software performs best at identifying white men and worst at identifying Black women.  ImageNet, which is widely used for object recognition purposes, employees a hierarchy of labels that include calling a child wearing sunglasses a “failure, loser, non-starter, unsuccessful person” but also differentiates between assistant and associate professors.  Despite these and many other problems, massive datasets are essential for training ML applications.

For this reason, datasets have been called “infrastructural” for machine learning, defined as follows:

“Infrastructure is characterized, we argue, by a set of core features: it is embedded into, and acts as the foundation, for other tools and technologies; when working as intended for a particular community, it tends to seep into the background and become incorporated into routines; the invisibility of infrastructure, however, is situated - what is natural or taken for granted from one perspective may be highly visible or jarring from another; though frequently naturalized, infrastructure is built, and thus inherently contextual, situated, and shaped by specific aims.”

If AI is cars and buses and trains that do what we want, the datasets it trains on shape the roads and paths where those roads go, provide their material basis, become thereby incorporated into higher level routines like search, and tend to disappear into the background when not actively used.  But just like other examples of infrastructure – say, the bridges over the Long Island Freeway – infrastructure can embed priorities and affordances.  In this sense, dataset infrastructures have a politics, and serve as platforms on which applications are built.

AI for Social Good: Calling out Amazon’s Lies to Congress

(story via Julia Angwin) You might remember that Amazon solemnly swore to Congress that they did not artificially elevate their own products in search results.  Except that they do.  Adrianne Jeffries and Leon Yin of The Markup used a machine learning algorithm to predict product placement in search results:

“We found that knowing only whether a product was an Amazon brand or exclusive could predict in seven out of every 10 cases whether Amazon would place it first in search results. These listings are not visibly marked as “sponsored” and they are part of a grid that Amazon identifies as “search results” in the site’s source code.”

Nothing else was anywhere close to as predictive.  Amazon products routinely get rated higher than better rated and better-selling competitors.  And it isn’t just for obvious Amazon products like “Amazon Basics.”  It turns out that a lot of products turn out to be Amazon:

“Using public records from the U.S. Patent and Trademark Office and Amazon’s own statements, we identified more than 150 brands registered by or owned by Amazon. These include both brands with an obvious connection, such as Amazon Basics and Amazon Commercial, and those that are generally known to be owned by the company, including Kindle and Zappos. But they also include dozens more, such as Happy Belly, Daily Ritual, and Society New York, where the connection to the company is not obvious. Those are in addition to the estimated hundreds of third-party brands that are exclusive to the site.”

Consumers are (understandably) totally confused about who makes the products they see.  Actual independent sellers are terrified to complain, because they’re worried about retaliation from Amazon.  Those who want to elevate their products can of course pay (one of the companies listed in the article was paying $10,000 a month, another $40,000 – not small change!), though even that might not get them listed above the Amazon products.  All of this is, not surprisingly, a potential antitrust problem:

“Bill Baer, a former assistant attorney general in charge of the antitrust division of the U.S. Department of Justice and former director of the Bureau of Competition at the FTC, said if consumers expect Amazon’s product search results to be neutral, but they are not, and the site is essentially a monopoly, that could be a violation of the FTC Act of 1914, which prohibits unfair competition and unfair or deceptive practices in commerce, or the U.S. Sherman Antirust Act, which prohibits monopolies from using their market power to harm competition.  If basically you’ve got somebody with market power that is restraining competition both in terms of site access or where things appear on the site,” he said, “that is potentially problematic.”

It’s also a problem because it suggests that Amazon executives lied to Congress.  Congress, you’ll recall, is completely dysfunctional and polarized on all topics except one: they all hate big tech.  Hence the “social good” part of the AI: the House Judiciary committee just sent a very threatening letter, which opens as follows:

“We write in response to recent, credible reporting that directly contradicts the sworn testimony and representations of Amazon’s top executives—including former CEO Jeffrey Bezos—to the Committee about their company’s business practices during our investigation last Congress. At best, this reporting confirms that Amazon’s representatives misled the Committee. At worst, it demonstrates that they may have lied to Congress in possible violation of federal criminal law.  In light of the serious nature of this matter, we are providing you with a final opportunity to provide exculpatory evidence to corroborate the prior testimony and statements on behalf of Amazon to the Committee. We strongly encourage you to make use of this opportunity to correct the record and provide the Committee with sworn, truthful, and accurate responses to this request as we consider whether a referral of this matter to the Department of Justice for criminal investigation is appropriate.”

Facebook as a Hostile Foreign Power

This piece, on Facebook's behaving more like an autocratic, hostile state than a large company, is worth the read.  Here's an excerpt:

"Perhaps Americans have become so cynical that they have given up on defending their freedom from surveillance, manipulation, and exploitation. But if Russia or China were taking the exact same actions to undermine democracy [that FB is], Americans would surely feel differently. Seeing Facebook as a hostile foreign power could force people to acknowledge what they’re participating in, and what they’re giving up, when they log in. In the end it doesn’t really matter what Facebook is; it matters what Facebook is doing."

Charles Mills (1951-2021)

I was both saddened and stunned this morning to read of the passing of Charles Mills.  I first met him at a SPEP years ago; I was having lunch at some random sandwich shop with friends.  He knew one of us, and asked if he could join.  Nevermind that we were all junior.  I managed to find him at a few later conferences, to join a meal or even just to say hi.  He was generous, warm and wickedly funny when he wanted to be.  And the stunning clarity with which he could call out ideal philosophy and other systems of domination is like nothing else I’ve read.

Daily Nous has more, and links to a really touching remembrance by Liam Kofi Bright.

New Paper: "The Death of the Data Subject"

Shameless self-promotion dept... here's the preprint for my new paper, "The Death of the Data Subject," now forthcoming in Law, Culture and the Humanities.  And here's the abstract:

This paper situates the data privacy debate in the context of what I call the death of the data subject.  My central claim is that concept of a rights-bearing data subject is being pulled in two contradictory directions at once, and that simultaneous attention to these is necessary to understand and resist the extractive practices of the data industry. Specifically, it is necessary to treat the problems facing the data subject structurally, rather than by narrowly attempting to vindicate its rights.  On the one hand, the data industry argues that subjects of biometric identification lack legal standing to pursue claims in court, and Facebook recently denied that that its facial recognition software recognizes faces.  On the other hand, industry takes consent to terms of service and arbitration clauses to create enforceable legal subject positions, while using promises of personalization to create a phenomenological subject that is unaware of the extent to which it is being manipulated.  Data subjects thus have no legal existence when it is a matter of corporate liability, but legal accountability when it is a matter of their own liability.  Successful reform should address the power asymmetries between individuals and data companies that enable this structural disempowerment.

 

 

When does the Supreme Court lose its legitimacy?

And what does that mean?  Now is a good time to ask.  The Court has let stand a 5^th Circuit decision upholding a Texas law that is plainly unconstitutional under current SCOTUS jurisprudence (it bans abortion at 6 weeks) and involves an enforcement mechanism that comes straight from Stalin’s playbook (it allows individuals to sue people they suspect of assisting a woman of obtaining an abortion).  This piece on Vox runs through how deeply perverse the Texas law in question in, how thoroughly Trump has corrupted the 5^th Circuit, and how alarming SCOTUS inaction is.

In Planned Parenthood v. Casey – which, along with Roe is apparently being overruled in Texas without a hearing and without a reasoned opinion – the Court favorably cites earlier opinion to the effect that the Court is supposed to give reasons when it overturns its precedent:

"A basic change in the law upon a ground no firmer than a change in our membership invites the popular misconception that this institution is little different from the two political branches of the Government. No misconception could do more lasting injury to this Court and to the system of law which it is our abiding mission to serve"

Justice O’Connor adds:

“The root of American governmental power is revealed most clearly in the instance of the power conferred by the Constitution upon the Judiciary of the United States and specifically upon this Court. As Americans of each succeeding generation are rightly told, the Court cannot buy support for its decisions by spending money and, except to a minor degree, it cannot independently coerce obedience to its decrees. The Court's power lies, rather, in its legitimacy, a product of substance and perception that shows itself in the people's acceptance of the Judiciary as fit to determine what the Nation's law means and to declare what it demands.  The underlying substance of this legitimacy is of course the warrant for the Court's decisions in the Constitution and the lesser sources of legal principle on which the Court draws. That substance is expressed in the Court's opinions, and our contemporary understanding is such that a decision without principled justification would be no judicial act at all. But even when justification is furnished by apposite legal principle, something more is required. Because not every conscientious claim of principled justification will be accepted as such, the justification claimed must be beyond dispute. The Court must take care to speak and act in ways that allow people to accept its decisions on the terms the Court claims for them, as grounded truly in principle, not as compromises with social and political pressures having, as such, no bearing on the principled choices that the Court is obliged to make. Thus, the Court's legitimacy depends on making legally principled decisions under circumstances in which their principled character is sufficiently plausible to be accepted by the Nation” (emphasis added)

O’Connor adds that if “the Court decides a case in such a way as to resolve the sort of intensely divisive controversy reflected in Roe” that “only the most convincing justification under accepted standards of precedent could suffice to demonstrate that a later decision overruling the first was anything but a surrender to political pressure, and an unjustified repudiation of the principle on which the Court staked its authority in the first instance.”

The Court’s so-called “shadow docket” – consequential decisions rendered in orders for cases not heard, as for example prioritizing religious claims over public health – has been on the rise over the last couple of years, and commentators have worried about the damage this does to the rule of law as an institution.  In the case of abortion, the Court has a case on the docket that would give it the opportunity to overturn Roe; it could have waited until then (and avoided the need to validate Texas’ enforcement mechanism).  You could argue that the Court didn’t “decide” anything last night, but it was faced with a clearly erroneous 5^th Circuit decision that it let stand.  I don’t see how last night’s failure to enjoin the Texas law doesn’t utterly gut its legitimacy in the sense articulated in Casey.  Texas just banned abortion, and SCOTUS offered no justification at all in letting the law stand.

Again, Casey: “to overrule under fire in the absence of the most compelling reason to reexamine a watershed decision would subvert the Court's legitimacy beyond any serious question.”  Court legitimacy is important:

“It is true that diminished legitimacy may be restored, but only slowly. Unlike the political branches, a Court thus weakened could not seek to regain its position with a new mandate from the voters, and even if the Court could somehow go to the polls, the loss of its principled character could not be retrieved by the casting of so many votes. Like the character of an individual, the legitimacy of the Court must be earned over time. So, indeed, must be the character of a Nation of people who aspire to live according to the rule of law. Their belief in themselves as such a people is not readily separable from their understanding of the Court invested with the authority to decide their constitutional cases and speak before all others for their constitutional ideals. If the Court's legitimacy should be undermined, then, so would the country be in its very ability to see itself through its constitutional ideals. The Court's concern with legitimacy is not for the sake of the Court, but for the sake of the Nation to which it is responsible.”

So much for Justice Roberts’ efforts to preserve the Court as an institution.

No Standing for You! (part 2)

By Gordon Hull

Last time, I started to look at the details of the Supreme Court’s recent TransUnion decision, which ruled that a credit agency that wrongly labeled someone as a match for a terrorist watch list (using only first and last names, with no effort at verification.  Sorry “John Smith”…) could only be sued if that person could show that the credit agency also distributed this information to others.  So basically you have to lose a job opportunity or a mortgage approval, and then you can sue.  There is nothing you can do to pre-emptively force them to correct the problem.  Worse, this ruling is in defiance of explicit statutory language in the FCRA.  The opinion directly says that SCOTUS can override Congressional determination of when something is legally actionable, and (just to be contrary, I guess), it does so in the name of separation of powers.

So what are we to make of this? As the various critical theories will remind you, law is a form of power. One thing that’s happening here is the protection of the powerful from the weak. With some high profile exceptions (like Justice Gorsuch’s endorsement of LGBTQ rights), the Roberts court has fairly consistently sided with the powerful, like corrupt politicians and conservative dark money donors by whittling away at laws designed to rein them in. One limitation to that thesis is to argue, as Erwin Chemerinsky has, that “Throughout history the court has overwhelmingly favored corporate power over employees, consumers, and the public, and has favored government power over individuals’ rights.” While it’s tempting to leave it at that here, I think there’s more nuance to be had. So two points.

No Standing for You! (part 1)

By Gordon Hull

In one of the Seinfeld episodes, the proprietor of a popular lunch stop would deny service to customers who offended his arbitrary sensibilities with a loud “No Soup for You!” This is basically the outcome of the Supreme Court’s June decision on standing, TransUnion v. Ramirez. “Standing” in this sense refers to access to the federal legal system; to simplify, to be able to sue somebody, you need to show that you’ve suffered a “concrete” injury. A number of recent standing cases have been about the outer limits of “concrete.” In principle, privacy harms can be concrete, but SCOTUS has been making standing harder, particularly in data breach cases.

And so it goes here. Per a 5-4 opinion authored by Justice Kavanaugh, you don’t have standing to sue a credit agency for flagging you as a terrorist, unless you can show that they also disseminated that information to somebody else. This opinion explicitly overrides a Congressional determination that you do, in fact, have standing in cases like that, arrogating to the Court the right to decide when Congress is and is not allowed to establish when statutory rights are actionable. This was too much for Justice Thomas, who, joined by the court’s liberals, points out in dissent that this both violates separation of powers in the name of preserving it, and turns historical standing doctrine on its head. As Dan Solove and Danielle Citron put it in a sharp critique that builds on Thomas’ dissent, TransUnion is “an activist decision that nullifies Congress’s power to protect consumers and that enables courts to rewrite privacy laws to alter how they are enforced.”

Foucault, Marx and Prophecy Part 3: On Bureaucrats

By Gordon Hull

In a previous post, I noted that Foucault strongly implies in a 1978 interview that his communist detractors are bureaucrats, and tied that to an earlier interview with Maoists in which he suggests that structuring populist tribunals on the model of bourgeois courts would fail to break with the power structure of the bourgeois court system: the model of impartiality is intrinsically bourgeois, and so importing that model into communist popular tribunals would iterate the very power structures that were to be replaced.  Here I want to flesh out a little more some of the resonances of an accusation of bureaucracy in the context of 1970s Marx debates in France.   I should say in advance that these are notes more than a complete assessment, designed to pick out highlights.  The back and forth polemics of Marxists are byzantine, and we should all be grateful that Foucault sets it as a rule not to engage in them.  Here I will mainly draw from Trotsky’s critique of Stalin and Lenin, with a closing gesture to a representative text of the French far-left that emerged in the aftermath of the 1968 student uprisings.

Patently Harmful

My further thoughts on the background through which we should interpret Covid patent waivers, at Real Life.

Foucault, Marx and Prophecy, Part 2: The Role of the Intellectual

By Gordon Hull

In a previous post, I made the case for reading Foucault’s 1978 comments on Marxism (especially in the Yoshimoto interview) in the context of theory/practice questions raised by the re-evaluation of Marx’s 11^th “Thesis on Feuerbach.”  Here I want to flesh out that position a little more, starting with reference to another 1978 piece.  Shortly after the Japan interviews, Foucault writes a response to the Italian communist Massimo Cacciari’s “Rationality and Irrationality of Politics in Delezue and Foucault” (the piece appeared in Italian; as far as I can tell, there is no translation of it).  Much of Foucault’s response is dedicated to accusing Cacciari of using the tactics of Stalinism: “always having a unique adversary” (created in this case by amalgamating Foucault, Deleuze and the New Philosophers), of engaging in a trial-like procedure, of assimilating ‘the enemy’ and danger, of reducing thought immediately to a system, and so forth.

He then defends the argument of Discipline and Punish, in particular with reference to the understanding of power; Foucault says he “would like precisely to show this heterogeneity of power, that is to say how it is always born of something other than itself” (D&E #238, II, 651 (2 vol. ed, 2001]).  He underscores that there is no metaphysics of power here, no “power with a capital P.”  Rather, “it is necessary to put the relations of power back into the interior of struggles and not suppose that there is, on the one hand, power, and on the other, that on which it is exercised, and that the struggle unfolds between power and non-power” (632).  In other words, Foucault’s entire effort is to resist the sort of abstraction that the Theses on Feuerbach warn against reifying.  The goal is rather to write a text that does something, that constitutes an intervention or a tool.  In language similar to that of the Japan interviews, and against the accusation that he is merely telling a story, Foucault proposes:

“In reality, what I am trying to do, and therein lies the difficulty of the attempt, is to work out an interpretation, a reading of a certain reality, which is such that, on the one hand, this interpretation is able to produce effects of truth and that, on the other hand, these effects of truth can become instruments at the heart of possible struggles.  To speak the truth so that it can be attacked [dire la vérité pour qu’elle soit attaquable].  It is a reality of possible struggles that I seek to make appear.” (633)

A few sentences later, he argues that “the effect of the truth that I seek to produce resides in this manner of showing that the real is polemical” (633).  However, “I do not speak of the current situation.  I effect an interpretation of history, and the problem – but I do not resolve it – is to know what the possible utilization of these analyses in the current situation is” (633-4).

This then leads to a very clear statement of the role of the intellectual.  “It is absolutely true that, when I write a book, I refuse to take a prophetic position which consists in saying to people: here is what you ought to do, or again, this is good, that is not.”  Instead, the goal is to show in a general way “it seems to me that these things have happened, but I describe them in such a manner that views of possible lines of attack are traced.  But in this I do not force or constrain anyone to attack” (634).  What to actually do – and he refers to his own decisions about throwing himself into certain political actions - “is a problem of groups, of physical and personal engagement,” and not of books.  Indeed, bossy Marxists are not radical: “one is not radical because one has pronounced some formulas; no, radicality is physical, radicality concerns existence” (634).

The Necropolitics of Intellectual Property

By Gordon Hull

Update (4/30).  Essential piece by Amy Kapczynski (who is one of my sources below).  Also see this Twitter thread by Dennis Crouch of PatentlyO, characterizing the issue as one of technology transfer more generally than patents specifically.  Trade secrets are an important issue - like patents, they allow companies to control access to innovation.  Companies also will use a strategy of combining patents and trade secrets in order to maximize their revenue (so if they think something can't be patented, they might claim it as a trade secret).  The theory behind trade secrets is based more in contracts than property, which means that they are arguably even worse if you're worried about neoliberalism, since they are an even further privatization of law and policy.

Update (4/29).  Here's more on IP and other barriers to vaccine distribution in India (and elsewhere):

• The Modi government is incompetent at every level, and India's base capacity was lower than it estimated.
• Most of the world wants IP waivers.  The U.S. and Europe and a few other countries do not. 
• This WaPo piece argues that waivers are bad but licensing (preferably voluntary, but with compulsory as a stick) is good.  Note that licensing is not what Bill Gates is talking about, as far as I can tell, which is buying vaccines and donating.

In my Biopolitics of Intellectual Property, I argue that IP policy has shifted from what I call a “public biopolitics” model to a neoliberal version.  In its briefest form: the public version treats IP as a necessary but limited monopoly to promote public goods, and the neoliberal version focuses on private wealth gain through proprietization (I summarize the argument here).

Something that I don’t particularly talk about in the book, but that one knows from Foucault, is that biopolitics comes with its inverse, necropolitics: if biopolitics is about promoting life and health for the “population,” it is also about who is allowed to die.  As Foucault puts it, “the ancient right to take life or let live was replaced by a power to foster life or disallow it to the point of death [au vieux droit de faire mourir ou de laisser vivre s'est substitué un pouvoir de faire vivre ou de rejeter dans la mort]” (History of Sexuality I, 138).  Governmental power goes from the right to kill to the power to cause people to live; death becomes something into which one is literally “thrown back.”  The leading examples of necropolitics are political, as for example Foucault’s discussion of state racism (of which Nazism is the apotheosis) in Society must be Defended.  Achille Mbembe’s “Necropolitics” article spends time on how post-colonial African states have dismantled populations, which are “disaggregated into rebels, child soldiers, victims or refugees, or civilians incapacitated by mutilation or simply massacred on the model of ancient sacrifices” (34).  Building on these, Ege Selin Islekel’s brilliant treatment of the disappeared in Turkey notes that in necropolitical spaces, “the entire content and the fact of living, constituted by the ethical, political, and epistemological conditions of life, are subsumed under death.”

However, as Ute Tellman has recently demonstrated, the political treatment of biopolitics needs to take seriously how it is co-configured with the economy.  On Tellman’s account, the notion of economic scarcity first appears in Malthus (it was missing in Smith!) as a way to police the behavior of the poor (and “savages” in the colonies) by training them to think in terms of futurity.  For Malthus, the poor have to be trained not to eat and procreate their way into oblivion by forcing them to think in terms of economic rationality.  This brings us to the neoliberal justification of IP, which is partly underpinned by the Schumpeterian thesis that innovation is to be pursued at all costs, because the gains of future innovation (“dynamic efficiency”) are more important than whatever short-term distribution problems (“static inefficiencies”) they entail.  Thus, more or less, is Harold Demsetz’s reply to Kenneth Arrow. 

It also subtends the argument being given for why IP rights around Covid vaccines shouldn’t be licensed to the poor in India (side note: Malthus served as professor at the East India Company College).  Developing countries have proposed a waiver of related IP rights to ensure the rapid production of generic Covid vaccines, and Pharma has responded with an army of lobbyists to explain that no, IP can’t possibly be the problem with Covid vaccine distribution, and it would be much better for philanthropies to purchase lots of drugs and then distribute them.  Other unrelated IP industries have followed with their own lobbyists.  Whatever other difficulties exist in getting vaccinations to people in developing countries, it seems hard to deny that insisting on IP rights and thereby limiting production of the drugs isn’t one of them.  Allowing generics – especially in India and Brazil – increases capacity.

Foucault, Marx and Prophecy: the 11th “Thesis on Feuerbach”

By Gordon Hull

Foucault thinks Marxism is bossy.  In Society must be Defended, he lays down the gauntlet clearly enough: totalizing theories get in the way of useful things at the local level.  As he notes, one should beware of:

“the inhibiting effect specific to totalitarian theories, or at least – what I mean is – all-encompassing and global theories.  Not that all-encompassing and global theories haven’t, in fairly constant fashion, provided – and don’t continue to provide – tools that can be used at the local level; Marxism and psychoanalysis are living proof that they can. But they have, I think, provided tools that can be used at the  local level only when, and this  is the real point ,the theoretical unity of their discourse is, so to speak, suspended, or at least cut up, ripped up, torn to shreds, turned inside out, displaced, caricatured, dramatized, theatricalized, and so on. Or at least that the totalizing approach always has the effect of putting the brakes on” (SMD 6).

That is, when you insist on your theoretical unities, you get in the way of actually doing anything.  What we need are to unearth “subjugated knowledges” and specific histories, and such activity requires the “removal of the tyranny of overall discourses” (SMD 8).  In his 1978 interviews with the Italian Communist Duccio Trombadori, Foucault underlines that “I absolutely will not play the part of one who prescribes solutions.  I hold that the notion of the intellectual today is not that of establishing laws or proposing solutions or prophesying, since by doing that one can only contribute to the functioning of a determinate situation of power that to my mind must be criticized” (Remarks on Marx, 157).

He similarly says at the end of an earlier 1978 interview in Japan that “I think that the role of intellectuals, in reality, absolutely does not consist in playing [the role of] prophets or legislators” (D&E #236; Vol II, 264-5 (2 vol, 2001 ed.)).  Marxism is theological, as he says in his 1979-80 Collège de France Lectures:

“With Marxism, it’s the same thing.  You have the model of the fall, alienation and dis-alienation.  You have the model of the two ways: Mao Zedong.  And you have, of course, the problem of the stain of those who are originally soiled and must be purified: Stalinism.  Marx, Mao, Stalin; the three models of the two ways, the fall, and the stain” (Government of the Living,  108)

At one level this is clear enough.   But Foucault also of course is an advocate of social change, and he wants his works to be picked up and used in local struggles, as he also says repeatedly.  Here I want to add a little specificity to the question about Foucault’s relation to Marxism (at least as he understands it around 1978) by picking up on his remarks in the Japan interview. Immediately after saying that intellectuals should not be prophets or legislators, he adds that: “for two thousand years, philosophers have always spoken of what we should have done [de ce qu’on devait faire].  But this always led to a tragic end.  What is important is that philosophers speak of what is currently happening, but not of what could happen” (624).  The first thing to note is the implicit reference to Lenin, at least in the context of a discussion of Marxism: Lenin’s What is to be Done is translated into French as Que Faire (the French rendering is correct: the book is Что Дълать, literally “What to Do”).  More significantly, the connection to prophetic discourse is something Foucault repeats in his slightly earlier interview with Yoshimoto, where he says that:

On Covid data, part n

I've been commenting off and on about the vagaries of Covid data - for example, in knowing what "covid cases" refers to (and here); states' early conflation  of PCR and antibody tests;  the vagaries of different testing technologies, or the ways that even death certificates can mislead about mortality.  This reflective piece by the co-founders of the Covid Tracking Project at the Atlantic - which became the de facto national source for Covid information as the federal government fell flat on its face - is absolutely worth the read.  Here's one short paragraph that says everything you need to know about data in general:

"Data are just a bunch of qualitative conclusions arranged in a countable way. Data-driven thinking isn’t necessarily more accurate than other forms of reasoning, and if you do not understand how data are made, their seams and scars, they might even be more likely to mislead you"

Marx explains how AI systems exploit workers (Part 1)

By Gordon Hull

“Factory work exhausts the nervous system to the uttermost; at the same time, it does away with the many-sided play of the muscles, and confiscates every atom of freedom, both in bodily and in intellectual activity” (Marx, Capital I [Penguin Ed.], 548).

A recent piece by Josh Dzieza in the Verge about the working conditions of those subject to discipline by AI in the workplace is wrenching.  It tells the story of Amazon workers and call center employees and others whose work lives have become an exhausting monotony of ever-intensifying robot-like demands.  The “AI boss” becomes ever more demanding, filling every moment at work with the requirement to be more productive and with absurd metrics for deciding when its demands are met.  Amazon workers report that they are basically ground down by the unsustainability of the pace of productivity; they are either fired or forced to quit, often with injuries.  Call center employees are subject to Orwellian monitoring by a machine that purports (often laughably inaccurately) to monitor and tweak their affect when speaking to customers.  Folks working from home find it impossible to go to the bathroom, because they are forced to sit in front of their computer all the time, which takes frequent pictures of them and reports on their productivity, as measured by lines of code or keystrokes per unit time.

All of this optimized misery reminds us of something important: AI systems are being deployed under capitalism.  A worker subject to an “AI Boss” is not some sort of new-fangled social form; they are fundamentally a worker subject to capitalist exploitation.  What AI has done is enable capitalism to be more intensively exploitative.  The stories in the article echo the stories Marx reports in Capital in his discussion of machinery, as does a lot of the logic.  This suggests that two popular narratives about AI need to be displaced.  First, it may be that AI will someday swoop in and take away everyone’s jobs.  As Dzieza notes, this narrative deflects attention from the pressing reality of deteriorating workplace conditions under AI supervision.  Second, Marx’s work on machines and technology is frequently read (as by the autonomists) through the Grundrisse “Fragment on Machines,” and in a way that emphasizes affective labor and cognitive capital.  The reality of AI today serves as an important reminder that the treatment in Capital is still very important.  Let me explain.

The Elephant in the Vaccine Room

You might have heard that minorities are hesitant about getting a Covid vaccine?  Well, about that.  According to polling reported by Axios, the group least likely to want a vaccine is White Republicans... to the point that "White Americans are now less likely than Black and Latino Americans to say they plan to get the vaccine."  I

Google fires another AI ethicist

This time Margaret Mitchell, one of the other authors on the fabulous "Stochastic Parrots" paper (that's my post on it.  The paper is here) on natural language processing.  This was obviously coming, since they'd suspended her email account weeks ago.  In case you haven't read the paper (and you really should!), it's worth mentioning that the paper does not mention Google, and the goal is to point to strategies to make AI better, i.e., less likely to entrench racism, sexism and so forth while destroying the planet in climate catastrophe.  It turns out that those are related, because bigger and bigger datasets are both carbon intensive and likely to pick up on and magnify hegemonic and toxic speech.  Attention to efficiency and curation moves the needle on both.

No commitment to even looking like they care about ethical AI.

Reading List: Bender and Gebru, “Stochastic Parrots”

By Gordon Hull

Not long ago, Google summarily dumped Timnit Gebru, one of its lead AI researchers and one of the few Black women working in AI.  Her coauthor Emily Bender has now posted the paper (to be presented this spring) that apparently caused all the trouble.  It should be required reading for anybody who cares about the details of how AI and data systems can perpetuate racism, or who cares more generally about the social implications of brute-force approaches to AI.  Bender and Gebru take up a common approach to natural-language processing (NLP), which involves an AI system learning how to anticipate what speech is likely to follow a given unit of speech.  If, for example, I say “Hello, how are,” the system learns by studying a dataset of existing phrases and text snippets that the next word is likely to be “you,” but almost certainly will not be “ice cream.”  How good the computer gets at this game is going to be substantially determined by the quantity and quality of its training data, i.e., the text that it examines.

Bender and Gebru outline the social costs of one approach to this problem, which is basically brute force.  As processing power increases, it’s possible to train computers with larger and larger datasets, and the use of larger datasets reliably improves system performance.  But should we be doing that?  Bender and Gebru detail several kinds of problems.  The first is environmental justice: all that processing power uses a lot of energy.  Although some of it may come from carbon-neutral sources, the net climate cost is significant.  Worse, the NLP systems being produced don’t benefit the people that are going to suffer the most from climate change.  As they memorably put it:

“Is it fair or just to ask, for example, that the residents of the Maldives (likely to be underwater by 2100) or the 800,000 people in Sudan affected by drastic floods, pay the environmental price of training and deploying ever larger English LMs, when similar large-scale models aren’t being produced for Dhivehi or Sudanese Arabic?”

Covid Deaths and the Politics of Obituaries

By Gordon Hull

As of this writing, approximately 421,000 people in the United States have officially died of Covid-19.  We also know that this number is fewer than the number that have actually died of Covid for a variety of reasons.  For example, early in the pandemic, there was nowhere near enough testing, and so many people who died of Covid-19 never received an official diagnosis.  For that sort of reason, measuring “excess deaths” during a given period is one way to try to get a handle on how many people actually died of Covid-19.  If n more people die during a given period this year than last year and/or typically, that number can be a useful indicator for how many deaths can be attributed to the pandemic.

Some of these will of course be indirect: people who didn’t go to the ER over a heart attack because they were afraid of contracting Covid, for example.  The isolation of Covid may be driving an increase in opioid overdose deaths.  Others will be direct, deaths where Covid was the underlying cause. STAT News is now reporting on the results of a recent study that takes a dive into this question, concluding that the overall direct Covid death toll is 31% higher than official figures.  That’s the top line, and it means that the national number of Covid deaths is approaching 550,000.  What should surprise no one (but is surprising, because deaths are so much more reliable for understanding infection rates than reported cases) is that the mortality data is deeply shaped by politics: excess deaths are a window into the sociology of the pandemic.

New Paper: "Infrastructure, Modulation, Portal: Thinking with Foucault about how Internet Architecture Shapes Subjects"

Now up on SSRN.  This paper uses Foucault's works on disciplinary power to develop a typology for understanding different models of Internet governance.  Here is the abstract:

Following Foucault’s remarks on the importance of architecture to disciplinary power, this paper offers a typology of power relations expressed in different models of Internet governance. Infrastructure governance understands the Internet as a common pool or public resource, on the model of traditional infrastructures like roads and bridges. Modulation, which I study by way of Net Neutrality debates in the U.S., understands Internet governance as traffic shaping. Portal governance, which I study by way of data collection policies of dominant platform companies, understands the Internet as creating a user experience that facilitates data mining. The latter two are forms of architectural disciplinary power that undermine the first. I then argue that the rise of portal and modulation governance primarily serves to remake parts of civil society by fostering market norms of consumption and entrepreneurialism. In that sense, efforts to shape Internet architecture need to be understood as techniques of subjectification.

 

Epistemic Justice with AI: indicators, diverse training data and knee pain

Knee pain is common and debilitating, and it’s often caused by osteoarthritis in the knee.  Treatment options range from analgesics (including opioids) to knee-replacement surgery.  If you go to the doctor with arthritic knee pain, you can get an x-ray which can then be interpreted using standard rubrics like the Kellgren–Lawrence Grade (KLG) to quantify damage to your knee and then guide treatment options. The KLG isn’t perfect in that the correlation between pain and objective scores of damage to the knee isn’t perfect.  Some people’s knees are a wreck and they report no pain; others have pain beyond what their KLG score indicate. But here’s the thing: Black patients consistently report more knee pain than white patients.  They also tend to have more knee damage on the KLG – but even when you factor that in, Black patients report much more knee pain than white patients with comparable KLG scores.  What’s going on?

One possibility is that factors external to the knee – stress, for example – explain the higher pain.  If that’s the case, then patients need less knee treatment.  But what if their knees were in worse shape?  To answer that question, you’d have to ask yourself what in an x-ray indicated poor knee condition.

Disease is often measured through indicators, and we know that these indicators can lead to all sorts of complexity.  In the context of Covid, for example, there are all sorts of questions about testing and sensitivity that I’ve talked about before.  Along the way, I referred to a fantastic paper on malaria testing in sub-Saharan Africa – suffice it to say that “cases of malaria” reported to donor organizations is a difficult number to parse for reasons having to do with vagaries in testing and diagnosis.

In a new paper in Nature Medicine, a team led by Emma Pierson makes ingenious use of artificial intelligence to tackle the problem of racial disparities in knee pain.  Since algorithms and data are so often implicated in increasing or magnifying racial disparities (see, for example, Safiya Noble on Google, or Timnit Gebru on facial recognition, or Margaret Hu’s chilling “Algorithmic Jim Crow”), it’s encouraging to learn about machine learning working to undermine racial disparities.  Ordinarily, you train an algorithm to perform like an excellent clinician.  In this case, that would mean training it to look at radiography and determine the correct KLG score.  The trick here was to instead train it to look at pain: to determine what features of the x-ray predicted that the patient would report pain.  It turns out that the algorithm’s diagnoses reduced racial disparities in diagnosis by a jaw-dropping 47%.