By Gordon Hull
Last time, I introduced a number of philosophy of law examples in the context of ML systems and suggested that they might be helpful in thinking differently, and more productively, about holding ML systems accountable. Here I want to make the application specific.
So: how do these examples translate to ML and AI? I think one lesson is that we need to specify what exactly we are holding the algorithm accountable for. For example, if we suspect an algorithm of unfairness or bias, it is necessary to specify precisely what the nature of that bias or unfairness is – for example, that it is more likely to assign high-risk status to Black defendants (for pretrial detention purposes) than it is white ones. Even specifying fairness in this sense can be hard, because there are conflicting accounts of fairness at play. But assuming that one can settle that question, we don’t need to specify tokens or individual acts of unfairness (or demand that each of them rise to the level where they would individually create liability) to demand accountability of the algorithm or the system that deploys it – we know that the system will have treated defendants unfairly, even if we don’t know which ones (this is basically a disparate impact standard; recall that one of the original and most cited pieces on how data can be unfair was framed precisely in terms of disparate impact).
Further, given the difficulties of individual actions (litigation costs, as well as getting access to the algorithms, which defendants will claim as trade secrets) in such cases, it seems wrong to channel accountability through tort liability and demand that individuals prove the algorithm discriminated against him (how could they? The situation is like the blue bus: if a group of people is 80% likely to reoffend or skip bail, we know that 20% of that group will not, and there is no “error” for which the system can be held accountable). Policymakers need to conduct regular audits or other supervisory activity designed to ferret out this sort of problem, and demand accountability at the systemic level.
Of course, ours is an adversarial judicial system based on individual rights (for the ways that this system completely fails to protect privacy, see this recent paper by Ari Waldman; much of Waldman’s reasoning applies here), and the ambiguities over whether a specific person was wronged has made it very hard for individuals to demand fair treatment: first they have to prove that they were unfairly treated as an individual. The legal construct that tries to manage this problem is class certification, and it is at the core of the recent Supreme Court standing decisions, most obviously in last summer’s TransUnion decision. In that case, the Supreme Court ruled that you don’t have standing to sue a credit agency for calling you a terrorist in its files, without also proving that they disseminated that information. The Court thereby removed a bunch of plaintiffs in a class action case against the agency. Justice Kavanaugh reasoned by analogy to a defamatory letter – if you write it and then keep it locked in your desk, then it’s hard for the person you defame to take action against you, because they won’t have been harmed by it – defamation has a publication requirement.
The various philosophy of law examples provide other ways of thinking. If we know that the algorithms of the credit agency erroneously label people as terrorists, maybe it’s less important to specify whether individuals whom the credit agency has labeled terrorists have had that information disseminated yet. The wrong for which we want accountability is the statistical tendency toward incorrect determination of individuals as terrorists; and we know that the credit agency is going to disseminate that information for at least some of them. Justice Kavanaugh is reasoning with the wrong analogy, because he begins and ends with a construct of individual tort liability. But the business model of the agencies is to disseminate information; like Susan’s speeding, we know that something bad is happening and we know who’s doing it, even if at a given moment we don’t know whether Susan was speeding or the credit agency is disseminating a specific person’s information. In other words, class certification needs to be easier to obtain against those who use algorithmic systems. It isn’t perfect, but it at least addresses the problem at the right level.
Beyond enabling class certification, it seems to me that this strategy points to a more general direction we need to go in establishing accountability for ML systems. An individual rights approach is neither necessary nor sufficient to do so. The philosophy of law examples show why it makes intuitive sense to hold a system accountable for an act of discrimination, if we are sure that discrimination is happening and that the system is doing it, even if we can’t identify a specific plaintiff or test case.
Law distributes social power, however, and in a paper on whether we should attribute mental states to algorithmic systems, Mala Chatterjee and Jeanne Fromer suggest the following:
“As a preliminary hypothesis … it might be that the law is interested in conscious properties of mental states when it seeks to treat the actor in question as a rightsholder (such as in copyright authorship) or an autonomous and responsible agent (such as in criminal punishment). But in contexts in which the law is seeking simply to protect the rights or interests of others from the actor (such as copyright infringement), functionality might be all that matters” (1915-16)
That is, we can and should treat algorithmic systems and the people they “accuse” of things differently. In particular, when a system is used to accuse an individual of some malfeasance, the bar ought to be much higher. The analogy to Schauer’s examples suggests that either there ought to be evidence specific to the person (gategcrasher) or it ought to be provable that the person did something, even if we cannot name the specific type (Susan’s driving). The difficult question will be to understand where the statistical bar for liability ought to lie; the examples about John the predator suggest that the statistical bar can be read differently, and ought to be read differently, pending a prior discussion of what the aims of the system are and the kinds of power it utilizes and distributes. To cite one possible example of how this thinking might work in practice: the gatecrasher example suggests that before someone lands on the “do-not-fly” list, there should be evidence specific to them, not a series of general inferences; pervasive big-data blacklisting needs to be reined in. Again, one of the topics of debate – substantive, political debate, not a technical debate – should be where the various standards of probability lie. As Aziz Huq notes, we’ve tended to operate on the assumption that false positives and false negatives in the ML context have equal costs – but that determination is a political one, and it need not be the same in each context (pp. 1916-17).
These are the sorts of questions that algorithmic accountability allows us to raise, and they focus attention on a different level of problem than individual liability and machine transparency or explainability. Explainability and transparency are on the right track insofar as they look for algorithmic accountability at the systemic level and draw effort away from a tort-based individual rights model. But they both model ways that we hold people accountable by asking them to explain what they have done, giving reasons that others can understand. But this isn’t how algorithmic systems arrive at their decisions, and there’s no reason to think that accountability for them needs to be narrowly modeled on the human process. At least, that's an assumption that we ought to seriously think about.
Recent Comments