By Gordon Hull
AI systems are notoriously opaque black boxes. In a now standard paper, Jenna Burrell dissects this notion of opacity into three versions. The first is when companies deliberately hide information about their algorithms, to avoid competition, maintain trade secrets, and to guard against gaming their algorithms, as happens with Search Engine Optimization techniques. The second is when reading and understanding code is an esoteric skill, so the systems will remain opaque to all but a very small number of specially-trained individuals. The third form is unique to ML systems, and boils down to the argument that ML systems generate internal networks of connections that don’t reason like people. Looking into the mechanics of a system for recognizing handwritten numbers or even a spam detection filter wouldn’t produce anything that a human could understand. This form of opacity is also the least tractable, and there is a lot of work trying to establish how ML decisions could be made either more transparent or at least more explicable.
Joshua Kroll argues instead that the quest for potentially impossible transparency distracts from what we might more plausibly expect from our ML systems: accountability. After all, they are designed to do something, and we could begin to assess them according to the internal processes by which they are developed to achieve their design goals, as well as by empirical evidence of what happens when they are employed. In other words, we don’t need to know exactly how the system can tell a ‘2’ from a ‘3’ as long as we can assess whether it does, and whether that objective is serving nefarious purposes.
I’ve thought for a while that there’s potential help for understanding what accountability means in the philosophy of law literature. For example, a famous thought experiment features a traffic accident caused by a bus. We have two sources of information about this accident. One is an eyewitness who is 70% reliable and says that the bus was blue. The other is the knowledge that 70% of the buses that were in the area at the time were blue. Epistemically, these ought to be equal – in both cases, you can say with 70% confidence that the blue bus company is liable for the accident. But we don’t treat them as the same: as David Enoch and Talia Fisher elaborate, most people prefer the witness to the statistical number. This is presumably because when the witness is wrong, we can inquire what went wrong. When the statistic is wrong, it’s not clear that anything like a mistake even happened: the statistics operate at a population level; when applied to individuals, the use of statistical probability will be wrong 30% of the time, and so we have to expect that. It seems to me that our desire for what amounts to an auditable result is the sort of thing that Kroll is pointing to.
A recent paper (preprint here, pagination below to the preprint) by Frederick Schauer deploys another famous philosophy of law thought experiment called the “paradox of the gatecrasher.” The scenario runs as follows: 1000 people attend a rodeo, but only 499 of them have paid for their admission. If any one of the spectators is sued for unlawful entry the probability that this spectator entered unlawfully is 501/1000, and therefore above the threshold for liability when that is the “preponderance of the evidence.” But almost everyone intuitively rejects this result. Schauer’s innovation is to argue that the paradox sets up the wrong question. There were 501 individual acts of illegal entry, and what the law does is identify a specific action (illegal entry #1) and then ask if an individual did that. Framed this way, a plaintiff would have to pick a specific act of illegal entry, and then accuse the defendant of having done that specific act. Since there’s 1000 people in attendance, that’s a 1/1000 chance that the defendant committed this specific act of illegal entry. Although he refrains from much discussion of the blue bus example, Schauer suggests that it’s a related paradox; “here [in the bus case] the paradox is presented not by an under-specified liability-generating act [as it is in gatecrasher], but rather, by an under-specified description of the defendant, and this is a different issue from that of under-specification of the wrong” (9).
Schauer then introduces another example to complicate the gatecrasher line of examples: Predator. “John has been accused by four different women of sexual assault. Each accuser is credible, but so too is John’s emphatic denial in each case” Let’s hypothesize that the probability is 80% for each accusation. That won’t be high enough to establish criminal liability, and so John will be acquitted in each case. But if we do the math differently, and ask what the odds are that John has committed at least one sexual assault, there is a 99.984% chance of that, well above any normal standard for criminal liability.
Another example concerns Susan’s driving. Susan is recorded by automatic cameras or LPRs at toll booths as having completed a trip between two cities fast enough that she had to be speeding at least part of the way – her travel time is less than the minimal travel time for someone who kept to the posted speed limits. Of course we don’t know where exactly she was speeding or by how much (she could have driven below the speed limit part of the way, and way above it for part of the way). Yet we are 100% certain she broke the law, even though the act is under-specified. If we contrast this with a second case in which Susan is also charged with running some stop signs along the way, we are far less likely to want to charge her with that, since the evidence of how fast she arrived doesn’t establish whether she ran any stop signs. Similarly, if John is accused by one person of sexual assault, another of embezzlement, a third with non-sexual assault and a fourth of burglary, it no longer makes intuitive sense to convict him of anything.
The lesson of all this? “it is right to require a specification of the type of wrong for which liability is sought, but not as obviously right that specification of precisely which token of a type of liability-generating wrong is necessary to produce liability’ (13). Behind that is the need to specify what a single act is. Sometimes, an “act” is correctly described as the accumulation of numerous instances of behavior – harassment, for example. That is, “implicit in law’s traditional requirement of specification is the law’s understanding of what makes a single act a single act, and when some combination of acts is a single act or instead multiple acts” (15). In the case of harassment, the act is specified as a “pattern” of recurrent behavior, and the Supreme Court ruled in the early 1990s that the aggregation of lesser actions were sufficient to create a hostile work environment:
“This standard, which we reaffirm today, takes a middle path between making actionable any conduct that is merely offensive and requiring the conduct to cause a tangible psychological injury. As we pointed out in Meritor, "mere utterance of an ... epithet which engenders offensive feelings in a employee," ibid. (internal quotation marks omitted) does not sufficiently affect the conditions of employment to implicate Title VII. Conduct that is not severe or pervasive enough to create an objectively hostile or abusive work environment-an environment that a reasonable person would find hostile or abusive-is beyond Title VII's purview. Likewise, if the victim does not subjectively perceive the environment to be abusive, the conduct has not actually altered the conditions of the victim's employment, and there is no Title VII violation. But Title VII comes into play before the harassing conduct leads to a nervous breakdown. A discriminatorily abusive work environment, even one that does not seriously affect employees' psychological well-being, can and often will detract from employees' job performance, discourage employees from remaining on the job, or keep them from advancing in their careers. Moreover, even without regard to these tangible effects, the very fact that the discriminatory conduct was so severe or pervasive that it created a work environment abusive to employees because of their race, gender, religion, or national origin offends Title VII's broad rule of workplace equality.”
Next time I’ll connect all this back to the ML context.
Recent Comments