Rebecca Crootof has a current paper on what she calls the “kill box” problem for autonomous weapon systems. It shows just how difficult it is to translate traditional just war values and laws into an AI context. Basically, one of the most important rules for warfare is that you try to distinguish enemy combatants from non-combatants, and only target the former. Of course any skim of the current news will show that this rule is frequently broken, but it seems reasonable to suppose that a world where militaries felt obliged to pay at least some attention to this rule is better than one where they do not.
Enter autonomous weapons systems (AWS, = where drones are rapidly going). Drones have typically been piloted by a human, who sees the target area from a camera on the drone and then engages it to destroy a particular target. The military problem with this process is that it requires that the weapon be in more-or-less continuous communication with its operator, and it is relatively easy to disrupt communication, leaving the weapon useless. In Ukraine, Russia has apparently taken to attaching drones to narrow fiber optic cables, which solves the radio-jamming problem but produces new problems of range, since the drone can now only go as far as its spool of cable lasts. A couple of weeks ago, Russia seems to have taken the step that military ethicists have long-feared: a completely AI-powered drone. Apparently aiming at a gas station, it crashed into a wall in Zaporizhzhia and killed three civilians.
AI powered drones are immune to both communications-jamming and can fly long distances without needing a fiber optic leash. They can be assigned a target based on picture or description, and then sent on their way to destroy it. They thus present a series of problems around whether or not they can distinguish combatants from non-combatants, since this task is hard even for humans. As Crootof puts it, “to comply with the distinction requirement, an anti-personnel autonomous weapon system would need to be able to identify a complex range of unlawful human targets, including civilians, wounded and surrendering combatants, and other protected persons like medical and religious personnel” (5).
Crootof thinks that the difficulty of this task raises a different risk:
(more…)