The Law-Giving Machines

This post also appears at Speculative Nonfiction. This article includes story spoilers.

There are now actual drones in our skies, both watchers and hunter-killers. But they're (so far) only semi-autonomous, and they're on missions to protect us legally and militarily, rather than sent by fellow machines to exterminate. Thought experiments in fiction about automatic law-giving devices have been much more interesting than apocalypse porn about bad AI.

Two short stories come to mind here, one of which has enjoyed recent attention, Robert Sheckley's Watchbird (1990) and Larry Niven's Cloak of Anarchy (1972). Both these stories involve surveillance drones with some degree of autonomy and that can hurt or kill their targets. In Watchbird, the drones are police devices, intended to kill murderers before they commit their crime; the drones are able to learn on the job and once released, they expand their definition and start protecting all living things and even some machines. In the end, another drone is released to kill the first drone species, but of course it soon expands its own definition of what it should kill. (Watchbird was adapted for film here.)

The drone in Cloak of Anarchy is the copseye. In this future world, there are "free parks" where anything is allowed except violence against another human being. The floating copseyes watch over the park,a nd if violence is imminent, the copseye stuns both the aggressor and aggressee, and both wake up later, calmed down and with a hangover. Then someone finds a way to short circuit the copseyes, and within hours factions have formed inside the park and violence breaks out.

The stories present us with two different sets of concerns, based on the problems that occur. In Watchbird, the central concern is the autonomy of the devices. Their ability to learn is what allows the problem to grow, but the protagonist is preoccupied with the very fact of machines executing laws without intervening humans. On one hand this could almost seem like a reactionary position: one of the greatest inventions of modernity was nations of laws and not of men. Intervening humans with narrow self-interest executing these laws have always been the problem! (Hence this proposal for a legal programming language in which to write laws that then have to compile with previous laws.) But even without that quibble, his point is well-taken that when autonomous law-givers are able to immediately carry out their sentence and we can no longer modify them, they might become paperclip maximizers, Less Wrong parlance: that is, a moral rule which seems universalizable has consequences that humans could not foresee when implementing it in all-powerful enforcers which can no longer be called back. The protagonist has no problem with more efficient enforcement, but the moral mutations allowed by the machines' autonomy.

To this end, naively, little mention is made of the interests of those authorizing and supporting the program. Still, Watchbird does peripherally make the point that technology allows concentrations of power in the hands of individuals in a way that distorts society. With sudden increments in enforcement power, some humans are able to apply laws with an all-pervasiveness and immediacy that had just never been possible before. Even someone with good intentions and what you would have called good values would suddenly find him or herself in a position of dictatorial authority. It's not even that power corrupts (although it does); it's that this centralization is so unnatural as to be impossible to handle with a good outcome. The best example is this exchange with the protagonist early in the bad behavior of the Watchbirds:

"One of the watchbirds went to work on a slaughterhouse man. Knocked him out."

Gelsen thought about it for a moment. Yes, the watchbirds would do that. With their new learning circuits, they had probably defined the killing of animals as murder.

"Tell the packers to mechanize their slaughtering," Gelsen said. "I never liked that business myself."

"All right," Macintyre said. He pursed his lips, then shrugged his shoulders and left.

One man is suddenly in the uncomfortable position of morally disapproving of whole industries and forcing them to change. What's more, this previously reluctant man does not seem so reluctant now.

It bears mentioning that the conclusion of the story, where the Watchbird-killers are now expanding their prey definition, is recapitulating one of the problems of a Singularity solution of building anti-AI AIs: there could conceivably be a parallel to an auto-immune reaction disease if humans fell into the definition of AIs.

At first glance the problem in Cloak of Anarchy is a curious one - that humans immediately revert to violent tribalism when the violence control mechanism is defunct - since Niven is elsewhere clearly sympathetic to libertarian concerns. The obvious interpretation of the story is the paternalistic one, that humans need authority to make them behave. But there's another interpretation, which is that the drones created the problem. That is to say, when we are coddled by perfect enforcement from drones, we lose the ability to exercise moral choices, as well as the ability to appreciate the consequences of poor choices. When it is physically impossible to harm another person, why learn restraint? Why worry about what happens when you pick a fight? When suddenly the daddies aren't around to break up the fights and bail everyone out, we shouldn't be surprised at what happens.

The watchbirds do exist today, although with less autonomy and more firepower. The changes are incremental; there won't be a red carpet unveiling of AI even as profound as the release of the watchbirds (or copseyes). They'll be to areas where there's the most pressure for advance, and the least opportunity for public awareness and understanding. It will be, and is, the addition of subroutines allowing a drone to apply the laws of war to a kill it's about to make (instead of getting slow permission from a JAG in an office in St. Louis who might be in the bathroom). It's the growth of autonomous stock trading algorithms. It will even be in advertising on porn sites.