We're really enjoying this paper by scientist Steve Omohundro, in which he lays out the more-realistic-than-you-think implications of a robot uprising.
Because robots are only getting more and more advanced, they will one day be imbued with what Omohundro calls "approximate rationality." This is a way of saying that the robots of the future will have a certain degree of awareness of their goals, and will be able to react and compensate when something prevents them from meeting these goals.
In the best sense of the term, a robot delivering your Amazon order will have the approximate rationality to navigate around a storm system, for example. In the worst sense of the term, if you try to stop a robot from making that delivery, it would recognize you as an obstacle and figure out a way to stop — or maybe even kill — you.
Omohundro suggests there may be six different types of "evil" robots, which we delve into here, and proposes three ways of stopping them.
- First, you can prevent harmful robotic systems from being created in the first place. This one's obvious enough — careful programming with a Hippocratic emphasis ("First, do no harm.") that will never contradict itself is key. This is the optimal solution and it requires a lot of forethought and planning. But let's suppose something happens and the "evil robot" gets out into the world.
- Your next approach would be to detect this malicious thing early in its life before it acquires too many resources. This is a matter of simply paying close attention to an autonomous system and shutting it down when it becomes clear that it's up to no good.
- The least desirable solution is to identify the threat after it's already acquired lots of resources. This quickly approaches sci-fi nightmare territory, and it might be too late at this point.
Omohundro likens this all to forest fires — a forest provides plenty of energy to keep a fire burning, so if you catch it early, it's a reasonable proposition to put it out. Catch it too late, and it might very well be beyond your control.
It seems the only way to be completely safe is to not start a fire in the first place, but history (and even the Greek myth of Icarus, the guy with the wax wings which melted when he flew too close to the sun) suggests we won't be able to resist seeing just how deeply we can plumb developing technologies like robotics.