“A Bomb Is Blown Up Only Once – But One Bot Can Become Many, And Quickly Get Out Of Control”

Computer scientist Bill Joy despised the violence of the Unabomber as any sane person would, so he felt great disquiet when he read a passage written by Ted Kaczynski and agreed with the domestic terrorist’s concerns for the future of humankind. In his famous 2000 Wired article, “Why the Future Doesn’t Need Us,” Joy meditates on the unease caused by his sympathies for the ideas of a madman. An excerpt:

“Part of the answer certainly lies in our attitude toward the new – in our bias toward instant familiarity and unquestioning acceptance. Accustomed to living with almost routine scientific breakthroughs, we have yet to come to terms with the fact that the most compelling 21st-century technologies – robotics, genetic engineering, and nanotechnology – pose a different threat than the technologies that have come before. Specifically, robots, engineered organisms, and nanobots share a dangerous amplifying factor: They can self-replicate. A bomb is blown up only once – but one bot can become many, and quickly get out of control.

Much of my work over the past 25 years has been on computer networking, where the sending and receiving of messages creates the opportunity for out-of-control replication. But while replication in a computer or a computer network can be a nuisance, at worst it disables a machine or takes down a network or network service. Uncontrolled self-replication in these newer technologies runs a much greater risk: a risk of substantial damage in the physical world.”

Tags: ,