Catastrophist philosopher Nick Bostrom believes machine superintelligence may be the greatest existential risk facing humankind, that it could, perhaps sooner than later, be the end of us if we’re not careful. There’s nothing theoretically impossible about that, though I seriously doubt the sooner part. First maybe McDonald’s will be fully automated, and then much, much, much later on we face a robot-inspired endgame. I actually think it’s more likely that such computer intelligence will help us engineer our own evolution into whatever it is we become in the long run, though miscalculation leading to a cascading disaster might become a plausible scenario at some point.
In a Washington Post piece, Joel Achenbach explores the so-called Artificial Intelligence threat and the professional worriers who analyze it and exhort us to shape the future. MIT computer scientist Daniela Rus is presented as a counterpoint to Bostrom, physicist Mark Tegmark and other thinkers who fear an AI-inspired end is near. The opening:
The world’s spookiest philosopher is Nick Bostrom, a thin, soft-spoken Swede. Of all the people worried about runaway artificial intelligence, and Killer Robots, and the possibility of a technological doomsday, Bostrom conjures the most extreme scenarios. In his mind, human extinction could be just the beginning.
Bostrom’s favorite apocalyptic hypothetical involves a machine that has been programmed to make paper clips (although any mundane product will do). This machine keeps getting smarter and more powerful, but never develops human values. It achieves “superintelligence.” It begins to convert all kinds of ordinary materials into paper clips. Eventually it decides to turn everything on Earth — including the human race (!!!) — into paper clips.
Then it goes interstellar.
“You could have a superintelligence whose only goal is to make as many paper clips as possible, and you get this bubble of paper clips spreading through the universe,” Bostrom calmly told an audience in Santa Fe, N.M., earlier this year.
He added, maintaining his tone of understatement, “I think that would be a low-value future.”
Bostrom’s underlying concerns about machine intelligence, unintended consequences and potentially malevolent computers have gone mainstream. You can’t attend a technology conference these days without someone bringing up the A.I. anxiety. It hovers over the tech conversation with the high-pitched whine of a 1950s-era Hollywood flying saucer.•
Tags: Joel Achenbach, Nick Bostrom