There are numerous financial and practical reasons for turning society into a giant, tentacled computer that connect us all, but even if mass surveillance and other elements of governmental and corporate fascism aren’t the chief driving forces of the transition, their danger is no less real. My main issue with the febrile fear of Strong AI—computers surpassing our brains and turning us into zoo animals or some such thing—is that this outcome is likely not happening today or tomorrow or perhaps ever, even if it isn’t theoretically impossible.

The bigger issue is that no such superintelligence need develop for humans to be diminished or doomed. Weak AI can do us in with a thousand cuts. The process of computerizing absolutely everything has already begun in earnest, and it hasn’t thus far made us better or wiser or happier. That may be because our best intentions have failed, but more likely is that this new system of digital capitalism has its own agenda and human well-being isn’t job one.

From Ian Bogost’s Atlantic article “You Are Already Living Inside a Computer“:

Newer dreams of what’s to come predict that humans and machines might meld, either through biohacking or simulated consciousness. That future also feels very far away—and perhaps impossible. Its remoteness might lessen the fear of an AI apocalypse, but it also obscures a certain truth about machines’ role in humankind’s destiny: Computers already are predominant, human life already plays out mostly within them, and people are satisfied with the results. …

Think about the computing systems you use every day. All of them represent attempts to simulate something else. Like how Turing’s original thinking machine strived to pass as a man or woman, a computer tries to pass, in a way, as another thing. As a calculator, for example, or a ledger, or a typewriter, or a telephone, or a camera, or a storefront, or a café.

After a while, successful simulated machines displace and overtake the machines they originally imitated. The word processor is no longer just a simulated typewriter or secretary, but a first-order tool for producing written materials of all kinds. Eventually, if they thrive, simulated machines become just machines.

Today, computation overall is doing this. There’s not much work and play left that computers don’t handle. And so, the computer is splitting from its origins as a means of symbol manipulation for productive and creative ends, and becoming an activity in its own right. Today, people don’t seek out computers in order to get things done; they do the things that let them use computers.

* * *

When the use of computers decouples from its ends and becomes a way of life, goals and problems only seem valid when they can be addressed and solved by computational systems. Internet-of-things gadgets offer one example of that new ideal. Another can be found in how Silicon Valley technology companies conceive of their products and services in the first place.•