“These Systems Leave No Room For Humanity, Yet They Define Our Daily Lives”

“Perhaps it’s the case that in order to live, we must process our experience first rationally and then irrationally,” Wallace Shawn has written. We still have a long way to go in 2017 with the Step One of that process in regard to our priorities and politics, but the playwright has, in his last two works, tried to “irrationally” address the rise of technology and authoritarianism. Despite promising premises, however, neither Grasses of a Thousand Colors nor Evening at the Talk House develop in profound ways.

Another dramatist, Jordan Harrison, has come much closer to processing these issues potently in a couple of his plays: Marjorie Prime, which thinks about the enhanced near-future of Virtual Reality, and Maple & Vine, which imagines a retreat from our connected technological society to a village that recreates the 1950s, a quaint place marked by repression and racism. Taken together, these works remind that we must go forward into a fraught tomorrow, can’t go back to a yesterday not nearly as bright as it might seem from a distance, but our tools will be powerful and we need to try our best to limit the damage they can do. One challenge will be that while the future arrives more quickly now than it once did, the process of getting there has fewer bumps and seams. It looks benign.

As Chelsea Manning writes in a New York Times op-ed: “The world has become like an eerily banal dystopian novel. Things look the same on the surface, but they are not.” Every now and then, with the Russian invasion during the election and today’s news that Facebook has been selling ads to people interested in the phrases “Jew hater” and “How to Burn Jews,” it becomes obvious that things have gone seriously awry, but we hardly noticed as we were building this Trojan horse inside our own gates. What to do now?

From Manning:

The real power of mass data collection lies in the hand-tailored algorithms capable of sifting, sorting and identifying patterns within the data itself. When enough information is collected over time, governments and corporations can use or abuse those patterns to predict future human behavior. Our data establishes a “pattern of life” from seemingly harmless digital residue like cellphone tower pings, credit card transactions and web browsing histories.

The consequences of our being subjected to constant algorithmic scrutiny are often unclear. For instance, artificial intelligence — Silicon Valley’s catchall term for deepthinking and deep-learning algorithms — is touted by tech companies as a path to the high-tech conveniences of the so-called internet of things. This includes digital home assistants, connected appliances and self-driving cars.

Simultaneously, algorithms are already analyzing social media habits, determining creditworthiness, deciding which job candidates get called in for an interview and judging whether criminal defendants should be released on bail. Other machine-learning systems use automated facial analysis to detect and track emotions, or claim the ability to predict whether someone will become a criminal based only on their facial features.

These systems leave no room for humanity, yet they define our daily lives. When I began rebuilding my life this summer, I painfully discovered that they have no time for people who have fallen off the grid — such nuance eludes them. I came out publicly as transgender and began hormone replacement therapy while in prison. When I was released, however, there was no quantifiable history of me existing as a transwoman. Credit and background checks automatically assumed I was committing fraud. My bank accounts were still under my old name, which legally no longer existed. For months I had to carry around a large folder containing my old ID and a copy of the court order declaring my name change. Even then, human clerks and bank tellers would sometimes see the discrepancy, shrug and say “the computer says no” while denying me access to my accounts.

Such programmatic, machine-driven thinking has become especially dangerous in the hands of governments and the police.

In recent years our military, law enforcement and intelligence agencies have merged in unexpected ways. They harvest more data than they can possibly manage, and wade through the quantifiable world side by side in vast, usually windowless buildings called fusion centers.

Such powerful new relationships have created a foundation for, and have breathed life into, a vast police and surveillance state. Advanced algorithms have made this possible on an unprecedented level. Relatively minor infractions, or “microcrimes,” can now be policed aggressively. And with national databases shared among governments and corporations, these minor incidents can follow you forever, even if the information is incorrect or lacking context.•