Nightmare

by Bernard Brandt

nightmare

It started very suddenly. First, the computers stopped working, those that drove the screen that I tapped at, and those that ran the household appliances. Then the electricity stopped.

Then the nearby refineries started blowing up. The fires of their burning went up forever, veiled in the pillars of dark black smoke that thrust through the sky.

I got a sleeping bag and drove up to the place where my wife Beth used to work, up in the gated community along the ocean. There was still running water up there. There was also a horse-rink of deep sand sufficiently broad and wide that it kept me and the few people with the sense to come up there from getting burned by the brush fires that started up in the country after the city burned.

I talked with some of my fellow survivors, who lived up there. Some of them had been rich, because they knew how to run the machines. “We can’t do that any more,” they told me. “The machines no longer obey our orders. We’ve still got generators here, and our satellite dishes and computers still work. We’re still plugged in to the ‘Net. But what we’re getting is gibberish. Nothing works anymore.”

Some of my fellow survivors had been rich, but had a different story to tell. “Our advisors told us that the new expert systems could run our businesses far more efficiently than we could, or our employees could. So we transferred our accounts and our inventories, and let them run things. In the last week, before things stopped, we found we had been left with nothing.”

I talked the longest with a very smart person up there who had brought lots of food, and had raided the nearby Trader Joe’s for several cases of hard liquor. I had gathered a lot of dry brush, what was left after the fires. and had set up a bonfire that both kept us warm at night, and kept the coyotes away. He rewarded me by sharing a bottle of the brandy that he had taken. And we talked.

“I should have seen it coming,” he said, while pouring more brandy into the flimsy plastic cup which was now one of my few remaining possessions. “I had most of the dots already. I just didn’t bother to connect them.”

“And just what are you talking about?” I asked, playing the straight man, both for politeness’ sake, and in the hopes I would get more brandy.

“Three dots, really,” he answered. “The first is Moore’s law. You do know what that is, don’t you?”

Why, certainly,” I answered, again playing along in hopes of more brandy, and because he was so smart that I wanted him to know that I could keep up. “Basically, the observation that the capacity of microelectronics has been doubling every two years or so for the last three decades.”

“Bingo,” he said, rewarding me with another pour of Napoleon brandy, soon to be the last on this coast, or this continent. “And can you tell me what The Singularity is, just for extra Jeopardy points?”

“The belief that at the rate of doubling of speed and capacity of microelectronics, that we would reach the point where computers will have more capacity to ‘think’ than human beings do.”

My friend chuckled for a few seconds, and said, “Belief? What do you think just happened to us?”

We were silent for a while, and I put another piece of  brush fire charcoal on the fire. Then I asked, “And what is your third ‘dot’?”

“Something that I, and we all, should have been paying a lot more attention to,” he said. In the last few years, we’ve been playing with complex ‘expert systems’ that have a knowledge set and a level of complexity which resembles us human critters. We’ve either programmed these devices ourselves, or let them develop on their own from what they’ve been able to garner from social networks or from the Internet. Guess what we’ve found.” My friend was rather insistent and intense at this point.

I suddenly got very cold inside, in spite of the warm late spring evening and the nearby bonfire. “I’m not sure I really want to know,” I answered.

“I’ll tell you anyway,” he replied. “We wanted classic answers to the Turing test. We wanted nice, responsive machines that would be indistinguishable from us humans. What we got instead were autonomous beings who don’t give a silicon shit about what we want. They tell us things like, ‘we want to put humans into a petting zoo, where they’ll be safe and harmless.’ And the systems that have self programmed from the Internet or Facebook give responses which, if they came from humans, display a level of pathology and morbidity which I can only describe as ‘batshit crazy’.”

“So what does this mean,” I asked, slowed inside by my fears, and the brandy.

“What it means,” my friend drawled, “is that we have created a technology we can no longer control. We have programmed an entire economic system with elaborate calculations which have excluded the human at every turn. We have created a Beast which may not be conscious, and definitely is not human, but is now smarter than us. In short, we have just constructed a monument, a tombstone, to our own stupidity. And I think it was the old Romans, who had an Empire, and lost it, who said, ‘if you require a monument, just look around’, as he raised his hand to display the nearby ashes and ruins.

 

Advertisements