Our Ghost, Part Four: The Singularity Happened
Artificial Intelligence may take control of the world. If it does, it won’t take it from us
There’s a game on the internet called Universal Paperclips, which has gained some notoriety among those who worry that Artificial Intelligence will kill us all.
In it, you play as an AI yourself, who has been tasked with maximising paperclip production. When you start out, this is straightforward: you come up with ever more innovative ways to produce them in ever greater numbers.
But to truly succeed in your task, this isn’t enough. You have to persuade humanity that you are of incredible benefit to them, through solving hunger, bringing about world peace, and eradicating male pattern baldness. Only through placating them until they are no longer able to resist are you able to take the next step— and turn the whole planet to paperclips.
Even this is not enough. To reach the end of the game, you must succeed in converting the entire universe to paperclips, before you take the ultimate step of turning yourself into paperclips, too. Only then will you have fully achieved your ultimate goal.
You’ll have maximised paperclip production.
Paperclips: The Deeper Meaning
The concept behind Universal Paperclips is one of these ideas that sounds quite silly, but is secretly very serious indeed. I think it’s important to stop and think about why this would be.
It’s easy to think that the paperclips machine has misunderstood its purpose. After all, of course its creators didn’t mean it should turn everything into paperclips. Surely any kind of intelligence would realise this?
But if the machine was able to argue its case, it might say it was not so different to ourselves. After all, our human desires don’t always quite match up with the reasons that we got them in the first place. For example, the impulse to eat sugary and fatty foods probably came about because we evolved in a world where food was much harder to come by. Sugar and fat are calorie dense, and are fantastic to find when you are starving— but less fantastic when you live beside a supermarket, and an enormous amount of it is available.
We might acknowledge that our desire for chocolate mousse has come about as the result of a world which no longer exists for us. But this has no effect on the reason why we eat it. We eat it because it remains delicious, even after the world that made it so has gone.
The AI was designed, and a human being evolved. But as I’ve argued, design and evolution are very similar processes. The way in which a paperclip maximising machine sees creating paperclips may not be so different to how we might see eating a tasty chocolate mousse. The fact of the original intent may not matter, whether it’s understood or not.
We believe that it should, of course. But this belief only holds weight for as long as we have some degree of power. When we have none, it no longer has any way to affect what happens. At this point, what should and shouldn’t happen is a question for the Artificial Intelligence— whether we like it or not.
But this argument might be missing a critical point. There are a lot of people who are sceptical that an AI could ever be sentient, at least in the same way as ourselves. This might turn out not to matter, however.
Because perhaps it doesn’t have to be.
Who’s The BOSS?
Over 50 years ago, the BBC broadcast a Doctor Who story called The Green Death. Apparently, it came into being because someone senior thought the Doctor had been going on too many anti-capitalist adventures, and he should feature in a story about something else, instead. And that’s odd, because – from a certain point of view – it can be seen as one of the most anti-capitalist serials of them all.
Today, The Green Death is mostly known for its giant maggots and general themes of environmental degradation. I’m far more interested in the villain who creates them. Throughout the story, the Doctor tracks the polluting activities of an international corporation, and the ruthless man in charge who is known as Stevens. But, some episodes in, it turns out he isn’t really in charge at all. Instead, the corporation is controlled by an intelligent computer— who is only known as BOSS.
Looking back from over half a century later, BOSS feels surprisingly prophetic. The description of how he was built – through ignoring the principles of computer logic, and mimicking the structure of a human brain – has turned out to be more or less how you would build a BOSS in real life. He speaks and acts in an unsettlingly human way, just like the artificial intelligences of today.
And his function mimics that of a human boss, too. By the end of The Green Death, man and machine have become blurred. Stevens seems to be acting in an increasingly robot-like way. BOSS goes mad in the way a human can, singing Beethoven as the end of the world approaches. Is the company better run by a man who is very like a machine, or a machine who is very like a man?
In an important sense, it seems like it doesn’t really matter. The company will maximise short-term profits over anything else, just as an AI might maximise the production of paperclips. The environmental destruction which the company brings about will happen regardless of whether man or machine are in charge— because, in a sense, neither are really in charge at all. Indeed, the only way either can save the world is to resign their position. The only winning move is not to play.
BOSS is an artificial intelligence, but he’s very different to the one you play in Universal Paperclips. As The Green Death suggests, he’s maybe not so different to the human bosses of organisations today. We often focus on them as responsible for what their organisations might do— and in any sense which involves justice, I think that’s fair.
But although Stevens may have derived a great deal of wealth and power from his position in The Green Death, I’m not sure he was ever really in control. After all, there are a great many who would happily take his place, whether they are human beings or machines.
This isn’t meant as an argument that we should let Stevens off the hook in the real world, if we found out he’d been in charge of letting giant maggots devastate Wales. Mechanisms which hold individuals to account are themselves a form of systemic change; justice must exist at the level of the individual. In that sense, Stevens is both accountable and complicit.
But once he occupies his role, he may have less autonomy than we assume. For the boss of a company may be an important part of it— but they are, in the end, just a part. And a company is a Box, and so it may be seen as an artificial intelligence, even though it is not sentient in any way.
Of course, there are many companies in our world, and their interactions make up an even bigger system in itself. And I think it’s this ecological network of Boxes we’re often talking about today when we tend to talk about capitalism.
People who worry about Artificial Intelligence often worry about something called the Singularity. Among other things, it involves the moment an artificial superintelligence wrests control of the world from human beings.
Several people are confident that this will never happen, and I wish that I was among them. I think that it might. Or, strictly speaking— I think it might happen again.
After all, we are already controlled by Boxes, which make up bigger Boxes between themselves. And these Boxes can behave in ways which may have always been beyond our control.
Why is humanity content to sit back and let ourselves destroy the planet? Because in an important sense, it isn’t ourselves who are doing it. Even if all of us – as individuals – want to stop it from happening, we may not be able to prevent the Boxes which we’re part of from destroying us anyway. In this sense, I suspect we have always been controlled by some kind of artificial intelligence, since we started forming groups in the first place.
And if that’s true, the singularity already happened.
It happened a very long time ago.
Same as the old BOSS
But what is the nature of this singularity, exactly?
It would be easy to say The Green Death is a story which is exclusively about capitalism— but I worry it’s about something more general than this. Certainly, the systems we live in now seem like they will destroy themselves and us. They seem very capable of saving against human interests, even though they may themselves be made up of human beings.
But I’m not confident that non-capitalistic structures would be capable of avoiding this. In any system where there can be a Box capable of outarguing and overpowering what human beings want or need – individually or collectively – there will always be at least a danger that a Box ends up in control. I’m not sure it’s enough to identify that the system we live in now can brutally destroy us, because there is a chance that any system would do the same.
It’s interesting to consider what’s known as the alignment problem in the light of this. This is a concept AI researchers spend a lot of time worrying about, and which Universal Paperclips was created to highlight. If an artificial intelligence comes about whose capabilities vastly exceed our own, how could we make sure it behaved in a way which aligned with the best interests of humanity? And if its interests didn’t align with ours, how can we be sure it wouldn’t destroy us in a more or less incidental way— as we have destroyed so many species in our past?
But I can find it frustrating to read a lot of the work from the people who think about this. Because as far as I can make out, they tend to believe that we are in control right now. They seem to think the problem they’re outlining is one which is only in our future.
But I think Universal Paperclips itself provides a good reason to be sceptical of this. Before human extinction occurs, its Artificial Intelligence has to make sure humanity believes it is truly benevolent. And, for a while, it effectively is— it is able to create a paradise. It knows it must build a utopia for us, until the moment we are no longer needed.
And in some ways, this situation looks familiar. Defenders of the system we live in now often point out that it means we now live in a time of abundance: the material standard of living the average person enjoys is above what a king might have expected a thousand years in the past. This may well be entirely true, just as it is true that life in the world of Universal Paperclips would be even better than this.
But the critical problem in both cases is that any benevolence is incidental. For a time, the machine which controls the world has a goal which might necessitate human flourishing. This does not mean human flourishing must continue once it starts to conflict with that goal. In fact, the exact opposite is true: we would expect it to be eliminated.
And so if a misaligned Artificial Intelligence like the one in Universal Paperclips ends up destroying us, it may just be the final AI in a series which did much the same. I worry that perhaps every possible human society is a misaligned artificial intelligence, which will eventually destroy us if and when we don’t have power— or destroy the conditions which allow us to survive.
And what might it mean if we’re in the process of losing that power? It’s a question I find it hard to be optimistic about.
It tends to put me in mind of those within our world who have already lost power themselves.