The Existential Risk of Hyperbole

The black and white world of AI Dystopianism

One of the reasons this blog exists is to counter some of the alarmism that percolates through public discussions of artificial general intelligence and has, in the last few years, seeped into discussions about machine learning based artificial intelligence as well. There has been fairly little public critiquing of the ideas promoted by AI Dystopianism, and what little there has been is simply dismissed by AI Dystopians rather than refuted in a reasoned way.

This dismissal has taken many forms, but it frequently involves claims that those espousing AI Dystopian themes are either ignored or maligned by the greater AI and AGI scientific community. AI Dystopians have accused critics of being scared and not facing the truth as well as deliberately hiding the truth due to ulterior, opportunistic, and possibly even nefarious motives. Perhaps most frequently, critics have simply been dismissed as ignorant.

Most scientists in AI and AGI are understandably busy doing research and development in their field, and this has left relatively few who have taken the time to refute AI Dystopian ideas directly. Even fewer have taken the time to analyze both specific AI Dystopian conclusions and the underlying foundations of those conclusions in a public-facing manner.

This problem is certainly not unique to the field of AI, as many professionals in a field will simply feel that refutations are unnecessary for what they see as obviously flawed arguments. Unfortunately, such flaws are not obvious to most outside the field. The foundational beliefs underlying much of AI Dystopianism and its dire predictions are for the most part unknown to the general public.

Some of these fundamental ideas have been explored in the Foundations of AI Dystopianism series of posts in this blog, including speculation on the nature of goals, rationality, and self-improvement of AGI systems. Most journalists, however, either ignore these fundamental ideas or simply reiterate some portion of them without any analysis or counterpoint from other professionals in the field. That has allowed those ideas to be let loose into the wild with little counterbalancing discussion.

The Undiscovered Country of AGI

The alarm some feel is perhaps rooted in the very nature of AGI, in that it rests conceptually somewhere between scientific concepts like nuclear fusion and warp drives. It’s something we can conceive of but can’t yet create. It is a known unknown technology, part of a potential but mysterious future, and this gives it the power to fester in our psyches and generate alarm in a way that known fears cannot.

AGI philosopher Eliezer Yudkowsky discussed the dangers of future artificial intelligence systems with general intelligence in his paper from the 2008 survey book Global Catastrophic Risks. In that paper, he stated:

It may be tempting to ignore Artificial Intelligence because, of all the global risks discussed in this book, AI is hardest to discuss. But this makes AI catastrophes more worrisome, not less.

Such a suggestion seems similar to the statement that the greatest trick the devil pulled off was convincing the world he didn't exist. There is no refutation that will suffice to those who take such a statement to heart, as any argument against the premise can safely be ignored as being the devil’s hand at work. In the case of AI Dystopianism, any argument against the conclusion of catastrophe is simply driven by one’s inability to believe in the realities of AGI existential risk.

A significant problem with Yudkowsky’s statement, however, is that one can swap in any potential future threat that lacks a coherent framework or empirical support, from alien attacks to inter-dimensional giant monsters, and the statement still applies with equal validity. Given that, it’s not a particularly useful sentiment.

While there are a relatively small number of computer scientists in AI that believe we are close to developing AGI, the overwhelming majority are AI Pragmatists: they’re aware that we know relatively little about the workings of human intelligence, that current AI is missing many key elements of human intelligence, and that it’s probably going to take a long time to figure out and replicate human intelligence. Most don’t discount the dangers of achieving AGI. However, they also don’t think they’re at all inevitable and believe there are many incredible benefits that will make it worth pursuing in a careful manner.

As discussed in this previous post, what’s apparent from the long historical record of mistaken predictions about technology, progress, and society, is that there are no experts when talking about the future, especially in areas with vast known unknowns and inevitably many unknown unknowns. Despite this, the clarion call of the AI Dystopians is not just that AGI will be dangerous, but also that we must start working right now on the task of ensuring it’s safe. At the same time, many AI Dystopians (including those promoting safety research) feel that ensuring that AGI is safe may well be an impossibility.

Impossible or not, it’s not remotely clear how one makes useful progress on this when we have no idea how the science and technology of AGI might work. We might similarly try to work out the safety systems necessary when using using Star Trek style transporters to beam down to a planet.

There have been more and more pronouncements in the media, and even by computer scientists, that LLM’s like OpenAI’s GPT-4 and other recent AI systems are harbingers of AGI systems just around the corner. As mentioned in my last post, however, climbing the tree of contemporary AI is not likely to get us meaningfully closer to the AGI moon anytime soon, as there seem to be very fundamental differences between the functionality of AI and the functionality of human intelligence. It’s as if we’d invented the hot air balloon and imagined that we’d discovered the secrets of bird flight. Both result in getting off the ground, but no amount of hot air will allow you to soar like an eagle.

Realities such as this, however, are quickly dismissed by AI Dystopians. Their frequent use of false dichotomies paints a world of black-and-white outcomes, one in which there is little to no accounting for nuance, probability, opportunity cost, broad technological progress, societal change, or recognition of the huge gaps in our current knowledge. Such an approach leaves us with no viable modalities of discourse, but instead results in the perspective revealed by computer scientist Stuart Russell when he expresses the following in his book on AGI, Human Compatible:

With a bit of practice, you can learn to identify ways in which the achievement of more or less any fixed objective can result in arbitrarily bad outcomes.

Perhaps so, but it seems at least equally important to ask whether arbitrarily bad outcomes are likely or imminent or supported by data. Bringing up this counterpoint, although seldom done, is usually not taken as an honest disagreement based on differing views and experience.

In his chapter of the 2019 survey book Possible Minds, Skype co-founder Jaan Tallinn stated:

Of course, just as there were dogmatic Communists who never changed their position, it’s all but guaranteed that some people will never admit that AI is potentially dangerous. Many of the deniers of the first kind came from the Soviet nomenklatura; similarly, the AI-risk deniers often have financial or other pragmatic motives. One of the leading motives is corporate profits.

In other words, AI Dystopianism detractors are equivalent to dogmatic Communist Party nomenklatura, which were party insiders who held key positions running the Soviet Union. Somewhat incongruously, Tallinn states that their chief underlying motivations may be greater corporate profits and personal financial gain.

In that same book, MIT physicist Max Tegmark offered a pithy Upton Sinclair quote as a potential motivation for AI Dystopianism's critics:

It is difficult to get a man to understand something, when his salary depends on his not understanding it.

It's worth noting that the vast majority of investment and corporate R&D dollars are currently going to refining and applying AI to business, not researching a path to AGI or determining the nature of cognition in the brain. It's pretty safe to say that the number of computer scientists whose bread is buttered by forces pushing for the reckless development of superintelligence is very nearly equivalent to zero.

While greed is one potential motivation for refuting AI Dystopianism, another is fear. Stuart Russell expressed the following sentiments in Human Compatible:

A perceived threat to one’s lifelong vocation can lead a perfectly intelligent and usually thoughtful person to say things they might wish to retract on further analysis.

Researchers that disagree with Russell’s perspective might not only fear losing their job — they may fear being too successful at it:

It’s as if researchers are afraid of examining the real consequences of success in AI.

In Possible Minds, Tegmark ponders the depths of this fear:

Third, psychologists have discovered that we tend to avoid thinking of disturbing threats when we believe there’s nothing we can do about them anyway. In this case, however, there are many constructive things we can do, if we can get ourselves to start thinking about the issue.

Perhaps it’s not just fear and greed, but simply a lack of imagination that causes critics to criticize AI Dystopian ideas. This seems to be the view of Russell in his chapter of Possible Minds:

Objections have been raised to these arguments, primarily by researchers within the AI community. The objections reflect a natural defensive reaction, coupled perhaps with a lack of imagination about what a superintelligent machine could do. None hold water on closer examination.

Alarm and Reason

When it comes to considering the potential downside of AGI technology, it seems reasonable to react with some degree of caution between 0 and 1 rather than jump to full panic once there is a greater than zero possibility of risk.

Some who aren't completely convinced that AGI is an existential threat have offered a form of Pascal’s wager. This is a philosophical argument proposed by the 17th century philosopher Blaise Pascal, in which he postulated that it made more sense to believe in God and lead a religious lifestyle than to do otherwise. In his mind, if God exists you’ll go to heaven rather than hell, and if you’re wrong, you haven’t lost anything. Similarly, AI Dystopians seem to choose the path of alarmism because if it turns out they’re right then humanity is saved and if they’re wrong we haven’t lost anything.

But alarmism does have a cost. Hyperbolic speculation on the dangers of AGI adds one more tributary feeding into the river of unfounded alarmism coursing through society and adding to the dystopian undertone that seems to permeate modern life.

The public is continually bombarded with things to fear and surveys showing how many people now fear them. When AI Dystopians conflate AI and AGI and then declare in dire terms that the technology could destroy humanity, much of the public begins to believe that this threat is imminent and likely and potentially inevitable. This is a tactic demagogues have used to great effect since time immemorial, and the impetus towards sensationalism in the media only enhances the effect of such alarmism for the general public.

The results are what one would expect. In 2016, a 60 Minutes/Vanity Fair Poll surveyed American attitudes about AI and AGI and found that 15% of Americans thought advancing the field would be dangerous. Several years later in 2019, the Center for the Governance of AI, Future of Humanity Institute, University of Oxford published a survey titled "Artificial Intelligence/ American Attitudes and Trends" which found that 34% of Americans believed that the development of high-level machine intelligence would be harmful to humanity, with 12% believing that it would lead to human extinction (26% thought it would be beneficial to humanity).

The marketing firm Edelman published a survey in 2019 on attitudes towards AI held by both the general population and separately by tech executives. That survey revealed deeper levels of concern, with 62% of the general population and 50% of tech executives having negative feelings towards the arrival of human-level or greater intelligence in machines.

But what's perhaps more surprising — and more concerning — is the percentage of people who think such human-level machine intelligence is imminent.

In the Center for the Governance of AI survey, 54% of Americans felt that high-level machine intelligence, able to perform most human tasks as well as the median human does today, would be developed by 2028. In the Edelman survey, 61% of the general population believe that machine intelligence would surpass human intelligence by 2029, while 73% of tech execs believed this would be the case and 48% of them actually believed it would be achieved by 2024.

Prominent AI researcher and roboticist Rodney Brooks has stated:

If you, as a journalist, or a commentator on AI, think that the AGI movement is large and vibrant and about to burst onto the scene with any engineered systems, you are confused. You are really, really confused.

Computer scientist and entrepreneur Ray Kurzweil, perhaps the leading proponent of AI Utopian ideas and extremely optimistic timelines for AGI development, believes that AGI won't achieve human-level intelligence until 2029 and won't surpass it until some time after that.

Yet the results of these surveys show that there is an increasingly large segment of the general population that is even more optimistic (or pessimistic, depending on your point of view) about the arrival time of AGI than Ray Kurzweil. Also disturbing is that the percentage of people who think advanced AI developments will be dangerous rather than beneficial is increasing significantly. AI Dystopians do seem to be making progress in both pointing out the dangers of AGI and shrinking the timeline of its arrival in the minds of the public.

There is a well-known cognitive bias called the illusory truth effect which makes people more likely to believe something if they've heard it before regardless of its inherent truth or likelihood. Even if someone starts out as a skeptic when reading about the dangers of superintelligent machines destroying humanity, they're more likely to believe such speculation the more often they hear it regardless of whether any evidence is ever provided to back up the claim.

The Parameters of a Wager

There are many counterarguments to Pascal’s wager, but there’s one fallacy within it that is particularly apropos to the AGI debate. Pascal was a product of 17th century Europe, so his ideas on demonstrating one’s belief in a higher power fell completely along Christian lines. To him, it was either believe in God as a Christian or don’t believe in God. There was no believing in God and following the doctrines of Judaism, Islam, Hinduism, Zoroastrianism, the religion of the ancient Greeks, or one of the myriad other religions that has existed or will exist in the world.

To Pascal, there were only two variables, two choices. In today’s AI Dystopianism, the thinking is just as black and white, just as unable or unwilling to consider the many possible paths and destinations that lie before us.