The Biggest Bad Beat: Living with the Possibility of Extinction
The Original Doomsday Clock
There is a particular kind of fear that doesn’t require anything bad to actually happen. You don’t need the missile to launch, the siren to sound or the mushroom cloud to appear on the horizon. Sometimes it is enough to know that these things could happen, and that the people capable of making them happen are sitting thousands of miles away in rooms you will never enter. This was the peculiar psychological burden of the Cold War, when an entire generation grew up under the shadow of an event that might never occur but could, in theory, end almost everything. It is difficult to imagine a more perfect recipe for anxiety: enormous stakes, uncertain probabilities and absolutely no control over the cards that will be dealt.
The Cuban Missile Crisis of October 1962 provided perhaps the purest expression of this anxiety. For thirteen days, the world watched John F. Kennedy and Nikita Khrushchev two-step around the possibility of nuclear confrontation, while ordinary people had little option beyond waiting, listening, clenching their sphincters and imagining the worst. The frightening thing was not merely that nuclear weapons existed but that a chain of misjudgements, misunderstandings or bad decisions could produce an outcome nobody actually wanted. Existential anxiety begins somewhere around this point: when the consequences become so enormous that your individual capacity to influence them approaches zero.
And the Cold War wasn’t merely an occasional news story. School children were taught civil-defence procedures, governments constructed fallout shelters and television repeatedly reminded families that nuclear war was not the stuff of science fiction but a contingency for which governments were actively preparing. The psychological paradox was obvious: the more seriously society prepared for catastrophe, the more vividly catastrophe entered everyday life. It was a bit like being told every morning that the building probably wouldn’t burn down, followed by another fire drill just to make sure you understood what would happen if it did.
The Fire Alarm
This is where the comparison with AI becomes considerably more interesting in 2026 than it might have seemed even a few years ago. The strongest argument against dismissing AI extinction anxiety as the latest manifestation of technological doomerism is that some of the people pulling the fire alarm are not outsiders, Luddites or science-fiction writers; they are people who have actually been inside the buildings where these systems are being developed. A few days ago, Jacob Coxon, a researcher who had worked at both OpenAI and Anthropic, resigned from Anthropic and published a viral warning that the industry was racing towards increasingly autonomous and potentially self-improving systems without having solved the problem of keeping them under control. His post has already reached more than 150 million people, turning what had previously been an esoteric argument among AI researchers into a mainstream existential question.
There is something psychologically different about hearing that warning from the man selling the fire insurance and the man who has spent three years wiring the building. Coxon’s decision is particularly striking because, according to Axios, he left shortly before equity was due to vest, meaning that walking away carried a tangible financial cost. And then came more calls from inside the house. Anthropic researcher Evan Hubinger publicly said he believed there was at least a 10% chance of AI causing human extinction within a decade, while another Anthropic researcher, Samuel Marks, said concern about catastrophic outcomes was widespread among experienced developers.
Suddenly the old question of whether AI doom is merely an internet fantasy becomes rather harder to answer with a shrug. This isn’t to say that Coxon is necessarily right, or that the odds being bandied around are accurate. 72 offsuit cracks aces about 10% of the time but probability estimates about unprecedented technological events are inherently slippery. Intelligent people inside and outside the industry disagree radically about timelines, mechanisms and the risk profile of AI. In a way, the uncertainty is part of what makes this psychologically so impactful. The Cold War citizen didn’t need to know the precise probability of nuclear war to feel anxious; the mere existence of a plausible route to catastrophe, combined with the knowledge that highly informed people were taking the danger seriously, was enough.
Cooler
When we sit down at a table in a poker tournament, we know that the dealer might have in their possession an ice-cold deck. We cannot see the cards, nor did we influence the shuffle but we get how there will sometimes be unavoidable situations when our actions are, in a way, compelled by a necessary and quite humdrum adherence to game theory. If they’ve got it, they’ve got it, and even if we’ve got it, they might get it! Those of us who are more grizzled have learned to accept this aspect of the game. Poker Gods, grant me the serenity to accept the coolers, the courage to make the right plays, and the wisdom to know the difference. That type of thing.
That method of coping was central to the psychology of the nuclear age. An ordinary citizen could vote, protest, campaign for disarmament or build a shelter, but ultimately the decision to launch nuclear weapons belonged to an extraordinarily small group of political and military leaders. The experience was therefore profoundly asymmetric: everyone could understand the consequences, but almost nobody possessed meaningful agency over the cause. Modern AI produces a different version of the same discomfort because a relatively small number of laboratories, corporations, researchers and governments are making decisions that could influence the technological trajectory of billions of people.
An important difference this time, however, is that the spectators can hear the players arguing. In 2023, more than 100 prominent AI researchers and industry figures, including the heads of OpenAI, Anthropic and Google DeepMind, signed a statement saying that mitigating the risk of extinction from AI should be a global priority alongside pandemics and nuclear war. The Future of Life Institute had already circulated an open letter calling for a pause on giant AI experiments, warning about systems that might eventually “outsmart, obsolete and replace” humans and arguing that such decisions should not simply be delegated to technology executives. These were not warnings coming from the tin-foil hat brigade. They were, at least in part, coming from knowledgeable people in close proximity to the action.
The Psychology of p(Doom)
During the nuclear age, people lived with what might be called ambient catastrophe. The bomb was always somewhere in the background, even when nothing was happening. AI has produced its own version, complete with its own vocabulary. p(Doom) is the shorthand used by AI researchers and enthusiasts for their subjective probability that advanced AI will cause human extinction or an equivalently catastrophic outcome. The very existence of the term is revealing because it transforms an almost metaphysical fear into something that presents as reassuringly mathematical. Put a percentage beside Judgment Day and, somehow Terminators killing us all feels like something we can spreadsheet.
But probabilities can also become a psychological trap. If someone tells you there is a 10% chance that the 72 off-suit will crack your aces and bust you, the rational response depends on the pot, your bankroll and the alternatives available. If someone tells you there is a 10% chance that humanity will cease to exist, there is no bankroll management strategy. This is why existential anxiety is so different from ordinary risk. The downside is so extreme that even a relatively small probability can become emotionally dominant. Nuclear deterrence worked partly because leaders understood this logic. Nobody needed to believe nuclear war was likely to understand that the consequences of getting the calculation wrong were unacceptable.
AI introduces an even more unsettling twist because the technology is simultaneously the thing being developed and the thing being used to help understand the danger. Researchers are attempting to build systems that can reason, code, act autonomously and improve their capabilities while also trying to determine whether increasingly capable systems will remain interpretable and controllable. Reuters reported this week that concerns are growing around reduced monitorability and increased autonomy in frontier systems, while researchers have publicly called for greater caution. OpenAI’s newly appointed safety figure Paul Christiano has gone further, warning that a failure to align superintelligent AI could potentially result in most people dying and arguing that the industry is not currently on track to mitigate the danger to an acceptable level.
Pause
As I read Jacob Coxon’s tweet for the fiftieth time yesterday, I thought about how there is an important distinction between taking a risk seriously and becoming psychologically consumed by it. The Cold War eventually demonstrated that living permanently in anticipation of catastrophe can become its own form of distortion. If every bad headline is interpreted as evidence that Armageddon is imminent, the absence of Armageddon itself starts to feel like evidence that we simply got lucky this time. AI discourse can fall into exactly the same trap, particularly when every new benchmark or model release is treated as another step towards either utopia or extinction. The result is a peculiar form of intellectual tilt in which uncertainty is mistaken for evidence.
That said, the existence of serious AI warnings does not mean that every alarm means there’s a fire. There are legitimate critics who argue that concentrating on hypothetical extinction scenarios can distract from immediate harms such as misinformation, discrimination, surveillance, labour displacement and concentration of power. There is also an obvious incentive problem when the executives and researchers building increasingly powerful systems simultaneously warn us that those systems might become extraordinarily dangerous. However, that cuts both ways as the possibility of incentives does not make the underlying risk imaginary. A poker player selling action and charging mark-up isn’t necessarily gouging the market.
Perhaps the deepest similarity between Cold War nuclear anxiety and the emerging fear of AI catastrophe is therefore not that the bomb and the algorithm are equivalent. It is that both confront ordinary human beings with the possibility that the most consequential event of their lifetime might be determined by people and forces beyond their control. Jacob Coxon’s viral warning is powerful partly because it punctures that distance. He has just walked out of the casino, leaving money on the table, because he believes that the game may be becoming unsafe. The fact that he is not the only one should give us pause and perhaps elicit a pause. When a bunch of experienced players have a strong hunch that the deck is rigged, the worst thing you can do is blindly deal the next hand.
Comments