I've refrained from providing my own estimates, as this is an imprecise exercise with wide confidence intervals and high tail-risks. I express relatively low confidence in these figures myself. Nonetheless, Tyler wants any numbers rather than precise: in other words, “don't let the perfect be the enemy of the good”. I agree on the need for starting points: all firms and governments will need to invest more in cybersecurity, so estimates for future costs will assist in making decisions on how much to invest in this area.
Disclosure: I've used Codex and ChatGPT 6 Astra to help with the calibration (in particular the sources and data gathering), yet the judgement is my own.
Gartner predicts total cybersecurity expenditures of roughly 0.2% of global GDP in 2026. Even if in a pessimistic scenario, you expect these costs to rise tenfold within the years to 2030, this would amount to no more than 2% of GDP by 2030. Similarly, the UK government estimates 0.5% of GDP. Even under plausibly pessimistic scenarios, it's hard to see AI-driven cyber risks being uniquely catastrophic in expectation1. On the more pessimistic end, Jamilov et al (2026) estimate total costs less than 1% of global GDP. We should also account for learning-by-doing dynamics and general productivity gains increasing the efficiency of cybersecurity measures, which should exert downward pressure on costs relative to GDP.
Suppose there is a small tail probability of an extreme cybersecurity event costing multiple trillions (e.g. a wipeout of the entire banking system). Great Depression saw an over 30% collapse in US GDP. Covid saw a near 20% fall. Disastrous, but hardly extinction. These remain realistic upper bounds for my tail-risk scenarios, unless anyone can point to better anchors?
As for my previous claims, it is apparent that there are more channels via which projected cybersecurity costs, and a “doom trade”, could influence the markets and hence affect optimal portfolio allocation. These include:
Cyber insurance premiums
Corporate debt spreads (those investing less in cybersecurity, or more vulnerable, might see higher yields)
Equities (more vulnerable firms should see lower valuations - indeed Jamilov et al tries to measure this)2
Prices and valuations for cybersecurity contractors
Moreover, perhaps Kelly's Criteron oversimplifies, as there will be many events before doom implying such a scenario. Doom will, if it happens, unfold sequentially, which opens up opportunities for shorting. Moreover, Bayesian agents can form subjective probabilities and account for this with higher uncertainty. This does not undermine optimal portfolio allocation nor standard asset-pricing models.
In general, I see cybersecurity as one of the primary risks associated with AI, yet extinction is highly unlikely on this basis. Scheming and misaligned agents are classic principal-agent and mechanism design problems that are solvable. Bioweapons manufacturing would rely on being able to go undetected in the physical world, which remains implausible (especially as AI would increase detection). Lab leaks are a far more salient concern, yet pandemic preparedness confounds, and AIs will likely assist in this. Disempowerment, and extreme totalitarian powers and government surveillance, alongside the integration of AIs into drone infrastructure, remain my greatest concerns, yet these are hardly existential.
I was one of the commentators that overreacted to the Hugging Face incident. Considering the general equilibrium social and economic dynamics more broadly, p(doom) cannot plausibly exceed 10%, and is probably below 5% (similar to most forecasts for nuclear warfare).
The mistake doomers are making is inferring the likelihood of extreme scenarios from capabilities. Yes, Hugging Face is consistent with a scheming paperclip maximiser. It's also consistent with more boring principal-agent misalignments that standard economics might predict. Capabilities trends can be extrapolated reasonably accurately from Moore's Law which tends to be invariant, yet doom scenarios rely on complex equilibrium interactions amongst individuals, firms, institutions, and agents. Ignore the Lucas Critique at your peril!
All the LW or EA scenarios consider extreme tail-risks, where there is high variation in forecasting accuracy. Neither microfounded theory nor superforecasters (even the Tetlock study leaves a lot of unexplained variation in performance - I call this “luck”, otherwise why aren't the hedge funds or Jane St rushing to hire them?) are that good at predicting these. Therefore for now, I restrict my analysis to the first moment. Note that extreme tail risks (extinction) have been ubiquitous throughout all of human history. We survived the Dark Ages. Shouldn't Bayesians update on this?
Doomers are right that real interest rates are high relative to historical averages. Fiscal policy and a massive CAPEX boom into data centres also push up real rates. In Halperin et al, high real rates are also a sign of AI optimism. Equities performance suggest markets are primarily pricing-in the optimistic scenario.