Researchers argue AI existential-risk probabilities are too unreliable for policymaking

Arvind Narayanan and Sayash Kapoor argue that public policy should not rely on quantified estimates of AI existential risk—often referred to as p(doom)—because these figures are methodologically unsound. They contend that such probabilities are frequently presented as authoritative without adequate justification, effectively laundering speculative fears through pseudo-quantification.

The authors examine three common approaches to estimating AI existential risk—inductive, deductive, and subjective—and find each method wanting. Inductive estimates, which draw on past observations to predict future risks, fail because there is no suitable reference class for an event as unprecedented as human extinction from AI. Unlike car accidents or geopolitical shifts, AI-driven extinction lacks historical analogs, making inductive reasoning unreliable.

They further argue that deductive and subjective methods also depend on unprovable assumptions about the world, which cannot be validated empirically. Even when forecasters intend no deception, the effect of presenting such probabilities as policy-relevant numbers is misleading, especially when governments consider costly interventions like restricting open releases of AI models.

Narayanan and Kapoor emphasize that probabilities alone carry no inherent authority; their legitimacy in policymaking depends on transparent, defensible methods. They warn that acting on unverified risk estimates risks undermining public trust, particularly when policies impose uneven costs on stakeholders who may reasonably reject the underlying assumptions.

The authors clarify that they do not oppose AI existential-risk forecasting as an academic exercise or for private decision-making. Their critique is directed specifically at the use of such estimates in public policy, where justification and empirical grounding are essential for legitimacy.