Philosopher Argues 97% AI Extinction Risk Is Mathematically Worth It
Nick Bostrom argues that current AI risks are worth taking if they extend human life by centuries, despite high extinction odds.
Key points
- Nick Bostrom calculates that a 97% extinction risk is acceptable if AI extends life to 1,400 years.
- The philosopher argues that current daily death tolls from aging justify taking high technological risks.
- Bostrom recommends a short pause of months or years to improve safety before full AI deployment.
Philosopher Nick Bostrom argues in Phoenix that building superintelligent AI is worth the risk. He suggests the potential life extension outweighs the danger of extinction for people alive today.
Bostrom, known for his 2014 book on AI dangers, presented this view at the Generation AI conference. His new working paper focuses strictly on maximizing life expectancy for the current human population.
High risks balanced by medical gains
Bostrom acknowledges that the risk of AI catastrophe is severe. He notes that some experts estimate the chance of disaster at over 10 percent. He believes the true figure might be even higher.
However, the status quo also claims lives. About 170,000 people die daily from disease and aging. Bostrom argues that superintelligence could accelerate medical research and potentially end aging itself.
Math behind the 97 percent threshold
His model assumes the average person has 40 years left to live. If AI reduces mortality to that of healthy young adults, lifespan could reach 1,400 years. This massive gain creates a high break-even point for risk.
Under these assumptions, building AI immediately is beneficial as long as the extinction risk stays below 97 percent. This is a mathematical limit, not a prediction of actual probability. If the life extension is smaller, the acceptable risk drops significantly.
Need for a short strategic pause
Bostrom suggests a brief pause in development to improve safety. This wait should last months to a few years, not centuries. This allows time to test the system before full rollout.
He warns that a poorly designed pause could backfire. It might push development to less careful countries or military programs. The paper advises moving quickly toward the technology, then pausing briefly to ensure safety.
