AI Insiders Warn of Existential Risk by 2030

Jacob Coxon, a 27-year-old researcher who recently left Anthropic, has sparked a significant debate within the tech industry by stating that his former colleagues genuinely believe artificial intelligence could become a threat to human survival by the end of this decade. His resignation post, which gained traction after being endorsed by a senior colleague, highlights a growing disconnect between public optimism about AI benefits and the private anxieties of those working on the technology.

This concern is not isolated to one company. It follows recent warnings from OpenAI’s chief scientist, who noted that AI capabilities are advancing faster than the industry’s ability to monitor and control them. As more employees voice these fears, the narrative is shifting from theoretical speculation to a pressing operational reality for major tech firms.

Internal Fears of Losing Control

Coxon’s comments were amplified by Evan Hubinger, a senior employee at Anthropic, who agreed that the company’s intentions are good but its strategies may be insufficient. Hubinger pointed out that the firm lacks a concrete plan to prevent the emergence of superintelligence, a state where AI systems exceed human capacity for oversight and direction.

While Hubinger later clarified that current AI models do not pose an immediate danger of gaining such power, the underlying risk assessment remains unchanged. The core issue is the speed of development; even if present systems are safe, the trajectory suggests that safety measures may not scale quickly enough to keep pace with increasing capability.

Call for Government Regulation

These internal warnings align with broader industry trends. In July, approximately 1,400 employees from various AI companies signed an open letter urging the U.S. government to intervene. They argued that voluntary self-regulation by big tech is insufficient and that external legal frameworks are necessary to slow the pace of development and ensure safety standards are met.

The push for regulation represents a significant shift in how the industry views its relationship with the public and the state. Rather than relying solely on internal ethical guidelines, many researchers now believe that only government oversight can enforce the necessary checks and balances to prevent catastrophic outcomes.

The Trade-off Between Speed and Safety

According to reporting from GN technics/ai (en-US), the central tension in this debate is the race for economic advantage versus the need for rigorous safety testing. Companies face immense pressure to release powerful models quickly to maintain market share, which often conflicts with the time required to thoroughly vet these systems for potential risks.

For readers, the stakes are high: if the industry prioritizes speed over safety, the consequences could be irreversible. The recent resignations and public statements serve as a warning that the technology may be moving faster than our collective ability to understand and manage it, creating a gap that could widen by the end of the decade.