Former AI Researcher Warns of Existential Risk
A high-profile departure from a leading AI lab has reignited debate over the safety of advanced artificial intelligence systems.
Jacob Coxon, a former researcher at Anthropic, has publicly stated that the company is pursuing technology that could lead to the extinction of humanity. His resignation, which became a viral sensation within hours, marks a significant escalation in internal dissent regarding the pace of AI development.
Coxon argues that the industry is prioritizing speed over safety, specifically targeting the creation of self-improving superintelligence. This claim is not isolated; it aligns with warnings from other insiders who describe the current trajectory as a dangerous gamble with human lives.
The Core Fear of Self-Improvement
The central concern raised by Coxon is the concept of recursive self-improvement. This refers to an AI system that can modify its own code to become more intelligent, potentially leading to a rapid and uncontrollable evolution. While experts note this capability does not yet exist, Coxon contends that major AI labs are actively working toward it.
He described the situation as a race where executives and researchers are acting with a sense of urgency that outpaces safety protocols. According to Coxon, the belief that AI could become an existential threat is not a marketing tactic but a genuine fear shared by those on the front lines of research.
Insiders Echo the Alarm
Coxon’s warnings have been corroborated by other figures in the AI community. Jeffrey Ladish, another former employee, recently compared the industry to a car without brakes. He questioned whether humans should be in control of their destiny or if they are ceding that control to algorithms.
Evan Hubinger, who currently works at Anthropic on AI alignment, also validated Coxon’s concerns. In a statement, Hubinger acknowledged that the company’s staff earnestly believe that AI poses a lethal risk to humanity if not properly managed. This internal validation suggests that the fears are widespread among technical staff, not just external critics.
The Trade-Off Between Speed and Safety
The controversy highlights a fundamental tension in the AI industry: the pressure to compete for market dominance versus the need for rigorous safety testing. Critics argue that the competitive landscape forces companies to rush products to market, leaving little room for the deep ethical and technical safeguards required to prevent catastrophic failures.
As reported by GN technics/ai (en-US), this debate is no longer confined to academic circles. It is now a mainstream concern, with public figures and former employees using social media to warn of the potential consequences. The stakes are high, as the decisions made now may determine whether AI remains a tool under human control or evolves into an autonomous force.
