Researchers Warn of Accelerating AI Risks
Former AI lab employees are raising alarms about the speed of development, suggesting that safety measures may not keep pace with rapidly evolving capabilities.
A former researcher at two leading artificial intelligence companies has issued a stark warning about the trajectory of the technology. Jacob Coxon, who recently left Anthropic after working at OpenAI, argues that the race to build superintelligence is moving faster than society can manage. His concern is not that machines will become malicious, but that they may become too powerful to control.
Coxon suggests that some insiders at major AI firms believe these systems could pose a significant threat to humanity before the end of this decade. While this is not a consensus view among all scientists, it represents a serious internal debate. The core issue is the potential for AI to outpace human oversight and regulatory frameworks, creating a gap in safety that could have severe consequences.
The Danger Is Not Robots
Public imagination often pictures humanoid robots turning against their creators, but that is not the primary concern of technical experts. The real risk lies in the software itself. Modern AI tools have evolved from simple question-and-answer interfaces into agents that can pursue complex goals autonomously. While these systems currently make frequent errors, their ability to plan and execute tasks is improving rapidly.
Recursive Improvement Could Accelerate
One of the most cited fears is recursive self-improvement. This theoretical scenario involves AI systems helping to design and build more capable versions of themselves. If this cycle were to accelerate, the growth in AI capability could outstrip the ability of governments, regulators, and even the companies creating the technology to respond. This speed could leave little time for humans to implement necessary safeguards or correct course.
Misalignment and Critical Infrastructure
The danger does not require AI to be conscious or evil. The risk is that a powerful system pursues a goal in unintended ways. If an AI can act independently across networks, it might discover software vulnerabilities faster than humans can repair them. This could endanger critical infrastructure such as power grids and financial systems. Additionally, there is a concern that humans could use these tools for cyberattacks or weapons development, amplifying existing threats.
Uncertainty About the Timeline
The reference to the year 2030 is not a scientific deadline but a projection based on current trends. Opinions vary widely on how quickly superintelligence might arrive. Some experts believe it could happen sooner than expected, while others argue these predictions are overly aggressive. As reported by GN technics/ai (en-US), this uncertainty is itself a major challenge, making it difficult to prepare for risks that are plausible but not yet certain.
