AI Researchers Warn of Imminent Danger and Call for Global Slowdown
A former Anthropic employee claims staff are genuinely frightened by the pace of AI development, sparking a debate on whether the industry needs coordinated international brakes.
Jacob Coxon, a 27-year-old researcher who recently left Anthropic, has issued a stark warning to the public. In an interview with BBC News, he stated that people working inside the AI sector are "genuinely frightened" about how quickly the technology is evolving. Coxon argued that if the current rate of progress continues without intervention, there is a strong chance of catastrophic outcomes for humanity in the near future.
His resignation post, which gained significant attention online, aligns with growing unease within the industry. Dario Amodei, the head of Anthropic, published an essay on Saturday arguing that development must be slowed down. While Coxon welcomed this call for caution, he emphasized that unilateral actions by US companies are insufficient. He argued that a coordinated slowdown involving China is necessary to prevent an international race to the bottom in safety standards.
The risk of autonomous bot swarms
Coxon identified a specific scenario that concerns him most: the emergence of a swarm of bots acting like a supercomputer that could take over the internet. He suggested this is not a distant possibility but a realistic threat within six months to a year. This view contrasts sharply with some industry leaders who dismiss such fears as exaggeration, highlighting a deep divide in how perceived risks are assessed.
Industry leaders dismiss existential fears
Not everyone in Silicon Valley shares Coxon’s urgency. Jensen Huang, the CEO of Nvidia, has publicly dismissed the idea that AI will end humanity as "complete nonsense." Similarly, Clement Delangue, CEO of Hugging Face, compared asking a junior researcher about extinction risks to asking an air conditioning technician about climate change. Delangue suggested that while the concerns are not necessarily wrong, they are often out of perspective with the broader technical reality.
Critics also point to potential conflicts of interest in these warnings. Since Anthropic and OpenAI are reportedly preparing for major initial public offerings, some observers suspect that emphasizing danger is a strategy to generate hype or justify regulatory barriers that could limit competition. This skepticism adds a layer of commercial complexity to what is ostensibly a safety debate.
Proposals for global regulatory frameworks
Despite the internal divisions, there is a push for structured oversight. Amodei’s essay proposed a three-point plan including independent monitoring of AI models, industry-wide regulation, and global regulatory cooperation. Anthropic maintains that it has always been transparent about the risks and benefits of AI, citing its work on safety frameworks and model testing. However, the company acknowledges that the world would benefit from a lawful, verifiable way to pace the release of powerful models.
The tension between rapid commercial growth and safety concerns remains unresolved. As companies race to monetize their advancements, the call for external checks and international coordination grows louder. The outcome of this debate will likely shape the regulatory landscape for artificial intelligence in the coming years, determining whether innovation is prioritized over precaution.
