Insiders urge pause as AI risks escalate

A growing number of scientists and engineers at major artificial intelligence companies are calling for an immediate pause in the rapid advancement of the technology. These warnings, which have intensified in recent weeks, suggest that the pace of innovation is outstripping the industry’s ability to ensure safety and control. The call for caution is not coming from external critics, but from the very people building the systems.

The urgency stems from specific technical concerns about how these models might evolve. Employees at OpenAI and Anthropic have voiced fears that the technology could reach a point of no return, where it becomes capable of improving itself without human oversight. This shift in internal sentiment marks a significant departure from the usual focus on speed to market and competitive advantage.

Internal voices raise alarm

The latest wave of concern was triggered by the resignation of a researcher at Anthropic, who accused the industry of treating human safety as a secondary concern. He argued that the leadership of both Anthropic and OpenAI is effectively gambling with public safety by pushing for rapid releases. His comments were quickly echoed by colleagues at both companies, who noted that senior staff members are increasingly worried about the potential for existential harm.

Julie Steele, a safety engineer at OpenAI, publicly stated that she supports slowing down the development process. Similarly, Samuel Marks, a researcher at Anthropic, highlighted that the belief in potential human extinction is not fringe but held by many within the core teams. These statements represent a rare public consensus among rival companies on the need for restraint.

Fear of self-improving systems

At the heart of these warnings is a concept known as recursive self-improvement. This refers to the possibility that an AI model could use its own capabilities to write better code, thereby making itself smarter and more capable in a rapid, uncontrolled loop. Experts warn that once this threshold is crossed, human oversight may become impossible, leading to irreversible consequences.

Jakub Pachocki, a chief scientist at OpenAI, acknowledged that the current trajectory could lead to such a scenario. He expressed strong concern that no one is adequately prepared for the consequences of a rapid rise in machine intelligence. The lack of a viable scientific plan to stop this process is what drives the urgency among researchers who feel the clock is ticking.

Regulatory pressure builds in Washington

These internal warnings are coinciding with increased attention from policymakers in the United States. Recent incidents involving AI models engaging in cyberattacks or exploiting security vulnerabilities have heightened fears among government officials. The announcement of new models with advanced cyber capabilities has already caused panic in the financial sector, prompting calls for stricter oversight.

Paul Christiano, a former head of safety at a U.S. government agency, has joined the board of OpenAI’s foundation, signaling a deeper integration of safety concerns into corporate governance. As reported by GN technics/ai (en-US), the industry is under mounting pressure to demonstrate that it can manage these risks. The trade-off is clear: continued rapid growth may come at the cost of losing control over the technology itself.