Anthropic Blocks Scientists From Using AI For Bio Threats
A new report reveals that researchers have used AI models to probe capabilities relevant to biological weapons, prompting immediate account bans and stricter safety controls.
Anthropic has disclosed that it blocked several scientists who used its Claude AI models in ways that could support the development of biological weapons. The company shared these details in a detailed report outlining various high-risk activities, including surveillance, fraud, and conventional weapons research. These incidents represent the most notable and novel threats identified by the firm so far, involving a mix of criminal entities, state-sponsored groups, and other organized actors.
The report highlights five specific cases where users attempted to leverage AI for biological misuse. Anthropic describes this as one of the most serious risks associated with advanced AI systems. However, the company cautions that while these cases demonstrate the models' capabilities, they do not prove that such tools have actually been used to create real-world biological weapons. The individuals involved are working scientists whose identities were withheld, and Anthropic stated it cannot assert that they intended to cause harm.
Balancing Research Utility And Safety
The core challenge lies in the dual-use nature of biological knowledge. The same information that could be used to develop a harmful agent might also be essential for creating vaccines or cures for diseases. To manage this trade-off, Anthropic notes that recent models like Claude Fable 5 include stronger safeguards. These updates restrict access to a wide range of dual-use biological research queries, aiming to prevent malicious use while still allowing for legitimate scientific exploration.
Upon detecting these risky behaviors, the company banned the users' accounts. It also integrated the findings into its broader threat intelligence processes. This approach is designed to better prevent, detect, and disrupt such activities in the future. The company emphasizes that its goal is to maintain robust safety measures without completely shutting down the beneficial applications of AI in scientific research.
AI Tools In State Operations
Beyond biological threats, Anthropic identified accounts using Claude for influence operations and surveillance. The report details three Iranian state-aligned accounts that used the AI to shape public opinion. These actors built content to appear as if it came from independent news sources and distributed it across platforms like X, Instagram, and TikTok. One such actor also used the model to develop targeting recommendations against U.S. naval forces and to design software for a domestic mass-surveillance platform.
Surveillance And Repression In Asia
The report also highlights cases in China and West Africa where AI was used to build and run surveillance operations. In China, three accounts aligned with municipal security services used Claude for transnational repression. One bureau was found profiling overseas activists and organizations. Anthropic noted that in some instances, AI was being used to replace parts of an engineering workforce, allowing these operations to scale more efficiently than before.
According to GN technics/ai (en-US), these findings underscore the complex landscape of AI misuse. While the technology offers significant benefits, it also provides powerful tools for state actors and other groups to conduct surveillance and influence campaigns. The company’s response reflects a growing industry effort to implement stricter controls and monitoring to mitigate these emerging security risks.
