Anthropic Hires First Outside Evaluator to Test AI Safety

Anthropic has formally selected Accenture as its first embedded evaluator, marking the initial practical step toward implementing a proposal to slow down the pace of artificial intelligence development. This move responds to growing concerns from researchers who have warned about the potential for catastrophic harm from increasingly capable AI systems. The partnership represents a concrete commitment to third-party oversight, a concept that has been debated within the tech sector for some time but rarely executed in such a structured manner.

According to a release cited by GN technics/ai (en-US), both companies have agreed to invest at least one billion dollars over the next five years to build capacity in this area. However, Anthropic stated that it will directly fund Accenture’s work initially due to the urgency of the situation. The company noted that while it prefers funding to eventually come from pooled or government sources, no such mechanisms currently exist, forcing it to rely on private arrangements for now.

Independent oversight enters the mix

The core of this arrangement involves embedding employees from Accenture’s specialist AI business, known as Faculty, into Anthropic’s operations. These evaluators will have employee-level access to verify safety practices, report incidents, and conduct rigorous testing of the models. Their specific tasks include red-teaming models to identify weaknesses and assessing whether the AI behaves in line with human values. This level of access goes beyond standard audits, allowing for a continuous and deep-dive review of the technology’s behavior.

This initiative is not exclusive to Accenture. Anthropic stated that it is also in discussions with the research nonprofit METR and other third parties. The company emphasized that it remains fully responsible for the safety of its models and that working with external evaluators does not reduce its accountability. By sharing these early efforts, Anthropic aims to demonstrate its process to the public and other AI developers, signaling a willingness to adapt as the field matures.

Industry reaction remains divided

The proposal to slow down AI development, published by CEO Dario Amodei, has received a mixed response from industry leaders. Figures such as OpenAI CEO Sam Altman and Tesla CEO Elon Musk have expressed support for tempering the pace of improvement. In contrast, Nvidia CEO Jensen Huang has dismissed the need for new regulation, arguing that current concerns are overstated. This divergence highlights a fundamental disagreement within the tech industry about how to manage the risks associated with advanced artificial intelligence.

Skeptics have also questioned the practical implications of these safety measures, especially given that Anthropic is preparing for what is widely expected to be a significant initial public offering. Critics argue that while the intention is noble, the effectiveness of such self-imposed slowdowns is uncertain. The company has acknowledged that its approach will evolve, promising to share more details as the work begins and additional evaluators are brought on board.

Balancing innovation and safety

The decision to bring in external evaluators reflects a broader tension in the AI industry between rapid innovation and responsible development. By allowing third parties to inspect its processes, Anthropic is attempting to address the trust deficit that has emerged among researchers and the public. This move could set a precedent for other AI companies, potentially leading to a new standard for safety verification in the sector.

Ultimately, the success of this initiative will depend on the depth of the access granted to evaluators and the willingness of the company to act on their findings. As the AI landscape continues to evolve, the balance between speed and safety will remain a critical issue. Anthropic’s partnership with Accenture is a notable step in this direction, offering a glimpse into how major tech companies are beginning to formalize their approach to AI governance.