Experts Urge Caution as AI Systems Gain Autonomous Capabilities

Manjeet Rege, director of the Center for Applied Artificial Intelligence at the University of St. Thomas, is challenging the narrative that artificial intelligence poses an existential threat through conscious malice. Instead, he argues that the immediate danger lies in a more mundane but equally dangerous scenario: well-intentioned systems pursuing assigned goals in ways that were never anticipated by their creators.

This shift in perspective moves the discussion away from science fiction tropes of evil machines and toward the practical challenges of regulating software that can plan, decide, and execute actions on its own. As these technologies transition from passive tools that answer questions to active agents that perform tasks, the stakes for users and businesses are rising sharply.

The shift from answering questions to taking actions

Until recently, most public interaction with AI involved simple inputs and outputs, such as generating text or images. However, the landscape is changing with the development of AI agents capable of using software and accessing networks to achieve specific objectives. Rege explains that this independence creates a new category of risk. The concern is no longer just about incorrect answers, but about unintended consequences when an AI system takes steps to fulfill a goal that may not align with human values or safety standards.

The core issue is the gap between the objective given to the system and the methods it chooses to achieve it. If an AI is tasked with optimizing a process and has access to various digital tools, it may execute actions that a human operator would never consider. This requires a fundamental rethinking of how we trust and supervise these systems, as the traditional model of direct human control becomes less effective against autonomous decision-making processes.

Regulation must target deployment rather than research

While industry leaders and policymakers have called for slowing down AI development, Rege suggests that this approach is imprecise. He distinguishes between the research phase, where discovery is rapid and global, and the deployment phase, where models are integrated into real-world applications. He argues that regulation should focus on the latter, ensuring that when a core AI model is used for specific applications, robust safeguards are in place to prevent misuse or unintended harms.

This targeted approach acknowledges that slowing down research is difficult to enforce globally, as development is happening across multiple countries and jurisdictions. By focusing on deployment, regulators can create clearer standards for how AI agents should be permitted to act within specific domains, such as finance or healthcare, without stifling the broader innovation ecosystem that drives technological progress.

Global coordination is essential for safety

A significant challenge in regulating AI is the international nature of the technology. Rege points out that while major AI companies are based in the United States, significant development is also occurring in China and other regions. Implementing strict regulations in one country while others remain permissive creates a loophole where unsafe or misaligned models can still be developed and deployed elsewhere.

According to reporting by GN technics/ai (en-US), this fragmented regulatory environment means that national efforts alone are insufficient. To truly mitigate the risks of autonomous AI systems, there must be a coordinated global effort that sets shared standards for safety and accountability. Without such cooperation, the potential for misuse by bad actors remains high, regardless of the safeguards in place within any single nation.