US and China Draft Principles for Human Control of Military AI

Security experts from the United States and China have outlined a set of joint principles aimed at preventing artificial intelligence from independently making decisions regarding nuclear weapons. The core of the proposal is to extend existing human-control norms, traditionally applied to nuclear arsenals, to the broader domain of military artificial intelligence, ensuring that critical strategic decisions remain firmly in human hands.

According to a report detailed by GN geopolitics/nuclear (en-US), the initiative stems from a collaboration between the Brookings Institution and Fudan University. The researchers argue that as AI systems become more capable and faster, the risk of unintended escalation increases, necessitating a clear boundary where machine autonomy ends and human responsibility begins.

Defining the Boundary of Autonomy

The primary focus of the proposed framework is the protection of Nuclear Command, Control, and Communications systems, often referred to as NC3. These systems are vital for threat detection, decision-making, and the transmission of launch orders. The report emphasizes that decisions to launch cyberattacks targeting these critical infrastructure components must be made exclusively by humans, not by automated systems.

Researchers note that any failure in the NC3 chain can lead to catastrophic outcomes, including misidentification of attacks or unauthorized launches. By mandating human oversight, the proposal seeks to mitigate risks arising from AI unpredictability, human error, or malicious actors. This approach is designed to reduce the chance of accidental or pre-emptive strikes that could have strategic consequences.

Addressing the Speed of Modern Conflict

A significant concern driving this proposal is the velocity at which AI can process information and execute actions. In the military sphere, there is growing anxiety that automated, millisecond-level decisions could turn a false alarm or a minor cyber incident into a full-scale nuclear crisis. The report warns that leaders may have insufficient time to intervene if systems operate beyond human reaction speeds.

To manage this risk, the researchers suggest establishing a dedicated military AI hotline between Washington and Beijing. Additionally, they propose creating a common definition of human control to ensure both nations interpret the requirement for human oversight in a consistent manner. This shared language is intended to prevent misunderstandings that could otherwise lead to rapid escalation.

Industry Calls for Restraint

The push for human control aligns with broader industry movements to moderate the pace of AI development. Earlier this month, Anthropic’s CEO Dario Amodei urged developers to slow down the performance race, a stance supported by other tech leaders. This sentiment reflects a growing recognition that the current trajectory of rapid advancement may outpace the development of necessary safety and governance structures.

Recent events have underscored the tension between technological capability and regulatory oversight. For instance, Anthropic recently rejected a U.S. Defense Department request to lift restrictions on autonomous weapons, leading to a legal dispute that highlighted the complexities of integrating advanced AI into military operations. The report suggests that governance of military AI is not a negotiation that can wait, but a race against time to establish minimum rules.