"Human-in-the-loop" is not enough.
In November 2024, U.S. President Joe Biden and Chinese President Xi Jinping released a joint statement agreeing that humans must always be “in the loop” when artificial intelligence (AI) is involved in the decision to use a nuclear weapon. This was a surprising yet welcome development for arms control advocates; still, many pointed out that the continuity of the agreement into the next U.S. government was uncertain, and that it did not include statements of intent on pursuing some form of binding agreement or expanding it to include more parties.
One year later, the Trump administration is emphasizing its desire for a stable, managed security relationship between the United States and China in the Asia-Pacific. That stability is threatened, in part, by unmanned systems. The two countries have experienced a series of incidents tied to different kinds of unmanned systems in recent years, especially in the South China Sea. As AI-driven autonomous systems become more common, the associated risks will only rise, and there is a distinct lack of bilateral crisis management tools for dealing with incidents involving autonomous systems.
Worryingly, it is also not unthinkable that elements in the U.S. government would consider developing autonomous nuclear-capable systems for deployment in the region. The conventional military balance is tilting in China’s favor, former officials have argued that the United States should consider nuclear first use against a Chinese invasion force bound for Taiwan, and the Congressionally-mandated Strategic Posture Review highlighted autonomous vehicles as a potential boost to U.S. nuclear “effectiveness.”
As President Donald Trump prepares to visit Beijing in May, cabinet and sub-cabinet officials consistently emphasize crisis communication, risk reduction, and stability with China in the security domain, as do strategy documents like the National Defense and National Security Strategies and the China Military Power Report. Formalizing the 2024 agreement, using it as a foundation for a ban on uploading nuclear weapons to autonomous vehicles, and agreeing to establish risk management channels for unmanned systems would help the administration achieve those goals.
Unmanned Systems: Categories and Previous Incidents
Generally, unmanned systems fall into one of three categories, defined by their technical specifications and situational use-cases. The first is long-range remote-controlled systems, from firefighting drones to military reconnaissance vehicles. The second is unmanned systems intended to be carried by wind or currents, like weather observation balloons or underwater buoys.
The third type is autonomous systems driven by artificial intelligence. These systems can independently patrol, collect intelligence and transmit data. In extreme situations, they can even independently decide whether to initiate military aggression. Unmanned systems with these types of military applications are a major point of focus for both tech developers and regulators.
The appeal of these emerging technologies is in large part due to their dual-use nature; they have wide civilian applications like weather observation or resource exploration but can also carry out easily concealable intelligence gathering. In crises or conflicts, many can also easily be modified into weapons systems.
In August 2025, China’s unmanned system mothership Zhuhai Yun arrived in the waters around Huangyan Island, working in coordination with the scientific research vessel Xiangyanghong 10 to conduct investigations on marine resources and the environment. The various unmanned vessels, submersibles and aerial vehicles it carried are representative of current trends in both the civilian and military application of autonomous tech: in different ways, both are used to solidify claims to maritime sovereignty.
This difficulty in distinguishing between military and civilian activities also puts unmanned systems in a gray area when it comes to international rules and regulations. There is a general lack of international or bilateral mechanisms for dealing with incidents, which happen relatively frequently off China’s southern coast.
In 2010, a fisherman discovered a “strange torpedo” off the coast of Hainan which turned out to be a line-controlled robot; in August 2015, a fisherman off the coast of Hainan recovered a UUV capable of underwater photography, optical fiber transmission and satellite communication capabilities. In December 2016, a Chinese navy lifeboat discovered an unmanned underwater vehicle (UUV) in the South China Sea. After salvaging and inspecting the device, the Chinese concluded that it was an American UUV, later returning it to the U.S. Navy.
A 2017 article from Chinese state media argued that because of the importance of the South China Sea to China’s submarine force, U.S. intelligence collection in the area poses a significant threat to Chinese national security. Such activities, the article said, are reminiscent of Japanese hydrological intelligence collection during World War II; both are intended to gain wartime advantages.
The “Balloon Incident” of February 2023 was the most serious autonomous system-related incident between the U.S. and China, causing a full-blown diplomatic crisis. A Chinese high-altitude balloon was blown off course by western winds, entering airspace off the Aleutian Islands on January 28 before continuing on to Alaska and Montana. China’s Ministry of Foreign Affairs clarified to the United States that the balloon only had civilian applications and no military purpose, but domestic U.S. outrage quickly caused the situation to spiral. While the U.S. Department of Defense initially stated that the balloon posed no security threat, the U.S. government soon began referring to it as a “Chinese spy balloon,” later suggesting that it was targeting strategic sites for intelligence collection before clarifying that it did not transmit any information back to China. On February 4, the U.S. military dispatched F-22s to shoot the balloon down off the coast of South Carolina.
“The Balloon Incident” was at its root an accident caused by force majeure, yet it led to the cutoff of many U.S.-China communication channels; China refused to answer a phone call from then-Secretary of Defense Lloyd Austin to Chinese Defense Minister Wei Fenghe after the balloon was shot down, Secretary of State Antony Blinken postponed a visit to Beijing, and it was not until Henry Kissinger’s visit to China several months later that normal meetings and communication began to resume.
Overall, the incident exposed many problems with bilateral crisis management of incidents involving autonomous systems. Firstly, technically speaking, existing defense and early warning systems are likely to have cracks in dealing with unmanned systems, which may lead to concerns about vulnerability to a surprise attack. In a crisis, these kinds of concerns can be a spark leading to uncontrolled escalation. Furthermore, unmanned systems, especially those that are not remote-controlled, can easily deviate from pre-intended courses or tasks, which can lead to unexpected consequences. On the legal side, there is also a lack of clarity about how exactly to deal with territorial infringement caused by accidental course deviations. Finally, on the diplomatic and inter-governmental side, there is a clear lack of communication channels and confidence-building measures dedicated to unmanned systems.
Unmanned Weapons and Nuclear Concerns
Misunderstandings emanating from unmanned vehicles, especially in a volatile political climate with underdeveloped risk management channels, carry high risks. Those risks grow exponentially with autonomous AI-driven vehicles, which are becoming more popular. As military use of AI grows more sophisticated, the temptation to upload weapons to fully autonomous platforms that make their own firing decisions is also growing. The recent messy split between the U.S. Department of Defense and AI company Anthropic is proof of that; Anthropic refused to comply with a request to remove its restrictions on building fully autonomous weapons systems, leading the government to ban Anthropic from government use and threaten to designate it a supply chain risk.
Unmanned systems are already a core part of the U.S. military’s strategy to counter China’s geographic and growing military advantage in East Asia. The main example of this is the military’s “Hellscape” tactic, whereby a swarm of low-cost, unmanned weapons would seek to harass and delay a Chinese invasion force of Taiwan, buying time for slower U.S. reinforcements (like aircraft carrier battle groups and amphibious assault ships) to arrive. This tactic is being developed within the Pentagon’s Defense Autonomous Warfare Group, which procures and trains with mainly longer-range drones.
U.S. allies in the Asia-Pacific are also investing in long-range autonomous systems meant to counter China, especially autonomous underwater vehicles (AUVs). American defense technology company Anduril and the Australian Defense Ministry recently announced their partnership on the Ghost Shark AUV submarine, an AI-driven AUV with reconnaissance and strike capabilities at “extremely long” distances from the Australian continent. Anduril describes Ghost Shark as something of a proof of concept that it hopes will push the United States and others to invest in AUVs as well.
To China, these kinds of deployments add to regional pressures and uncertainties. The U.S. military deploying autonomous weapons around the Taiwan Strait may be seen as a precursor to military intervention. Moreover, the Defense Department is pushing ahead with other AI companies after Anthropic refused to work on autonomous weapons systems because current technology is simply not mature enough to be safe. If the U.S. deploys an unsafe autonomous weapon, its independent decision-making features could unexpectedly trigger a military confrontation.
What worries China even more is the possibility that U.S. forward-deployed autonomous systems may eventually carry nuclear weapons.
In 2023, two reports from the Atlantic Council’s Scowcroft Center explored the possibility of the use of nuclear weapons in a Taiwan conflict, with one arguing that United States should consider limited nuclear use against a Chinese invasion force. The report argued that traditional nuclear platforms, being easier to track and relatively slower to respond, lack something in constituting a credible deterrent.
The Strategic Posture Commission, a group of former officials, policymakers and analysts tasked by Congress in 2023 with reviewing the state of U.S. nuclear capabilities, also argued that generative AI and autonomous vehicles “could contribute to the survivability and effectiveness of US nuclear forces” by providing “new opportunities to defend, survive and prevail.” Their report also argued that theater U.S. nuclear posture needed more “militarily effective response options to deter or counter Russian or Chinese nuclear use.”
While U.S.-China relations stabilized somewhat in 2025, the relationship can sour quickly. A future administration may, like the authors of the Scowcroft reports, see China taking Taiwan as a borderline civilizational crisis for the United States. It may also see autonomous weapons carrying tactical nuclear warheads, due to their covert, rapid-response nature, as an effective way to maintain a deterrent in the First and Second Island Chains. Indeed, Anduril’s promotion of Ghost Shark shows that many of the use-cases for the AUV overlap with the arguments for considering nuclear use in a Taiwan conflict, as well as the reasoning behind Hellscape: asymmetric tactics intended to negate Chinese advantages.
The United States may come to see nuclear-capable autonomous systems as strategic risk-taking, cost-effectively maintaining its forward-deployed regional presence, while conveying a stronger albeit riskier deterrent signal. On the other hand, this approach may be seen by China not as deterrence but as coercion, thus leading to strategic instability. Given the numerous uncertainties associated with artificial intelligence, especially the risks of data poisoning and adversarial attacks, the loss of control of autonomous weapon systems may lead to a nuclear crisis.
Policy Recommendations
Historical examples show that the United States and China are vulnerable to tension and incidents resulting from the use of unmanned systems. U.S. and allied ambitions in the Asia-Pacific, coupled with Chinese resistance to containment or encroachment on its territory, raise the risk of incidents. Meanwhile, autonomous, AI-driven systems are becoming increasingly common and will likely be deployed by many actors in the region. Moreover, countering China is a central selling point for these systems. All of these factors highlight the importance of improved crisis management and agreements dedicated specifically to unmanned systems.
On the nuclear side, the United States is uncertain about its conventional capabilities in the region, New START has expired without extension or replacement, and prominent current and former government officials have advocated exploring limited nuclear use in a Taiwan contingency and the use of nuclear weapons with autonomous vehicles. These are all factors that may lead the United States to seriously consider nuclear-capable autonomous systems in the future. This would be extremely destabilizing to the regional stability the United States claims to be seeking.
With all this in mind, bilateral dialogues on strategic stability should take the following steps.
First, China and the United States should reiterate the 2024 Lima agreement on maintaining human control of nuclear weapons and strengthen it by formally agreeing to a ban on uploading nuclear weapons to autonomous systems. This understanding is key to maintaining strategic stability; as the U.S.-China relationship continues to undergo change, uncertainties and challenges, the consensus should be reinforced as much as possible. Considering the U.S. Defense Department’s current interest in fully autonomous weapons, the U.S. is unlikely to agree to restrictions on conventional systems. But a formal agreement on banning nuclear-capable autonomous systems would enhance strategic stability and hopefully serve as a vital trust-building block
The two sides should also establish crisis management systems and practices focused on unmanned systems. One possibility is using the China-U.S. Maritime Military Security Consultation Mechanism (MMCA) as a model for a similar system dealing specifically with these kinds of accidental encounters. This would establish concrete procedures for how to handle unmanned systems crossing borders during peacetime, set procedures for the transfer and return of damaged unmanned systems, and reduce the likelihood of governmental responses being swayed by political sentiment.
Finally, both sides can take the consensus on maintaining human control of nuclear weapons further by promoting it in multilateral settings. Creating a multilateral system involving both state and non-state actors, one that focuses on enhancing transparency and building trust, can help build guardrails around the military use of autonomous systems.
| # | Наименование новости | Тональность | Информативность | Дата публикации |
|---|---|---|---|---|
| 1 | The ‘harvest’ China wants is one India cannot afford | -2 | 6 | 02-06-2026 |
| 2 | China Is Indispensable, But Not Dominant | 0 | 5 | 17-06-2026 |
| 3 | Rubio’s remarks and the limits of strategic empathy | 0 | 5 | 18-06-2026 |
| 4 | I Met With China’s Top AI Experts. They’re Freaking Out, Too | -2 | 6 | 24-06-2026 |
| 5 | What the West Misses About China’s Nuclear Build-up | 0 | 5 | 27-03-2026 |
| 6 | Opinion: It's madness not to annex Greenland | 2 | 6 | 09-01-2026 |
| 7 | The Trump-Xi Summit Produced Stability, But It Won’t Last Forever | 0 | 5 | 02-06-2026 |
| 8 | The Staged Death of China’s Military-Civil Fusion | 0 | 5 | 24-04-2026 |
| 9 | The Mirage of China’s Military Edge | 0 | 2 | 23-06-2026 |
| 10 | Постпред РФ: идея США о трехсторонних переговорах с Россией и КНР по СНВ не имеет шансов | 0 | 0 | 18-11-2019 |