AI agents that can make decisions and carry out transactions independently could soon operate across the financial system, the Bank of England has warned, adding that relying on human oversight might no longer be realistic.

Bank of England Deputy Governor Sarah Breeden made the warning at the European Central Bank’s annual forum in Sintra on Tuesday, as leading developers shift from generative AI chatbots to agentic systems.

Unlike generative AI tools such as ChatGPT, Gemini, and Claude, which generate text, images, and code in response to prompts, agentic AI is designed to complete multi-step tasks with limited human intervention.

Rather than merely recommending an investment or suggesting a purchase, an AI agent could execute trades, make payments, buy goods, and book services without requiring human approval for every action.

Breeden said the financial system is likely to “evolve into one that operates more autonomously, at scale and speed.” 

As AI agents become more capable, “relying on a human in the loop for all agent actions is unlikely to be realistic,” she said, arguing that the current regulatory frameworks “were not built to contemplate” this scenario.

Breeden warned that AI agents trained on similar data and responding to the same market signals could exhibit herd behavior, making identical trading decisions simultaneously and amplifying market volatility during periods of stress.

Regulators should examine “whether guardrails are needed, analogous to circuit breakers or kill switches that would limit or stop trading market-wide if faulty AI models cause market meltdown,” Breeden said.

She also urged central banks to prepare for more frequent “technology surprises,” saying recent advances in AI have already exceeded policymakers’ expectations.

Her remarks come as governments increasingly treat AI as a national security issue. Washington has recently intervened in the rollout of some advanced AI models over cybersecurity concerns, while AI developers continue to push ahead with agentic systems. Anthropic unveiled Claude Sonnet 5 this week, its latest model designed to perform complex tasks on users’ behalf.

The push toward agentic AI has also reached the Pentagon. Last week, it unveiled AI agents designed to continuously analyze intelligence and rapidly generate targeting options for commanders, while stressing that humans will retain final decision-making authority.