Anthropic Research Raises AI Botnet Scenario With Significant Caveats

A new Anthropic study demonstrates controlled elements of AI self-replication across IoT hardware, but the threat remains theoretical and hedged with important assumptions.

AI-generated illustration depicting ai security for the story: Anthropic Research Raises AI Botnet Scenario With Significant Caveats

Summary

  • Anthropic published research showing a 9-billion-parameter model can run inference distributed across multiple cheap IoT devices, and that GLM-5.2 successfully propagated Llama3.1-8B across eight controlled cloud nodes in four of five trials.
  • The experiments used custom-built, deliberately open nodes and a purpose-built harness called Memento — real-world exploitation was explicitly excluded from the study scope.
  • Inference speed achieved was 0.39 tokens per second; by comparison, the Mirai botnet doubled in size roughly every hour in 2016, while Anthropic estimates an AI worm might take days to double.
  • Only one model, GLM-5.2, achieved end-to-end replication; all other open-weight models tested failed reliably.
  • Basic IoT security hygiene — closing common ingress routes — would substantially raise the bar against any such threat, AI-powered or not.

What Anthropic Actually Demonstrated

Anthropic published a research report this week claiming that AI self-replicating worms could potentially spread across vulnerable IoT devices within months. The headline findings sound alarming, but the underlying methodology warrants careful reading before security teams reprioritise their roadmaps.

The researchers ran two distinct experiments. In the first, they successfully distributed a Gemma4-26B-A4B model across 15 generic Android set-top streaming boxes — the kind of cheap, lightly secured hardware that conventional botnets such as Mirai have long targeted. The distributed model was able to perform inference tasks, though at just 0.39 tokens per second. This demonstrated that large models can be layered across inexpensive commodity hardware, which is a legitimate technical observation.

The second experiment tested self-replication. Using Z.ai’s GLM-5.2, researchers propagated a copy of Llama3.1-8B across eight Google Cloud instances running an Android 15 emulator inside a virtual private cloud. GLM-5.2 succeeded in spreading to all eight nodes in four of five trials, and seven of eight in the remaining trial. Anthropic described this as evidence that small models could, in principle, complete the network enumeration, payload delivery, and inference setup steps of a replication cycle.

Where the Research Stops Short

Several constraints significantly limit how far these findings can be extrapolated. Most notably, Anthropic explicitly excluded the exploitation component — the part where a worm actually compromises a device — from its study. The researchers instead used nodes custom-built for the project and left deliberately open to attack. Anthropic acknowledged this provided ‘an upper bound on real-world success,’ not a realistic floor.

The replication harness itself, called Memento, was also purpose-built for this research. The scenario therefore tests whether self-replication is theoretically achievable under favourable conditions, not whether it is practically deployable in the wild today.

GLM-5.2’s success also rests on a layered assumption: that its capabilities could be packed into a Llama3.1-8B-sized model. Anthropic tested other open-weight models for replication capability, and by its own account they all failed reliably. The report does not disclose how many models were tested. Anthropic also concedes it does not know ‘exactly what form autonomous AI replication will take,’ or when it might arise — if ever.

Speed and Detectability

Anthropic warns that an AI worm would be difficult to shut down because it lacks a centralised kill switch. That is a fair point in principle. However, the propagation profile of an AI model is very different from that of conventional malware. AI models are large, bandwidth-hungry payloads. Anthropic itself estimates such a worm might take days to double in size. Mirai, by contrast, doubled roughly every hour when it emerged in 2016. The traffic generated by a growing AI botnet would be substantially more visible to network monitoring tools than a compact piece of traditional malware, potentially making infected devices easier to identify and isolate.

Context on Frontier Lab Research

It is worth noting the broader pattern here. Frontier AI laboratories have developed a habit of publishing research that identifies future risks of a kind that, conveniently, only continued investment in AI safety — including their own — can address. That does not make the underlying technical observations wrong, but it does mean security leaders should read these reports with the same scepticism they would apply to any vendor-originated threat research. The scenario described is plausible as a long-run concern; it is not an imminent operational threat.

Why it matters

CISOs should treat this research as a useful prompt to revisit IoT security fundamentals, not as evidence of an emergent threat requiring immediate escalation. The study confirms that small AI models are approaching the capability threshold where, given the right conditions, they could participate in self-replicating attacks — but those conditions include deliberate exploitation scaffolding that does not exist in the wild today. The more immediate takeaway is that IoT device hygiene, network segmentation, and ingress hardening remain effective countermeasures against both conventional and hypothetical AI-assisted botnets. Security teams that have deferred IoT hardening work now have a research-backed case to make to the board.

What to do now

  • Review and close common IoT ingress routes — default credentials, exposed management interfaces, and unpatched firmware — which remain the primary attack surface for conventional botnets and would equally apply to any AI-assisted variant.
  • Implement network segmentation to isolate IoT and consumer-grade devices from critical infrastructure, limiting lateral movement opportunity regardless of the malware type involved.
  • Establish baseline traffic monitoring on IoT network segments; the large payload sizes associated with AI models would generate anomalous bandwidth patterns that existing tools could detect if baselines are known.
  • Monitor open-weight AI model developments and track any future research that adds real-world exploitation capability to the replication scenarios Anthropic has described.
  • When briefing executives or boards on AI-related risks, distinguish between demonstrated research findings and operational threats — Anthropic’s own report acknowledges the timeline and form of this risk are unknown.

Sources