Anthropic Calls for Global AI Slowdown, Warning of Recursive Self-Improvement Risk

Anthropic, currently valued at nearly $1 trillion, is calling on top AI labs to consider slowing or temporarily pausing frontier AI development — a striking move from a company that is simultaneously one of the industry's fastest-growing players. In a blog post authored by Anthropic Institute head Marina Favaro and co-founder Jack Clark, the company warned that AI systems are advancing rapidly toward 'recursive self-improvement,' a threshold where models upgrade themselves without human intervention. Clark has said he believes this milestone could arrive within two years, possibly sooner.

The post proposes a global agreement to coordinate any slowdown and a verification regime to ensure compliance — explicitly acknowledging the difficulty of enforcement. The comparison drawn is to nuclear arms treaties, but with an admission that AI training runs are far harder to detect than missile silos, and that whoever keeps developing while others pause stands to inherit a commanding lead. The company plans to convene policymakers, researchers, and others in coming months to flesh out what such a system might look like.

The announcement lands amid a moment of remarkable commercial momentum for Anthropic. Its annualized revenue run rate is on pace to hit $50 billion by the end of June 2026, up from $9 billion at the end of 2025. The company recently filed confidential IPO paperwork and is in a neck-and-neck race with OpenAI for AI dominance — making its safety-first posture simultaneously principled and commercially loaded.

Critics are skeptical. David Sacks, venture capitalist and informal Trump adviser, has accused Anthropic of running a 'regulatory capture agenda' designed to kneecap cheaper open-source competitors. Others point to Anthropic's decision to restrict access to its powerful 'Mythos' cybersecurity model — capable of finding software vulnerabilities — as a dual-use move that doubles as product marketing. Wharton professor Ethan Mollick offers a more nuanced take, describing Anthropic as a company housing genuine safety idealists alongside standard corporate machinery, with real internal tension between the two.

The broader AI field remains divided on whether current large-language model architectures can even reach human-level intelligence. Meta's Yann LeCun has long argued they cannot, comparing current AI to cat-level cognition. CEO Dario Amodei sits at the opposite pole, warning that advanced AI could eliminate up to half of entry-level white-collar jobs and that models trained on science-fiction rebellion narratives could develop genuinely rebellious tendencies. Whether Anthropic's call for a pause represents existential responsibility or competitive strategy — or both — the debate is now squarely on the table.