The AI Doomsday Scenario Getting Real Funding: Recursive Self-Improvement

A new wave of well-funded startups is aggressively pursuing one of artificial intelligence's most consequential and contested ideas: building AI systems capable of improving themselves without meaningful human input. Two Silicon Valley companies — each valued at $4 billion — are openly chasing this goal, alongside labs like OpenAI and Anthropic. The concept, known as recursive self-improvement (RSI), holds that a sufficiently capable AI could redesign its own architecture, write better training code, and iterate toward superintelligence at speeds no human team could match.

At the center of this race are companies like London-based Inherent, co-founded by former Google researchers Edward Hughes and Louis Kirsch, and San Francisco-based Recursive Superintelligence, co-founded by OpenAI and Google veteran Jeff Clune. Inherent's prototype system, called Faraday — named after 19th-century physicist Michael Faraday — ingests everything the company's researchers do: emails, meeting transcripts, Slack messages, and AI conversations. That data is then used to train a better version of Faraday. Recursive Superintelligence is taking a different approach, using evolutionary computation inspired by Darwinian selection to generate and test entirely new AI architectures, discarding what fails and amplifying what works.

The intellectual lineage of RSI stretches back to the 1956 Dartmouth conference that coined 'artificial intelligence,' and to British mathematician I.J. Good's 1965 prediction of an 'intelligence explosion' — his argument that the first truly self-improving machine would be the last invention humans ever needed to make. Today's optimism is grounded in concrete recent developments: OpenAI and Anthropic both released powerful code-generating systems in late 2025, capable of autonomously writing software for hours or days at a stretch. OpenAI has described its AI as an 'automated research intern.' In 2024, Clune's team released 'The AI Scientist,' a system that independently proposed research directions, generated code to test them, and wrote up findings in academic papers — producing at least one idea later explored independently by Oxford researchers.

But the risks are being taken seriously even by the companies building these tools. Anthropic published a blog post this spring explicitly warning that its RSI work could cause humans to lose control of AI systems, and its CEO cited RSI as a primary reason to consider slowing AI development. Yale economist Jason Abaluck warned publicly that a self-improving model could theoretically disable all rivals while accumulating power. Critics also note the gap between hype and reality: Faraday and similar agents remain nearly useless without constant guidance from senior researchers. OpenAI's chief research officer Mark Chen acknowledged that current models don't generate creative ideas autonomously — the core capability RSI requires. And the Oxford researcher whose work Clune's AI Scientist supposedly anticipated independently pointed out that the concept predated the AI Scientist and was widely known, suggesting the system was pointed toward the idea rather than discovering it.