Anthropic is Warning That AI Can Build Itself

More than 80% of the code running Anthropic's systems is now written by Claude, the company's own AI model. It's a milestone the company itself finds alarming.

In a report released this week, the Anthropic Institute disclosed internal data showing how quickly its artificial intelligence is learning to improve itself — and raised an urgent question: What happens when AI systems start designing their own successors?

The numbers suggest change is accelerating.

An Anthropic engineer who once merged small, incremental improvements to code now processes eight times as much per day as two years ago. The engineer isn't working harder. Claude is doing the writing while humans review and redirect.

This is not recursive self-improvement yet. But it's the early warning signs of a path toward it. "We are not there yet," the report's authors, Marina Favaro and Jack Clark, write. "But it could come sooner than most institutions are prepared for."

We are not there yet. But it could come sooner than most institutions are prepared for - Anthropic Institute

The speed of progress defies the pace of a year ago.

Claude Opus 3, tested in March 2024, could handle software tasks requiring about four minutes of human work. By spring 2026, Claude Opus 4.6 completed tasks of up to 12 hours. An independent evaluator found a newer version capable of at least 16 hours of continuous work.

The company isn't celebrating in private. If the current trajectory holds, its models could tackle tasks requiring days of work by year's end and weeks by 2027.

Anthropic's internal benchmarks show why engineers are nervous.

Claude solved open-ended research problems with a 76% success rate in May 2026, up from 26% six months earlier. In one incident, a code upgrade triggered crashes across tens of thousands of jobs. An engineer described the problem to Claude, gave it access to the system, and went home. Two hours later, Claude had isolated an obscure debugging flag, reproduced the crash, and confirmed the fix. A human researcher would typically need two to three days.

The widest gap opened in pure research tasks.

Claude Opus 4, tested in May 2025, improved code performance by roughly 3 times. By April 2026, a newer version was achieving roughly 52 times speedup on identical tests. A skilled human researcher working four to eight hours typically reaches 4 times. "Claude has gone from super helpful to superhuman in under a year," the report states.

Claude has gone from super helpful to superhuman in under a year - Anthropic Institute

The most striking evidence came from April.

Anthropic's AI agents were given an unsolved problem in AI safety and left alone to solve it. They designed experiments, ran tests, and iterated. Over about a week, two human researchers recovered 23% of the measurable performance they'd hoped to close. The agents recovered 97%, consuming roughly $18,000 in computing power.

Anthropic is candid about what happens if this continues.

The company outlined three scenarios. The first: everything slows down. The second, which Anthropic considers most likely: AI development becomes substantially automated while humans set the direction. That would enable small teams to do the work of armies. A 100-person company could match a 100,000-person organization, the report suggests — accelerating everything from drug discovery to AI-powered manipulation "that tailor deception to each individual and run at a scale no human team could match."

The third scenario is the one that keeps executives awake at night: a world driven by machines improving themselves faster than humans can observe, let alone control.

The company faces a strategic bind.

A unilateral pause on development, the authors argue, wouldn't solve anything. It would only shift which company leads the field. A meaningful slowdown would require multiple AI labs across multiple countries to stop simultaneously, verifying one another's compliance. Anthropic compares the challenge to nuclear arms control — a process that took decades to establish. "We don't have that long," the report says.

Anthropic says it will spend the coming months meeting with policymakers, researchers, and rival AI companies to explore how credible verification might work.

The window for building consensus, the company suggests, is open now. After that, the window may close.

• The autohr is the Head of Research and Analysis at Icarus Asia, a Hong Kong-based risk and advisory business.

Sources

• Anthropic Institute, "When AI builds itself" June 6, 2026