rotative
the decoder · June 26, 2026

Claude coded for 19 days nonstop — and it only passed one test

epoch ai and metr released mirrorcode, a benchmark where models reimplement programs without source code. claude opus 4.7 leads with a 56% solve rate, but the hardest tasks still stump every model.

read in 简体中文 · 繁體中文 · 日本語 · 한국어

epoch ai and metr launched mirrorcode to test long-horizon coding capabilities by forcing models to reimplement 25 open-source programs without seeing the original code. claude opus 4.7 leads with a 56% solve rate, demonstrating the ability to rebuild a 16,000-line bioinformatics toolkit in 14 hours—a task that would take humans weeks. however, every model failed on the largest tasks, with one extreme run costing $2,600 after operating autonomously for 19 days. the results highlight a growing gap in reliability: while models handle small utilities easily, complex architectural challenges remain unsolved. researchers have open-sourced the scaffold and 22 programs to measure progress without relying on memorized training data.

read at source ↗

← rotative