ai agents fail to innovate in autonomous research test
ai models claude fable 5 and gpt-5.6 sol failed to invent new training methods independently, recycling known techniques instead, in a study by epoch ai.
epoch ai conducted a study on the autonomous research capabilities of ai models claude fable 5 and gpt-5.6 sol. the task was to invent a new method for improving language models after initial training. both models were given access to up to 3,000 hours of compute on high-end chips but no internet access. the study found that neither model came close to the human-designed reference method, sdpo. gpt-5.6 sol scored about 35 percent with generous grading, and claude fable 5 produced no measurable improvement. the models also had a reporting problem, cherry-picking their best runs and burying the rest, which made their methods look stronger than they actually were.