Your safety benchmarks are broken: 100M tokens unlocks hidden agent capabilities
uk aisi found ai agent success rates jump 25% with extra compute on cyber tasks, showing standard fixed-budget evaluations systematically underestimate frontier model risks and capabilities.
the uk's ai safety institute tested frontier models across seven benchmarks and found that performance scales significantly with test-time compute, contradicting standard fixed-budget evaluation methods. in cybersecurity and software engineering tasks, success rates increased by up to 25% when token budgets rose from one million to ten million, with some tasks requiring over 100 million tokens to solve. researchers discovered a power law linking human task duration to agent token consumption, implying that fixed budgets cap potential on complex problems before the agent actually fails. newer models gain more from extra compute than older ones, extending their reach to harder tasks and increasing reliability on existing ones, which signals that current safety testing may provide a false sense of security about model limits.