Frontier models fail 50% of agentic enterprise IT tasks
claude opus 4.7 leads at 47% on itbench-aa sre, a new benchmark of 59 kubernetes incident troubleshooting tasks where all models scored below 50%.
artificial analysis and ibm introduced itbench-aa sre, a benchmark evaluating frontier models on 59 site reliability engineering tasks requiring root cause diagnosis of kubernetes incidents. claude opus 4.7 leads with 47%, followed by gpt-5.5 at 46%, while glm-5.1 topped open weights models at 40%. results show that longer reasoning trajectories do not guarantee accuracy, as models penalized for false positives often underperformed more concise agents.