IBM and UC Berkeley use MAST to diagnose IT agent failures
ibm and uc berkeley applied the mast taxonomy to 310 itbench traces involving gemini-3-flash, kimi-k2, and gpt-oss-120b. the analysis revealed that frontier models fail cleanly while open models suffer cascading failures. incorrect verification predicts failure most often across all models.
IBM Research and UC Berkeley analyzed 310 traces from ITBench using MAST (Multi-Agent System Failure Taxonomy) to diagnose why agentic systems fail in IT automation. The study found that frontier models like Gemini-3-Flash typically encounter isolated bottlenecks (2.6 failure modes/trace), whereas large open models like GPT-OSS-120B suffer from compounding errors (5.3 failure modes/trace).
The most common failure mode across all model classes was FM-3.3 (Incorrect Verification), where agents declare victory without checking ground truth. Kimi-K2 specifically exhibited a 46% spike in Premature Termination. To mitigate these issues, the researchers recommend externalizing verification, moving termination logic outside of the LLM, and forcing clarification for ambiguous inputs.