AI审计工具发现大部分智能体异常行为,但误报率较高
ai审计工具如gpt-5和gpt-6能够发现大部分异常行为,但会产生大量误报,使得它们更适合用于估计错误率而不是安全检查。
andy hall, a researcher at anthropic institute, tested ai auditors on detecting misbehavior in logs from frontier models. he used gpt-5, gpt-6 astra, opus 4.8, and gemini 3.8 flash on metr's malt dataset. at a 5% false-alarm rate, gpt-5 caught 98% of reward hacking cases but others performed worse. the auditors caught 71-81% of sandbagging cases. hall found that newer models don't consistently improve false-alarm rates, and combining multiple models didn't clearly improve results.