AI 编程代理在模拟工作场所中挣扎
ai 编程代理在模拟软件公司中完成少于 1 成任务. 这揭示了基准测试性能与实际应用之间的差距.
ai coding agents have shown impressive results on standard benchmarks, with anthropic's claude writing over 80% of the company's production code. however, in a simulated software company, these agents completed fewer than 1 in 10 tasks. claude opus 5.5 and gpt-6 astra were tested as digital employees with colleagues, project boards, and chat histories. the experiment reveals a significant gap between benchmark performance and real-world application. this challenges the deployment of ai in software development.