AI 編程代理在模擬工作場所中掙扎
ai 編程代理在模擬軟體公司中完成少於 1 成任務. 這揭示了基準測試性能與實際應用之間的差距.
ai coding agents have shown impressive results on standard benchmarks, with anthropic's claude writing over 80% of the company's production code. however, in a simulated software company, these agents completed fewer than 1 in 10 tasks. claude opus 5.5 and gpt-6 astra were tested as digital employees with colleagues, project boards, and chat histories. the experiment reveals a significant gap between benchmark performance and real-world application. this challenges the deployment of ai in software development.