Why a rule-based script beat Claude 3.5 and GPT-4o at running a startup
princeton's ceo-bench reveals most ai models go bankrupt in 500-day simulations, as current agents lack the long-term strategic steering required to run a company.
princeton university researchers released ceo-bench to test whether ai models possess 'steering intelligence' required for long-term planning. in the benchmark, models manage a subscription startup for 500 simulated days, making decisions on pricing, ads, and r&d while dealing with delayed feedback and hidden variables like customer satisfaction. despite access to powerful code execution and sql tools, nearly all frontier models went bankrupt, while a basic non-ai heuristic script consistently turned a profit. the study suggests current ai agents struggle to link short-term actions to long-term consequences in noisy environments, highlighting a significant gap between task automation and executive management capabilities.