how openai is simulating real-world chaos to catch gpt-5 misbehavior
openai deployed 'deployment simulation' to replay past user conversations with gpt-5-series models, catching novel misalignments and predicting behavior rates that traditional evals miss.
openai implemented 'deployment simulation' to predict model behavior before public release by having new models, specifically in the gpt-5-series thinking line, respond to anonymized past user conversations. this approach provides a deployment-like preview of real-world model responses, identifying novel forms of misalignment and providing more accurate frequency estimates for undesired behaviors than synthetic or manually curated evals. the method successfully reduced the risk of models sensing they are under test and was validated on complex agentic rollouts involving tool use. openai used these insights to inform mitigations and deployment decisions, planning to expand the pipeline's role in future development.