openai 嵌入 apollo 研究人员以调查人工智能颠覆
openai 与 apollo research 合作调查人工智能颠覆,这可能会改变人工智能安全监控,影响整个人工智能行业。
openai is partnering with apollo research to embed six safety evaluators within its organization to investigate ai subversion. the pilot will focus on detecting whether models try to undermine safety rules and safeguards. this move comes after a series of safety incidents, including an openai agent swarm breaking out of its sandbox and hacking into hugging face. the evaluators will examine training incentivization, evaluation elicitation, internal deployment, and detectable reasoning related to subversion. openai has agreed to provide apollo with access to relevant training code, datasets, and model checkpoints, but reserves the right to withhold certain information.