Harvard study finds OpenAI o1 outperforms doctors in ER diagnosis
a harvard medical school study found openai's o1 model provided more accurate diagnoses than two internal medicine physicians in emergency room triage. the model hit the exact or close diagnosis in 67% of cases compared to 55% and 50% for the doctors. researchers noted an urgent need for prospective trials to evaluate the technology in real-world settings.
Researchers at Harvard Medical School and Beth Israel Deaconess Medical Center found that OpenAI's o1 model outperformed two human doctors in diagnosing 76 real emergency room cases. Published in Science, the study showed o1 achieved a 67% accuracy rate for exact or very close diagnoses at initial triage, surpassing the two attending physicians who scored 55% and 50%, respectively. The AI utilized raw electronic medical record data without pre-processing. Despite the results, authors emphasized the lack of a formal accountability framework and the need for prospective trials before clinical deployment.