Nature: AI agents outperform doctors despite running on outdated models
two nature studies show ai agents matching or beating doctors in diagnosis and treatment planning using older base models, signaling that medical agents are approaching clinical utility despite not using the latest llms.
tud dresden's mira agent operated autonomously within a simulated electronic health record, utilizing 11 tools to diagnose conditions like appendicitis with 98.6% accuracy, while a google study demonstrated amie achieving 95% appropriate treatment plans versus 72% for human doctors. the researchers found mira made zero dangerous drug interactions or dosing errors and perfectly identified necessary hospitalizations, highlighting the safety potential of these autonomous systems. importantly, both systems ran on base models that are already outdated, implying that architecture and domain-specific fine-tuning matter more for medical tasks than access to the newest generation of general-purpose llms. while these results are limited to simulated environments, they set a new benchmark for the reliability of ai agents in complex, high-stakes decision-making processes.