Why accuracy drops when doctors use explainable AI
mit researchers found that ai explainability tools often improve patient accuracy but reduce physician performance. this suggests that medical ai interfaces require dynamic adaptation to the user's expertise level.
mit researchers and collaborators evaluated ai explainability tools within the context of skin disease diagnosis, discovering that their utility varies drastically by user expertise. non-expert users demonstrated improved accuracy by deferring to the model, but primary care providers frequently experienced reduced performance because they were misled by the explanations. the study indicates that standard explainability methods can disrupt the clinical judgment of skilled professionals. this signals a shift away from static tools toward adaptive systems that account for the user's baseline medical knowledge. future ai deployments in healthcare must customize interfaces to support, rather than confuse, expert clinicians.