Tao Tu, Mike Schaekermann, Anil Palepu, Khaled Saab, Jan Freyberg, Ryutaro Tanno, Amy Wang, Brenna Li et al.
AMIE is an LLM-based AI system that learns diagnostic dialogue skills through self-play simulation and automated feedback, demonstrating superior diagnostic accuracy and communication abilities compared to primary care physicians in a comparative study.
Physician-patient dialogue is central to diagnosis and treatment, but AI systems require complex clinical expertise including history-taking, diagnostic reasoning, and empathy. Existing AI systems have limitations in conversational diagnostic capabilities.
AMIE uses a self-play-based simulated environment to learn dialogues across diverse diseases, specialties, and contexts. An automated feedback mechanism optimizes diagnostic accuracy, communication, and empathy. Evaluation was conducted through a randomized double-blind crossover study comparing AMIE with 20 primary care physicians using 159 case scenarios from Canada, the UK, and India, assessed by specialist physicians and standardized patients.
AMIE outperformed primary care physicians on 30 out of 32 axes according to specialist physicians and 25 out of 26 axes according to standardized patients. It showed significant differences in diagnostic accuracy and received high scores in empathy and communication. This marks an important milestone for conversational diagnostic AI, suggesting potential for future translation to real-world clinical settings.