During the pandemic, many patients at NYU Langone Health began using electronic health record (EHR) tools to communicate with their doctors, refill prescriptions, and review test results. These messages, sent through a tool called In Basket in NYU Langone’s EHR system, EPIC, increased by over 30% annually.
Doctors, already spending significant time managing these messages, saw a surge, receiving more than 150 messages daily. This overwhelming load contributed to physician burnout, with half of doctors reporting high stress levels.
A new study by NYU Grossman School of Medicine shows that an AI tool can draft responses to patient queries as accurately and empathetically as human doctors. This AI tool could significantly reduce the message load on doctors and improve patient communication, provided doctors review the AI drafts before sending them.
NYU Langone has been experimenting with generative AI (genAI), which predicts the next word in a sentence based on internet usage. In 2023, they licensed GPT-4, allowing doctors to test it with real patient data while maintaining privacy.
The study, published in JAMA Network Open, compared AI-generated responses to human responses for accuracy, relevance, completeness, and tone. Sixteen primary care doctors evaluated 344 pairs of responses without knowing which were AI-generated.
Results showed no significant difference in accuracy, completeness, or relevance between AI and human responses. However, AI responses were 9.5% better in tone and understandability and were seen as more empathetic and positive. AI responses were also longer and more complex, suggesting the need for further training.
The study indicates that using private patient information makes the AI tool more effective. Future studies will confirm if private data improves performance.
“This AI tool can create high-quality draft responses to patient queries,” said Dr. Devin Mann, senior director of Informatics Innovation at NYU Langone. “With continued refinement, AI message quality will match that of human responses.”
The research team included experts from NYU Langone, NYU Tandon School of Engineering, and NYU Stern School of Business.
Source: news-medical
