Study: AI Outperforms Humans in Writing Medical Summaries

There’s no shortage of debate as to which is more accurate and effective: AI or human documentation and output. AI is proving its mettle with everything from sports reporting to financial statements. Now, a recent study suggests AI may now have an edge in at least one area of reporting: medical summaries.

This is a case in which no one should fear for their jobs, and most medical professionals – from heart surgeons to neonatal care nurses – will likely welcome the respite from completing forms and reports on patient visits, enabling them to focus on the wellness challenge at stake.

AI-generated medical documents offer “a promising path to improve clinical workflows and patient care,” the study, published in Nature, states. It means healthcare professionals will be able to spend more quality time with patients. “Doctors have less time for patient care and there is always the possibility of error anytime you are summarizing information from EHR,” the study’s team of authors, led by Dave Van Veen of Stanford, states. “Analyzing vast textual data and summarizing key information from electronic health records imposes a substantial burden on how clinicians allocate their time.”

See also: AI Chatbot Outperforms Human Clinicians in Probabilistic Diagnosis

Applying AI in diverse medical areas

The Stanford study looked at AI’s emerging role in four areas: radiology reports, patient questions, progress notes and doctor–patient dialogue.

The researchers applied eight LLMs to the challenge, finding that in 81% of the instances studied, AI was just as good or surpassed human-written medical summaries. Ten physicians participated in the study, evaluating the machine-versus-human summaries on the basis of completeness, correctness and conciseness. “Summaries from our best-adapted LLMs were deemed either equivalent (45%) or superior (36%) compared with summaries from medical experts,” they found.

LLMs “outperform medical experts in clinical text summarization across multiple tasks.,” the study continues. “This suggests that integrating LLMs into clinical workflows could alleviate documentation burden, allowing clinicians to focus more on patient care.

In addition, while there is some risk of hallucinations or AI bias, the risk is far lower than human error in completing these documents, the study showed.