Quality and efficiency of integrating customised large language model-generated summaries versus physician-written summaries: a validation study

Abstract

Objectives

To compare the quality and time efficiency of physician-written summaries with customised large language model (LLM)-generated medical summaries integrated into the electronic health record (EHR) in a non-English clinical environment.

Design

Cross-sectional non-inferiority validation study.

Setting

Tertiary academic hospital.

Participants

52 physicians from 8 specialties at a large Dutch academic hospital participated, either in writing summaries (n=42) or evaluating them (n=10).

Interventions

Physician writers wrote summaries of 50 patient records. LLM-generated summaries were created for the same records using an EHR-integrated LLM. An independent, blinded panel of physician evaluators compared physician-written summaries to LLM-generated summaries.

Primary and secondary outcome measures

Primary outcome measures were completeness, correctness and conciseness (on a 5-point Likert scale). Secondary outcomes were preference and trust, and time to generate either the physician-written or LLM-generated summary.

Results

The completeness and correctness of LLM-generated summaries did not differ significantly from physician-written summaries. However, LLM summaries were less concise (3.0 vs 3.5, p=0.001). Overall evaluation scores were similar (3.4 vs 3.3, p=0.373), with 57% of evaluators preferring LLM-generated summaries. Trust in both summary types was comparable, and interobserver variability showed excellent reliability (intraclass correlation coefficient 0.975). Physicians took an average of 7 min per summary, while LLMs completed the same task in just 15.7 s.

Conclusions

LLM-generated summaries are comparable to physician-written summaries in completeness and correctness, although slightly less concise. With a clear time-saving benefit, LLMs could help reduce clinicians’ administrative burden without compromising summary quality.

Strengths and limitations of this study.

• Real-world clinical data extracted from routine hospital care without stimulation were used.

• The study compared large language model-generated and physician-written summaries using both a blinded evaluation by medical professionals and computational metrics.

• Input data were limited to a Dutch-language setting, which may affect the generalisability of findings to other healthcare systems.

Introduction

The use of large language models (LLMs) in healthcare has the potential to reduce the administrative burden of clinicians without compromising quality of care, thus supporting a sustainable healthcare system.¹,³ LLMs, which employ deep learning to process and interpret human language, have many potential applications in healthcare, such as medical note summarisations and clinical decision-making.^{3 4} These models undergo extensive pretraining and fine-tuning to generate tailored outputs for specific use cases. Currently, the Generative Pretrained Transformer 4-o (GPT-4-o) model by OpenAI (San Francisco, California, USA), Google DeepMind‘s MedPaLM 2 and Anthropic‘s Claude 3 Opus are considered top-tier LLMs.⁵,⁷ LLMs were predominantly trained and tested using the English language and with a US-centric perspective.^{8 9} As their potential in healthcare is explored, robust clinical validation is essential to ensure their safe integration into electronic health records (EHRs) and everyday clinical practice. However, the efficacy of such integrated EHR functionalities remains untested in non-English clinical settings, highlighting the need for equitable LLM applications across different languages and nations.

Healthcare professionals spend a significant amount of time on administrative duties, impacting their ability to deliver high-quality care, career satisfaction, burnout rates and even their willingness to remain in clinical practice.¹⁰ A large part of the administrative tasks consists of summarising patient files, such as for outpatient visits or hospital discharge letters—a complex, time-consuming process that is prone to inconsistencies and errors, particularly when patient care is fragmented across multiple providers. Accurate and consistent LLM-generated medical summaries could significantly reduce this administrative burden.¹¹,¹⁴ Generative artificial intelligence (AI) presents a compelling opportunity to revolutionise note summarisation, streamline workflow efficiency and improve patient outcomes.¹⁵ Integrating data from shared care centres into these summaries can provide a holistic view of patient health, further enhancing care quality and safety.

To ensure the safe use of LLMs for medical summarisation, it is crucial to validate and benchmark these technologies against current practices, that is, physician-written summaries. In this study, we compared LLM-generated medical summaries with physician-written medical summaries, using a non-inferiority study design. LLM-generated medical summaries were embedded in EHR within a non-English clinical environment. Summaries were compared by independent physicians as well as objective protocols. Furthermore, we assessed the clinicians’ preference and level of trust for each summary.

Methods

Physician-written summaries

For this non-inferiority validation study, 60 Dutch physicians across 8 departments in a large Dutch academic hospital were recruited to participate in this study in February 2024. Their experience was defined as the number of years of clinical practice since attaining the medical licence. For each (sub)department, 5 patients were selected (figure 1), to a total of 50 patient records.

Figure 1. The non-inferiority study design. Online supplemental material includes further explanation on the numbers. LLM, large language model.

The participating physicians (physician writers) were instructed to write a summary of the patient files, as if they were preparing for an outpatient visit. The instructions were to write the summaries in a similar time frame as they would normally use for outpatient clinic preparation. Additionally, they were asked to time the duration of these preparations for each summary. They were instructed to use at maximum the last 20 notes in the patient files, which consisted of structured medical notes within the same specialty (including medical history, laboratory and radiology results, physical exams, previous treatments, etc) as well as paramedic notes or medical notes from different specialties. Baseline characteristics of the participating physician population were collected.

LLM-generated summaries: LLMs within the EHR

LLM-generated summaries were generated for the selected patients via Microsoft’s Azure OpenAI, using the GPT-4 model, through the EHR (Epic Systems Corporation, Verona, Wisconsin, USA).

Prompt engineering (the act of writing sound instructions to the model) was performed by a team of multicentre medical and technical experts in an iterative manner. The LLM was given the same instructions as the physician through the prompt and was instructed to have a maximum word count of 100 for the output.

To control token length and model performance, we limited the LLM input to a maximum of 20 most recent structured clinical notes per patient. These included relevant free-text sections such as anamnesis, physical examination, specialist consultations and progress notes. Diagnostic reports (eg, lab or radiology) were referenced only if documented in the clinical notes. The context window length at the time of the study was set to 8000 tokens. It was estimated that 1 token was ±3 characters, so the token limit enforced was 18 000 characters. This translates up to a minimum of 3 and maximum of 15 notes per summary as note length differs per clinical setting due to local practices. The lookback period was a maximum of 3 years.

Comparing physician-written versus LLM-generated summaries

A subset (online supplemental material) of the physician-written summaries was compared with LLM-generated summaries using objective and subjective measures. For objective measures, the validated ROUGE and BLEU scores for analysis of computational linguistics and natural language processing were used.¹⁶,¹⁸ A higher ROUGE and BLEU score reflects higher textual similarity between summaries. The ROUGE-1 recall represents the percentage of words that match between the LLM-generated summary and the physician-written summary (where 100 represents 2 equal texts). The BLEU score reflects the number of similar words divided by the total words (as percentage) with 100 representing 2 equal texts.

Since automated metrics do not directly reflect summary quality, readability and accuracy, a selection of the summaries (total: n=400) was evaluated by ten physician evaluators. These physician evaluators reviewed paired LLM-generated summaries and physician-written summaries, presented in a blinded and randomised order to a total of 40 summaries per evaluator. Physicians’ evaluation was measured using a 5-point Likert scale of three domains (derived from Van Veen et al)³:

• Completeness: captures recall, amount of relevant clinical details. ‘Which summary more completely captures important information?’

• Correctness: captures precision, a summary without errors. ‘Which summary includes less false information?’

• Conciseness: decreasing the amount of irrelevant information. ‘Which summary contains less non-important information?’

Furthermore, these physician evaluators were asked which summary they believed was the LLM-generated summary and which summary they would trust during clinical practice.

Statistical analysis

SPSS Statistics V.28.0 (IBM) was used to analyse data. Descriptive statistics for baseline characteristics are presented depending on their distribution. For comparison, Student’s t-tests, the Mann-Whitney U test or the χ² test were used, depending on the type of variable studied. A p<0.05 was considered statistically significant.

For the natural language processing analyses (baseline characteristics, ROUGE, BLEU scores), Python (V.3.12.2) was used (Natural Language Toolkit (NLTK) packages). NLTK scores are presented as a percentage and are designed to compare two sets of texts.

The combinations of Likert scale scores were calculated for each summary for each observer, and paired t-tests were used for statistical analyses. Two-sided p values are used for comparisons of paired data. Preference for either the physician-written summary, the LLM-generated summary or equal assessment was established for each pair of summaries for each observer. Inter-rater reliability was calculated using the intraclass correlation coefficient (ICC).

Patient and public involvement

Patients are informed of the use of AI-assisted tools in our clinic through the hospital website. Through the institutional patient panel, patients are encouraged to contribute to the setup and design of our studies. All results of our generative AI scientific research will be shared through multidisciplinary group meetings and publications.

Results

Baseline characteristics

A total of 42 physicians (70%) enrolled, and 209 summaries were written for 50 individual patient records (online supplemental material). For each patient record, three LLM summaries were generated (n=150). The mean age and years of experience for the physician writers (n=42) and the physician evaluators of the subset (n=10) were comparable (39.9 vs 39.1 years of age, and 12.5 vs 13.7 years of expertise, p=0.981 and p=0.422, respectively, table 1).

Table 1. Baseline characteristics of the physician writers and the physician evaluators.

Baseline characteristics	Physician writers n=42	Physician evaluators n=10
Age, mean (SD), years	39.9±8.9	39.1±6.7
Departments (no. %)
ENT surgery	5	1
Radiotherapy	4
Orthopaedics	5	1
Paediatrics
General	15	4
Cardiology	5
Intensive care		3
Gynaecology	2
Urology	3
Internal medicine	3	1
Years of experience (SD)	12.5±8.6	13.7±6.7
Time to write summary
Mean (SD), min	7.2±5.0

ENT, ear, nose and throat.

The mean writing time for the summary for the physicians was 7 min (±5). The mean generating time for the LLM summaries was 15.7 s (95% CI 2.1).

Objective measures

Physician-written summaries were significantly shorter compared with LLM-generated summaries in both word count (60 vs 100 words, p<0.001) and characters (463 vs 696 characters, p<0.001).

The overall ROUGE recall score was 24.8, and the overall BLEU score was 14.2 (table 2A). Hallucinations were not observed.

Table 2. Performance of LLM versus physician using a 5-point Likert scale of three domains.

Baseline		ROUGE-1		BLEU
Mean word count (n)	Mean characters (n)	Recall	Precision	BLEU-1
A: Objective measures
80	580	24.8	14.7	14.2
	LLM	Physician		P value
B: Subjective measures
Completeness	3.5±0.5	3.3±0.6		0.517
Correctness	3.3±0.3	3.3±0.4		0.938
Conciseness	3.0±0.4	3.5±0.5		0.001
Overall scores	3.3±0.3	3.4±0.3		0.373

A higher ROUGE and BLEU score reflects higher textual similarity between summaries. The ROUGE-1 recall represents the percentage of words that match between the LLM-generated summary and the physician-written summary (single word overlap, where 100 represents 2 equal texts). The BLEU score reflects the number of similar words divided by the total words (as percentage) with 100 representing. Precision reflects the overlapping unigrams divided by the total unigrams in the generated summary. The BLEU score reflects the number of similar words divided by the total words (as percentage) with 100 representing 2 equal texts.

LLM, large language model.

Performance of LLM-generated summaries versus physician-written summaries

The combined scores of completeness, correctness and conciseness for LLM vs physician summaries were not significantly different (3.3 vs 3.4, p=0.373). Completeness and correctness of the LLM-generated summaries did not significantly differ compared with the physician-written summaries (table 2B). LLM-generated summaries were significantly less concise (3.0 vs 3.5, p=0.001).

An example of the scoring system is provided in figure 2. Overall, there was a preference (or equal score) for the LLM-generated summary compared with the physician-written summary (57% vs 43%, figure 3). Evaluators were able to correctly identify the LLM-generated summaries in the majority of the summaries (84%). Trust in both the physician and LLM summaries was similar: 77 vs 81% of the summaries were trusted enough to be used in clinical decision making (p=0.187). Interobserver variability showed excellent reliability (ICC 0.975).

Figure 2. Example of summaries (left panel) and their corresponding evaluation by the physician evaluators (right panel). In red: mistake by physician, in green: additional valuable information in physician summary. Translated from Dutch to English for illustration purposes: original Dutch text available (online supplemental material). AI, artificial intelligence.

Figure 3. Recognition, preference and trust infographic. AI, artificial intelligence.

Discussion

In this non-inferiority validation study, conducted in a large academic hospital, LLM-generated summaries were compared with physician-written summaries based on real world patient data in a non-English clinical environment. The tool used for generating LLM summaries is embedded in the EHR.

Our results show that LLM-generated summaries are non-inferior to physician-written summaries in terms of completeness and correctness. Although LLM-generated summaries were less concise, they were trusted as much as those written by physicians. Notably, physicians were able to discern whether a summary was created by an LLM or a human in most cases.

Objective and subjective comparison of physician-written versus LLM-generated summaries

The low ROUGE and BLEU scores indicate substantial differences in word-level matching between LLM-generated and physician-written summaries. These metrics focused on n-gram overlap and do not capture semantic similarities. They were mainly included to allow for comparability with prior studies, in the absence of widely accepted objective instruments that quantify factual consistency without relying on human evaluation. Therefore, physician evaluators assessed summaries on completeness, correctness and conciseness. Our findings show that LLM-generated summaries are non-inferior to the physician-written summaries on completeness and correctness, the most clinically important parameters.

LLM-generated summaries were significantly less concise, reflected in the word count with an average of 100 words compared with 60 words in physician-written summaries. Despite the statistical significance, the clinical relevance of this finding is minimal, given the minor difference in reading time.

Physicians wrote the summaries in a realistic timeframe, but with the knowledge that their summaries would be used as gold standard for the LLM-generated summaries. Therefore, the level of comparison was considerably higher compared with earlier studies,^{3 4} where already existing data were used as reference. LLM-generated summaries were generated 28 times faster than physician-written ones, suggesting significant potential for reducing clinical hours. Although not all clinicians typically produce written summaries in preparation for outpatient consultations, they are expected to review patient information beforehand. Its impact may vary depending on individual clinical practices.

The majority of physician evaluators could identify the LLM-generated summaries. This suggests that the characteristics of LLM-generated summaries are distinct enough to be noticed by professionals, potentially influencing their trust, acceptance and reliance on these tools. Furthermore, our results show a high level of trust for the LLM-generated summaries to be used in clinical practice, paralleling that of physician-written summaries. The majority of physician evaluators either preferred LLM-generated summaries or found them equivalent to physician-written summaries.

The absence of harmful hallucinations, potentially due to setting the model’s temperature to 0, further supports the reliability of the LLM outputs.

Comparison with previous studies

In the field of research for LLM applications in healthcare, only 5% of studies use real patient care data such as in this study.¹⁶ Only one other study has been published that evaluated LLM summarisation on clinical notes,³ reporting an overall preference or equivalent score of 81% for summaries generated by the best LLM versus the readily available progress notes from the MIMIC-III dataset. In contrast, we asked physicians to summarise medical files specifically for the purpose of our study as if they were preparing for a patient visit. It is therefore likely that the physician-written summaries are of superior quality to those already available in the notes. Furthermore, this is the first study to evaluate LLM-generated medical summaries in a non-English clinical environment and the Dutch language. Note that the Dutch language is only spoken by less than 0.5% of the world population and the Dutch word count in the GPT model is only 0.34%.⁹ This study shows that foundation models can be used in a non-English clinical setting without further linguistic adaptations, underscoring the language-agnostic characteristics of LLMs.

Another study analysed discharge summaries in a similar manner, reporting a significant benefit in concision and coherence for LLM-generated summaries, and increased comprehensiveness in physician summaries.¹⁹ Naturally, the requirements of a discharge summary differ from an outpatient clinic summary.

Strengths and limitations

This study has several limitations. First, physicians were used as the gold standard for evaluating LLM-generated summaries. Although physicians are highly skilled and knowledgeable, their summaries may inherently vary due to individual differences in experience, expertise and subjective interpretation of clinical data. This variability can introduce inconsistencies in the benchmark against which the LLM-generated summaries are measured, potentially skewing the assessment of AI performance. Due to the low temperature settings used for the GPT model, the LLM-generated summaries showed more consistency. Additionally, prompt engineering plays a crucial role in generating high-quality summaries. The effectiveness of LLM-generated summaries heavily depends on the design and specificity of the prompts given to the LLM, and the amount of text in the context window. For this study, this was approximately 18 000 characters; relatively low in this rapidly increasing field.¹⁹ This might limit generalisability.

Another limitation is the potential presence of the Hawthorne effect: participating physicians were aware that their summaries would be compared against LLM-generated summaries, which may have led to more meticulous summary writing than in clinical practice. As such, the comparison may understate the potential benefits of LLMs in time-constrained clinical conditions.

Furthermore, medical summaries are inherently context specific, as, for example, an ear, nose and throat surgeon would have a different focus and preference for the content of the summary than an orthopaedic surgeon. In this study, this effect will be reflected in the physician-written summaries but not in the LLM-generated summaries. We aimed to find a prompt which serves every specialty or only would need minor fine-tuning for end-users. The desired summary length might vary according to national healthcare standards, which were only tested in a Dutch clinical context, potentially limiting generalisability.

Implications for clinical practice

These results suggest that integrating LLMs into EHR systems may be an effective strategy to reduce the administrative burden on clinicians without compromising the quality of documentation. The results allow us to test the functionality in real-world scenarios by providing evidence of its performance and reliability, enabling us to achieve even greater accuracy and efficiency through feedback loops.

Furthermore, the potential of differentiated prompting—customising LLM prompts to cater to specific medical specialties and disciplines such as nursing—can enhance the relevance and accuracy of the summaries. Further development should focus on prompt engineering to increase the conciseness of LLM-generated summaries without loss in quality (completeness and correctness), for example, by reducing word count. The collected set of summaries can function as a validation set for future prompts. The scoring system used by the physician evaluators is quite labour-intensive, and future endeavours may focus on automating this process through LLMs and validating this approach.

This LLM was trained on English language and ingested and generated output in Dutch, even though there is a discrepancy between the amount of text available on the internet in English vs in Dutch (estimated 55% vs 1%).¹⁷ Future studies should explore the long-term impacts of LLM integration on clinical workflow and patient outcomes, as well as the adaptability of these models across different languages and healthcare settings.

Conclusions

The findings in our study indicate that LLM-generated medical summaries are a viable alternative to physician-written summaries. Additionally, our study shows that LLM-generated summaries are trustworthy in clinical practice. This suggests that this functionality can be safely integrated and used in the EHR with significant potential for reducing administrative burdens and enhancing clinical efficiency, potentially saving valuable time for clinicians.

Data availability statement

Data are available on reasonable request.