Abstract
A retrospective review of 7185 South Australian discharge summaries revealed that 37.6% of discharge summaries were released at least a day after discharge, and per day of delay of medical discharge summary release, the chance of hospital 30‐day readmission increased by 1.60% (P < 0.001). Delays may be associated with the task being time‐consuming. The workforce impact of junior doctor time and increased readmission rates favours interventions to change discharge summary workflows.
Keywords: artificial intelligence, machine learning, discharge summary, hospital efficiency
Discharge summaries are a critical but occasionally overlooked component of patient care. 1 As summaries of patients' admissions, their primary purpose is to direct care transition from hospital to community care. They also may be used to coordinate readmissions, provide information for research databases, apply for hospital funding and communicate with patients. The timeliness of discharge summary completion is an important consideration when viewing their potential utility. 2
In the era before electronic medical records (EMRs), errors were identified in 35% of discharge summaries from a New South Wales hospital. 3 Additionally, for up to 30% of discharges, a discharge summary was not received by a patient's general practitioner. 3 While recent data on discharge summary quality are sparse, a recent survey of South Australian general practitioners responded they either ‘often’ or ‘sometimes’ detected omissions or discrepancies in discharge summaries. 4
The mechanism for generating discharge summaries is inconsistent between hospitals. In Australia – including at our institution – trainee medical officers (TMOs) are responsible for typing a narrative summary of an admission, based on EMR and other data. At times, a senior medical officer may review these summaries. Internationally, other methods are employed, including automatic generation from an EMR and dictation. 5 With the advent of generative artificial intelligence (gAI), there is the opportunity to re‐evaluate discharge summary workflows. Given the potential costs (both in time used and material and electricity expense) associated with implementing and testing AI workflows, there must be signals of clinical equipoise before testing such systems. 6
The aims of this study were to determine the frequency and extent of delays in discharge summary completion at a tertiary hospital, along with the time burden of discharge summary completion and association with readmission.
We performed a retrospective review of discharge summaries created at a tertiary hospital in South Australia for patients admitted on medical and surgical wards between 1 January 2024 and 31 March 2024. Data were extracted from the hospital's EMR system, Sunrise (Allscripts), through the internal clinical analytics team using a secure institutional data request process. Descriptive statistics were employed on the time between creation and submission for discharge summaries, with a cutoff of 100 min to exclude non‐writing time delays. Time taken to write was linearly regressed against word count to separate thinking and editing time from typing time. Binomial regression was performed on the 30‐day readmission outcome against delay between discharge and discharge summary finalisation. Age, gender and measures of disease severity (e.g. length of stay and discharge summary word count) were included in the regression to isolate the delay effect. Frequency of discharge summary delay (days between discharge and discharge summary finalisation) was also calculated. This study received institutional human research ethics committee approval with a waiver of individual consent. CALHN Human Research Ethics Committee approval number: 18665.
We found 7185 medical records in which there were 5070 medical and 2019 surgical discharge summaries. The median word count was 443. Median patient age was 71; 50.7% were female. Other collected data included date of discharge, time that a discharge summary was first created, time the discharge summary was finalised, length of stay in hospital and whether a patient was readmitted after 30 days.
The median discharge summary took 14.5 min to complete (interquartile range (IQR) 7.56–28.9 min). Medical discharge summaries had a median of 15.9 min (IQR 7.91–32.3 min). Surgical discharge summaries took a median of 11.7 min (IQR 6.76–22.1 min). For medical discharge summaries, linear regression of word count and time taken revealed an average of 5.9 min of intrinsic delay per discharge summary not associated with typing more words (i.e. thinking time or searching/loading information).
A total of 37.6% of discharge summaries was released at least a day after discharge, 31.0% two or more days and 27.4% were finalised three or more days after discharge. When accounting for covariates, each day of delay of medical discharge summary release (using the day of discharge as a baseline) was associated with a 1.60% increased rate of 30‐day readmission per day (P < 0.001). For surgical discharge summaries, there was a 0.66% increased risk per day (P = 0.0162).
Discussion
Our results demonstrate that current discharge workflows at a tertiary hospital in Adelaide consume significant amounts of doctor time and may have negative implications for readmission. It is likely that the workflow employed at this hospital, i.e. discharge summaries created and sent through an EMR, applies to other tertiary institutions. The harms identified have been previously observed, albeit in the pre‐EMR era. 7
We note the time taken to create discharge summaries is significant. We calculated the equivalent full‐time employment rate of discharge summary creation at 3.5 (135 doctor‐hours per week). For an institution with 76 medical interns, this amounts to 4.6% of the non‐overtime labour available. This use of clinician time may compete with other clinical tasks. 8
Delay in discharge summary completion was associated with increased rates of 30‐day readmission. This association appeared independent of age, gender and proxy measures of patient complexity, such as discharge summary word count and hospital length of stay. These, and incomplete treatment, have previously been identified as risk factors for readmission. 9 However, as only indirect measures were included, unmeasured patient complexity may remain a confounder. The differing magnitude of association between medical and surgical wards is consistent with a causal relationship. For example, medical patients are more likely to have chronic conditions, where timely communication to primary care may influence outcomes more significantly. Nevertheless, as this is a retrospective study, we cannot exclude collinear factors, such as higher workload, driving both delayed documentation and increased readmissions.
Prior work has cited the burden of manual data entry, limited EMR usability, competing clinical demands, lack of standardisation and inadequate TMO engagement, including during handovers, as possible causes of discharge summary delay. 2 It is likely that the impact of these delays is meaningful both to a patient, where readmission implies a deterioration in health, and to hospitals, where admission contributes to additional costs and access block. 10 Despite the late discharge summary rate being lower for medical discharge summaries, the higher number of medical patients, combined with greater risks associated with any delay, imply that discharge summary workflow interventions may find most patient benefit in medical specialities.
Student assistants in medicine may be able to write drafts of discharge summaries that can be approved by the treating team; however, their summaries may be limited in accuracy. 11 Large language models can interpret significant bodies of text including admission and ward notes, pathology and radiology reports and medication lists to provide a template discharge summary. 12 Strategic prompting and fine tuning of existing large language models could produce summaries that incorporate all relevant patient information. 13 gAI systems require validation, both in quality of output and systems benefits, noting that benefits of AI systems can be site‐specific. 14 It is also possible that a gAI could alter human behaviour in a way that harms health outcomes. 15 Given the clinical impact and current low performance of discharge summaries, 3 after preliminary testing, there may be compelling clinical equipoise for a randomised trial of gAI‐generated and human‐edited compared with human‐generated discharge summaries. Our study also suggests that the readmission rate is an appropriate outcome measure for such studies.
The descriptive data on discharge summary time are limited as the data are from a single site and were collected over a short period of 3 months. The association between discharge summary delay may not be causative as this study is retrospective. Given the study design, it was not possible to assess upstream causes of delay such as clinician workload variability, EMR system downtime or other factors. It is worth noting that collinear effects between discharge summary delays and 30‐day readmission (e.g. an overworked department) cannot be excluded; however, these may still favour an interventional study to examine whether reducing the documentation burden of discharge summaries improves readmission rates.
Acknowledgements
Open access publishing facilitated by The University of Adelaide, as part of the Wiley ‐ The University of Adelaide agreement via the Council of Australian University Librarians.
Funding: S. Bacchi is supported by a Fulbright Scholarship. O. Kleinig, L. Hains and A. Murugappa were supported by the NALHN Young Health Innovators Scholarship.
Conflict of interest: None.
Data availability statement
The data that support the findings of this study are available from the corresponding author upon reasonable request.
References
- 1. Kind AJH, Smith MA. Documentation of mandated discharge summary components in transitions from acute to subacute care. In: Henriksen K, Battles JB, Keyes MA and Grady ML, eds Advances in Patient Safety: New Directions and Alternative Approaches (Vol. 2: Culture and Redesign). Rockville, MD: Agency for Healthcare Research and Quality (US); 2008. (cited 2024 Aug 9). Available from URL: http://www.ncbi.nlm.nih.gov/books/NBK43715/. [PubMed] [Google Scholar]
- 2. O'Leary KJ, Liebovitz DM, Feinglass J, Liss DT, Evans DB, Kulkarni N et al. Creating a better discharge summary: improvement in quality and timeliness using an electronic discharge summary. J Hosp Med 2009; 4: 219–225. [DOI] [PubMed] [Google Scholar]
- 3. Wilson S. General practitioner‐hospital communications: a review of discharge summaries. J Qual Clin Pract 2001; 21: 104–8. [DOI] [PubMed] [Google Scholar]
- 4. Scarfo NL, Dehghanian S, Duong M, Woodman RJ, Shetty P, Lu H et al. General practitioners' perspectives on discharge summaries from a health network of three hospitals in South Australia. Aust Health Rev 2023; 47: 433–440. [DOI] [PubMed] [Google Scholar]
- 5. Walraven CV, Laupacis A, Seth R, Wells G. Dictated versus database‐generated discharge summaries: a randomized clinical trial. CMAJ 1999; 160: 319–326. [PMC free article] [PubMed] [Google Scholar]
- 6. Kleinig O, Sinhal S, Khurram R, Gao C, Spajic L, Zannettino A et al. Environmental impact of large language models in medicine. Intern Med J 2024; 54: 2083–2086. [DOI] [PubMed] [Google Scholar]
- 7. Li JYZ, Yong TY, Hakendorf P, Ben‐Tovim D, Thompson CH. Timeliness in discharge summary dissemination is associated with patients' clinical outcomes. J Eval Clin Pract 2013; 19: 76–79. [DOI] [PubMed] [Google Scholar]
- 8. Szulewski A, Howes D, van Merriënboer JJG, Sweller J. From theory to practice: the application of cognitive load theory to the practice of medicine. Acad Med 2021; 96: 24–30. [DOI] [PubMed] [Google Scholar]
- 9. Shalchi Z, Saso S, Li H, Rowlandson E, Tennant R. Factors influencing hospital readmission rates after acute medical treatment. Clin Med 2009; 9: 426–430. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Richardson DB. Access block in Australian emergency departments 2017–2020. Emerg Med Australas 2021; 33: 529–533. [DOI] [PubMed] [Google Scholar]
- 11. Monrouxe LV, Hockey P, Khanna P, Klinner C, Mogensen L, O'Mara DA et al. Senior medical students as assistants in medicine in COVID‐19 crisis: a realist evaluation protocol. BMJ Open 2021; 11: e045822. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Thirunavukarasu AJ, Ting DSJ, Elangovan K, Gutierrez L, Tan TF, Ting DSW. Large language models in medicine. Nat Med 2023; 29: 1930–1940. [DOI] [PubMed] [Google Scholar]
- 13. Kleinig O, Gao C, Kovoor JG, Gupta AK, Bacchi S, Chan WO. How to use large language models in ophthalmology: from prompt engineering to protecting confidentiality. Eye 2023; 38: 649–653. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Zech JR, Badgeley MA, Liu M, Costa AB, Titano JJ, Oermann EK. Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: a cross‐sectional study. PLoS Med 2018; 15: e1002683. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. He G, Buijsman S, Gadiraju U. How stated accuracy of an AI system and analogies to explain accuracy affect human reliance on the system. Proc ACM Hum‐Comput Interact 2023; 7: 3610067. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
The data that support the findings of this study are available from the corresponding author upon reasonable request.
