Effect of a generative artificial intelligence digital scribe on pediatric provider documentation time, cognitive burden, and burnout

Jonathan H Pelletier; Kevin Watson; Jenny Michel; Robert McGregor; Sarah Z Rush

doi:10.1093/jamiaopen/ooaf068

. 2025 Jul 3;8(4):ooaf068. doi: 10.1093/jamiaopen/ooaf068

Effect of a generative artificial intelligence digital scribe on pediatric provider documentation time, cognitive burden, and burnout

Jonathan H Pelletier ^1,^2,^✉, Kevin Watson ^3,⁴, Jenny Michel ⁵, Robert McGregor ⁶, Sarah Z Rush ^7,⁸

PMCID: PMC12225174 PMID: 40620477

Abstract

Objective

To assess the effect of a digital scribe among pediatric providers on documentation time, cognitive load, burnout, and caregiver satisfaction.

Materials and Methods

Six-month pilot of digital scribe use in pediatric generalists and subspecialists between June 27, 2024 and February 22, 2025. We included all pilot providers with EHR record logs. Using pretest-postest and interrupted time series analysis (ITSA), we analyzed associations between digital scribe use and documentation time, NASA task load index, mini-Z burnout inventory, and caregiver satisfaction. We conducted thematic analysis of provider feedback.

Results

The digital scribe was used to generate notes for 69.5% of encounters (31 931/45 914) across 84 providers during 1454 provider-weeks. Compared with before scribe adoption, there was a mean time savings of 2.8 (95% CI, 2.1-3.5) minutes in the EHR notes activity per appointment following digital scribe adoption, averaging 1.5 (95% CI, 1.1-1.8) hours per provider-week, totaling 2143 (95% CI, 1607-2678) hours during the pilot. ITSA demonstrated 20.9% (95% CI, 17.2-24.6) time savings with digital scribe adoption. In provider surveys, there were significant reductions in all task load components (all P < .001). Burnout decreased from 54.9% (28/51) to 33.3% (17/51), P = .01. Provider-specific caregiver’s likelihood-to-recommend increased from 92.3% (IQR, 88.1%-95.5%) to 94.3% (IQR, 89.7%-100.0%), P = .01. Thematic analysis of provider feedback frequently identified decreased cognitive burden and time savings.

Discussion

Digital scribe use was associated with decreased documentation time, task load, provider burnout, and improved caregiver satisfaction. Further study is needed to determine if pilot results are scalable.

Keywords: artificial intelligence, documentation, burnout, pediatrics

Introduction

Clinicians’ notes serve myriad functions.¹ In addition to facilitating care, documentation is used for reimbursement, quality improvement, research, safety reporting, regulatory audits, and malpractice defense.^2–4 Implementation of electronic health records (EHRs) was coupled with pressure to lengthen documentation to support these functions.⁵^,⁶ Together, electronic data entry, regulatory requirements, and social pressures resulted in a higher burden on clinicians.⁷ Electronic tools such as “copy and paste” have further lengthened notes without reducing clinician time.^8–10 The overall effect has been increased documentation time compared to paper records, resulting in less face-to-face time between providers and patients.^11–13 The problem is most notable in the United States, where documentation is 4-times longer and 25% more time-consuming than peer countries.⁵^,¹⁴

Documentation burden is one important factor in clinician burnout.^15–23 Approximately half of providers report burnout, though results are heterogenous by specialty.^24–27 Clinicians suffering from burnout consistently report EHR frustrations as a contributing factor.^28–31 Increased EHR usage time is associated with burnout, particularly charting outside of scheduled work hours.^15–23 However, burnout appears to be related both to task duration and complexity. Improving EHR usability reduces provider cognitive load, which improves clinical performance and reduces burnout.^32–40 Thus, documentation burden should be considered both in terms of time and cognitive strain.

One method of reducing documentation burden is by using medical scribes. Human scribes have been associated with decreased provider EHR time and clinician burnout.^41–44 However, human scribes typically cost tens of thousands of dollars per year, which must be offset by increasing patient encounters.⁴⁵^,⁴⁶ Additionally, human scribes require dozens of hours of training and still may not achieve competency.⁴⁷ Human scribes may also suffer burnout themselves.⁴⁴

Digital scribes are a promising alternative to human scribes.^48–50 Digital scribes receive recordings of clinical conversations and use automatic speech recognition (ASR) algorithms to generate transcriptions, which are subsequently used as input to generative artificial intelligence models to create a structured clinical note.⁴⁸^,^51–54 Digital scribes are associated with decreased EHR time and improved workflow satisfaction.^55–58 However, existing studies are limited to adult providers.^55–58 While EHR documentation time is similar between adult and pediatric providers, time savings from digital scribe use may vary in pediatrics due to challenges in simultaneously obtaining history from patients and caregivers.⁵⁹ Furthermore, existing studies have not systematically assessed provider cognitive burden when using a digital scribe and have come to inconsistent conclusions regarding provider burnout.^55–58 Prior work at our institution had noted reports of provider burnout consistent with national averages. Prior informatics projects (eg, redesigning note templates to reduce EHR time) had not shown significant improvement in provider burnout. Given that our institutional experience appeared to mirror US national healthcare trends, we sought to analyze the effect of implementing a digital scribe in a heterogenous sample of pediatric healthcare providers.⁶⁰

Methods

Study population

This protocol was approved by the Akron Children’s Hospital Institutional Review Board. In 2025 Akron Children’s Hospital underwent a staged pilot trial of a commercial digital scribe (Abridge, Philadelphia, PA) to generate clinical notes and patient instructions from ambulatory pediatric encounters.⁶⁰ A business associate agreement was signed between the health system and the digital scribe vendor, and a cybersecurity review was completed as per institutional policy. As the goal of the program was to reduce clinician EHR time and burnout, the decision to undertake a staged pilot was motivated by a desire to constrain program costs while not requiring funding from individual physicians or changes in clinical productivity expectations.⁶¹ The choice of digital scribe vendor was determined by independent rankings of commercially available digital scribe products.⁶² This study included clinicians who completed a pilot trial of the digital scribe software program between June 27, 2024 and February 22, 2025. Clinical psychologists were excluded, as they do not have EHR usage records in our system. The health system initially offered digital scribe licenses to all clinicians from 3 high-volume outpatient pediatric subspecialty departments (cardiology, endocrinology, and gastroenterology). Licenses from those who declined to participate were reallocated to other clinicians on a voluntary basis. An initial analysis of digital scribe adoption was performed after 30 days. Because only volunteer status and clinical volume predicted scribe adoption, the organization then serially increased the license count. During the rolling expansion, volunteer recruitment was broadened to multiple other high-volume outpatient offices (allergy and immunology, dermatology, orthopedics, general pediatrics, nephrology, neurology, neurosurgery, psychiatry, and sports medicine).

Digital scribe workflow

The use of the digital scribe in clinical practice is outlined in Figure 1. Clinicians received a 1-hour virtual training on software use prior to implementation. Caregivers provided verbal consent and patients provided assent prior to audio recording. Following consent, clinicians recorded clinical encounters using personal smartphone devices’ speakerphone capabilities.⁵² HIPAA-compliant encrypted recordings were transmitted to United States-based cloud servers with no local storage on personal devices.⁵² All data were encrypted in transit using Transport Layer Security 1.2 or higher and at rest using Advanced Encryption Standard with a 256-bit key length.⁵² Recordings were transcribed using commercial automated speech recognition (ASR) models trained on a dataset of clinical audio with associated human transcription, clinical notes, and patient metadata.⁵² ASR model derivation and performance has been previously published.^52–54^,⁶⁰ Following ASR, clinical notes were generated from transcripts by commercial large language models (LLMs) tuned with proprietary prompt engineering and reinforcement learning by clinical reviewers.^52–54^,⁶⁰^,⁶³ Following the patient encounter, clinicians opened the electronic health record (EHR; Epic, Verona, WI) and a web-based portal provided by the digital scribe vendor (Abridge, Philadelphia, PA). Clinicians reviewed and edited the LLM-generated progress note. Clinicians also had the option to review the ASR-generated encounter transcript and to generate a patient summary designed at an 8th-grade reading level. After review, clinicians transferred the progress note from the web-based portal into the EHR via embedded SmartPhrases. The notes were signed by the clinician at the close of the encounter as per the standard workflow. Notes were intermittently audited by health information management and revenue cycle specialists per typical institutional workflow.

Workflow diagram for digital scribe use. Rectangles represent clinician workflow. Green rectangles represent process start and stop, while blue rectangles represent intermediate steps. Purple octagons represent computations, and manilla files represent computational products. Red cylinders represent data storage for the present study. Abbreviations: ASR: automated speech recognition; EHR: electronic health record; LLM: large language model.

Data sources

Study data were provided from 4 sources. First, data regarding software use, number of notes generated, and provider note feedback were extracted at the encounter-level from the digital scribe vendor. Second, weekly provider-level documentation length and time in notes and monthly provider-level EHR usage logs were extracted from the EHR analytics portal (Epic Signal, Verona, WI). EHR usage logs were queried from November 30, 2023 through February 22, 2025 (the latest date through which data were available) to ensure at least 30 weeks of pre-scribe data for interrupted time series analysis. Third, optional surveys were sent to providers before and 4 weeks after use of the digital scribe. These surveys measured self-reported EHR time, note-specific cognitive burden, burnout, self-assessed visit quality, and job satisfaction. Note-specific cognitive burden was measured using the mental demand, temporal demand, and performance indices of the NASA Task Load Index. The NASA Task Load Index is a measure of cognitive task burden that has been previously validated in multiple healthcare settings.^64–68 It is the most commonly used index to measure EHR burden; higher scores have been correlated with worse clinical performance and provider burnout.³³^,³⁴^,³⁸^,³⁹^,⁶⁹^,⁷⁰ Burnout was assessed using the burnout index of the Mini-Z survey. The Mini-Z survey has been previously validated against the Maslach Burnout Inventory and is commonly used for healthcare research due to brevity.²⁰^,⁷¹^,⁷² Fourth, monthly provider-level caregiver satisfaction data was extracted from post-encounter survey results provided routinely after ambulatory clinical encounters at our institution (NRC Health, Lincoln, NE). These surveys are sent electronically or via mail to patient’s caregivers following a subset of encounters.

Statistical analysis

Digital scribe use and note generation were described with summary statistics. EHR usage, provider survey responses, and caregiver satisfaction survey data before and after digital scribe implementation (“pre-scribe” and “post-scribe,” respectively) were compared using paired t-tests with grouping at the provider-level. The reporting period during which the software was implemented by each provider was censored to ensure that pre- and post-implementation periods were representative. As before-and-after analysis may be subject to confounding by other similarly timed unknown practice changes, EHR usage data were subsequently normalized at the provider level prior to digital scribe implementation to allow for interrupted time series analysis and determine the immediate effect of digital scribe adoption.^73–76 Last, because multiple studies have demonstrated the ability of LLMs to expedite analyses of qualitative results,^77–82 we used a general purpose LLM (GPT-4, OpenAI, San Francisco, CA, United States) to assist in thematic analysis of free-text provider comments. The LLM was used to generate lists and counts of frequently represented themes in provider survey data and requested to generate representative quotes. The LLM was specifically prompted to include both emotionally positive and negative themes.^77–82 All representative provider comments were human verified to protect against the low rates of hallucinations reported in the literature for these use cases.^77–82

Competing interest

The digital scribe vendor had no role in the decision to publish the manuscript. The vendor supplied data for the manuscript as outlined above and was given the opportunity to review the methods for accuracy in description. No financial payment or in-kind support was provided for the study.

Results

Initial digital scribe adoption and refusal

There were a total of 65 providers to whom a digital scribe license was offered in the initial 30 days. Of those, 47/65 (72.3%) adopted the digital scribe. At baseline, those who adopted the virtual scribe had a higher proportion of days with clinic appointments (median 46% [IQR, 36%-55%] vs 37% [IQR, 26%-51%], P = .049) and a lower proportion of same-day chart closure (median 79% [IQR, 42%-94%] vs 94% [IQR, 64%-98%], P = .049) compared to those who declined the virtual scribe. In addition, 92.9% (13/14) volunteers compared with 66.7% (34/51) of randomly assigned providers adopted the virtual scribe (P = .052). In multivariable regression, only volunteer status (OR 8.93 [95% CI, 1.42-179]) and percentage of days with clinic appointments (OR 1.06 [95% CI, 1.01-1.11]) remained significant predictors of scribe adoption. Because of this, the institution began recruiting volunteers from any high-volume outpatient department for subsequent rolling pilot expansion.

Digital scribe use and time savings

The rolling expansion of the pilot included 87 total providers (47 that adopted the digital scribe during the first 30 days and 40 that adopted the scribe after the first 30 days). There were 3 clinical psychologists who used the digital scribe but were excluded from the analysis due to lack of EHR usage log data. Thus, analysis was performed on 96.6% (84/87) providers who completed the digital scribe pilot. The digital scribe was used to generate notes for 69.5% of encounters (31 931/45 914) across 84 providers between June 27, 2024 and February 22, 2025. Individual providers used the digital scribe for a median of 79.0% of their encounters (IQR, 50.8%-91.2%). EHR logs are shown in Figure 2. There was a mean time savings of 2.8 (95% CI, 2.1-3.5) minutes in the EHR notes activity per appointment following digital scribe adoption. There were 45 914 appointments across 1454 provider-weeks in the pilot period. Based on this mean time savings of 2.8 (95% CI, 2.1-3.5) minutes per appointment, the total time savings across the pilot was 2143 (95% CI, 1607-2678) hours in notes. This represented a savings of 1.5 (95% CI, 1.1-1.8) hours per provider-week. When considering all EHR time (in any activity), there was a mean time savings of 14.8 (95% CI, 10.4-19.1) minutes per day. There were 7277 provider-workdays in the pilot period. Based on this mean time savings of 14.8 (95% CI, 10.4-19.1) minutes per day, the total time savings across the pilot was 1795 (95% CI, 1261-2316) hours in the EHR. This represented a savings of 1.2 (95% CI, 0.9-1.6) hours per provider-week. There was no difference in time outside scheduled hours per day (26.4 [SD 22.6] minutes vs 28.5 [SD 23.2] minutes per day, P = .11). Interrupted time series analysis is shown in Figure 3. Digital scribe implementation was associated with an immediate 20.9% (95% CI, 17.2-24.6) reduction in time in notes per appointment (P < .001). There was no significant effect of time after digital scribe implementation, though time in notes decreased by 0.2% (95% CI, 0.0-0.3) in the 30 weeks prior to scribe implementation (P = .049).

Graphs of electronic health record usage logs for digital scribe pilot users. The graph shows statistically significant reductions in Time in Notes per Appointment and Time in EHR per Day. There is no difference in Appointments Closed Same Day, and a significant increase in Progress Note Length. — Electronic health record usage logs for digital scribe pilot users. The x-axis shows the time period, and the y-axis shows the metric for each usage log. Each panel has a separate y-axis as shown. The usage log type is displayed in the gray bar above each panel. For each panel, the horizontal lines and numbers represent the mean value for each time period, and the P-values between the groups represent paired t-test values. Each individual point represents a single provider per time period. The shapes and colors of the points correspond to the time periods.

Graph of interrupted time series analysis of time in notes per appointment. The graph shows a 20.9% time savings at the time of digital scribe adoption. — Interrupted time series analysis of time in notes per appointment. The x-axis shows the reporting weekly EHR usage log reporting period start date relative to an individual provider’s adoption of the digital scribe. The y-axis shows the weekly time in notes per appointment normalized to 100% of each provider’s pre-implementation weekly time. Each point represents an individual provider-week. Colors denote points before and after digital scribe implementation. The red line shows a time series model trained on pre-implementation data, and the blue line shows a time series model trained on post-implementation data. The gray-shaded regions represent the 95% confidence interval of each model. The red dashed line shows the time in notes predicted by the pre-implementation model (the time in notes that would have been expected if the digital scribe had not been implemented).

Provider and patient surveys

Both pre- and post-scribe surveys were completed by 60.7% (51/84) providers. After digital scribe use, there was a decrease of 1.1 (95% CI, 0.4-1.8) in self-reported hours writing notes outside work per week (2.8 [SD 2.3] hours vs 3.9 [SD 2.7] hours, P = .004). Results of the provider NASA Task Load Index surveys are shown in Figure 4. On a 0-20 scale, there was a reduction of 6.9 points (95% CI, 5.7-8.1) in providers’ scoring of the mental demand of note writing. There was an 8.0 (95% CI, 6.6-9.3) point reduction in providers’ scoring of how rushed they felt during note writing, and a 7.0 (95% CI, 5.5-8.5) point reduction in providers’ scoring of how hard providers felt they had to work to accomplish their level of note-writing performance. Results of the provider burnout survey are shown in Figure 5. When responses were dichotomized, 33.3% (17/51) providers reported feeling burned out on the post-scribe survey compared to 54.9% (28/51) in the pre-scribe survey [P = .01]. In addition, 76.5% (39/51) providers agreed or strongly agreed that “[the digital scribe] has increased my satisfaction at work.” The proportion of providers who agreed or strongly agreed that they could “give my patients my undivided attention” increased after digital scribe adoption (94.1% [48/51] vs 56.9% [29/51], P < .001); 56.9% (29/51) providers agreed or strongly agreed that “[the digital scribe] has improved the quality of my notes.” However, the proportion of providers who agreed or strongly agreed that patients could understand their care plan did not significantly change (74.5% [38/51] vs 66.7% [34/51], P = .48). The proportion of providers who agreed or strongly agreed that they could add another patient to their daily schedule did not significantly change (66.7% [34/51] vs 70.6% [36/51], P = .80).

Graph of NASA task load index survey results for digital scribe pilot users. The graph shows statistically significant reductions in provider evaluations of hard work, mental demand of note writing, and rushed pace of note writing. — NASA task load index survey results for digital scribe pilot users. The x-axis shows the time period, and the y-axis shows the score. All panels use a 0-20 score. For each panel, the horizontal lines and numbers represent the mean value for each time period, and the P-values between the groups represent paired t-test values. Each individual point represents a single provider per time period. The shapes and colors of the points correspond to the time periods.

Graph of alluvial plot of burnout survey results for digital scribe pilot users. The graph shows significant improvements in reported burnout symptoms after digital scribe use. — Alluvial plot of burnout survey results for digital scribe pilot users. The x-axis shows the time period, and the y-axis shows the number of respondents in each category. The color-shaded regions represent the number of respondents in each category, shown by the color scale on the right. The numbers within the bars show the number and percent of respondents in each category. Each horizonal line represents the serial answers for a given survey respondent.

Caregiver survey results were available for 96.4% (81/84) of providers. The median provider-specific likelihood to recommend increased after digital scribe implementation to 94.3% (IQR, 89.7%-100.0%) compared with 92.3% (IQR, 88.1%-95.5%) before digital scribe use (P = .01). There was no significant change in the median provider-specific response to “my provider listened” after digital scribe implementation (87.9% [IQR, 80.0%-95.8%] vs 86.5% [IQR, 82.6%-89.7%], P = .36). There was no significant change in the median provider-specific response to “I trust my provider with my care” after digital scribe implementation (88.5% [IQR, 82.9%-95.0%] vs 87.9% [IQR, 83.3%-90.7%], P = .26).

Provider feedback

Providers submitted 47 unstructured comments on the use of the digital scribe. Overall 83.0% (39/47) comments were positive, and 17.0% (8/47) were negative. Thematic analysis of provider comments is shown in Table 1. There were a total of 7 themes identified with 61 total thematic occurrences (some comments addressed multiple themes). The most frequently identified themes were that the digital scribe allowed for reduced documentation cognitive burden, decreased documentation time, and improved work-life balance. Negative themes related to concerns related to overly general summarizations, lack of customization in behavioral health notes, and patient privacy and consent logistics.

Table 1.

Thematic analysis of provider comments.

Theme, occurrence, and description	Representative quote
Reduced documentation burden (13 occurrences)—Providers frequently reported that the scribe significantly reduced the cognitive and time burden associated with notetaking, making clinical documentation faster and less stressful.	“[The scribe] has lifted a layer of dread that I felt when faced with each patient encounter. It allows me to enjoy what I do.” “I'm not worried that I am going to forget some important detail. If I get pulled away by some emergent OR case, I don't have the burden of documentation hanging over my head.”
Increased efficiency and speed (11 occurrences)—Many providers noted significant time savings, estimating a 40%-50% reduction in documentation time.	“[The scribe has] cut my actual documentation time down by probably 40-50% JUST by producing my HPI for me.”
Improved work-life balance (10 occurrences)—The scribe helped providers complete documentation before leaving work, reducing the need for after-hours charting and improving personal time.	“I was consistently taking home 3 or 4 patient notes per day, without being able to address my inbox at all. Now I leave work on Fridays with my inbox empty and all patient notes signed.”
Enhanced patient engagement (9 occurrences)—Clinicians appreciated the ability to focus more on their patients rather than typing, leading to improved rapport and communication.	“It makes the families feel like the physician is more engaged because we actually are. I have been holding more babies and coloring with more toddlers that ever before.”
AI Integration and formatting issues (7 occurrences)—Some providers experienced challenges with how the scribe structured notes, including items in the wrong sections, difficulties capturing pertinent negatives, and challenges with template integration.	“…even though I discuss specifics of pathophysiology with parents and patients (in lay language), the app only includes general discussions of our assessment and management plans and omits specifics.”
Challenges in behavioral health use (6 occurrences)—Behavioral health providers reported that the scribe struggled to capture suicide risk assessments, mental status exams, and small progress indicators.	“I cannot figure out how to do [a mental status exam]. I attempted to record this after the patient left, but the AI does not pick it up.”
Concerns over AI bias and consent (5 occurrences)—Some patients and families declined to participate due to privacy concerns. Pediatricians also questioned whether parental consent was required for each visit or if one-time consent was sufficient.	“Most patients and families have been happy to allow me to use it, though some have expressed concerns about AI and have declined.” “…if parents consent at the first session, do they need to continue to consent each session? Is there a certain age that the patient could consent without parent being there?”

Open in a new tab

Discussion

This is the first study to analyze digital scribe use among pediatric healthcare providers. We found significant reductions in documentation time, cognitive burden, and provider burnout, with neutral provider assessment of note quality. Time savings were similar to prior adult studies.²⁷^,⁵¹^,^56–58 Similarity with the adult literature is reassuring, as digital scribe performance may be meaningfully different in pediatric contexts. While EHR documentation time for pediatricians is similar to adult providers’, pediatric healthcare visits themselves are markedly different from adults’.⁵⁹ Children are typically accompanied by an adult, requiring a digital scribe to differentiate between history relayed by the child and that of the caregiver. Furthermore, children experience both illness and relay medical history differently than adults depending on developmental stage, adding to summarization challenges for a digital scribe.

This study was also novel in demonstrating a significant reduction in provider burnout associated with digital scribe implementation, and in relating those reductions in burnout to decreased temporal and cognitive burden of documentation.⁵¹^,^55–58 While prior studies of human scribes have shown improvements in provider burnout, previous studies of digital scribes have failed to reliably demonstrate improvements in burnout despite differences in documentation time.^41–44^,⁵¹^,^55–58 In the present study, more than three-quarters of providers directly attributed improved job satisfaction to use of the digital scribe. These improvements are meaningful, as burnout is correlated with intentions to leave medicine among pediatricians.⁸³ Interestingly, in thematic analysis, decreased cognitive burden occurred more often than decreased documentation time, with several comments suggesting that providers experienced cognitive distraction from incomplete documentation when trying to perform more clinically important tasks (such as operating). These comments may offer insight as to why there was a large reduction in provider burnout despite relatively modest time savings of only 1.5 hours per provider-week. Prior work has shown improvements in burnout when evaluating LLM drafts of patient message replies despite no significant time savings.⁸⁴ Taken together, these data may suggest that reductions in cognitive load of EHR task are more important than absolute time savings when attempting to reduce provider burnout. Importantly, our study specifically did not require clinicians to increase patient volume in order to access a digital scribe, differing from some prior work.⁶¹ Whether this decision impacted clinician burnout is unclear, though physicians have previously cited increased productivity targets as a concern regarding digital scribe use.⁵⁵ Furthermore, in the present study, there was no significant increase in the proportion of providers who felt able to add another patient to their clinic day after digital scribe adoption. Thus, “replacing” cognitive burden of EHR tasks with increased productivity requirements may also be counterproductive when attempting to reduce provider burnout.

In the present study, providers estimated 1.1 hours of time saved outside of work hours. EHR usage records suggested approximately 1.5 hours of total weekly time savings, but no significant savings outside of scheduled hours. This is similar to prior studies, where provider estimates of time outside of work are often higher than those calculated in EHR logs.²⁹^,⁸⁵ Given this, others have expressed concern that EHR usage records may underestimate documentation burden due to excluding 30 minutes before and after the last scheduled appointment, ignoring work done between clinic appointments (eg, during lunch breaks or administrative time), and using a 5-second inactivity period to censor active session time.⁸⁶^,⁸⁷ Providers may consider all of these periods to be “outside scheduled hours” even if not calculated as such by the EHR. Regardless of which method is more accurate, prior work correlating EHR use with burnout has used self-reported time.¹⁵^,¹⁷

In the case of the present study, reductions in provider burnout prompted purchase of an enterprise license of the digital scribe, and rollout is ongoing across the organization. Our system is now beginning phased implementation across multiple outpatient clinics. Additionally, we plan to study the use of the tool in both emergency department and inpatient settings during the upcoming rollout. Further study is necessary to determine whether time savings and reductions in cognitive load and burnout are generalizable to these settings. While medical documentation is ubiquitous across specialties, EHR documentation time and workflows (eg, whether notes are completed during or after an encounter) varies widely depending on clinical context and provider training.^88–90 Additionally, demographic differences in subspecialty composition may influence documentation time.⁹¹

This study has important limitations. First, all pilot users were volunteers; benefits seen in volunteer samples are often less impressive when applied across an entire organization. Further studies are needed to determine whether the effect is maintained as the health system deploys the digital scribe broadly. Second, scribe use in the present pilot study was only approximately 6 months in duration. Long-term analysis is needed to determine if effects are sustained. Third, this study did not assess cost-effectiveness. Digital scribes cost several thousand dollars per year, though improved physician productivity and retention related to decreased burnout is valued at hundreds of thousands of dollars per year.⁹² Fourth, long-term studies are needed to determine if benefits are sustained beyond a pilot period. Fifth, caregiver satisfaction surveys were collected from a random sampling of clinical encounters per routine clinical practice, so may be subject to random variation that may underestimate any effect of digital scribe implementation. Sixth, during the pre-scribe period, some clinical departments underwent note template revisions to improve documentation efficiency. This may reduce the measured effect of the digital scribe on documentation time in pretest-posttest analysis and bias towards the null.

Conclusion

Digital scribe use was associated with significant improvements in documentation time, cognitive burden, and burnout among pediatric healthcare providers in this pilot study. More work is needed to determine whether these findings will be representative of ongoing organization-wide implementation.

Contributor Information

Jonathan H Pelletier, Department of Clinical Informatics, Akron Children’s Hospital, Akron, OH, 44308, United States; Division of Pediatric Critical Care Medicine, Department of Pediatrics, Akron Children’s Hospital, Akron, OH, 44308, United States.

Kevin Watson, Department of Clinical Informatics, Akron Children’s Hospital, Akron, OH, 44308, United States; Division of Pediatric Gastroenterology, Department of Pediatrics, Akron Children’s Hospital, Akron, OH, 44308, United States.

Jenny Michel, Division of Pediatric Emergency Medicine, Department of Pediatrics, Akron Children’s Hospital, Akron, OH, 44308, United States.

Robert McGregor, Department of Pediatrics, Akron Children’s Hospital, Akron, OH, United States.

Sarah Z Rush, Department of Clinical Informatics, Akron Children’s Hospital, Akron, OH, 44308, United States; Division of Pediatric Hematology/Oncology, Department of Pediatrics, Akron Children’s Hospital, Akron, OH, 44308, United States.

Author contributions

Jonathan H. Pelletier (Data curation, Formal analysis, Investigation, Methodology, Writing—original draft), Kevin Watson (Conceptualization, Methodology, Project administration, Writing—review & editing), Jenny Michel (Data curation, Methodology, Project administration, Writing—review & editing), Robert McGregor (Conceptualization, Project administration, Writing—review & editing), and Sarah Z. Rush (Conceptualization, Project administration, Writing—review & editing)

Funding

This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.

Conflicts of interest

The authors have no competing interests to declare.

Data availability

The data underlying this article were provided by Epic (Verona, WI) and Abridge (Philadelphia, PA) under license/by permission. Data will be shared on request to the corresponding author with permission of Epic and Abridge.

References

1. Johnson KB, Neuss MJ, Detmer DE. Electronic health records and clinician burnout: a story of three eras. J Am Med Inform Assoc. 2021;28:967-973. 10.1093/jamia/ocaa274 [DOI] [PMC free article] [PubMed] [Google Scholar]
2. Rosenbloom ST, Denny JC, Xu H, et al. Data from clinical notes: a perspective on the tension between structure and flexible documentation. J Am Med Inform Assoc. 2011;18:181-186. 10.1136/jamia.2010.007237 [DOI] [PMC free article] [PubMed] [Google Scholar]
3. Ebbers T, Kool RB, Smeele LE, et al. The impact of structured and standardized documentation on documentation quality; a multicenter, retrospective study. J Med Syst. 2022;46:46. 10.1007/s10916-022-01837-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
4. Ghaith S, Moore G, Colbenson K, et al. Charting practices to protect against malpractice: case reviews and learning points. West J Emerg Med. 2022;23:412-417. 10.5811/westjem.2022.1.53894 [DOI] [PMC free article] [PubMed] [Google Scholar]
5. Downing NL, Bates DW, Longhurst CA. Physician burnout in the electronic health record era: are we ignoring the real cause? Ann Intern Med. 2018;169:50-51. 10.7326/M18-0139 [DOI] [PubMed] [Google Scholar]
6. Teichman PG. Documentation tips for reducing malpractice risk. Fam Pract Manag. 2000;7:29-33. [PubMed] [Google Scholar]
7. Rule A, Bedrick S, Chiang MF, et al. Length and redundancy of outpatient progress notes across a decade at an academic medical center. JAMA Netw Open. 2021;4:e2115334. 10.1001/jamanetworkopen.2021.15334 [DOI] [PMC free article] [PubMed] [Google Scholar]
8. Apathy NC, Hare AJ, Fendrich S, et al. Early changes in billing and notes after evaluation and management guideline change. Ann Intern Med. 2022;175:499-504. 10.7326/M21-4402 [DOI] [PubMed] [Google Scholar]
9. Apathy NC, Rotenstein L, Bates DW, et al. Documentation dynamics: note composition, burden, and physician efficiency. Health Serv Res. 2023;58:674-685. 10.1111/1475-6773.14097 [DOI] [PMC free article] [PubMed] [Google Scholar]
10. Huang AE, Hribar MR, Goldstein IH, et al. Clinical documentation in electronic health record systems: analysis of similarity in progress notes from consecutive outpatient ophthalmology encounters. AMIA Annu Symp Proc. 2018;2018:1310-1318. [PMC free article] [PubMed] [Google Scholar]
11. Gaffney A, Woolhandler S, Cai C, et al. Medical documentation burden among US office-based physicians in 2019: a national study. JAMA Intern Med. 2022;182:564-566. 10.1001/jamainternmed.2022.0372 [DOI] [PMC free article] [PubMed] [Google Scholar]
12. Asan O, D. Smith P, Montague E. More screen time, less face time—implications for EHR design. Eval Clin Pract. 2014;20:896-901. 10.1111/jep.12182 [DOI] [PMC free article] [PubMed] [Google Scholar]
13. Joukes E, Abu-Hanna A, Cornet R, et al. Time spent on dedicated patient care and documentation tasks before and after the introduction of a structured and standardized electronic health record. Appl Clin Inform. 2018;9:46-53. 10.1055/s-0037-1615747 [DOI] [PMC free article] [PubMed] [Google Scholar]
14. Holmgren AJ, Downing NL, Bates DW, et al. Assessment of electronic health record use between US and Non-US health systems. JAMA Intern Med. 2021;181:251-259. 10.1001/jamainternmed.2020.7071 [DOI] [PMC free article] [PubMed] [Google Scholar]
15. Eschenroeder HC, Manzione LC, Adler-Milstein J, et al. Associations of physician burnout with organizational electronic health record support and after-hours charting. J Am Med Inform Assoc. 2021;28:960-966. 10.1093/jamia/ocab053 [DOI] [PMC free article] [PubMed] [Google Scholar]
16. Li C, Parpia C, Sriharan A, et al. Electronic medical record-related burnout in healthcare providers: a scoping review of outcomes and interventions. BMJ Open. 2022;12:e060865. 10.1136/bmjopen-2022-060865 [DOI] [PMC free article] [PubMed] [Google Scholar]
17. Gardner RL, Cooper E, Haskell J, et al. Physician stress and burnout: the impact of health information technology. J Am Med Inform Assoc. 2019;26:106-114. 10.1093/jamia/ocy145 [DOI] [PMC free article] [PubMed] [Google Scholar]
18. Giess CS, Ip IK, Cochon LR, et al. Predictors of self-reported burnout among radiology faculty at a large academic medical center. J Am Coll Radiol. 2020;17:1684-1691. 10.1016/j.jacr.2020.01.047 [DOI] [PubMed] [Google Scholar]
19. Harris DA, Haskell J, Cooper E, et al. Estimating the association between burnout and electronic health record-related stress among advanced practice registered nurses. Appl Nurs Res. 2018;43:36-41. 10.1016/j.apnr.2018.06.014 [DOI] [PubMed] [Google Scholar]
20. Olson K, Sinsky C, Rinne ST, et al. Cross-sectional survey of workplace stressors associated with physician burnout measured by the Mini-Z and the Maslach Burnout Inventory. Stress Health. 2019;35:157-175. 10.1002/smi.2849 [DOI] [PubMed] [Google Scholar]
21. Sharp M, Burkart KM, Adelman MH, et al. A national survey of burnout and depression among fellows training in pulmonary and critical care medicine. Chest. 2021;159:733-742. 10.1016/j.chest.2020.08.2117 [DOI] [PMC free article] [PubMed] [Google Scholar]
22. Tiwari V, Kavanaugh A, Martin G, et al. High burden of burnout on rheumatology practitioners. J Rheumatol. 2020;47:1831-1834. 10.3899/jrheum.191110 [DOI] [PubMed] [Google Scholar]
23. Adler-Milstein J, Zhao W, Willard-Grace R, et al. Electronic health records and burnout: time spent on the electronic health record after hours and message volume associated with exhaustion but not with cynicism among primary care clinicians. J Am Med Inform Assoc. 2020;27:531-538. 10.1093/jamia/ocz220 [DOI] [PMC free article] [PubMed] [Google Scholar]
24. Ortega MV, Hidrue MK, Lehrhoff SR, et al. Patterns in physician burnout in a stable-linked cohort. JAMA Netw Open. 2023;6:e2336745. 10.1001/jamanetworkopen.2023.36745 [DOI] [PMC free article] [PubMed] [Google Scholar]
25. Shanafelt TD, Boone S, Tan L, et al. Burnout and satisfaction with work-life balance among US physicians relative to the general US population. Arch Intern Med. 2012;172:1377-1385. 10.1001/archinternmed.2012.3199 [DOI] [PubMed] [Google Scholar]
26. Shanafelt TD, Hasan O, Dyrbye LN, et al. Changes in burnout and satisfaction with work-life balance in physicians and the general US working population between 2011 and 2014. Mayo Clinic Proc. 2015;90:1600-1613. 10.1016/j.mayocp.2015.08.023 [DOI] [PubMed] [Google Scholar]
27. Rotenstein LS, Torre M, Ramos MA, et al. Prevalence of burnout among physicians: a systematic review. JAMA. 2018;320:1131-1150. 10.1001/jama.2018.12777 [DOI] [PMC free article] [PubMed] [Google Scholar]
28. Yan Q, Jiang Z, Harbin Z, et al. Exploring the relationship between electronic health records and provider burnout: a systematic review. J Am Med Inform Assoc. 2021;28:1009-1021. 10.1093/jamia/ocab009 [DOI] [PMC free article] [PubMed] [Google Scholar]
29. Tajirian T, Stergiopoulos V, Strudwick G, et al. The influence of electronic health record use on physician burnout: cross-sectional survey. J Med Internet Res. 2020;22:e19274. 10.2196/19274 [DOI] [PMC free article] [PubMed] [Google Scholar]
30. Budd J. Burnout related to electronic health record use in primary care. J Prim Care Community Health. 2023;14:21501319231166921. 10.1177/21501319231166921 [DOI] [PMC free article] [PubMed] [Google Scholar]
31. Robertson SL, Robinson MD, Reid A. Electronic health record effects on work-life balance and burnout within the I3 population collaborative. J Graduate Med Educ. 2017;9:479-484. 10.4300/JGME-D-16-00123.1 [DOI] [PMC free article] [PubMed] [Google Scholar]
32. Iskander M. Burnout, cognitive overload, and metacognition in medicine. Med Sci Educ. 2019;29:325-328. 10.1007/s40670-018-00654-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
33. Mazur LM, Mosaly PR, Hoyle LM, et al. Relating physician’s workload with errors during radiation therapy planning. Pract Radiat Oncol. 2014;4:71-75. 10.1016/j.prro.2013.05.010 [DOI] [PubMed] [Google Scholar]
34. Lowndes BR, Forsyth KL, Blocker RC, et al. NASA-TLX assessment of surgeon workload variation across specialties. Ann Surg. 2020;271:686-692. 10.1097/SLA.0000000000003058 [DOI] [PubMed] [Google Scholar]
35. Gandhi TK, Classen D, Sinsky CA, et al. How can artificial intelligence decrease cognitive and work burden for front line practitioners? JAMIA Open. 2023;6:ooad079. 10.1093/jamiaopen/ooad079 [DOI] [PMC free article] [PubMed] [Google Scholar]
36. Staal M. Stress, Cognition, and Human Performance: A Literature Review and Conceptual Framework. National Aeronautics and Space Administration; 2004. [Google Scholar]
37. Harry E, Sinsky C, Dyrbye LN, et al. Physician task load and the risk of burnout among US physicians in a national survey. Jt Comm J Qual Patient Safety. 2021;47:76-85. 10.1016/j.jcjq.2020.09.011 [DOI] [PubMed] [Google Scholar]
38. Melnick ER, Harry E, Sinsky CA, et al. Perceived electronic health record usability as a predictor of task load and burnout among US physicians: mediation analysis. J Med Internet Res. 2020;22:e23382. 10.2196/23382 [DOI] [PMC free article] [PubMed] [Google Scholar]
39. Mazur LM, Mosaly PR, Moore C, et al. Toward a better understanding of task demands, workload, and performance during physician-computer interactions. J Am Med Inform Assoc. 2016;23:1113-1120. 10.1093/jamia/ocw016 [DOI] [PMC free article] [PubMed] [Google Scholar]
40. Asgari E, Kaur J, Nuredini G, et al. Impact of electronic health record use on cognitive load and burnout among clinicians: narrative review. JMIR Med Inform. 2024;12:e55499. 10.2196/55499 [DOI] [PMC free article] [PubMed] [Google Scholar]
41. Abbondanza L, White P. Evaluating a scribe program in reducing provider burnout. J Nurse Pract. 2024;20:104913. 10.1016/j.nurpra.2023.104913 [DOI] [Google Scholar]
42. Micek MA, Arndt B, Baltus JJ, et al. The effect of remote scribes on primary care physicians’ wellness, EHR satisfaction, and EHR use. Healthcare. 2022;10:100663. 10.1016/j.hjdsi.2022.100663 [DOI] [PubMed] [Google Scholar]
43. Stephens J, Kieber-Emmons AM, Johnson M, et al. Implementation of a virtual asynchronous scribe program to reduce physician burnout. J Healthcare Manag. 2022;67:425-435. 10.1097/JHM-D-21-00329 [DOI] [PMC free article] [PubMed] [Google Scholar]
44. Corby S, Ash JS, Mohan V, et al. A qualitative study of provider burnout: do medical scribes hinder or help? JAMIA Open. 2021;4:ooab047. 10.1093/jamiaopen/ooab047 [DOI] [PMC free article] [PubMed] [Google Scholar]
45. Miksanek TJ, Skandari MR, Ham SA, et al. The productivity requirements of implementing a Medical Scribe Program. Ann Intern Med. 2021;174:1-7. 10.7326/M20-0428 [DOI] [PubMed] [Google Scholar]
46. Carnes KM, De Riese CS, De Riese WTW. A cost-benefit analysis of medical scribes and electronic medical record system in an academic urology clinic. Urol Pract. 2015;2:101-105. 10.1016/j.urpr.2014.10.006 [DOI] [PubMed] [Google Scholar]
47. Walker KJ, Dunlop W, Liew D, et al. An economic evaluation of the costs of training a medical scribe to work in emergency medicine. Emerg Med J. 2016;33:865-869. 10.1136/emermed-2016-205934 [DOI] [PubMed] [Google Scholar]
48. Van Buchem MM, Boosman H, Bauer MP, et al. The digital scribe in clinical practice: a scoping review and research agenda. NPJ Digit Med. 2021;4:57. 10.1038/s41746-021-00432-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
49. Ghatnekar S, Faletsky A, Nambudiri VE. Digital scribe utility and barriers to implementation in clinical practice: a scoping review. Health Technol (Berl). 2021;11:803-809. 10.1007/s12553-021-00568-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
50. Falcetta FS, De Almeida FK, Lemos JCS, et al. Automatic documentation of professional health interactions: a systematic review. Artif Intell Med. 2023;137:102487. 10.1016/j.artmed.2023.102487 [DOI] [PubMed] [Google Scholar]
51. Wang J, Lavender M, Hoque E, et al. A patient-centered digital scribe for automatic medical documentation. JAMIA Open. 2021;4:ooab003. 10.1093/jamiaopen/ooab003 [DOI] [PMC free article] [PubMed] [Google Scholar]
52. Oberst M, Liang D, Lipton ZC. Pioneering the Science of AI Evaluation. Abridge; 2024. [Google Scholar]
53. Krishna K, Pavel A, Schloss B, et al. Extracting structured data from physician-patient conversations by predicting noteworthy utterances. 2020. https://arxiv.org/abs/2007.07151
54. Krishna K, Khosla S, Bigham J, et al. Generating SOAP notes from doctor-patient conversations using modular summarization techniques. In: Zong C, Xia F, Li W, et al., Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Association for Computational Linguistics; 2021:4958-4972.
55. Bundy H, Gerhart J, Baek S, et al. Can the administrative loads of physicians be alleviated by AI-facilitated clinical documentation? J Gen Intern Med. 2024;39:2995-3000. 10.1007/s11606-024-08870-z [DOI] [PMC free article] [PubMed] [Google Scholar]
56. Liu T-L, Hetherington TC, Stephens C, et al. AI-powered clinical documentation and clinicians’ electronic health record experience: a nonrandomized clinical trial. JAMA Netw Open. 2024;7:e2432460. 10.1001/jamanetworkopen.2024.32460 [DOI] [PMC free article] [PubMed] [Google Scholar]
57. Galloway JL, Munroe D, Vohra-Khullar PD, et al. Impact of an artificial intelligence-based solution on clinicians’ clinical documentation experience: initial findings using ambient listening technology. J Gen Intern Med. 2024;39:2625-2627. 10.1007/s11606-024-08924-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
58. Owens LM, Wilda JJ, Hahn PY, et al. The association between use of ambient voice technology documentation during primary care patient encounters, documentation burden, and provider burnout. Fam Pract. 2024;41:86-91. 10.1093/fampra/cmad092 [DOI] [PubMed] [Google Scholar]
59. Rotenstein LS, Holmgren AJ, Downing NL, et al. Differences in clinician electronic health record use across adult and pediatric primary care specialties. JAMA Netw Open. 2021;4:e2116375. 10.1001/jamanetworkopen.2021.16375 [DOI] [PMC free article] [PubMed] [Google Scholar]
60. Abridge. Generative AI for clinical conversations. 2024. Accessed 29 January 2024. https://www.abridge.com/
61. Rotenstein L, Melnick ER, Iannaccone C, et al. Virtual scribes and physician time spent on electronic health records. JAMA Netw Open. 2024;7:e2413140. 10.1001/jamanetworkopen.2024.13140 [DOI] [PMC free article] [PubMed] [Google Scholar]
62. KLAS. Best in KLAS Software & Services 2025. KLAS; 2025.
63. Ramprasad S, Ferracane E, Selvaraj SP. Generating more faithful and consistent SOAP notes using attribute-specific parameters. In: Deshpande K, Fiterau M, Joshi S, et al. , eds. Proceedings of the 8th Machine Learning for Healthcare Conference. PMLR; 2023:631–649. [Google Scholar]
64. Hernandez R, Roll SC, Jin H, et al. Validation of the National Aeronautics and Space Administration Task Load Index (NASA-TLX) adapted for the whole day repeated measures context. Ergonomics. 2022;65:960-975. 10.1080/00140139.2021.2006317 [DOI] [PMC free article] [PubMed] [Google Scholar]
65. Said S, Gozdzik M, Roche TR, et al. Validation of the Raw National Aeronautics and Space Administration Task Load Index (NASA-TLX) questionnaire to assess perceived workload in patient monitoring tasks: pooled analysis study using mixed models. J Med Internet Res. 2020;22:e19472. 10.2196/19472 [DOI] [PMC free article] [PubMed] [Google Scholar]
66. Tubbs-Cooley HL, Mara CA, Carle AC, et al. The NASA Task Load Index as a measure of overall workload among neonatal, paediatric and adult intensive care nurses. Intensive Crit Care Nurs. 2018;46:64-69. 10.1016/j.iccn.2018.01.004 [DOI] [PMC free article] [PubMed] [Google Scholar]
67. Mohamed R, Raman M, Anderson J, et al. Validation of the National Aeronautics and Space Administration Task Load Index as a tool to evaluate the learning curve for endoscopy training. Can J Gastroenterol Hepatol. 2014;28:155-159. 10.1155/2014/892476 [DOI] [PMC free article] [PubMed] [Google Scholar]
68. Kennedy-Metz LR, Wolfe HL, Dias RD, et al. Surgery Task Load Index in cardiac surgery: measuring cognitive load among teams. Surg Innov. 2020;27:602-607. 10.1177/1553350620934931 [DOI] [PMC free article] [PubMed] [Google Scholar]
69. Wilbanks BA, McMullan SP. A review of measuring the cognitive workload of electronic health records. Comput Inform Nurs. 2018;36:579-588. 10.1097/CIN.0000000000000469 [DOI] [PubMed] [Google Scholar]
70. Hudson D, Kushniruk AW, Borycki EM. Using the NASA Task Load Index to assess workload in electronic medical records. Stud Health Technol Inform. 2015;208:190-194. [PubMed] [Google Scholar]
71. Linzer M, McLoughlin C, Poplau S, et al. ; AMA-Hennepin Health System (HHS) burnout reduction writing team. The Mini Z Worklife and Burnout Reduction Instrument: psychometrics and clinical implications. J Gen Intern Med. 2022;37:2876-2878. 10.1007/s11606-021-07278-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
72. Linzer M, Smith CD, Hingle S, et al. Evaluation of work satisfaction, stress, and burnout among US internal medicine physicians and trainees. JAMA Netw Open. 2020;3:e2018758. 10.1001/jamanetworkopen.2020.18758 [DOI] [PMC free article] [PubMed] [Google Scholar]
73. Hategeka C, Ruton H, Karamouzian M, et al. Use of interrupted time series methods in the evaluation of health system quality improvement interventions: a methodological systematic review. BMJ Glob Health. 2020;5:e003567. 10.1136/bmjgh-2020-003567 [DOI] [PMC free article] [PubMed] [Google Scholar]
74. Lopez Bernal J, Cummins S, Gasparrini A. Interrupted time series regression for the evaluation of public health interventions: a tutorial. Int J Epidemiol. 2017;46:348-355. 10.1093/ije/dyw098 [DOI] [PMC free article] [PubMed] [Google Scholar]
75. Penfold RB, Zhang F. Use of interrupted time series analysis in evaluating health care quality improvements. Acad Pediatr. 2013;13:S38-S44. 10.1016/j.acap.2013.08.002 [DOI] [PubMed] [Google Scholar]
76. Fretheim A, Tomic O. Statistical process control and interrupted time series: a golden opportunity for impact evaluation in quality improvement. BMJ Qual Saf. 2015;24:748-752. 10.1136/bmjqs-2014-003756 [DOI] [PMC free article] [PubMed] [Google Scholar]
77. Prescott MR, Yeager S, Ham L, et al. Comparing the efficacy and efficiency of human and generative AI: qualitative thematic analyses. JMIR AI. 2024;3:e54482. 10.2196/54482 [DOI] [PMC free article] [PubMed] [Google Scholar]
78. Towler L, Bondaronek P, Papakonstantinou T, et al. Applying machine-learning to rapidly analyze large qualitative text datasets to inform the COVID-19 pandemic response: comparing human and machine-assisted topic analysis techniques. Front Public Health. 2023;11:1268223. 10.3389/fpubh.2023.1268223 [DOI] [PMC free article] [PubMed] [Google Scholar]
79. Somani S, Van Buchem MM, Sarraju A, et al. Artificial intelligence–enabled analysis of statin-related topics and sentiments on social media. JAMA Netw Open. 2023;6:e239747. 10.1001/jamanetworkopen.2023.9747 [DOI] [PMC free article] [PubMed] [Google Scholar]
80. Wosny M, Hastings J. Applying large language models to interpret qualitative interviews in healthcare. In: Mantas J, Hasman A, Demiris G, et al. , eds. Studies in Health Technology and Informatics. IOS Press; 2024;316:791-795. [DOI] [PubMed] [Google Scholar]
81. Deiner MS, Honcharov V, Li J, et al. Large language models can enable inductive thematic analysis of a social media corpus in a single prompt: human validation study. JMIR Infodemiol. 2024;4:e59641. 10.2196/59641 [DOI] [PMC free article] [PubMed] [Google Scholar]
82. Mathis WS, Zhao S, Pratt N, et al. Inductive thematic analysis of healthcare qualitative interviews using open-source large language models: how does it compare to traditional methods? Comput Meth Programs Biomed. 2024;255:108356. 10.1016/j.cmpb.2024.108356 [DOI] [PubMed] [Google Scholar]
83. Grossman Z, Chodick G, Kushnir T, et al. Burnout and intentions to quit the practice among community pediatricians: associations with specific professional activities. Isr J Health Policy Res. 2019;8:2. 10.1186/s13584-018-0268-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
84. Garcia P, Ma SP, Shah S, et al. Artificial intelligence–generated draft replies to patient inbox messages. JAMA Netw Open. 2024;7:e243201. 10.1001/jamanetworkopen.2024.3201 [DOI] [PMC free article] [PubMed] [Google Scholar]
85. Gilleland M, Komis K, Chawla S, et al. Resident duty hours in the outpatient electronic health record era: inaccuracies and implications. J Grad Med Educ. 2014;6:151-154. 10.4300/JGME-D-13-00061.1 [DOI] [PMC free article] [PubMed] [Google Scholar]
86. Sinsky CA, Rule A, Cohen G, et al. Metrics for assessing physician activity using electronic health record log data. J Am Med Inform Assoc. 2020;27:639-643. 10.1093/jamia/ocz223 [DOI] [PMC free article] [PubMed] [Google Scholar]
87. Arndt BG, Micek MA, Rule A, et al. Refining vendor-defined measures to accurately quantify EHR workload outside time scheduled with patients. Ann Fam Med. 2023;21:264-268. 10.1370/afm.2960 [DOI] [PMC free article] [PubMed] [Google Scholar]
88. Crawford S, Kushner I, Wells R, et al. Electronic health record documentation times among emergency medicine trainees. Perspect Health Inf Manag. 2019;16:1f. [PMC free article] [PubMed] [Google Scholar]
89. Iscoe MS, Venkatesh AK, Holland ML, et al. Benchmarking emergency physician EHR time per encounter based on patient and clinical factors. JAMA Netw Open. 2024;7:e2427389. 10.1001/jamanetworkopen.2024.27389 [DOI] [PMC free article] [PubMed] [Google Scholar]
90. Reznek MA, Mangolds V, Kotkowski KA, et al. Accuracy of physician self‐estimation of time spent during patient care in the emergency department. JACEP Open. 2023;4:e12923. 10.1002/emp2.12923 [DOI] [PMC free article] [PubMed] [Google Scholar]
91. Rotenstein LS, Fong AS, Jeffery MM, et al. Gender differences in time spent on documentation and the electronic health record in a large ambulatory network. JAMA Netw Open. 2022;5:e223935. 10.1001/jamanetworkopen.2022.3935 [DOI] [PMC free article] [PubMed] [Google Scholar]
92. Shanafelt T, Goh J, Sinsky C. The business case for investing in physician well-being. JAMA Intern Med. 2017;177:1826-1832. 10.1001/jamainternmed.2017.4340 [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Data Availability Statement

[ooaf068-B1] 1. Johnson KB, Neuss MJ, Detmer DE. Electronic health records and clinician burnout: a story of three eras. J Am Med Inform Assoc. 2021;28:967-973. 10.1093/jamia/ocaa274 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B2] 2. Rosenbloom ST, Denny JC, Xu H, et al. Data from clinical notes: a perspective on the tension between structure and flexible documentation. J Am Med Inform Assoc. 2011;18:181-186. 10.1136/jamia.2010.007237 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B3] 3. Ebbers T, Kool RB, Smeele LE, et al. The impact of structured and standardized documentation on documentation quality; a multicenter, retrospective study. J Med Syst. 2022;46:46. 10.1007/s10916-022-01837-9 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B4] 4. Ghaith S, Moore G, Colbenson K, et al. Charting practices to protect against malpractice: case reviews and learning points. West J Emerg Med. 2022;23:412-417. 10.5811/westjem.2022.1.53894 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B5] 5. Downing NL, Bates DW, Longhurst CA. Physician burnout in the electronic health record era: are we ignoring the real cause? Ann Intern Med. 2018;169:50-51. 10.7326/M18-0139 [DOI] [PubMed] [Google Scholar]

[ooaf068-B6] 6. Teichman PG. Documentation tips for reducing malpractice risk. Fam Pract Manag. 2000;7:29-33. [PubMed] [Google Scholar]

[ooaf068-B7] 7. Rule A, Bedrick S, Chiang MF, et al. Length and redundancy of outpatient progress notes across a decade at an academic medical center. JAMA Netw Open. 2021;4:e2115334. 10.1001/jamanetworkopen.2021.15334 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B8] 8. Apathy NC, Hare AJ, Fendrich S, et al. Early changes in billing and notes after evaluation and management guideline change. Ann Intern Med. 2022;175:499-504. 10.7326/M21-4402 [DOI] [PubMed] [Google Scholar]

[ooaf068-B9] 9. Apathy NC, Rotenstein L, Bates DW, et al. Documentation dynamics: note composition, burden, and physician efficiency. Health Serv Res. 2023;58:674-685. 10.1111/1475-6773.14097 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B10] 10. Huang AE, Hribar MR, Goldstein IH, et al. Clinical documentation in electronic health record systems: analysis of similarity in progress notes from consecutive outpatient ophthalmology encounters. AMIA Annu Symp Proc. 2018;2018:1310-1318. [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B11] 11. Gaffney A, Woolhandler S, Cai C, et al. Medical documentation burden among US office-based physicians in 2019: a national study. JAMA Intern Med. 2022;182:564-566. 10.1001/jamainternmed.2022.0372 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B12] 12. Asan O, D. Smith P, Montague E. More screen time, less face time—implications for EHR design. Eval Clin Pract. 2014;20:896-901. 10.1111/jep.12182 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B13] 13. Joukes E, Abu-Hanna A, Cornet R, et al. Time spent on dedicated patient care and documentation tasks before and after the introduction of a structured and standardized electronic health record. Appl Clin Inform. 2018;9:46-53. 10.1055/s-0037-1615747 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B14] 14. Holmgren AJ, Downing NL, Bates DW, et al. Assessment of electronic health record use between US and Non-US health systems. JAMA Intern Med. 2021;181:251-259. 10.1001/jamainternmed.2020.7071 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B15] 15. Eschenroeder HC, Manzione LC, Adler-Milstein J, et al. Associations of physician burnout with organizational electronic health record support and after-hours charting. J Am Med Inform Assoc. 2021;28:960-966. 10.1093/jamia/ocab053 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B16] 16. Li C, Parpia C, Sriharan A, et al. Electronic medical record-related burnout in healthcare providers: a scoping review of outcomes and interventions. BMJ Open. 2022;12:e060865. 10.1136/bmjopen-2022-060865 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B17] 17. Gardner RL, Cooper E, Haskell J, et al. Physician stress and burnout: the impact of health information technology. J Am Med Inform Assoc. 2019;26:106-114. 10.1093/jamia/ocy145 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B18] 18. Giess CS, Ip IK, Cochon LR, et al. Predictors of self-reported burnout among radiology faculty at a large academic medical center. J Am Coll Radiol. 2020;17:1684-1691. 10.1016/j.jacr.2020.01.047 [DOI] [PubMed] [Google Scholar]

[ooaf068-B19] 19. Harris DA, Haskell J, Cooper E, et al. Estimating the association between burnout and electronic health record-related stress among advanced practice registered nurses. Appl Nurs Res. 2018;43:36-41. 10.1016/j.apnr.2018.06.014 [DOI] [PubMed] [Google Scholar]

[ooaf068-B20] 20. Olson K, Sinsky C, Rinne ST, et al. Cross-sectional survey of workplace stressors associated with physician burnout measured by the Mini-Z and the Maslach Burnout Inventory. Stress Health. 2019;35:157-175. 10.1002/smi.2849 [DOI] [PubMed] [Google Scholar]

[ooaf068-B21] 21. Sharp M, Burkart KM, Adelman MH, et al. A national survey of burnout and depression among fellows training in pulmonary and critical care medicine. Chest. 2021;159:733-742. 10.1016/j.chest.2020.08.2117 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B22] 22. Tiwari V, Kavanaugh A, Martin G, et al. High burden of burnout on rheumatology practitioners. J Rheumatol. 2020;47:1831-1834. 10.3899/jrheum.191110 [DOI] [PubMed] [Google Scholar]

[ooaf068-B23] 23. Adler-Milstein J, Zhao W, Willard-Grace R, et al. Electronic health records and burnout: time spent on the electronic health record after hours and message volume associated with exhaustion but not with cynicism among primary care clinicians. J Am Med Inform Assoc. 2020;27:531-538. 10.1093/jamia/ocz220 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B24] 24. Ortega MV, Hidrue MK, Lehrhoff SR, et al. Patterns in physician burnout in a stable-linked cohort. JAMA Netw Open. 2023;6:e2336745. 10.1001/jamanetworkopen.2023.36745 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B25] 25. Shanafelt TD, Boone S, Tan L, et al. Burnout and satisfaction with work-life balance among US physicians relative to the general US population. Arch Intern Med. 2012;172:1377-1385. 10.1001/archinternmed.2012.3199 [DOI] [PubMed] [Google Scholar]

[ooaf068-B26] 26. Shanafelt TD, Hasan O, Dyrbye LN, et al. Changes in burnout and satisfaction with work-life balance in physicians and the general US working population between 2011 and 2014. Mayo Clinic Proc. 2015;90:1600-1613. 10.1016/j.mayocp.2015.08.023 [DOI] [PubMed] [Google Scholar]

[ooaf068-B27] 27. Rotenstein LS, Torre M, Ramos MA, et al. Prevalence of burnout among physicians: a systematic review. JAMA. 2018;320:1131-1150. 10.1001/jama.2018.12777 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B28] 28. Yan Q, Jiang Z, Harbin Z, et al. Exploring the relationship between electronic health records and provider burnout: a systematic review. J Am Med Inform Assoc. 2021;28:1009-1021. 10.1093/jamia/ocab009 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B29] 29. Tajirian T, Stergiopoulos V, Strudwick G, et al. The influence of electronic health record use on physician burnout: cross-sectional survey. J Med Internet Res. 2020;22:e19274. 10.2196/19274 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B30] 30. Budd J. Burnout related to electronic health record use in primary care. J Prim Care Community Health. 2023;14:21501319231166921. 10.1177/21501319231166921 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B31] 31. Robertson SL, Robinson MD, Reid A. Electronic health record effects on work-life balance and burnout within the I3 population collaborative. J Graduate Med Educ. 2017;9:479-484. 10.4300/JGME-D-16-00123.1 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B32] 32. Iskander M. Burnout, cognitive overload, and metacognition in medicine. Med Sci Educ. 2019;29:325-328. 10.1007/s40670-018-00654-5 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B33] 33. Mazur LM, Mosaly PR, Hoyle LM, et al. Relating physician’s workload with errors during radiation therapy planning. Pract Radiat Oncol. 2014;4:71-75. 10.1016/j.prro.2013.05.010 [DOI] [PubMed] [Google Scholar]

[ooaf068-B34] 34. Lowndes BR, Forsyth KL, Blocker RC, et al. NASA-TLX assessment of surgeon workload variation across specialties. Ann Surg. 2020;271:686-692. 10.1097/SLA.0000000000003058 [DOI] [PubMed] [Google Scholar]

[ooaf068-B35] 35. Gandhi TK, Classen D, Sinsky CA, et al. How can artificial intelligence decrease cognitive and work burden for front line practitioners? JAMIA Open. 2023;6:ooad079. 10.1093/jamiaopen/ooad079 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B36] 36. Staal M. Stress, Cognition, and Human Performance: A Literature Review and Conceptual Framework. National Aeronautics and Space Administration; 2004. [Google Scholar]

[ooaf068-B37] 37. Harry E, Sinsky C, Dyrbye LN, et al. Physician task load and the risk of burnout among US physicians in a national survey. Jt Comm J Qual Patient Safety. 2021;47:76-85. 10.1016/j.jcjq.2020.09.011 [DOI] [PubMed] [Google Scholar]

[ooaf068-B38] 38. Melnick ER, Harry E, Sinsky CA, et al. Perceived electronic health record usability as a predictor of task load and burnout among US physicians: mediation analysis. J Med Internet Res. 2020;22:e23382. 10.2196/23382 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B39] 39. Mazur LM, Mosaly PR, Moore C, et al. Toward a better understanding of task demands, workload, and performance during physician-computer interactions. J Am Med Inform Assoc. 2016;23:1113-1120. 10.1093/jamia/ocw016 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B40] 40. Asgari E, Kaur J, Nuredini G, et al. Impact of electronic health record use on cognitive load and burnout among clinicians: narrative review. JMIR Med Inform. 2024;12:e55499. 10.2196/55499 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B41] 41. Abbondanza L, White P. Evaluating a scribe program in reducing provider burnout. J Nurse Pract. 2024;20:104913. 10.1016/j.nurpra.2023.104913 [DOI] [Google Scholar]

[ooaf068-B42] 42. Micek MA, Arndt B, Baltus JJ, et al. The effect of remote scribes on primary care physicians’ wellness, EHR satisfaction, and EHR use. Healthcare. 2022;10:100663. 10.1016/j.hjdsi.2022.100663 [DOI] [PubMed] [Google Scholar]

[ooaf068-B43] 43. Stephens J, Kieber-Emmons AM, Johnson M, et al. Implementation of a virtual asynchronous scribe program to reduce physician burnout. J Healthcare Manag. 2022;67:425-435. 10.1097/JHM-D-21-00329 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B44] 44. Corby S, Ash JS, Mohan V, et al. A qualitative study of provider burnout: do medical scribes hinder or help? JAMIA Open. 2021;4:ooab047. 10.1093/jamiaopen/ooab047 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B45] 45. Miksanek TJ, Skandari MR, Ham SA, et al. The productivity requirements of implementing a Medical Scribe Program. Ann Intern Med. 2021;174:1-7. 10.7326/M20-0428 [DOI] [PubMed] [Google Scholar]

[ooaf068-B46] 46. Carnes KM, De Riese CS, De Riese WTW. A cost-benefit analysis of medical scribes and electronic medical record system in an academic urology clinic. Urol Pract. 2015;2:101-105. 10.1016/j.urpr.2014.10.006 [DOI] [PubMed] [Google Scholar]

[ooaf068-B47] 47. Walker KJ, Dunlop W, Liew D, et al. An economic evaluation of the costs of training a medical scribe to work in emergency medicine. Emerg Med J. 2016;33:865-869. 10.1136/emermed-2016-205934 [DOI] [PubMed] [Google Scholar]

[ooaf068-B48] 48. Van Buchem MM, Boosman H, Bauer MP, et al. The digital scribe in clinical practice: a scoping review and research agenda. NPJ Digit Med. 2021;4:57. 10.1038/s41746-021-00432-5 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B49] 49. Ghatnekar S, Faletsky A, Nambudiri VE. Digital scribe utility and barriers to implementation in clinical practice: a scoping review. Health Technol (Berl). 2021;11:803-809. 10.1007/s12553-021-00568-0 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B50] 50. Falcetta FS, De Almeida FK, Lemos JCS, et al. Automatic documentation of professional health interactions: a systematic review. Artif Intell Med. 2023;137:102487. 10.1016/j.artmed.2023.102487 [DOI] [PubMed] [Google Scholar]

[ooaf068-B51] 51. Wang J, Lavender M, Hoque E, et al. A patient-centered digital scribe for automatic medical documentation. JAMIA Open. 2021;4:ooab003. 10.1093/jamiaopen/ooab003 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B52] 52. Oberst M, Liang D, Lipton ZC. Pioneering the Science of AI Evaluation. Abridge; 2024. [Google Scholar]

[ooaf068-B53] 53. Krishna K, Pavel A, Schloss B, et al. Extracting structured data from physician-patient conversations by predicting noteworthy utterances. 2020. https://arxiv.org/abs/2007.07151

[ooaf068-B54] 54. Krishna K, Khosla S, Bigham J, et al. Generating SOAP notes from doctor-patient conversations using modular summarization techniques. In: Zong C, Xia F, Li W, et al., Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Association for Computational Linguistics; 2021:4958-4972.

[ooaf068-B55] 55. Bundy H, Gerhart J, Baek S, et al. Can the administrative loads of physicians be alleviated by AI-facilitated clinical documentation? J Gen Intern Med. 2024;39:2995-3000. 10.1007/s11606-024-08870-z [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B56] 56. Liu T-L, Hetherington TC, Stephens C, et al. AI-powered clinical documentation and clinicians’ electronic health record experience: a nonrandomized clinical trial. JAMA Netw Open. 2024;7:e2432460. 10.1001/jamanetworkopen.2024.32460 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B57] 57. Galloway JL, Munroe D, Vohra-Khullar PD, et al. Impact of an artificial intelligence-based solution on clinicians’ clinical documentation experience: initial findings using ambient listening technology. J Gen Intern Med. 2024;39:2625-2627. 10.1007/s11606-024-08924-2 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B58] 58. Owens LM, Wilda JJ, Hahn PY, et al. The association between use of ambient voice technology documentation during primary care patient encounters, documentation burden, and provider burnout. Fam Pract. 2024;41:86-91. 10.1093/fampra/cmad092 [DOI] [PubMed] [Google Scholar]

[ooaf068-B59] 59. Rotenstein LS, Holmgren AJ, Downing NL, et al. Differences in clinician electronic health record use across adult and pediatric primary care specialties. JAMA Netw Open. 2021;4:e2116375. 10.1001/jamanetworkopen.2021.16375 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B60] 60. Abridge. Generative AI for clinical conversations. 2024. Accessed 29 January 2024. https://www.abridge.com/

[ooaf068-B61] 61. Rotenstein L, Melnick ER, Iannaccone C, et al. Virtual scribes and physician time spent on electronic health records. JAMA Netw Open. 2024;7:e2413140. 10.1001/jamanetworkopen.2024.13140 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B62] 62. KLAS. Best in KLAS Software & Services 2025. KLAS; 2025.

[ooaf068-B63] 63. Ramprasad S, Ferracane E, Selvaraj SP. Generating more faithful and consistent SOAP notes using attribute-specific parameters. In: Deshpande K, Fiterau M, Joshi S, et al. , eds. Proceedings of the 8th Machine Learning for Healthcare Conference. PMLR; 2023:631–649. [Google Scholar]

[ooaf068-B64] 64. Hernandez R, Roll SC, Jin H, et al. Validation of the National Aeronautics and Space Administration Task Load Index (NASA-TLX) adapted for the whole day repeated measures context. Ergonomics. 2022;65:960-975. 10.1080/00140139.2021.2006317 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B65] 65. Said S, Gozdzik M, Roche TR, et al. Validation of the Raw National Aeronautics and Space Administration Task Load Index (NASA-TLX) questionnaire to assess perceived workload in patient monitoring tasks: pooled analysis study using mixed models. J Med Internet Res. 2020;22:e19472. 10.2196/19472 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B66] 66. Tubbs-Cooley HL, Mara CA, Carle AC, et al. The NASA Task Load Index as a measure of overall workload among neonatal, paediatric and adult intensive care nurses. Intensive Crit Care Nurs. 2018;46:64-69. 10.1016/j.iccn.2018.01.004 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B67] 67. Mohamed R, Raman M, Anderson J, et al. Validation of the National Aeronautics and Space Administration Task Load Index as a tool to evaluate the learning curve for endoscopy training. Can J Gastroenterol Hepatol. 2014;28:155-159. 10.1155/2014/892476 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B68] 68. Kennedy-Metz LR, Wolfe HL, Dias RD, et al. Surgery Task Load Index in cardiac surgery: measuring cognitive load among teams. Surg Innov. 2020;27:602-607. 10.1177/1553350620934931 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B69] 69. Wilbanks BA, McMullan SP. A review of measuring the cognitive workload of electronic health records. Comput Inform Nurs. 2018;36:579-588. 10.1097/CIN.0000000000000469 [DOI] [PubMed] [Google Scholar]

[ooaf068-B70] 70. Hudson D, Kushniruk AW, Borycki EM. Using the NASA Task Load Index to assess workload in electronic medical records. Stud Health Technol Inform. 2015;208:190-194. [PubMed] [Google Scholar]

[ooaf068-B71] 71. Linzer M, McLoughlin C, Poplau S, et al. ; AMA-Hennepin Health System (HHS) burnout reduction writing team. The Mini Z Worklife and Burnout Reduction Instrument: psychometrics and clinical implications. J Gen Intern Med. 2022;37:2876-2878. 10.1007/s11606-021-07278-3 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B72] 72. Linzer M, Smith CD, Hingle S, et al. Evaluation of work satisfaction, stress, and burnout among US internal medicine physicians and trainees. JAMA Netw Open. 2020;3:e2018758. 10.1001/jamanetworkopen.2020.18758 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B73] 73. Hategeka C, Ruton H, Karamouzian M, et al. Use of interrupted time series methods in the evaluation of health system quality improvement interventions: a methodological systematic review. BMJ Glob Health. 2020;5:e003567. 10.1136/bmjgh-2020-003567 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B74] 74. Lopez Bernal J, Cummins S, Gasparrini A. Interrupted time series regression for the evaluation of public health interventions: a tutorial. Int J Epidemiol. 2017;46:348-355. 10.1093/ije/dyw098 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B75] 75. Penfold RB, Zhang F. Use of interrupted time series analysis in evaluating health care quality improvements. Acad Pediatr. 2013;13:S38-S44. 10.1016/j.acap.2013.08.002 [DOI] [PubMed] [Google Scholar]

[ooaf068-B76] 76. Fretheim A, Tomic O. Statistical process control and interrupted time series: a golden opportunity for impact evaluation in quality improvement. BMJ Qual Saf. 2015;24:748-752. 10.1136/bmjqs-2014-003756 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B77] 77. Prescott MR, Yeager S, Ham L, et al. Comparing the efficacy and efficiency of human and generative AI: qualitative thematic analyses. JMIR AI. 2024;3:e54482. 10.2196/54482 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B78] 78. Towler L, Bondaronek P, Papakonstantinou T, et al. Applying machine-learning to rapidly analyze large qualitative text datasets to inform the COVID-19 pandemic response: comparing human and machine-assisted topic analysis techniques. Front Public Health. 2023;11:1268223. 10.3389/fpubh.2023.1268223 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B79] 79. Somani S, Van Buchem MM, Sarraju A, et al. Artificial intelligence–enabled analysis of statin-related topics and sentiments on social media. JAMA Netw Open. 2023;6:e239747. 10.1001/jamanetworkopen.2023.9747 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B80] 80. Wosny M, Hastings J. Applying large language models to interpret qualitative interviews in healthcare. In: Mantas J, Hasman A, Demiris G, et al. , eds. Studies in Health Technology and Informatics. IOS Press; 2024;316:791-795. [DOI] [PubMed] [Google Scholar]

[ooaf068-B81] 81. Deiner MS, Honcharov V, Li J, et al. Large language models can enable inductive thematic analysis of a social media corpus in a single prompt: human validation study. JMIR Infodemiol. 2024;4:e59641. 10.2196/59641 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B82] 82. Mathis WS, Zhao S, Pratt N, et al. Inductive thematic analysis of healthcare qualitative interviews using open-source large language models: how does it compare to traditional methods? Comput Meth Programs Biomed. 2024;255:108356. 10.1016/j.cmpb.2024.108356 [DOI] [PubMed] [Google Scholar]

[ooaf068-B83] 83. Grossman Z, Chodick G, Kushnir T, et al. Burnout and intentions to quit the practice among community pediatricians: associations with specific professional activities. Isr J Health Policy Res. 2019;8:2. 10.1186/s13584-018-0268-2 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B84] 84. Garcia P, Ma SP, Shah S, et al. Artificial intelligence–generated draft replies to patient inbox messages. JAMA Netw Open. 2024;7:e243201. 10.1001/jamanetworkopen.2024.3201 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B85] 85. Gilleland M, Komis K, Chawla S, et al. Resident duty hours in the outpatient electronic health record era: inaccuracies and implications. J Grad Med Educ. 2014;6:151-154. 10.4300/JGME-D-13-00061.1 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B86] 86. Sinsky CA, Rule A, Cohen G, et al. Metrics for assessing physician activity using electronic health record log data. J Am Med Inform Assoc. 2020;27:639-643. 10.1093/jamia/ocz223 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B87] 87. Arndt BG, Micek MA, Rule A, et al. Refining vendor-defined measures to accurately quantify EHR workload outside time scheduled with patients. Ann Fam Med. 2023;21:264-268. 10.1370/afm.2960 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B88] 88. Crawford S, Kushner I, Wells R, et al. Electronic health record documentation times among emergency medicine trainees. Perspect Health Inf Manag. 2019;16:1f. [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B89] 89. Iscoe MS, Venkatesh AK, Holland ML, et al. Benchmarking emergency physician EHR time per encounter based on patient and clinical factors. JAMA Netw Open. 2024;7:e2427389. 10.1001/jamanetworkopen.2024.27389 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B90] 90. Reznek MA, Mangolds V, Kotkowski KA, et al. Accuracy of physician self‐estimation of time spent during patient care in the emergency department. JACEP Open. 2023;4:e12923. 10.1002/emp2.12923 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B91] 91. Rotenstein LS, Fong AS, Jeffery MM, et al. Gender differences in time spent on documentation and the electronic health record in a large ambulatory network. JAMA Netw Open. 2022;5:e223935. 10.1001/jamanetworkopen.2022.3935 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ooaf068-B92] 92. Shanafelt T, Goh J, Sinsky C. The business case for investing in physician well-being. JAMA Intern Med. 2017;177:1826-1832. 10.1001/jamainternmed.2017.4340 [DOI] [PubMed] [Google Scholar]

PERMALINK

Effect of a generative artificial intelligence digital scribe on pediatric provider documentation time, cognitive burden, and burnout

Jonathan H Pelletier, MD, MS

Kevin Watson, MD

Jenny Michel, DNP

Robert McGregor, MD

Sarah Z Rush, MD

Roles

Abstract

Objective

Materials and Methods

Results

Discussion

Introduction

Methods

Study population

Digital scribe workflow

Figure 1.

Data sources

Statistical analysis

Competing interest

Results

Initial digital scribe adoption and refusal

Digital scribe use and time savings

Figure 2.

Figure 3.

Provider and patient surveys

Figure 4.

Figure 5.

Provider feedback

Table 1.

Discussion

Conclusion

Contributor Information

Author contributions

Funding

Conflicts of interest

Data availability

References

Associated Data

Data Availability Statement

ACTIONS

PERMALINK

RESOURCES

Similar articles

Cited by other articles

Links to NCBI Databases