Abstract
Introduction
Physician burnout persists in the American healthcare system. In part, this burnout is believed to be driven by the Electronic Health Record (EHR) and its fraught role in the clinical work of physicians. Artificial intelligence (AI)‐enabled healthcare technologies are often promoted on the basis of their promise to reduce burnout by introducing efficiencies into clinical work, particularly related to EHR utilization and documentation. Where documentation is perceived as the problem, AI scribes are offered as the solution.
Methods
This essay looks closely at existing studies of AI scribes in clinical context and draws upon experience and understanding of healthcare delivery and the EHR to anticipate how AI may related to provider burnout.
Results
We find that it is premature to assert that AI tools will reduce physician burnout. Considering the integration of AI scribes into Learning Health Systems healthcare delivery becomes a starting point for understanding the challenges faced in safely adopting AI tools more generally, with attention to the healthcare workforce and patients.
Conclusion
It is not a foregone conclusion that AI‐enabled healthcare technologies, in their current state and application, will lead to improved healthcare delivery and reduced burnout. Instead, this is an open question that demands rigorous evaluation and high standards of evidence before we restructure the work of physicians and redefine the care of our patients.
Keywords: artificial intelligence, documentation, electronic health records, physician burnout, program evaluation
1. INTRODUCTION
Significant physician burnout persists in the American healthcare system. For physicians, burnout can mean a diminished sense of purpose, reduced quality of life, or the end of a career. For employers, burnout can mean lower productivity and the costs of physician turnover. For patients, burnout can mean the difference between the care they deserve and the care they receive. Artificial intelligence (AI)‐enabled healthcare technologies, particularly those that assist with documentation burden, have been a focus of proposals to reduce burnout. However, studies of AI scribes in real‐world settings have not yet sufficiently demonstrated that AI will yield time savings and reductions in cognitive burden commensurate with reducing burnout.
When technologies that should revolutionize productivity fail to do so until years (or decades) after their introduction, economists call this a Productivity Paradox. 1 , 2 , 3 This Paradox—driven by the processes, structures, and cultures surrounding the new technology—may partly explain the delayed impact of AI scribes. However, reducing physician burnout to a question of documentation burden reflects the “premature certainty” and “silver bullet thinking” that neither appreciates the complex role of the Electronic Health Record (EHR) in the work of physicians nor anticipates how such tools could exacerbate burnout. 4 Furthermore, the adoption of other, future AI tools may similarly be complicated by the Productivity Paradox, the inherent limitations of AI, or complex human and socio‐technical interactions. It is premature to assume these impressive tools will lead to impressive efficiencies that lead to impressive reductions in provider burnout and improvements in patient care.
In Learning Health Systems (LHS), science, informatics, incentives, and culture are aligned toward continuous improvement and innovation. 5 The ideal LHS implementation cycle begins with the establishment of a multistakeholder learning community that embraces a process of discovery and uncertainty. 4 , 6 Yet, many LHS will evaluate AI tools already procured from vendors through purchase or partnership. Thus, LHS face, the challenge of building the capability to evaluate AI tools in real‐time during a period of distinct AI enthusiasm. To support LHS in their discovery and data interpretation processes, we review the current evidence for AI scribes and highlight the complexity of their interactions with the socio‐technical—and largely human—infrastructure in which they are embedded. We also situate physician burnout within the broader context of the LHS role in the safe adoption of AI.
2. STUDIES OF AI SCRIBES AND EHR UTILIZATION
AI has been promoted as a means of alleviating burnout by improving physician EHR experience and reducing physician administrative and clinical documentation time. To evaluate this proposition, we searched OVID Medline for peer‐reviewed studies of AI scribes in clinical practice using keywords and MeSH terms (exploded and focused) for AI, EHR, and documentation combined with modifiers and synonyms of “scribes.” After removing publications before 2020 and excluding reviews, 3805 titles resulted. Searching the World Wide Web yielded two additional studies. The “Cited by” and “Similar articles” features of PubMed were then applied to each publication to check for additional articles. While this essay is not a formal review, we identified and evaluated 13 publications. 7 , 8 , 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 Initially, the studies portend a potential to alleviate physician documentation burden and burnout with reductions in documentation time ranging from 0.7 to 2.1 min per encounter (Table 1). However, on close reading, it appears premature to assert that AI scribes will generate meaningful time savings for physicians.
TABLE 1.
Studies using Electronic Health Record (EHR) metric‐based analyses of artificial intelligence (AI) scribes use and impacts.
| Author | Year | Study period | AI products | Human editor of AI document a | Potential AI scribe users (n=) | Non‐AI scribe users, control (n=) | Enrollment | Primary outcome classification | Analysis adjusted for confounders | Rate of utilization | Rate of attrition | Utilization outcome | ||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Metric | Intervention | Control | ||||||||||||
| Rotenstein et al. | 2024 | Jan 2020 to Sep 2022 | Mixed tools | Yes | 144 (108 users of AI Scribes) | None | Voluntary | EHR metrics | Unadjusted | Not available | Consistent scribe utilization was an inclusion criteria | Total EHR time per encounter | Down 4.1 m (14%) | n/a |
| Time in notes per encounter | Down 0.7 m (9%) | n/a | ||||||||||||
| After hours EHR time per appointment (5:30 p.m.–7:00 a.m.) | Down 0.5 m (9%) | n/a | ||||||||||||
| Coa et al. | 2024 | Feb 2021 to Jan 2023 | DAX, Nuance | Unspecified, unlikely fully automated | 12 | None | Voluntary | EHR metrics and self‐report | Unadjusted | 17% (n = 2) used scribe on >70% of encounters. 66.7% (n = 8) used scribe on <50% of encounters | 17% (n = 2) withdrew from scribe use. | Total EHR time per day | Down 19.8 m (22%) | n/a |
| Time in notes per day | Down 12.4 m (23%) | n/a | ||||||||||||
| Time in notes per encounter | Down 1.4 m (22%) | n/a | ||||||||||||
| After hours EHR time per day (5:30 p.m.–7:00 a.m.) | Down 7.4 m (20%) | n/a | ||||||||||||
| Haberle et al. | 2024 | Mar 2022 to Jul 2022 | DAX, Nuance | Yes | 99 | 76 | Voluntary | Mixed objective (e.g., EHR metrics, RVU metrics) and self‐report | Unadjusted | 59% (n = 58) used scribe on 47% of encounters | 41% (n = 41) stopped or lost privileges due to low utilization | Time in note per patient (encounter) | Down 0.76 m (14%) | Down 0.15 m (3%) |
| After hours EHR time per day (6:00 p.m.‐6:00 a.m.) | Up 5% | Down 1% | ||||||||||||
| Owens et al. | 2023 | Apr 2022 to Aug 2022 | DAX, Nuance | Unspecified, unlikely fully automated | 110 (n = 19 for EHR analysis) | None | Unspecified. Authors refer to “All primary care providers” undergoing training | EHR metrics and self‐report | Unadjusted | 21% (n = 23) used scribe in 60% or more of encounters | Unspecified. Time analysis limited to “high” Dax user with 3 months of pre/post‐Dax data | Time in notes per encounter (for “high” users only) | Down 1.8 m (29%) | n/a |
| After hours note time per encounter (unspecified, for “high” users only) | Down 4 m (12%) | n/a | ||||||||||||
| Liu et al. | 2024 | Unspecified, beginning after June 2023 | DAX, Nuance | Unspecified, likely fully automated | 112 | 103 | Unspecified | Mixed objective (e.g., EHR metrics, RVU metrics) | Adjusted | 60% (n = 67) used scribe in 60% or more of encounters | 25% (n = 28) excluded from analysis for no/low utilization | Total EHR time per 8 h day | No between‐group difference | |
| Time in notes per 8 h day | No between‐group difference | |||||||||||||
| After hours EHR time per 8‐h day (undefined) | No between‐group difference | |||||||||||||
| Time in notes, high users c | Decrease 7% | |||||||||||||
| Tierney et al. | 2024 | Oct 2023 to Dec 2023 | Unspecified | Unspecified, likely fully automated | 3442 b | Yes, unspecified number | Voluntary | EHR metrics, self‐report, quality assessment | Adjusted | 28% (n = 968) used scribe 100 or more times in 10 weeks | 72% (n = 2474) used scribe in less than 10 encounters per week | Time in clinical review per appointment | No between‐group difference | |
| Time in in‐basket per encounter | No between‐group difference | |||||||||||||
| Time in notes per encounter | Scibe users with greater decrease | |||||||||||||
| Time in orders per encounter | No between‐group difference | |||||||||||||
| Time in EHR outside scheduled hours | No between‐group difference | |||||||||||||
| Time in EHR on unscheduled days | No between‐group difference | |||||||||||||
| Pajama time in EHR, not defined | No between‐group difference | |||||||||||||
| After hours EHR time (7:00 p.m.–7:00 a.m.) | Scibe users with greater decrease | |||||||||||||
| Ma et al. | 2025 | Oct 2023 to Jan 2024 | DAX, Nuance | Unspecified, likely fully automated | 50 | None | Voluntary | EHR metric | Unadjusted | 90% (n = 45) with median utilization of 55.5% of encounters | 8% (n = 4) removed for no/low utilization | Average time per note per physician | Down 0.74 m (15%) | n/a |
| Time in documentation per day | Down 11.1 m (17%) | n/a | ||||||||||||
| After hours documentation time (5:30 p.m.–7:00 a.m.) | Down 9.5 m (14%) | n/a | ||||||||||||
| Total EHR time per day | Down 24.97 m (15%) | n/a | ||||||||||||
| Duggan et al. | 2025 | Unspecified, 5 weeks | Dax, Nuance | Yes | 46 | None | Voluntary | EHR metrics, self‐report | Unadjusted | 67% (average, n = 31) used scribe for at least one encounter during a each week of the study | Not available | Time in notes per appointment | Down 2.1 m (20%) | n/a |
| After hours work per workday (unspecified) | Down 15.2 m (30%) | n/a | ||||||||||||
Note: Effort was made to use terminalogy or reflect measures that facilitate comparison between studies. Where definitions were unclear or this was not possible, author's own words or terms were left in place.
Likelihood of human reviewers paired with AI scribes is based on the technological standard at the time of the study period and what other studies using the same tools in similar study periods reported. If AI note results were reported to be available immediately, fully, the AI scribe was assumed to be fully automated without a human reviewer.
Ten thousand physicians and staff offered use of AI scribe; 3442 individuals had any use, and 984 individuals used it for more than 100 encounters in the 10‐week study period. Actual n = for analysis not specified.
Part of an exploratory analysis of as many as 40 comparisons, potentially without penalized confidence intervals. DAX, Dragon Ambient EXperience; RVU, Relative Value Unit.
These studies have significant limitations, often highlighted by their authors. First, early studies did not evaluate fully automated AI scribes. In Haberle et al. and Rotenstein et al., documents generated by AI scribes were first edited by human reviewers before returning to physicians for review and signing. 8 , 19 Although not specified, Cao et al. and Owens et al. may have used similar human reviewers: they used the same AI technology during overlapping study periods, before the company's fully automated AI scribes were announced. 15 , 18 Second, universally, AI users were self‐selected, with little information on why specific physicians utilized the AI scribes, which limits generalization of results either to a broader physician population or to compulsory AI scribe use. Third, few studies used a control group. In Tierney et al. and Liu et al., time in documentation and outside of regular working hours decreased for both non‐users and users of AI scribes. 7 , 12 After adjustment for confounders, between‐group differences became significant for two of eight measures in Tierney et al., but remained non‐significant for the eight primary outcomes in Liu et al. Only these two studies attempted adjusting for confounders.
Across the studies, outcome measures are heterogeneous, overlapping, and sometimes not clearly defined. Tierney et al. found that documentation time from 7:00 p.m. to 7:00 a.m. decreased by a larger (but unspecified) amount for AI scribe users but found no associated decrease in EHR use during “Pajama time” (undefined), unscheduled days, or outside of scheduled hours. 7 Whereas Rotenstein et al. found an average 0.5 min per visit decrease in documentation from 5:30 p.m. to 7:00 a.m. with AI scribe utilization, Haberle et al. found a 4.7% increase in documentation time from 6:00 p.m. to 6:00 a.m. 8 , 19 Moreover, most physicians do not schedule patients for 12 h per day, and defining the workday as a 12‐h period underestimates (and normalizes) after‐hours EHR utilization simply by defining the workday to be longer.
Finally, several studies presented favorable survey‐based assessments of AI scribes, but participants are non‐randomly selected, and results may be subject to the Hawthorn Effect and Demand Characteristic biases (Table 2). 9 , 11 , 13 , 16 , 17 For example, Liu et al. found that clinicians using AI scribes reported more time savings than non‐users. 11 However, their quantitative metrics did not identify additional time savings for AI scribe users versus non‐users. 12 In another example of unintentional bias, when patients knew an AI scribe was present, they were confident that their doctors were more personable, more focused on them, and typed less; however, when patients were blinded, the effect disappeared. 20
TABLE 2.
Studies using survey‐based analyses of artificial intelligence (AI) scribe use and burnout.
| Author | Year | Study period | AI products | Human editor of AI document* | Potential AI scribe users (n=) | Non‐AI scribe users, control (n=) | Enrollment | Primary outcome classification |
|---|---|---|---|---|---|---|---|---|
| Liu et al. | 2024 | Unspecified, beginning after June 2023 | DAX, Nuance | Unspecified, likely fully automated | 112 | 117 | Voluntary | Self‐report |
| Shah et al. | 2025 | Oct 2023 to Jan 2024 | DAX, Nuance | Unspecified, likely fully automated | 50 | None | Voluntary | Self‐report |
| Galloway et al. | 2024 | Unspecified | Abridge | Unspecified | 117 | None | Unspecified | Self‐report |
| MIsurac et al. | 2024 | Unspecified | Nabla | Unspecified | 38 | None | Voluntary | Self‐report |
| Nguyen et al. | 2023 | Unspecified | DAX, Nuance | Yes | 21 | None | Voluntary | Self‐report |
When not specified in methods, degree of automation is estimated based on technology in evaluates conducted during similar study peridos.
LHS must distinguish enthusiasm from real impacts on physicians and patients. Additional considerations for LHS beginning to evaluate the impact of AI scribes are in Table 3.
TABLE 3.
Considerations for learning health systems evaluating artificial intelligence (AI) scribes.
| Multistakeholder team | A robust data analysis and interpretation will include multiple stakeholders. The analytical team should extend beyond the product's customer service team or the direct reports of the individuals overseeing its adoption. |
| Personnel time | Labor may become a bottleneck for improvement cycle evaluations. Many LHS will not have the labor bandwidth for rapidly extracting, organizing, and analyzing information from the EHR for meaningful conclusions. |
| Hardware and software infrastructure | As AI scribes and tools become more pervasive, active monitoring may require attention to large amounts of data that require processing power to organize and analyze in interactive processes, which may tax existing infrastructure. |
| Create a control group for comparison | Trends—in appointment times and productivity, for example, may emerge that have little to do with the technologies being evaluated. Creating a control group for comparison is an important step in contextualizing results. |
| Stratify or adjust data | While adjusting by basic demographic information is important, trends may be confounded by additional user characteristics, such as patient volume, clinical specialty, and Full‐Time Equivalent (FTE). |
| Include objective measurements | Using objectively measured data is important. Most AI tool are adopted on a largely voluntary basis; it may be difficult to discern enthusiasm from measurable impacts. |
| Include a measure total EHR use per encounter | A total encounter time measure should include work on non‐index appointment days. or this can be captured in additional measures. |
| Use the 5:30 p.m.–7:00 a.m. definition of “After Hours” | A metric that reflects a healthy workday should be the standard in analyses of burnout, though this may not be the default metric for existing analytical tools. The time from 5:30 p.m. to 7:30 a.m. may be its own metric as well, as this may have separate, significant meaning. |
| Include a measure of delinquency or time to encounter closure | Time to encounter closure may be significant both to compliance and to burnout, be sure that the metric includes closure on non‐work days. |
| Monitor utilization rate and attrition rate | Understanding if utilization increases, decreases or stays the same will yield information about ease of use and clinician preference; attending to rates of tool adoption and withdrawal of user participation will help clarify a tool's potential impact on burnout and invite opportunities to explain patterns. |
| Monitor quality of AI scribe output | Few evaluations of AI scribes have included any form of documentation quality assurance, and still fewer have assessed relationships with measured patient outcomes. This could include a direct link to error reporting, daily or weekly reporting requests, or development of measurement tools. |
| Additional considerations | Is participation targeted to volunteers or all‐comers? Is participation associated with funding or productivity expectations? |
Abbreviations: EHR, Electronic Health Record; LHS, Learning Health Systems.
3. ARTIFICIAL INTELLIGENCE SCRIBES AND PHYSICIAN WORKLOAD IN THE EHR ECOSYSTEM
Reducing the EHR's effect—good or bad—on physician burnout to a question of documentation burden alone fails to appreciate that the EHR is a complex ecosystem tasked with balancing the needs of patient care, health systems operations, and billing. Physicians spend an average of 43%–52% of a workday in the EHR, yet documentation represents only 23%–30% of that time. 21 , 22 , 23 , 24 , 25 The EHR is a clearinghouse of increasing amounts of data that physicians navigate to make medical decisions and the platform where non‐procedural physicians execute their essential functions: review, plan, document, order, communicate. Disruptions to EHR workflows can introduce friction that undermines physicians and healthcare teams as they work.
Sepsis early warning systems (SEWS) illustrate how an EHR function can unintentionally introduce friction into physician workflows with unclear benefit to patients. Kamran et al. recently reported that after removing model inputs that depended on physician identification or anticipation of sepsis, the widely adopted sepsis prediction system evaluated was no more accurate than a coin flip (Receiver‐operating Characteristic [ROC] 0.49). 26 The costs of these alerts are not negligible: in addition to alert fatigue, they generate substantial work, such as new communication within a healthcare team, patient reevaluations, and lab work to be ordered, collected, processed, reviewed, and billed. Records‐based alerts have been identified as the least useful part of the EHR and contribute to burnout. 27 Kamran et al. also found that 95% of alerts sent were for patients who did not become septic. Given that interruptions undermine high‐quality cognitive work, and that cognitive work is important to clinical care, tools that randomly generate additional tasks for a patient care team may harm patients or delay care in ways not yet measured. 28 , 29
Low utilization and high discontinuation rates suggest that AI scribes could similarly introduce friction (Table 1). For example, in Tierney et al., of the 3442 individuals who enabled the AI scribe, less than 30% (n = 968) used it for more than 100 encounters in the 10‐week study period. 7 Across the 13 studies, little data are presented to explain low utilization. Even for self‐selected physicians, an unidentified aspect of AI scribe use caused sufficient friction to reduce utilization.
A possible explanation is that editing AI‐generated documents exacts a triple tax on physicians. First, supervising AI‐generated output is demanding. In a random sample of 35 AI‐generated documents, Tierney et al. found at least three significant errors: AI documented a clinical exam as performed that had only been discussed; diagnosed a patient with hand‐foot‐mouth disease based on discussions of a patient's hand, feet, and mouth; and, omitted discussions of chest pain and anxiety from an encounter. 7 AI errors may be both predictably biased and substantially unpredictable: one study of multiple AI‐voice recognition transcription tools consistently found nearly twice the errors for African American speakers as for white speakers, while another study of a single AI transcription tool found high variability in what key data points were successfully documented. 30 , 31 Humans signing AI documents must remain vigilant.
Second, that the editing of AI‐generated documents is likely to happen later may exacerbate burnout. Already, much physician EHR utilization happens later. For example, 10%–17% of documentation time and as much as 42% of total EHR use for a given patient encounter occur after hours. 22 , 23 , 25 , 32 Importantly, recent work suggests that the cumulative time a patient chart is open (e.g., from the beginning of the clinic visit until whenever it is finally closed) is a better predictor of burnout than total EHR use time. 33 Open charts may function like open tabs on an internet browser: out of sight and seemingly out of mind, but nevertheless taxing the physician's “working memory.” Despite time savings, shifting work to be done later may lead to more tasks that remain unfinished at the end of a day, greater background strain on physician cognition, and more burnout.
Third, editing AI notes may complicate cognitive work by changing key steps in how physicians practice medicine and introducing a new kind of task. Before, during, and after patient appointments, physicians draw upon layers of information recall, foraging, interpretation, and integration to understand their patients and document these assessments. Time reviewing, organizing, and acting upon data supports high‐quality patient care. Conversely, less time in the EHR and shorter clinical encounters are associated with lower quality patient care, such as poorer diabetes control, lower cancer screening, and increased prescribing of inappropriate antibiotics and narcotics. 34 , 35 Yet, AI scribes appear to reduce EHR time by a greater magnitude than they reduce documentation time (Table 1). It is not clear what EHR activities were reduced or discontinued and whether patient care was impacted. Finally, AI scribes ask physicians not only to perform the additional work of ensuring AI has built a meaningful representation of the patient, but require notes to be adjusted for accuracy. Beyond reallocation of time, editing AI documents broadens the scope of a physician's work that impacts patient care, the accuracy of the medical record and the personal liability of a physician's signature.
4. THE HUMAN WORKFORCE IN A LEARNING HEALTH SYSTEM
The human benefits and costs of AI scribes are still largely unproven and undiscovered, yet we are beginning to see their widespread implementation. With growing patient populations and increasing competition, healthcare systems are under pressure to innovate and to perform financially. Embracing AI is promoted toward these ends. However, rapid AI adoption could introduce new stress within LHS as the apparent needs of healthcare systems are not always aligned with those of their human workforce.
The financing of healthcare in the United States makes reimbursement a central activity of healthcare systems. 36 A longstanding critique of the EHR is that the particular state of documentation reflects this financial priority at the expense of bloated, convoluted clinical notes and the time clinicians expend foraging for information relevant to patient care. 37 , 38 , 39 If AI scribes generate higher accounts receivable than human clinicians, they may become the standard of documentation required for employed physicians, regardless of any association with burnout.
For healthcare systems adopting AI tools, efficiency and time savings alone do not guarantee a return on investment; increased productivity does. If AI scribes create dependable time savings in clinical care, a parallel question will be how that time and its value will be allocated. Documentation support may enable physicians to see more patients per day and increase access to care, but, as noted, quality may suffer as appointment times shrink and EHR utilization is reduced. 34 , 35 If clinicians feel they are providing lower quality care to more patients, burnout may increase. 40
A challenging feature of the EHR is the relative ease of changing workflows and insufficient incentives to account fully for their human impacts can allow unintended consequences to arise and persist without full evaluation. For example, SEWS are assumed to have little or no harms. However, it seems more likely that their costs—the burden on the workforce (namely nurses, physicians, and auxiliary services), the additional phlebotomy pokes to patients and blood forfeited, the additional traffic on laboratory equipment, and potential delays in care for non‐“septic” patients—are unmeasured. Because these costs are disaggregated, borne by individuals (e.g., nurses, physicians, and patients), hard to measure and reasonably attributed to normal hospital operations, they can be overlooked and there are few incentives to anticipate or account for them entirely.
When tools introduce friction, human labor—with its flexibility, common sense, and problem‐solving—adjusts to smooth operations and absorb the costs of additional time and energy expenditure to make healthcare delivery work. However, given this inherent capacity of the workforce to adapt, the difficulty of measuring dispersed impacts, and the lack of direct incentives to do so, it may be hard to know when new tools—including future generations of AI—generate negative impacts. In an environment demanding rapid change within healthcare systems, the specter of burnout asks that we consider how unintended consequences could arise and persist unmeasured and unquantified.
5. THE NEED FOR LEARNING HEALTH SYSTEMS
AI adoption appears likely to proceed without the pretesting, publication, and peer review of clinical application to which medicine is accustomed. Whereas AI scribes have the relatively narrow application of reducing documentation burden and commensurate limited impact on patient care, the implications of future AI tools may reach much farther. Consensus is building that health systems will themselves be responsible for keeping patients safe when implementing AI tools. While it is essential to understand the limits of AI (Table 4), a full review is beyond the scope of this essay. Nevertheless, the Productivity Paradox and Dataset Shift, as examples of AI limitations, underscore the challenges facing AI and motivate a discussion of the LHS infrastructure required for safe AI adoption.
TABLE 4.
| Artificial intelligence tools are built on algorithms that employ various machine learning and large language model techniques for processing data to generate an output. We continue to learn about the limits of AI tools and how to address them | |
| Hallucinations | AI has been observed to fabricate information (output) that would seem to fit well given the context of the task at hand, but is not factual. For example, in a recent study of AI scribes, AI hallucinated a positive exam finding and assessment, as well as that a treating procedure had been performed. |
| Bias | Model performance may reflect or amplify biases embedded within training data sets or the choices engineers make during training in ways that users are unable to detect or anticipate. |
| Brittleness | The performance of even powerful AI tools may suffer when small, unanticipated or irregular inputs are presented. |
| Forgetting | As models are updated to improve performance on specific new tasks, performance of tasks for which it was previously trained can decline. |
| Accountability | It remains unclear whether the company developing an tool, the healthcare system providing it to clinicians, or the clinician using the tool in patient care is accountable for deleterious effects of AI, should tool failure occur. |
| Dataset shift | Model performance can degrade if input data differs from the data upon which the model was trained. This “shift” in a dataset may come from a model being applied to a different population, demographic changes within a population, or novel application of the tool to a population for which it was not trained. |
| Explainability | It has been observed that AI model accuracy sometimes comes at the expense of explainability, or the ability to explain how the model arrived at its output. Novel machine learning methods occupy parametric spaces that are difficult to explain, making it difficult to explain model output. |
| Out of the loop problem | How humans and AI will work together remains unknown, but there remain concerns that the particular ways AI are integrated into operations can degrade human competence. For example, the “out of the loop” problem reflects the observation as automatic tools become more intelligent and capable, human skill degrades and humans may become less able to oversee credibly automated tool function or to stand‐in should the automated tool fail. |
| Data privacy | AI models use large amounts of patient data for training and operation. Ethical questions about patient consent for data sharing and logistic questions about data security through the stages of AI tool development are unresolved. |
| Environmental wellbeing | Currently, AI models and the computing infrastructure that support them are energy intensive. Though future AI models will likely use less energy per process, the widespread application of AI models may entail significant demands on global energy and water resources. |
The Productivity Paradox highlights that how a new tool—AI or analog—performs depends on the context into which it is introduced. 1 , 2 , 3 A team of researchers identified this critical role for local socio‐technical infrastructure after moving their AI retinopathy screening tool from the laboratory to the clinic. Because the real clinical settings were more varied and less ideal than a laboratory for retinal photography, tool performance declined. The potential time and financial costs to patients that followed were averted though human intervention. 49 The authors concluded, “… end‐users and their environment determine how a new system will be implemented … implementation is of equal importance to the accuracy of the algorithm itself, and cannot always be controlled through careful planning.”(Italics added) Successful AI adoption will require healthcare organizations have the capacity to tailor integration to their own socio‐technical infrastructures.
Algorithmic accuracy in one setting is no guarantee of performance in another. Instead, accuracy can decline when tools are applied in novel settings or even in the same settings, but at later times. 41 , 42 This phenomenon, “Dataset Shift,” occurs when training data are sufficiently different from input data that model performance declines. Moreover, we continue to learn of new AI limits: recent work showed that AI tool performance can suffer when inputs shift from training data to AI‐generated data. 50 By extension, the performance of the current generation of AI tools may decline as AI‐generated documents and data replace human‐generated data in the EHR.
Each organization may be responsible for ensuring the algorithmic accuracy of its own tools through continuous monitoring and updates. However, how and when a LHS will know an algorithm needs updating, or who will experience the performance decline first—patients, physicians, or yet‐to‐be‐established monitors—remains unknown. Additionally, guidance on what methods a Learning Health System could use to monitor tool performance is scant. In 2024, Andersen et al. reviewed published methods for monitoring AI and found 39 suggestions, only 23% of which had been tested or implemented in a clinical setting. 51 The authors also noted that the most proposed metrics depend on assumptions about underlying data structures that are unlikely to hold with AI tools.
Similarly, frameworks for the safe implementation of AI tools articulate what should be expected of organizations and AI when building AI‐enabled healthcare systems. For example, Novak et al. argue that after developing the competencies, resources, and infrastructures for routinely assessing, implementing, monitoring, and evaluating AI tools, organizations can accomplish the degree of control over processes that yields safe operations. 52 To this end, the FUTURE‐AI consensus guidelines proposes best practices for each step in the healthcare AI lifecycle to ensure “trustworthy and deployable” AI. 53 However, even just one FUTURE‐AI recommendation can imply large potential LHS investments needed to do the task well across multiple AI tools and different clinical settings. For example, the recommendation to enhance traceability during AI by “[Implementing a] logging framework capturing all interactions; User action, AI inputs, AI outputs, clinical decisions,” could necessitate large amounts of processing power and person power (informaticists to extract this data routinely across myriad tools and applications, statisticians to interpret performance metrics, computer engineers to localize and fix problems within models, and oversight committees to discuss risk and utility) required to do the task well.
Operating without established best practices is daunting, but precisely the challenge for which the LHS was conceptualized. 54 Moreover, existing infrastructure can be leveraged to accomplish basic steps toward safe AI adoption, such as active monitoring. LHS are uniquely positioned both to leverage implementation cycles to collect rapidly the data needed to assess AI tool impact on their patients and their workforce and to generate guidance as implementation continues. Kamel Rahimi et al. provide a useful outline of the organizational infrastructure needed for monitoring AI tool performance. 55 Their review outlines 28 healthcare system attributes that enable safe and productive AI adoption and 13 system barriers that must be overcome to effectively monitor AI in real time. Though the best methods for actively monitoring AI are undefined and doing so actively at scale may require significant investment, these frameworks help LHS understand the capabilities that will be needed to utilize AI safely.
6. CONCLUSION
There are no foregone conclusions on how AI will affect the organization of clinical work, the role of the EHR ecosystem, and the nature of physician burnout and patient care. Studies of AI scribes suggest time savings and potential burnout reductions but have inherent limitations that undermine these conclusions. LHS must independently assess AI scribes in real time, including if and how physician time is rearranged or cognitive labor changed, and if patient care is impacted. We provide considerations for LHS undertaking this work (Table 3).
While it is reasonable to be optimistic about AI tools, many steps remain between high‐functioning algorithms, successful integration into LHS, and improved patient care. Nearly every aspect of AI integration—from assessing a tool before purchasing, to training clinicians for safe use, to identifying what metrics can measure a model's performance—requires further study. Additionally, beyond merely monitoring AI, LHS need new tools for measuring the impact of EHR changes on clinician workflow, the shape of human‐AI collaboration in clinical practice, the quality of physician information processing, and whether these impacts affect burnout or patient care. 56 As individual LHS begin navigating the implementation of AI tools, each step toward real‐time AI monitoring, even of AI scribes, is an opportunity to begin addressing the enablers and barriers to AI tool adoption. LHS that aspire to adopt more powerful AI tools safely must start building their institutional capacity now.
CONFLICT OF INTEREST STATEMENT
Dr Hogarth is an advisor for Medeloop.ai. The remaining authors declare no conflicts of interest.
ACKNOWLEDGMENTS
Our team is supported by grant R01CA273058 from the National Cancer Institute. The contents of this manuscript are solely the responsibility of the authors and do not represent the official view of the National Cancer Institute. Our team would like to acknowledge Dr. Richard Kravitz for his time, feedback, and thoughts on the manuscript and the nature of physician burnout and the Electronic Health Record.
Goodson DA, Garcia B, Hogarth M, Tu S‐P. Artificial intelligence and physician burnout: A productivity paradox. Learn Health Sys. 2025;9(4):e70013. doi: 10.1002/lrh2.70013
REFERENCES
- 1. Wachter RM, Brynjolfsson E. Will generative artificial intelligence deliver on its promise in health care? Jama. 2024;331(1):65‐69. doi: 10.1001/jama.2023.25054 [DOI] [PubMed] [Google Scholar]
- 2. Brynjolfsson E, Rock D, Syverson C. Artificial intelligence and the modern productivity paradox: a clash of expectations and statistics. NBER Working Paper Series; 2017.
- 3. Brynjolfsson E. The productivity paradox of information technology. Bus Comput. 1993;36(12):66‐67. doi: 10.1145/163298 [DOI] [Google Scholar]
- 4. Friedman CP. What is unique about learning health systems? Learn Health Syst. 2022;6(3):e10328. doi: 10.1002/lrh2.10328 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Olsen L, Aisner D, McGinnis J. The Learning Healthcare System. Institute of Medicine; 2007. [PubMed] [Google Scholar]
- 6. Friedman CP, Lomotan EA, Richardson JE, Ridgeway JL. Socio‐technical infrastructure for a learning health system. Learn Health Syst. 2024;8(1):e10405. doi: 10.1002/lrh2.10405 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Tierney AA, Gayre G, Hoberman B, et al. Ambient artificial intelligence scribes to alleviate the burden of clinical documentation. NEJM Catal. 2024;5(3). doi: 10.1056/cat.23.0404 [DOI] [Google Scholar]
- 8. Rotenstein L, Melnick ER, Iannaccone C, et al. Virtual scribes and physician time spent on electronic health records. JAMA Netw Open. 2024;7(5):e2413140. doi: 10.1001/jamanetworkopen.2024.13140 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Nguyen OT, Turner K, Charles D, et al. Implementing digital scribes to reduce electronic health record documentation burden among cancer care clinicians: a mixed‐methods pilot study. JCO Clin Cancer Inform. 2023;7:e2200166. doi: 10.1200/cci.22.00166 [DOI] [PubMed] [Google Scholar]
- 10. Ma S, Liang A, Shah S, et al. Ambient artificial intelligence scribes: utilization and impact on documentation time. J Am Med Inform Assoc. 2024;32(2):381‐385. doi: 10.1093/jamia/ocae304 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Liu TL, Hetherington TC, Stephens C, et al. AI‐powered clinical documentation and clinicians' electronic health record experience: a nonrandomized clinical trial. JAMA Netw Open. 2024;7(9):e2432460. doi: 10.1001/jamanetworkopen.2024.32460 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Liu T‐L, Hetherington TC, Dharod A, et al. Does AI‐powered clinical documentation enhance clinician efficiency? A longitudinal study. N Engl J Med AI. 2024;1(12). doi: 10.1056/AIoa2400659 [DOI] [Google Scholar]
- 13. Galloway JL, Munroe D, Vohra‐Khullar PD, et al. Impact of an artificial intelligence‐based solution on clinicians' clinical documentation experience: initial findings using ambient listening technology. J Gen Intern Med. 2024;39(13):2625‐2627. doi: 10.1007/s11606-024-08924-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Duggan MJ, Gervase J, Schoenbaum A, et al. Clinician experiences with ambient scribe technology to assist with documentation burden and efficiency. JAMA Netw. 2025;8(2):e2460637. doi: 10.1001/jamanetworkopen.2024.60637 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Cao DY, Silkey JR, Decker MC, Wanat KA. Artificial intelligence‐driven digital scribes in clinical documentation: pilot study assessing the impact on dermatologist workflow and patient encounters. JAAD Int. 2024;15:149‐151. doi: 10.1016/j.jdin.2024.02.009 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Shah SJ, Devon‐Sand A, Ma SP, et al. Ambient artificial intelligence scribes: physician burnout and perspectives on usability and documentation burden. J Am Med Inform Assoc. 2025;32(2):375‐380. doi: 10.1093/jamia/ocae295 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Misurac J, Knake LA, Blum JM. The effect of ambient artificial intelligence notes on provider burnout. Appl Clin Inform. 2024;16(2):252‐258. doi: 10.1055/a-2461-4576 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Owens LM, Wilda JJ, Hahn PY, Koehler T, Fletcher JJ. The association between use of ambient voice technology documentation during primary care patient encounters, documentation burden, and provider burnout. Fam Pract. 2024;41(2):86‐91. doi: 10.1093/fampra/cmad092 [DOI] [PubMed] [Google Scholar]
- 19. Haberle T, Cleveland C, Snow GL, et al. The impact of nuance DAX ambient listening AI documentation: a cohort study. J Am Med Inform Assoc. 2024;31(4):975‐979. doi: 10.1093/jamia/ocae022 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Owens LM, Wilda JJ, Grifka R, Westendorp J, Fletcher JJ. Effect of ambient voice technology, natural language processing, and artificial intelligence on the patient‐physician relationship. Appl Clin Inform. 2024;15(4):660‐667. doi: 10.1055/a-2337-4739 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Sinsky C, Colligan L, Li L, et al. Allocation of physician time in ambulatory practice: a time and motion study in 4 specialties. Ann Intern Med. 2016;165(11):753‐760. doi: 10.7326/M16-0961 [DOI] [PubMed] [Google Scholar]
- 22. Overhage JM, McCallie D Jr. Physician time spent using the electronic health record during outpatient encounters: a descriptive study. Ann Intern Med. 2020;172(3):169‐174. doi: 10.7326/M18-3684 [DOI] [PubMed] [Google Scholar]
- 23. Arndt BG, Beasley JW, Watkinson MD, et al. Tethered to the EHR: primary care physician workload assessment using EHR event log data and time‐motion observations. Ann Fam Med. 2017;15(5):419‐426. doi: 10.1370/afm.2121 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Pinevich Y, Clark KJ, Harrison AM, Pickering BW, Herasevich V. Interaction time with electronic health records: a systematic review. Appl Clin Inform. 2021;12(4):788‐799. doi: 10.1055/s-0041-1733909 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. Rotenstein LS, Holmgren AJ, Horn DM, et al. System‐level factors and time spent on electronic health records by primary care physicians. JAMA Netw Open. 2023;6(11):e2344713. doi: 10.1001/jamanetworkopen.2023.44713 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Kamran F, Tjandra D, Heiler A, et al. Evaluation of sepsis prediction models before onset of treatment. N Engl J Med AI. 2024;1(3). [Google Scholar]
- 27. Holmgren AJ, Hendrix N, Maisel N, et al. Electronic health record usability, satisfaction, and burnout for family physicians. JAMA Netw Open. 2024;7(8):e2426956. doi: 10.1001/jamanetworkopen.2024.26956 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Scientists need more time to think. Nature. 2024;631:709.39048687 [Google Scholar]
- 29. McCoy L, Manrai A, Rodman A. Large language models and the degradation of the medical record. N Engl J Med. 2024;391(17):1561–1564. [DOI] [PubMed] [Google Scholar]
- 30. Koenecke A, Nam A, Lake E, et al. Racial disparities in automated speech recognition. Proc Natl Acad Sci U S A. 2020;117(14):7684‐7689. doi: 10.1073/pnas.1915768117 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31. Kernberg A, Gold JA, Mohan V. Using ChatGPT‐4 to create structured medical notes from audio recordings of physician‐patient encounters: comparative study. J Med Internet Res. 2024;26:e54419. doi: 10.2196/54419 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32. Holmgren AJ, Sinsky CA, Rotenstein L, Apathy NC. National comparison of ambulatory physician electronic health record use across specialties. J Gen Intern Med. 2024;39(14):2868‐2870. doi: 10.1007/s11606-024-08930-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33. Shah M, De Arrigunaga S, Forman LS, West M, Rowe SG, Mishuris RG. Cumulated time to chart closure: a novel electronic health record‐derived metric associated with clinician burnout. JAMIA Open. 2024;7(1):ooae009. doi: 10.1093/jamiaopen/ooae009 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34. Neprash HT, Mulcahy JF, Cross DA, Gaugler JE, Golberstein E, Ganguli I. Association of primary care visit length with potentially inappropriate prescribing. JAMA Health Forum. 2023;4(3):e230052. doi: 10.1001/jamahealthforum.2023.0052 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35. Rotenstein LS, Holmgren AJ, Healey MJ, et al. Association between electronic health record time and quality of care metrics in primary care. JAMA Netw Open. 2022;5(10):e2237086. doi: 10.1001/jamanetworkopen.2022.37086 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36. Starr P. The Social Transformation of American Medicine. Basic Books; 1984. [Google Scholar]
- 37. Rule A, Bedrick S, Chiang MF, Hribar MR. Length and redundancy of outpatient Progress notes across a decade at an Academic Medical Center. JAMA Netw Open. 2021;4(7):e2115334. doi: 10.1001/jamanetworkopen.2021.15334 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38. Steinkamp J, Kantrowitz JJ, Airan‐Javia S. Prevalence and sources of duplicate information in the electronic medical record. JAMA Netw Open. 2022;5(9):e2233348. doi: 10.1001/jamanetworkopen.2022.33348 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39. Downing NL, Bates DW, Longhurst CA. Physician burnout in the electronic health record era: are we ignoring the real cause? Ann Intern Med. 2018;169(1):50‐51. doi: 10.7326/M18-0139 [DOI] [PubMed] [Google Scholar]
- 40. Rosenbaum L. Beyond moral injury – can we reclaim agency, belief and joy in medicine? N Engl J Med. 2024;390(10):951–955. [DOI] [PubMed] [Google Scholar]
- 41. Lea A, Phil D, Jones D. Mind the gap – machine learning, dataset shift, and history in the age of clinical algorithms. N Engl J Med. 2024;390(4):293‐295. [DOI] [PubMed] [Google Scholar]
- 42. Finlayson S, Subbaswamy A, Singh K, et al. The clinician and dataset shift in artificial intelligence. N Engl J Med. 2021;385(3):283‐286. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43. Baker HP, Dwyer E, Kalidoss S, Hynes K, Wolf J, Strelzow JA. ChatGPT's ability to assist with clinical documentation: a randomized controlled trial. J Am Acad Orthop Surg. 2024;32(3):123‐129. doi: 10.5435/JAAOS-D-23-00474 [DOI] [PubMed] [Google Scholar]
- 44. Ren S, Wierman A. The uneven distribution of AI's environmental impacts. Harv Bus Rev. 2024;15:2024. [Google Scholar]
- 45. Yang Q, Zhang C, Hines K, Calder LA. Improved hospital safety performance and reduced medicolegal risk: an ecological study using 2 Canadian databases. CMAJ Open. 2018;6(4):E561‐E566. doi: 10.9778/cmajo.20180077 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46. Evans BJ, Bihorac A. Co‐creating consent for data use—AI‐powered ethics for biomedical AI. N Engl J Med AI. 2024;1(7). doi: 10.1056/AIpc2400237 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47. Jeyaraman M, Balaji S, Jeyaraman N, Yadav S. Unraveling the ethical enigma: artificial intelligence in healthcare. Cureus. 2023;15(8):e43262. doi: 10.7759/cureus.43262 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48. National Academies of Sciences, Engineering, and Medicine . Human‐AI Teaming: State‐of‐the‐Art and Research Needs. The National Academies Press; 2022. [Google Scholar]
- 49. Beede E, Baylor E, Hersch F, et al. A human‐centered evaluation of a deep learning system deployed in clinics for the detection of diabetic retinopathy. Paper presented at: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems; 2020.
- 50. Shumailov I, Shumaylov Z, Zhao Y, Papernot N, Anderson R, Gal Y. AI models collapse when trained on recursively generated data. Nature. 2024;631(8022):755‐759. doi: 10.1038/s41586-024-07566-y [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51. Andersen ES, Birk‐Korch JB, Hansen RS, et al. Monitoring performance of clinical artificial intelligence in health care: a scoping review. JBI Evid Synth. 2024;22:2423‐2446. doi: 10.11124/jbies-24-00042 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52. Novak LL, Russell RG, Garvey K, et al. Clinical use of artificial intelligence requires AI‐capable organizations. JAMIA Open. 2023;6(2):ooad028. doi: 10.1093/jamiaopen/ooad028 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53. Lekadir K, Frangi AF, Porras AR, et al. FUTURE‐AI: international consensus guideline for trustworthy and deployable artificial intelligence in healthcare. BMJ. 2025;388:e081554. doi: 10.1136/bmj-2024-081554 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 54. McGinnis J, Fineberg H, Dzua VJZ. Advancing the learning health system. N Engl J Med. 2021;385(1):1‐5. [DOI] [PubMed] [Google Scholar]
- 55. Kamel Rahimi A, Pienaar O, Ghadimi M, et al. Implementing AI in hospitals to achieve a learning health system: systematic review of current enablers and barriers. J Med Internet Res. 2024;26:e49655. doi: 10.2196/49655 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 56. Tu SP, Garcia B, Zhu X, et al. Patient care in complex sociotechnological ecosystems and learning health systems. Learn Health Syst. 2024;8(suppl 1):e10427. doi: 10.1002/lrh2.10427 [DOI] [PMC free article] [PubMed] [Google Scholar]
