A learning health systems approach to implementation and evaluation of an academic emergency department single shared note workflow

Abstract

Introduction

The application of implementation science (IS) frameworks to evaluate quality process improvement initiatives in an academic emergency department (ED) setting offers a promising approach for learning health systems (LHS). This report describes the University of Colorado Department of Emergency Medicine's LHS partnership among clinical operations, data analytics, and IS experts to evaluate a novel shared note workflow.

Methods

The University of Colorado Health (UCH) ED, staffed with scribes, advanced practice providers, and physicians, implemented a shared note workflow in summer 2022. This workflow involved all parties in a clinical encounter using a single note template, unlike previous separate notes. An IS expert guided the evaluation using the implementation of change model and the theoretical domains framework, with surveys and Epic electronic health record (EHR) encounter data. Analysis included descriptive statistics and regression analysis, comparing note completion rates pre (summer 2021) and post (summer 2022 and 2023) shared notes.

Results

Among 146 survey respondents, knowledge [M (SD) = 6.0 (0.9)] and confidence [5.5 (1.3)] following the new workflow were high (1–7 scale). Acceptability [M (SD) = 4.42 (0.80)], appropriateness [4.48 (0.70)], and feasibility [4.5 (0.61)] were also high (1–5 scale). EHR data showed note completion during the shift increased to 48.3% in 2022 from 32.2% in 2021, a 50% improvement (OR (95% CI) = 2.34 (2.28–2.40), p < 0.0001). By summer 2023, note completion attenuated to 41.3%, still significantly higher than pre‐shared note workflow (OR (95% CI) = 1.96 (1.90–2.01), p < 0.0001).

Conclusion

This IS‐enhanced LHS evaluation of a shared note workflow in an academic ED demonstrated high satisfaction and a positive impact on note completion during shifts. Timely note completion is crucial for reducing clinician burnout and providing high‐quality care. IS methods provided data‐driven insights to justify the sustainment of the shared note workflow to organizational leaders.

1. INTRODUCTION

Learning Health Systems (LHS) provide an opportunity to use rigorous research methods—notably from data science and implementation science—to test innovative and potentially scalable solutions to clinical and operational needs in health care settings. ¹ From improving patient access to addressing provider burnout, an LHS approach can speed identification of high value systems and processes of care. Ideally this process can unfold with minimal health system burden and resources. For instance, LHS scientists may use administrative data collected in the course of routine care to evaluate impact of workflow modifications on efficiency metrics. Furthermore, implementation science methods, such as application of theories, models, and frameworks and validated measures to evaluate implementation processes and outcomes, combined with familiar quality and process improvement methods, enable assessing and targeting implementation outcomes and determinants of adoption, implementation, and sustainment to inform continuous improvement of clinical and operational practices. ¹

This paper adds to the growing literature on LHS application case examples that leverage implementation science, in the context of a quality process improvement initiative to simultaneously address a priority quality metric (timely completion of patient encounter notes) and a source of documentation burden (multiple electronic health records notes created by multiple care team members for a single patient encounter) in an academic emergency department (ED). Documentation burden has been linked to clinical quality and safety outcomes, job attrition, and burnout among clinicians. ² The purpose of this initiative was to apply LHS methods and principles and apply implementation science frameworks and measures to evaluate implementation of a strategy designed to enhance ED provider well‐being through improved efficiencies in the use of electronic health records (EHRs) for clinical documentation. The strategy used was a novel “shared note” workflow in which residents, advanced practice providers, attending physicians, and scribes all worked within a single note in the EHR for a given ED patient encounter.

With the near‐universal adoption of EHRs, we can prioritize systematic investigation of ways to optimize EHRs' impacts on clinical efficiency, quality, and provider burnout. One of the most common sources of EHR‐related provider burnout is the amount of time spent on documentation tasks. ³ Documentation burden is typically quantified in terms of time and effort, including factors such as average and proportion of time spent on documentation, and timeliness of note completion. ² Provider time spent completing clinical notes outside of clinical shifts interferes with personal well‐being, family priorities, institutional non‐clinical service, and scholarship. Simultaneously, ensuring timely completion of notes is critical for accurate documentation, continuity of care, and clinical decision making, as well as efficient billing and coding. ⁴

EHR‐related workflows can negatively impact team function, especially when there is role confusion and lack of norms around use of the EHR for team communication and clinical conflict resolution. ⁵ Further, clinicians and patients have long noted that interacting with the EHR during an encounter negatively impacts the clinical experience. ⁶ Intervention opportunities for reducing impact of EHRs on well‐being include those that target total EHR time, after‐hours EHR time, and documentation burden, among others. ⁷ There are many examples of models of “documentation support” to reduce EHR documentation burden such as support from scribes and medical assistants ⁸ , ⁹ and reported benefits of “mutual access” EHRs that are shared between primary care and behavioral health clinicians. ¹⁰ However, to our knowledge this is the first published instance of a full team shared note approach. We evaluated implementation and impact of a single shared note workflow for all care team members involved in each ED patient encounter using a LHS approach.

The University of Colorado Hospital Emergency Department (UCH ED) has long had an inefficient and duplicative notes workflow, initially driven by the local health system's internal billing and coding policies. The UCH ED's previously existing process had learners (residents and medical students) and advanced practice providers (APPs) document in separate notes from the attending physicians, who would eventually attest to those notes as well as write their own note. Downstream consequences of this approach included increased overall charting burden across the team, duplicative charting leading to the potential for contradictory information, a high barrier to review and critique learners' notes leading to limited learning and teaching of the critical skillset of proper note writing, and a non‐cohesive patient encounter narrative that was difficult for other caregivers to interpret and glean concise clinical information. When billing and coding requirements changed in 2021 with the aim of reducing administrative burden, many centers, including ours, observed that those changes did not automatically translate to reductions in documentation time or length. ¹¹ As local perspectives on the new requirements evolved, the opportunity to revisit the documentation process arose.

Joint goals for this initiative were improving time to completing clinical notes and reducing time spent on documentation outside of clinical hours. The proposed novel solution—a single shared note workflow—was identified as a priority for a new LHS‐aligned collaboration among academic department clinical operations and research leaders with expertise in informatics and implementation science. This paper reports on our LHS approach to implementing and evaluating impact of ED the shared note workflow on operations metrics, care team member attitudes, and perceptions of documentation burden.

2. QUESTIONS OF INTEREST

There were dual objectives of this initiative. For ED operations, the goal was to answer the question of how to evaluate the impact of the shared note workflow to inform its implementation and sustainment. For the ED research group, the goal was to answer the question of how might we use implementation science methods to improve implementation and effectiveness of the shared note workflow.

3. METHODS

3.1. Context and rationale

The UCH ED is an academic setting with a dedicated 4‐year emergency medicine residency as well as staffing by APPs and attending physicians. The site sees over 110 000 ED patient visits per year and is a level 1 trauma center and regional quaternary care hospital. This organization uses the Epic EHR (Verona, MI). In partnership with the University of Colorado Anschutz Medical Campus, the UCD ED has long embodied core dimensions of an academic health center‐affiliated LHS, ¹² especially use of scientific methods, a robust informatics platform for data capture, monitoring, and reporting, and a continuous learning culture that yield both knowledge production and translation to practice both locally and beyond. The rationale for this initiative was the need to demonstrate the value of collaboration between clinical operations and implementation science (IS) experts in the context of an IS‐enhanced LHS. This work was conducted as program evaluation/quality improvement and thus did not require institutional review board review or approval.

3.2. Shared note workflow and implementation

Prior to implementation of the shared note workflow, attendings had assigned scribes who documented a full note template including history of present illness, review of systems, physical exam, medical decision making, and procedure documentation where applicable. Residents or APPs either duplicated that work entirely or completed a shorter note with only their personal medical decision making included. The new shared note template combined all these notes into a single document where all parties could contribute during the patient encounter. Under the new workflow, attending physicians remained ultimately responsible for all aspects of care, including documentation, but residents and APPs were able to create the shared note and begin documenting in it as soon as they engaged in the care of any given patient. Our ED uses a “round robin” patient distribution scheme, where arriving patients are assigned to one of many pre‐defined ED care teams (each comprised of one attending physician, one or more residents and APPs, one scribe, and a dedicated group of nurses and technicians). Medical scribes underwent standardized training provided by a third party contractor (Scribe America, Fort Lauderdale, FL) and could be assigned by the attending to follow the attending, resident, or APP to perform contemporaneous patient care documentation in the shared note. Depending on hour of day and anticipated arrival curves, patients requiring care in the main part of the ED would be distributed among three or four ED care teams in a sequentially rotating fashion. Typically, a resident or APP might evaluate a patient initially and discuss the case with their assigned attending physician shortly thereafter. Under the shared note workflow, the resident or APP would most commonly begin documentation of history of present illness, physical examination, and initial medical decision making immediately after seeing the patient, even before the attending's direct involvement. As care continued throughout a patient's ED visit, the resident or APP and the attending were all able to make incremental contributions to the shared note, such as documenting review of laboratory or imaging results, re‐evaluations, procedures, and further medical decision making.

All providers on the care team had real‐time access to the current version of the shared note via their individual EHR logins, and the shared note resided within the context of each patient's EHR visit record. This allowed for transparency of information already entered, elimination of duplicative charting, as well as editing, review and feedback by more senior clinicians and decreased the overall documentation burden. At the time a patient was dispositioned from the ED, the attending was responsible for finalizing the shared note with an electronic signature action that prevented future edits by any other EHR user.

The shared note workflow was implemented throughout the UCH ED in August 2022 after a brief 3‐week pilot in July 2022. The UCH ED change management process included several steps. First, the ED operations team engaged 4–5 early adopters (both residents and attending physicians), known locally as savvy EHR users, as well as scribes and APPs to work together in‐person on developing the workflow and drafting a new note template. This consisted of human centered design sessions outside of the clinical environment, followed by live iterative testing by the pilot participants in live clinical practice. Next, ED operations conducted a pilot with select ED care teams who received asynchronous didactic education before their ED shifts in the form of a 5‐min video and static tipsheet and were supported “at the elbow” during live clinical shifts by operational physician leaders well versed in the new workflow. As part of the proof of concept, the goal was to develop a cohort of clinicians with comfort and confidence in the process, gain feedback for iterative improvement of the shared note template, and garner buy‐in for an expectation of having all shared notes completed by the end of a shift. After this initial pilot was deemed successful, it was extended to all ED provider teams on every shift. Education was both asynchronous via email using tipsheets and explainer videos that described the why, what, and when, as well as synchronous at resident conference and faculty meetings with interactive question‐and‐answer sessions. Concerns about this change were addressed transparently through dialog with ED operations leadership and frontline providers. The share note process did not alter existing workflows or expectations around defining final diagnoses or problem lists, medical decision making, nursing team communication, order entry, or other usual elements of ED care.

3.3. Implementation science‐enhanced LHS approach

As shown in Table 1, we applied the implementation of change model (ICM) as a process framework to guide this LHS initiative. ¹⁴ We applied the theoretical domains framework (TDF) ¹⁵ as an implementation determinants framework to guide assessment of individual skills building and to gain an understanding of the barriers and facilitators to implementation of the shared note workflow. The TDF includes 14 domains of behavioral determinants identified as factors in health care professional implementation behavior, which originated from 33 behavior change theories. In spring of 2022, the ED operations director and ED LHS research group's implementation scientist met to discuss priority opportunities for integration of implementation science with change management processes (ICM step 1). The upcoming shared note workflow implementation was identified as aligning with ED operations' current priorities and the skills and expertise of the implementation scientist. After getting buy‐in from other department leadership, the next step was to identify the potential targets for change (the outcomes of interest) and readily obtainable data sources for these priority outcomes (ICM step 2). Then, ICM steps 3–5 (problem analysis, practice change, implementation plan) proceeded in an iterative, parallel fashion, with ED operations using existing processes for implementing and monitoring change in the ED, ED research conducting a survey to evaluate ED care team experience based on domains of the TDF identified by the team as most relevant to this implementation initiative, and a joint presentation and discussion with the ED faculty. After this iterative process, the ED operations team made the decision to fully integrate the shared note workflow into all routine care processes (ICM step 6); a data and analysis team pulled the requisite data and then conducted an analysis of impact on target outcomes using EHR data (ICM step 7).

TABLE 1.

University of Colorado Emergency Department Learning Health Systems approach.

Implementation of change model steps	The shared note workflow evaluation methods
Step 1. Development of proposal for change	Operations and research leadership joint decision to implement and evaluate the shared note workflow as a learning health system project enhanced with implementation science methods
Step 2. Outcome/performance analysis to identify and measure potential targets for change	Select priority implementation outcomes, determinants, and measures (including key TDF constructs and validated implementation science feasibility, acceptability, and appropriateness measures)
Step 3. Problem analysis to describe stakeholders and context, assess barriers and facilitators/determinants of change, assess adaptations, describe subgroups	TDF‐based survey administered to identify implementation determinants; presented findings, feedback received during faculty meeting—did experience vary by “generation” or job type? Who needed extra help?
Step 4. Development and selection of strategies and measures to change practice based on problem analysis	Recommendations for implementation strategies and adaptations to address implementation determinants (lower knowledge and confidence among certain team members, especially attending physicians more resistant to change)
Step 5. Development, testing, and execution of implementation plan	Operations‐led implementation using standard change management principles and initial analysis of implementation and impact on notes completion using internal data monitoring processes and platforms
Step 6. Integration into routine care	Shared notes workflow expanded to include “SuperTrack” cases 13 ; formal decision by the health system leadership for shared notes to be institutionalized and sustained
Step 7. Evaluation and adaptation	Research team‐led EHR data analysis of longer‐term impact on timely completion of notes during the associated shift compared to pre‐shared notes

Abbreviations: EHR, electronic health records; TDF, theoretical domains framework.

3.4. Outcomes and measures

3.4.1. Outcomes and performance analysis step

Two types of outcomes were determined to be important to informing operations decisions. First, for the problem analysis, there was interest in evaluating impact on common implementation science outcomes (acceptability, appropriateness, and feasibility of the shared note workflow), select barriers and facilitators as defined by the TDF (knowledge, confidence domains), and perceived impact on outcomes related to clinician burnout (impact on documentation burden and charting time). The primary outcome of interest was the rate of notes being completed during the clinical shift. Secondary outcomes included the pattern of time for notes completed within and beyond 48‐h of the end of the shift.

3.4.2. Problem analysis step

We used survey methods to assess overall attitudes, experience with, and impact of the shared note workflow. In fall 2022, approximately 3 months after implementation of the shared note workflow, all ED attending physicians, APPs, residents, and scribes received survey invitations from the Vice Chair of Operations via Qualtrics (Qualtrics, Provo, UT). The survey contained a total of 40 items; respondents completed a subset of items depending on role. The survey assessed respondent role, number of ED shifts with shared note workflow since July 2022 (none, 1–10, more than 10, what is shared note workflow?), demographics, and clinical experience (time in role/at organization). We used select items adapted from the TDF questionnaire to assess knowledge of the new shared note workflow (3 items, e.g., “I know how to follow the shared notes workflow,” “With regard to the shared notes workflow, I know what my responsibilities are”) and confidence (4 items, e.g., “I am confident that I can follow the shared notes workflow, even when there is little time”) in their ability to follow the shared note workflow. ¹⁶ We used items adapted from the validated Acceptability of Intervention, Feasibility of Intervention, and Intervention Appropriateness Measures (4 items each) to assess perceived acceptability, feasibility, and appropriateness of the shared note workflow in the ED context, in response to the prompt, “What do you think about the shared notes workflow in the context of our ED?”. ¹⁷ We developed novel items to assess perceived impact of the shared note workflow on documentation burden (1 item, 5‐point scale, much lower burden to much higher burden), charting time (1 item, 5‐point scale, much less time to much more time), feedback given on resident, APP, and scribe notes (by attendings) or feedback received by APPs, residents, and scribes (1 item each, 5‐point scale, decreased significantly to increased significantly). The survey also provided the opportunity for open‐ended comments. The survey is available in the Supporting Information.

3.4.3. Evaluation step

For the analysis of impact on timely notes completion—a longer term outcome—we used data available in the EHR. We initially requested an extract of EHR data from summer (July–September) 2021 (pre‐shared note workflow) and 2022 (post‐shared note workflow including the July pilot period and August and September full implementation). The data request included every ED encounter July–September 2021 and July–September 2022 encounter dates, E&M level codes including observation codes and critical care codes, procedure codes, notes edits history (timestamps of who edited and when), notes final sign off (time and date of final sign off by attending physician), and time and date of shift. These parallel 3‐month time periods from the year prior to the shared note implementation and the year when shared notes were implemented were selected to allow for analysis of impact to proceed quickly (3 months after implementation) given the health system wanted answers in a timely manner (rather than waiting for a full year's worth of data) and to account for seasonality in ED encounter volume and reasons for visits. For the final analysis presented in this paper, we also included data from the parallel summer months from July to September 2023, for comparison with a period fully post shared note workflow implementation. Patients included are those who were roomed in the main part of the ED; we excluded those who were discharged as part of our front‐end rapid assessment and discharge process because shared note implementation in that area occurred later. ¹³ , ¹⁸ Included note author types were Physician, Nurse Practitioner (NP), Physician Assistant (PA), Resident, or Scribe. We excluded notes with CURRENT NOTE AUTHOR TYPE = Coordinator, Fellow, Medical Student, Midwife, Nursing Student, Pharmacy Resident, Professional Research Assistants, Technician, Therapist, Clinical Social Worker, Case Managers, Respiratory Therapist, Registered Nurse, or Blank. Note, fellows serve as attending physicians in this ED.

3.5. Analysis

We used descriptive statistics and regression analyses to summarize and compare perceptions of the shared note workflow among attending physicians, residents, APPs, and scribes. Analyses were conducted using SAS Version 9.4. We compared outcomes by respondent role using non‐parametric ANOVA and Kruskal–Wallis tests. Normality of the measures was questionable, so comparisons were run using a non‐parametric ANOVA run on ranks rather than values. We used descriptive statistics to calculate percent note completion during the shift and within and beyond 48 h post‐shift. We used logistic regression methods to examine differences in odds of notes completion pre‐ versus post‐implementation of the shared note workflow, presenting results both combining across all 3 months included each year and separately by month. We counted a note as having been completed during the shift if it was signed or co‐signed by a physician within the shift in which it was created. We calculated the percentage of notes completed during the shift as well as percentage of notes completed in segments of time following the shift (0–12 h, 12–24 h, 24–48 h, more than 48 h after the shift).

4. RESULTS

4.1. Survey analysis

Figure 1 shows number of responses and response rate by respondent role and number of shifts with the shared note workflow in place. Overall, on a scale of 1 to 7 (low to high), respondents reported high knowledge of the workflow (M = 5.95, SD = 0.90) and confidence in following the workflow (M = 5.48, SD = 1.28) following the shared note implementation. Similarly, on a scale of 1 to 5 (low to high), respondents reported high acceptability (M = 4.42, SD = 0.80), appropriateness (M = 4.48, SD = 0.70), and feasibility (M = 4.5, SD = 0.61) of following the shared note workflow in the ED. On a scale of 1 to 5 (lower/less to higher/more), respondents generally perceived the ED the shared note workflow decreased documentation burden (M = 2.19, SD = 1.03) and post‐shift charting time (M = 2.06, SD = 1.06).

FIGURE 1.

The shared note workflow evaluation survey responses and response rate by respondent role and number of shifts with shared note workflow in place.

However, there were significant differences by respondent type. Based on non‐parametric ANOVA results and as shown in Figure 2, scribes reported significantly higher knowledge and confidence than both residents and attendings (all pairwise comparisons, p < 0.05). As shown in Figure 3, attending physicians reported significantly lower acceptability, feasibility, and appropriateness than scribes and residents and lower appropriateness than APPs (all pairwise comparisons, p < 0.05), but still reported overall positive attitudes toward the new process. Figure 4 shows perceived impact of the shared note workflow on notes feedback given (from the perspective of attendings) and received (from the perspective of residents, APPs, and scribes). On a scale from 1 (feedback significantly decreased) to 5 (feedback significantly increased), respondents perceived there was slightly more feedback given and received compared to before the shared note workflow was implemented, especially for the residents. Figure 5 shows attending physicians on average perceived slight decreases in documentation burden and post‐shift charting time, while residents, APPs, and scribes reported significantly greater improvements in both outcomes relative to attendings (all pairwise comparisons, p < 0.05).

FIGURE 2.

Knowledge and confidence in following the shared note workflow by respondent role.

FIGURE 3.

Acceptability, feasibility, and appropriateness of the shared note workflow by respondent role.

FIGURE 4.

Perceived impact of the shared note workflow on documentation burden and post‐shift charting time by respondent role.

FIGURE 5.

Perceived impact of the shared note workflow on feedback given (attending physician perspective) and received by respondent role.

4.1.1. Open comments synthesis

Perceived benefits described in free text by survey respondents included opinions that the shared note workflow were good for teamwork and decreased charting burden. The shared note workflow were described as a “huge asset,” “life changing,” “significantly improve charting,” and a “huge quality of life boon for residents.” That said, there were growing pains, such as learning new auto‐text tools embedded in the new note templates. Regarding implementation, there was a need for role clarity and clarifying expectations for attending versus resident responsibilities. There were some inconsistencies in experience of the benefits, with some describing the experience as “attending‐dependent” and dependent on the availability and quality of a scribe. It was noted that it was important to discuss the documentation plan and status at the beginning and end of the shift. There were also some initial technical difficulties reported, such as notes not showing up in the in‐basket for all contributors, which were resolved.

4.2. EHR analysis

There were 44 970 unique patient encounters July–September 2021 (pre‐shared note workflow implementation) and 44 227 in July–September 2022 (July reflecting the pilot period, August and September reflecting full shared note workflow implementation) and 39 982 in July–September 2023 (post‐shared note workflow implementation) meeting inclusion criteria. In the pre‐the shared note workflow period, 32.2% of notes were completed (signed/co‐signed by a physician attending) during the clinical shift in which it was started. Upon implementation of the shared note workflow in summer 2022, note completion during the shift increased to 48.3%, a 50% improvement (OR (95% CI) = 2.34 (2.28–2.40); p = <0.0001). By summer of 2023, the improvement had slightly attenuated, with 41.3% of notes completed during the shift, but still significantly higher than pre‐the shared note workflow (OR (95% CI) = 1.96 (1.90–2.01); p = <0.0001). A breakdown of note completion month‐by‐month during the evaluated periods is shown in Table 2. Notably, the 2022 improvement in timely note completion was smaller in July 2022 during the pilot phase when only some teams were using the shared note workflow (40.0% of notes completed during the clinical shift in July 2022 vs. 32.7% in July 2021) than in August (53.5% in 2022 vs. 29.9% in 2021) and September (52.4% in 2022 vs. 34.2% in 2021) when all teams started using the shared note workflow. By 2023, all evaluated months showed similar improvement over 2021. The pattern of notes completion after a shift was similar pre‐ versus post‐the shared note workflow (Figure 6), with an even distribution of note completion over the ensuing 48 h. Both pre‐ and post‐the shared note workflow, about a quarter to a third of notes completed after the shift were signed in the ensuing 12‐h period (26.8% in 2021; 31.9% in 2022; 27.3% in 2023).

TABLE 2.

Notes completion rates by month and year, pre‐ and post‐shared note implementation.

Month	2021 (pre shared notes): n (%) notes completed during shift	2022 (post‐pilot + full implementation): n (%) notes completed during shift	2023 (post‐full implementation): n (%) notes completed during shift	2021 versus 2022: OR (95% CL)	2023 versus 2021: OR (95% CL)
July	14 884 (32.7%)	15 690 (40.0%)	13 540 (41.8%)	1.48 (1.41–1.54)	1.90 (1.81–1.99)
August	15 397 (29.9%)	14 431 (53.5%)	13 404 (40.2%)	3.35 (3.21–3.51)	2.13 (2.04–2.24)
September	14 687 (34.2%)	14 107 (52.4%)	13 037 (41.8%)	2.75 (2.63–2.88)	1.85 (1.76–1.93)

FIGURE 6.

Time to note completion pre‐ and post‐shared note workflow implementation.

5. CONCLUSION

In our academic ED setting, an LHS partnership among the clinical operations team, a data and analysis team, and an implementation scientist provided enhanced insights into implementation of a shared note workflow compared to the department's historical change management process. In particular, application of the implementation of change model for identification of priority implementation and effectiveness metrics and administration of validated implementation determinants and outcomes survey measures helped systematically assess care team member perspectives on the shared note workflow. Use of encounter data generated from the EHR to analyze impact on notes completion over 3 years pre‐ and post‐shared note workflow implementation provided empirical evidence useful for reporting to institutional leadership to justify sustaining the change. The results were strongly supportive of the shared note workflow approach, with both strongly positive attitudes across care team members involved and a demonstrable impact on time to note completion. Results revealed several opportunities for improvement in both the shared note workflow itself and the implementation process—corroborating “on the ground” feedback received by the operations leadership. Notably, in the problem analysis phase of the ICM, assessment of shared note workflow implementation determinants (knowledge and confidence in shared note processes, EHR interfaces, and responsibilities) informed the need for additional education and support for certain members of the team. Specifically, we identified that while most team members felt confident in the workflow and believed it was feasible and acceptable, certain team members (especially more senior attending physicians, and those with a longer tenure in the department more used to established routines) desired follow‐up and guidance from the operations team leading the implementation. These insights informed focused education and support for transition to the shared note workflow among specific subgroups of clinicians. Insights gained from this additional follow‐up by the operations teams also revealed minor changes to the EHR templates to improve usability. The success of this initiative led to expansion of shared notes to the ED “SuperTrack” (initial triage) beyond the main ED and a decision at the health systems to formally institutionalize and sustain the shared note workflow approach.

Often, operational changes are implemented based on the subject matter expertise of a small team but without a clear plan to evaluate the actual impact on end users. For this project, employing implementation science methods alongside change management and/or quality and process improvement techniques had multiple benefits. Use of an implementation process framework, identification of implementation determinants and use of validated implementation outcomes measures and a rigorous analysis of EHR data has not been the historical norm in the UCH ED. This more systematic scientific approach was widely seen as providing novel insights and driving more empirically sound operational decision making. This multidisciplinary approach thus enabled us to assess the value and impact of the change and empowered the front‐line teams affected by the change to share their experience leading to iterative improvement on new workflows. This approach also yielded the opportunity for dissemination of our findings for other academic EDs seeking evidence‐based and operationally tested approaches to quality improvement (more timely notes completion) and reducing documentation burden. There is a growing literature on the benefits of integrating quality improvement science and implementation research. ¹⁹ , ²⁰ , ²¹ , ²² This project adds to the literature demonstrating that use of implementation science frameworks can be effectively and efficiently integrated into LHS‐style projects focused on addressing factors contributing to burnout.

This project also identified benefits of adoption of a single shared note workflow with respect to reducing documentation burden and improving timeliness of note completion and education around documentation. Other strategies designed to enhance efficiencies in clinical documentation have shown mixed effects in ED settings. For instance, use of auto‐expanding text phrases has reported to be unstandardized and to significantly increase note length, while also being associated with higher billing levels. ²³ To further address documentation burden, there are opportunities to explore artificial intelligence‐based approaches. Sezgin et al. evaluated the use of digital scribes for summarizing ED clinical conversations, involving use of four trained large language models (LLMs), finding that the “BART‐Large‐CNN” model demonstrated a high level of understanding of clinical dialog structure. ²⁴ Given live scribes can have significant cost, despite benefits for reducing documentation burden ²⁵ and enhancing resident educational experience, ²⁶ it is likely that further exploration into the role of AI‐based documentation support is forthcoming.

Challenges associated with an LHS approach involving implementation science methods combined with clinical operations often pertain to misalignment between typical research timelines (which are slower and more methodical) and operations timelines (which are faster and more amenable to iteration). ²⁷ In this effort, we made joint decisions about timing and scope of data collection that would support focused and accurate assessments of impact while minimizing burden to the clinical teams and allowing the results to drive rapid process improvement. While there were many potential outcomes that could have been assessed; we narrowed down the options to those most relevant to our goals: improving a key quality and process metric (timely note completion) while addressing a noted source of clinician burnout and poor job satisfaction (documentation burden). These analyses were conducted using resources already available to the department. While additional data collection using more robust qualitative methods (e.g., focus groups) was of interest, the time required to produce qualitative results would not have been useful to operational decision making given the rapid implementation time frame. In our LHS, these types of design decisions were essential for ensuring mutual acceptability among clinical operations and research team members.

Altogether, this was a successful demonstration of an IS‐enhanced LHS partnership in the ED setting.

6. LIMITATIONS

Limitations include the observational nature of these analyses and the timing relative to historical events such as COVID‐19 that may have affected changes in clinician experience. The selection of the 3‐month summer periods for analysis of EHR data and the logistic regression analysis used to compare pre‐ and post‐shared notes implementation across years was driven by the health system's priority for timely answers. Given a common LHS concern is the time for the research process to unfold, timeliness of answers was prioritized. Additionally, the EHR data analysis component of the evaluation included July of 2022, during the pilot phase of the shared note implementation when some ED team members were still using the old duplicative notes workflow; the analysis is thus considered a more conservative evaluation of the impact of shared notes. However, analysis at the month level showed a smaller effect in July 2022 (compared to the same month in 2021) than for August and September 2022 (compared to the same months in 2021), suggesting that the improvements in timely notes completion were partially observed during the pilot and then fully observed after all ED teams implemented the shared note workflow. While a longer‐term period for analysis and use of analytic approaches such as an interrupted time series analysis might have been a more sophisticated approach, this approach was deemed most pragmatic and in alignment with health system priorities Local contextual factors such as financial incentives to complete notes within 48 h (in place both before and after implementation of the shared note workflow) may impact note completion rates and timing. Potential limits to generalizability of this LHS approach include the personnel and resources specific to this institution, such as the availability of implementation scientists in typical academic emergency departments.

CONFLICT OF INTEREST STATEMENT

The authors declare no conflicts of interest.

Supporting information

Data S1.

Kwan BM, Hosokawa P, Resnick‐Ault D, et al. A learning health systems approach to implementation and evaluation of an academic emergency department single shared note workflow. Learn Health Sys. 2025;9(4):e70017. doi: 10.1002/lrh2.70017