Selection and Implementation of Virtual Scribe Solutions to Reduce Documentation Burden: A Mixed Methods Pilot

Abstract

Electronic health record (EHR) documentation is a leading reason for clinician burnout. While technology-enabled solutions like virtual and digital scribes aim to improve this, there is limited evidence of their effectiveness and minimal guidance for healthcare systems around solution selection and implementation. A transdisciplinary approach, informed by clinician interviews and other considerations, was used to evaluate and select a virtual scribe solution to pilot in a rapid iterative sprint over 12 weeks. Surveys, interviews, and EHR metadata were analyzed over a staggered 30 day implementation with live and asynchronous virtual scribe solutions. Among 16 pilot clinicians, documentation burden metrics decreased for some but not all. Some clinicians had highly positive comments, and others had concerns regarding scribe training and quality. Our findings demonstrate that virtual scribes may reduce documentation burden for some clinicians and describe a method for a collaborative and iterative technology selection process for digital tools in practice.

Introduction

Despite significant investment in electronic health records (EHRs) and other health information technology aimed at improving patient care, clinical documentation is a leading cause of burnout among physicians. For example, one study demonstrated that a typical family physician may spend on average 4.5 hours per day on documentation (1,2). A spectrum of modalities exists for assisting clinicians with clinical documentation, ranging from dictation technologies and documentation templates to in-person medical scribes. While in-person scribes have been shown to reduce documentation burden, there are barriers to their use, including cost, training requirements, and high turnover among the scribes (2–4). The COVID-19 pandemic also impacted the scribe labor force, causing more difficulty with scribe staff retention and forcing the shift from many scribes being physically present to virtually present (3,5).

Most virtual scribe services have individuals that work remotely, using various Health Insurance Portability and Accountability Act (HIPAA)-complaint technology platforms. The virtual scribe will populate a note for clinicians to review, edit, and sign. This can happen in real-time, with a scribe virtually present over a video or audio connection to the visit and documenting relevant information from the encounter into the EHR. Alternatively, some solutions work asynchronously, where information from the clinical encounter is recorded and the scribe documents using the recording from the encounter. Advances in machine learning (ML) and natural language processing (NLP) capabilities have added the opportunity to automate parts of the virtual scribe process so that information from the clinical encounter can flow passively in real time through speech-to-text software to generate note text (6). In some cases, this is coupled with a scribe reviewing the generated documentation for further quality control before providing the document to the physician for final review, though newer technologies have begun to emerge that are fully digital and with no human scribe component (7). The advancement of this latter approach will likely continue to rapidly expand and change with the rise of large language models and other ML advancements (6).

To date, few organizations have formally evaluated the usability or effectiveness of virtual and/or digital scribe solutions. Most reports to date have described virtual scribe programs in certain specialties such as primary care or orthopedics, and in some cases these programs have been associated with reduced physician burnout (8–10). Other reviews have focused specifically on fully digital and automated scribe technologies incorporating speech-to-text and ambient listening. These current reports highlight the need for further evaluation of these models in different settings and healthcare systems and the need to identify unique implementation challenges and opportunities, including factors around cost, linguistic variation, and interoperability with existing EHRs (11,12). These scribe-specific challenges exist alongside the typical challenges of incorporating any digital health tool into a complex healthcare ecosystem and workflow(s), including deciding on the appropriate solution, implementing the solution and testing in a real world setting, and broader operationalization factors. In particular, strengthening the evidence-base and better characterizing implementation processes for digital health technologies in practice is an imperative for optimizing their use and value within clinical care (13-14)

The aim of this health system quality improvement project was to select and test virtual scribe solution(s) with live and asynchronous approaches best-suited to the needs of our healthcare system. This paper describes that process and the pilot implementation among a diverse group of healthcare providers within a single academic healthcare system, an integrated health system of 10 hospitals and 60 clinics.

Methods

Virtual Scribe Solution Selection

While numerous implementation frameworks exist, processes for selection of technologies have largely been derived from the business and IT literature. We use a framework of key considerations for adoption of digital health tools by Marwaha et al in conceptualizing our IT selection and adoption process (13), described in Table 1.

Table 1.

Key Considerations For Adoption of Digital Health Tools (taken directly from Marwaha et al. (2022))

Key Considerations	Description
Institutional priorities	Does the tool align with institutional goals or help with regulatory compliance?
Executive sponsors	Is there a senior executive-level advocate who will help support and pay for the tool’s adoption and deployment?
Internal champion	Is there an advocate within the health system with the leverage and motivation to facilitate adoption and implementation?
Product selection	Was the tool developed internally, by a third-party, by the health system’s existing EHR vendor, or through a private sector partnership?
Financial value	What framework does the tool leverage to generate financial value, and does it outweigh the costs associated with deployment and maintenance?
Clinical value	Is there a clear, meaningful outcome metric consistent with the Quadruple Aim that would be improved by adopting this tool?
Implementation	What IT, training, and workflow modification resources are needed for implementation?
Data assets	Does the health system have access to the data necessary for tool functionality? Is the tool interoperable with the health system’s IT infrastructure? Is training data needed to tune the tool to the local environment? Has data governance been established?
Long-term operational home	Who will continue to provide technical support and quality assessment in the future?

These efforts were motivated by an institutional priority to reduce documentation burden and an interest specifically in exploring the use of virtual scribes. In October 2022, a core group of leaders including information technology specialists, clinical informaticists, operations leaders, and clinicians were engaged to understand documentation practices, previous experiences with live or virtual scribes (if any), and features they would consider to be important in selecting a virtual scribe technology. A semi-structured interview guide was created with questions for clinicians regarding previous documentation practices, experiences (if any) with scribes, and features they would be looking for or need to assist them with their documentation workflow. Through one-on-one semi-structured interviews, a clinical informatics fellow (C.H.) conducted a needs assessment of 10 healthcare providers across specialties regarding their documentation practices, previous experiences with live or virtual scribes (if any) and features that they would consider to be important in using a virtual scribe technology. Some of these key-informant providers were selected because they had had previous experience with in-person scribes but were no longer able to access this service. Notes were taken during interviews, results were compiled, and general themes analyzed to inform requirements for potential solutions. These requirements, along with a review of the market landscape and current technology dependencies, were used to inform a process for IT selection and implementation with a core virtual scribe selection group. The steps of this process were guided by the different dimensions and questions important for digital health adoption by Marwaha et al. (Table 1).

Implementation and Pilot

Sixteen clinicians were selected to participate in the pilot conducted from April through July 2023. Clinicians were not necessarily participants in the core group, but rather were selected based on willingness to participate and specialty, with the goal of representing a diverse array of clinical specialties in a time-limited live environment. Emails were sent out to certain specialty groups requesting volunteer participants. Clinician participants were allocated to either a live virtual scribe arm or an asynchronous virtual scribe arm, largely based on what best fit their clinical needs and specialty workflow. For instance, given that participants from emergency medicine and dermatology indicated that they needed the scribe to place orders to best support their work flow, they were allocated to the live scribe arm. Scribes were provided and trained by the vendor. The live virtual scribe connected through an institutionally supported virtual meeting platform on a computer or tablet in the room during the clinical encounter. Scribes were then able to populate the note directly in the EHR and enter orders in a pended state in real-time for the clinician to review and sign. In the asynchronous virtual scribe arm, an audio recording of the clinical encounter was made via the vendor application on a smartphone or tablet. This recording then flowed through an ambient listening program using machine learning to pre-populate text into the note. Subsequently, a scribe worked asynchronously to review the note accuracy, edit as needed, and finalize the note for the physician to review with an estimated four hour turnaround time. Clinician participants received no compensation for participating in the pilot. All technology utilized in the pilot was HIPAA compliant. At the beginning of the clinical encounter, either the clinician or a member of the clinical staff obtained verbal consent for the use of the virtual scribe solution from each patient.

Onboarding of clinicians occurred prior to pilot initiation and involved meeting virtually with institutional health IT staff and vendor staff. During this meeting, pilot participants were given an overview of the technology and provided with best practices and optimal workflows to utilize. Clinicians also spoke with vendor staff and virtual scribes themselves, to discuss documentation preferences and templates prior to the first clinical encounter. Clinicians had the opportunity to provide feedback to the scribes via the vendor application (asynchronous arm) or verbal, real time feedback (live arm) during the pilot.

For the evaluation, a pre- and post-intervention survey and semi-structured interviews with participating clinicians were conducted by informatics fellows (C.H. and M.G.) with the goal of understanding the perceived usability as well as any subjective changes in documentation practices, clinical efficiency, and burnout. Survey questions were adapted with permission from a survey of in-person scribes and effects on documentation burden conducted by Pozdnyakova et al (15). They were reviewed by a group of physician informatics leaders from primary care and surgery, informatics fellows and an implementation scientist and focused on measures related to documentation practices, documentation burden, and clinical burnout. Final survey question selection was performed by an implementation researcher and informatics fellow (M.G. and D.P.). Survey data was collected from March through June 2023 and managed using REDCap electronic data capture tools hosted at the University of Minnesota (16-17). Semi-structured interviews with clinicians were recorded and transcribed; in cases where interviews could not be transcribed, the researcher took detailed notes during the conversation. NVivo12 was used for qualitative thematic analysis (18), with two informatics fellows (M.G and C.H) independently coding. The first several were coded by both fellows and discussed to resolve internal discrepancies in the coding process. The research team built a codebook of terms using an inductive theme generation process related to aspects of using scribe, clinician-reported patient perspectives of virtual scribe, and facilitators of successful adoption. An informatics fellow (M.G.) also analyzed provider-level EHR metadata (Signal, Epic Systems) for clinicians participating in the pilot. This project was deemed non-human subjects research and a quality improvement protocol by the University of Minnesota Institutional Review Board (IRB ID STUDY00017124).

Results

Virtual Scribe Solution Selection

We apply our process of selection and implementation to schema created by Marwaha et al for considering different components of selecting and evaluating new digital health tools within complex health systems (13-14). Table 2 outlines different considerations and how they were addressed within our system. A virtual scribe selection group of 25 people made up of clinical informatics leaders, information technology (IT) specialists, and physician leaders across specialties was formed around this initiative. Several themes emerged in clinician interviews which were included as criteria in the selection process, including: accuracy of documentation, quality of the audio/visual connection of the virtual scribe technology platform, ability to understand and work with specialty-specific language, and ability to distinguish general from medically relevant conversation. Other features, such as the ability to pend orders or utilize consistent scribes, were considered helpful but only necessary in a subset of high volume clinicians who performed procedures.

Table 2.

Application of Key Considerations for Digital Health Tool Adoption (from Marwaha et al (2022))

Key Consideration	Description of our process
Aligning with institutional priorities	Documentation burden is a leading cause of clinician burnout in the U.S., and relieving that burden had been identified as an institutional priority. The use of virtual or digital solutions was also of interest as several in-person scribe vendors were either decreasing or stopping their services locally.
Executive sponsorship	Resources were assigned given this high priority. Sponsorship for the pilot included the Chief Digital Officer and Chief Medical Officer.
Internal champion	The internal champion was the Associate Chief Medical Information Officer for Provider Efficiency. A core group of 25 individuals including clinician champions across specialties, IT, and clinical informatics was formed for virtual scribe selection with the goal to reduce documentation burden
Product selection	The core group conducted an internal needs assessment, clear outlines of current technology and vendor dependencies, and created a scoring system and systematic process for evaluating different vendors, involving clinicians across specialties.
Financial value	Extensive information was obtained on different cost models in marketscape review. Value models included certain assumptions on administrative time with documentation and clinician clinical volume.
Clinical value	A literature review was conducted to understand prior research on clinical value with scribe solutions. The evaluation in this pilot study included interview questions to explore the effects of the scribe solution on clinician perceived efficiency and clinician well-being
Implementation	The core team engaged directly with the vendor to align the IT technology owner, the training team, and the operational workflow staff.
Data assets	The underlying technology required minimal data integration/assets. Most technology dependencies were ensuring devices were HIPAA compliant and scribes were properly credentialed and provisioned to utilize Epic. The interview pilot data, planning documents for technology solution, and Epic Signal data were stored in a secure central database.
Long-term operational home	The technology owner was identified at the start of the pilot. Operational leadership was distributed across service line leaderships, with a process identified for approval of clinicians to have scribes ongoing beyond the pilot and an intake assessment in the training and support team for the provider, as well as onboarding of new virtual scribes.

IT and informatics teams conducted an extensive review of over 15 different solutions in the market landscape related to documentation augmentation, with a particular interest in virtual scribes. Information captured included cost, functionality, and systems IT requirements. These data were used to compare features of different vendors and inform the selection process. Notable differences among solutions were observed in cost models, documentation turnaround time, specialty-specific language, training, and end-user platforms. Based on these factors, five final vendors were chosen for further consideration. The five vendors presented their virtual scribe solutions, including demonstrations of the technology itself, to the full group of 25 individuals with representatives from IT, informatics, and clinical practice. This group then used an internally developed scoring system to rank the vendors on the following parameters: overall impression, end-user experience, training, cost, specialty coverage, and IT considerations. Overall scores were used to select a final vendor. Ultimately, one vendor (ScribeAmerica, Fort Lauderdale, FL) with two virtual scribe solutions – a live virtual scribe (Telescribe) and an asynchronous virtual scribe tool (Speke)– were selected for piloting within our healthcare system.

Implementation and Pilot

Sixteen clinicians (15 physicians and one advanced practice provider) participated in the pilot. Specialties represented include primary care, dermatology, orthopedic surgery, emergency medicine, pain medicine, and cardiology. Six providers tested the live virtual scribe and 10 tested the asynchronous virtual scribe.

In pre-pilot surveys, clinicians reported completing 36.5% of documentation during clinical shifts (SD 22%, range 5-90%). Eight clinicians reported spending 6-15 hours per week completing charting outside of normal business hours, whereas seven reported spending 0-5 hours, and one reported 25+ hours. Most clinicians reported using the computer in the exam room with patients, typically for reasons like ordering labs, documenting the note, reviewing online resources and updating the problem list or history. Most clinicians also reported using several different methods for documentation, including direct typing (12/16), speech recognition software (13/16), and personalization tools like Smartphrases (10/16). Three clinicians reported having a scribe or office staff help enter their documentation. Most clinicians agreed or strongly agreed with the statement “I spend too much time on documentation” (13/16). Half of clinicians reported they were “under stress… but I don’t feel burned out,” while the remaining eight clinicians reported feeling that they were “definitely burning out,” having “symptoms of burnout that wouldn’t go away” or feeling “completely burned out.”

After 30 days in the post-pilot surveys, the number of clinicians reporting more than 5 hours of documentation outside of their normal business hours dropped from 9 to 5. The number of clinicians who agreed or strongly agreed with the statement “I have enough time for documentation” went from two prior to the pilot to 12 afterward. On average, clinicians reported the virtual scribe saved 3 hours per shift (SD 3.0). Regarding burnout, fewer clinicians reported feeling one or more symptoms of burnout post-pilot compared to pre-pilot, with no clinicians reporting they felt ‘completely burned out’ and three reporting no symptoms of burnout.

Interviews with clinicians demonstrated several themes related to the effects of using scribes, including increased efficiency and reduced stress on clinicians (with additional detail in Table 3). However, clinicians voiced concerns regarding variable scribe quality and the training process, given it was more burdensome to provide ongoing feedback to virtual scribes when there was a lack of consistency between scribe and provider. Clinicians’ feedback regarding patient perceptions of the scribe was positive or neutral. Consistency with the same scribe(s) emerged as a theme facilitating successful use, whereas needing to train new scribes was a limiting factor. Additional positive aspects of virtual scribe use included: more face-to-face time with patients, easy incorporation into clinical workflow, and scribes’ ability to work within existing note templates. Concerns arose related to accuracy and sensitivity of language used by the scribe and how patients might perceive that language given Open Notes with patients now being able to directly view clinician’s notes after being signed. Additional clinician concerns in some cases related to decreased utility for certain types of encounters (e.g. pre-operative assessments with very specific templates were perceived as less useful for a scribe to complete) and certain types of complex workflows, and concerns regarding hardware including device size and battery life. The short duration of the pilot of only 30-days was also mentioned as a limitation by a number of participants. Participants also noted strengths of the app itself such as overall ease of use and ease of use with interpreters, as well as the ability to provide free-form feedback to scribes after patient encounters. While there were uncommon instances where the app did not pick up the recording, participants noted few other weakness of the application itself

Table 3:

Thematic Analysis of Qualitative Interviews with Clinicians on Use of Virtual Scribe

Themes	Example Quotations
Positive/Negative Aspects of Virtual Scribe Use
• Efficiency	“I was seeing 4 more patients at least” (Ambulatory Subspecialty) “There was pretty minimal difference” (Emergency Medicine) “In general it was tweaking a couple of things and then signing it, so that was a time saver” (Primary Care) “Documentation time saved was 2.5 to 3 hours per clinic” (Ambulatory Subspecialty)
• Clinician wellbeing	“(it) made me a happier provider” (Primary Care) “I was able to listen to the patient and focus. I think it was much better visits” (Ambulatory Subspecialty)
• Variable scribe quality	“Some scribes really knocked it out of the park” (Primary Care) “The scribes are different...some of them are better than others” (Ambulatory Subspecialty)
Clinician’s Perception of Patient View of Virtual Scribe	“Almost all the patients were grateful that it existed…they’re aware of the amount of work that I have to do and…the potential for burnout because they’ve lost other providers” (Primary Care) “Most people pay pretty minimal attention to it” (Emergency Medicine)
Facilitators of or Barriers to Successful Use
• Consistency of scribes	“The thing that did not go well is that I must have had five different individuals scribing for me” (Primary Care) “If I was going to use it on an ongoing basis, it is greatly helpful to have … the same person as often as possible…” (Ambulatory Subspecialty) “one to two or maybe three scribes … a smaller number of people would be most helpful” (Primary Care)
• Training and feedback	“We never really got to a place where it was super effective. We are kind of still in that learning curve at the end of the trial” (Ambulatory Subspecialty) “Some physicians need more training than others with a scribe” (Primary Care)
• Technology issues	“The ipads need to be newer or have longer battery life” (Amb. Subspecialty) “The scribe didn’t have reliable connectivity” (Ambulatory Subspecialty)

In examining EHR data during the 30-day pilot, average time in notes per appointment among clinicians decreased slightly, from 8.54 minutes to 7.04 minutes. Progress note length slightly increased, from 4,930 characters to 5,589 characters. In a smaller analysis of participating emergency medicine physicians, time in notes and note length were comparable before and during the pilot.

Discussion

The ongoing crisis of clinical burnout and moral injury among clinicians in the United States poses major challenges for every healthcare system and is of urgent concern (19). Given that administrative tasks related to the EHR are a major contributor, it is troubling that relatively few studies explore the potential role of evolving virtual or digital scribes technologies in reducing this burden. There is a need for better reporting of efficient technology selection and implementation processes among healthcare systems interested in using these technologies. We aimed to provide a useful case study by describing our institution’s process for rapidly selecting virtual scribe solutions tailored to the needs of our clinicians and exploring the usability of this technology. Our efforts align with the goals put forth by the American Medical Informatics Association’s 25x5 Symposium, which aims to aggressively reduce documentation burden to 25% of the current state within five years (20). Importantly, these efforts were occurring alongside other efforts to reduce documentation burden within our healthcare system, and within the context of other systemic shifts including ongoing shortages of the in-person scribe labor supply (21).

Our solution selection applied a range of principles important for implementation of new digital health technologies within complex systems, guided by a schema by Marwaha et al. Diverse sets of stakeholders are often not involved in the IT selection process within complex health systems; however, this was felt to be important for buy-in and gaining diverse perspectives on needs, which was useful in selecting an appropriate technology to pilot. The selection process and pilot were conducted over a relatively short amount of time, which was helpful for having momentum with this project across operational, IT, and clinical stakeholders.

Results from the pilot study suggest that our selected virtual scribe solutions were acceptable and useful by some clinicians. Results of other virtual scribe studies also support this (9-10, 22). Given the small number of participants in this time-limited pilot, we are not able to draw broad conclusions or extract trends by specialty. However, the highly positive comments among some clinicians suggest that the technology could play an important clinical role. Larger studies are warranted to explore the effect of clinician- or system-level factors moderating any association between virtual scribe interventions and outcomes important for understanding documentation burden. Some clinicians also voiced concerns related to variable scribe quality, number of scribes with whom they worked, and language used by scribes in the note. Scribe and note quality were not directly measured in this study, only healthcare providers’ perceptions of quality. Concerns regarding variable scribe proficiency have been reported in other pilot studies evaluating virtual scribes (22). It is possible that longer pilot duration would allow for greater consistency with scribes and more opportunities for scribe training to learn clinician preferences, which could influence clinicians’ views of overall scribe quality. Longer-term studies are warranted to understand the association between scribe consistency and time spent using various scribe solutions on outcomes that capture overall value.

Cost is an important consideration in the evaluation of long-term virtual scribe programs. In the review of different vendors on the market, there was considerable heterogeneity in cost-models, with some charging based on number of providers versus various metrics of time. High upfront costs are a noted barrier in implementation of digital scribes (12). In a different study of in-person medical scribes, cost models for offsetting scribe costs were estimated to only be a modest increase of two additional patient encounters per day of productivity (23). In our experience, however, measuring true costs and recuperation of costs is more involved and varies significantly by clinician and specialty. While 14 clinicians in this pilot reported efficiency gains from using the virtual scribe solution, measuring return on investment can be a challenge and requires careful organizational scrutiny. This is further complicated by the varying training time it can take for virtual scribes to adapt to clinician preferences (though this could become more efficient with further automation of digital scribes) and the fact that certain gains - such as avoiding a clinician from leaving the workforce - might not be directly measured in traditional cost-savings models. These factors pose a challenge for healthcare systems in making decisions regarding resource allocation for virtual or digital scribe adoption.

Given that this was an institutional quality improvement project with a relatively small group of participants over a short time period and without true time-concordant controls, it is difficult for us to draw broad conclusions from this experience despite some strong indications that many of the clinicians using virtual scribes derived significant value from this work. As such, our results may not be generalizable to other clinician groups and healthcare systems. Metrics of interest such as documentation quality or patient attitudes toward virtual scribes were not directly measured; these are important areas for further investigation. Future studies should also focus on understanding the effect of virtual scribes in comparison to other tools for assisting providers with documentation, including digital scribes and speech recognition software.

Conclusion

We describe our process for IT selection and pilot implementation for a virtual scribe tool to reduce the burden of documentation among a diverse group of clinicians. More research is warranted to understand the optimal implementation process of a virtual scribe solution, the impact of virtual scribes on clinical practice and clinician burnout, and among whom virtual scribes would be most useful in reducing documentation burden. Documenting IT selection and implementation processes for these tools can shed light on how to meaningfully incorporate them into complex health systems given varying clinician needs, cost, IT requirements, and other considerations.