Abstract
Purpose
To observe the impact of using scribes on documentation efficiency in ophthalmology clinics.
Design
Single-center retrospective cohort study.
Participants
A total of 29 997 outpatient visits conducted by 7 attending ophthalmologists between January 1, 2018, and December 31, 2019, were included in the study: 18 483 with a scribe present during the encounter and 11 514 without a scribe present.
Methods
Use of a scribe.
Main Outcome Measures
Total physician documentation time, physician documentation time during and after the visit, visit length, time to chart closure, note length, and percentage of note text edited by physician.
Results
Total physician documentation time was significantly less when working with a scribe (mean ± standard deviation, 4.7 ± 2.9 minutes/note vs. 7.6 ± 3.8 minutes/note; P < 0.001), as was documentation time during the visit (2.8 ± 2.2 minutes/note vs. 5.9 ± 3.1 minutes/note; P < 0.001). Physicians also edited scribed notes less, deleting 1.9 ± 4.4% of scribes’ draft note text and adding 14.8 ± 11.4% of the final note text, compared with deleting 6.0 ± 9.1% (P < 0.001) of draft note text and adding 21.2 ± 15.3% (P < 0.001) of final note text when not working with a scribe. However, physician after-visit documentation time was significantly higher with a scribe for 3 of 7 physicians (P < 0.001). Scribe use was also associated with an office visit length increase of 2.9 minutes (P < 0.001) per patient and time to chart closure of 3.0 hours (P < 0.001), according to mixed-effects linear models.
Conclusions
Scribe use was associated with increased documentation efficiency through lower total documentation time and less note editing by physicians. However, the use of a scribe was also associated with longer office visit lengths and time to chart closure. The variability in the impact of scribe use on different measures of documentation efficiency leaves unanswered questions about best practices for the implementation of scribes and warrants further study of effective scribe use.
Keywords: Electronic health record, Note, Office visit, Scribe
Abbreviations and Acronyms: EHR, electronic health record
As electronic health records (EHRs) gained wide adoption over the past decade, clinicians expressed concern over increased documentation time.1, 2, 3 New billing and documentation requirements that coincided with EHR adoption have often resulted in time away from patients and greater documentation burden. In response, the profession of medical scribe rapidly emerged with the aim of freeing clinicians from clerical tasks and EHR-associated burnout.2,4 Such documentation assistance from scribes may be particularly critical in high-volume specialties like ophthalmology.5
Much of the published literature on scribes has focused on quantitatively assessing economic feasibility and efficiency or qualitatively assessing the effect of scribe use on physicians. In emergency departments, quantitative studies associated scribe use with increased patient volumes,6 relative value units,6,7 and reductions in documentation time of up to 1 hour per 8-hour shift.8 In the outpatient setting, scribes were similarly associated with reduced time to chart closure9,10 and increased patients seen per hour.11 Qualitative studies in outpatient clinics showed increased physician satisfaction when using scribes,5,6,9,10,12 as well as decreased perceived EHR burden, documentation time, and documentation time at home.13,14 In ophthalmology, however, very few studies have examined the use of scribes, either quantitatively or qualitatively, and we are not aware of any published analyses of the impact of scribe use in ophthalmology on clinical documentation or note editing.
This study aimed to expand our understanding of scribe use in ophthalmology. We hypothesize that the use of scribes increases documentation efficiency for ophthalmologists by reducing time spent on documentation, as well as by reducing the amount of editing of office visit notes. To test this hypothesis, we retrospectively analyzed documentation from just fewer than 30 000 outpatient visits with 7 attending ophthalmologists and compared 4 measures of documentation efficiency—and 2 of note editing specifically—between visits with a scribe present versus those without. Understanding how scribes affect documentation will help ophthalmologists to better evaluate the potential benefits and drawbacks of scribes in their practices.
Methods
Institutional review board approval was obtained for this study at Oregon Health & Science University, including a waiver of informed consent for analysis of EHR records. All research adhered to the tenets of the Declaration of Helsinki.
Setting
This study was conducted at the Casey Eye Institute, Department of Ophthalmology, Oregon Health & Science University, an academic medical center in Portland, Oregon. Casey Eye Institute provides primary eye care and is a major tertiary referral center in the Pacific Northwest and nationally. Oregon Health & Science University implemented an institution-wide EHR (EpicCare; Epic Systems) in 2006. All ambulatory practice management, documentation, order entry, medication prescribing, and billing are performed with this EHR. Casey Eye Institute began using scribes in some outpatient clinics in 2015.
Study Dataset and Inclusion Criteria
We collected data from Oregon Health & Science University’s clinical data warehouse on all office visits for study physicians occurring between January 1, 2018, and December 31, 2019. This included visit type, check-in time, check-out time, chart closure time, usage of “smart phrases” (templated text that can be inserted into a note using a single keyword), and EHR use data from audit logs. For each visit, we also acquired data about each saved version of the visit progress note, including the user who saved it, the time at which it was saved, and the text that version contained.
We included attending ophthalmologists who consistently used scribes in their weekly schedule during outpatient clinics at Casey Eye Institute during the study period. We excluded preoperative and postoperative visits and visits where a trainee (resident or fellow) was identified as being present based on a previously validated method.15 We also excluded visits that were missing visit or timing data.
Scribe Presence and Identity
We determined if a scribe was present during each office visit using smart phrase use data. These data allowed us to identify visits in which the word scribe appeared in a smart phrase text and was invoked by a user who was not the ophthalmology physician. This method was manually validated in a previously published study.16 We labeled a visit with a scribe present as a scribed visit and a visit without a scribe present as a nonscribed visit.
Documentation Efficiency
To assess documentation efficiency, we computed 4 efficiency metrics: documentation time (during the visit, after the visit, and total), office visit length, time to chart closure, and note length. All documentation time calculations were performed using EHR audit log data, as validated in a prior study.16 Office visit length was calculated as the difference between check-in and check-out times, and time to chart closure was calculated as the difference between check-in time and chart closure time. Note length was obtained directly from the clinical data warehouse.
Overall Note Editing
The typical workflow for clinical documentation during office visits at Casey Eye Institute involves multiple edited and saved versions of each progress note. Comparing these versions allowed us to compute 2 editing metrics: percentage of final note added by the physician and percentage of draft note deleted by the physician for scribed and nonscribed visits. Technicians typically initiate documentation by starting a progress note from a template during both scribed and nonscribed visits. If a scribe is present, they further edit the saved technician note and resave it before the physician edits and signs the final version of the note. For nonscribed visits, we compared the final version of physicians’ progress notes with the last version saved by the technician, and for scribed visits, we compared the final physician progress note with the last version saved by the scribe. We excluded any visit that did not follow the typical documentation workflow (i.e., any visits with no physician note, no scribe note when a scribe was present, no technician note when a scribe was not present, or multiple progress notes).
We computed the percentage of note text that the physician added to and deleted from the last scribe or technician note (for scribed and nonscribed visits, respectively) using sequence alignment via the modified Levenshtein edit-distance algorithm.17 The percentage of note text added was calculated as the percentage of words in the physician’s note that did not align with (i.e., match) the last scribe or technician note, and percentage of note text deleted was calculated as the percentage of words in the last scribe or technician note that did not align with the physician note. These calculations were performed in Python software version 3.7 (Python Software Foundation).
Statistical Analysis
Mean efficiency metrics (documentation time, office visit length, time to chart closure, and note length), as well as documentation metrics (percent of note text added and deleted for visits) with and without scribes were compared for each individual physician using 2-tailed Welch’s t tests. P values for these t tests were adjusted using the Bonferroni correction for multiple comparisons. To estimate the effect of scribe use on the 4 efficiency metrics across all physicians, we built multivariate linear mixed-effects models where scribe presence was the fixed effect and patient and physician identity were random effects. All data processing and statistical calculations were conducted in R software version 3.5.0 (R Foundation for Statistical Computing), and linear mixed-effects models were constructed using lme4.18
Results
Study Data Set
Seven physicians met the inclusion criteria for the study: 4 comprehensive ophthalmologists and 3 ophthalmology subspecialists. The decision to use a scribe was made by both the subspecialty department and the provider; no substantial difference was found between the practices of the 7 study ophthalmologists and those of their peers. The study ophthalmologists performed 45 570 office visits from 2018 through 2019. We excluded a total of 15 560 unique visits based on our study criteria: 2994 (19.2%) had incomplete visit or timing data, or both; 5471 (35.2%) were operative visits; 5629 (36.2%) were visits with a trainee; and 3977 (25.6%) were visits with no evidence of a technician or a provider participating in the note or visit (note: several excluded visits met multiple exclusion criteria). We further excluded a small number of outlier visits (n = 4) with an office visit length of more than 20 000 minutes above any other visit length, which we attributed to errors in the check-in and check-out data fields. We also excluded visits that had missing values for any 1 of the variables reviewed (n = 9). We were unable to further determine a reason for the missing note data, and we assumed these to be missing at random. After applying all exclusion criteria, 29 997 visits were analyzed: 18 483 with a scribe present (scribed visits) and 11 514 without a scribe present (nonscribed visits).
Each physician used scribes throughout the entirety of the study time frame. The 4 comprehensive ophthalmologists and 1 subspecialist used established technicians as scribes; when a technician was not busy with clinical duties, they served as scribes for the physicians during office visits. The remaining 2 subspecialists used a preprofessional dedicated scribe during office visits (Table 1). The preprofessional scribes received formal training with the physicians, but the technicians did not because they were already working with the physicians and were familiar with their documentation practices. On most days, a physician either used scribes for all of the visits or none of the visits. For the 7 physicians, days in which they had no scribe for most visits occurred approximately once per week throughout the study time frame.
Table 1.
Summary of Physician Data by Specialty and Scribe Use
| Physician | Specialty | Scribe Identity | No. of Nonscribed Visits | No. of Scribed Visits |
|---|---|---|---|---|
| 1 | Subspecialty | Preprofessional | 4064 | 1215 |
| 2 | Subspecialty | Preprofessional | 727 | 1762 |
| 3 | Subspecialty | Technician | 1541 | 366 |
| 4 | Comprehensive | Technician | 1129 | 4296 |
| 5 | Comprehensive | Technician | 1436 | 3580 |
| 6 | Comprehensive | Technician | 1540 | 3807 |
| 7 | Comprehensive | Technician | 1077 | 3457 |
Efficiency Metrics
Total physician documentation time (mean ± standard deviation) was 4.7 ± 2.9 minutes for scribed visits (2.8 ± 2.2 minutes during the visit and 1.9 ± 2.0 minutes after the visit) versus 7.6 ± 3.8 minutes for nonscribed visits (5.9 ± 3.1 during the visit and 1.7 ± 2.6 minutes after the visit; Fig 1). Linear models predicted a 2.3-minute/patient reduction in total documentation time for scribed visits across physicians (P < 0.001), as well as a 2.4-minute/patient reduction in documentation time during the visit (P < 0.001) for scribed visits. The direction of this effect was consistent throughout the study sample, with all 7 physicians having significantly less total documentation time (P < 0.001) and less documentation time during the visit (P < 0.001) for scribed visits. However, 4 physicians showed increases in documentation time after the visit for scribed visits, 3 of whom (physicians 1, 3, and 4) were significant (P < 0.001), whereas 2 physicians (physicians 5 and 6) showed significant decreases in documentation time after the visit when using a scribe (P < 0.001; Fig 2).
Figure 1.
Mean clinical documentation time among 7 ophthalmologists for scribed and nonscribed office visits. Overall significance was tested with a mixed-effects linear model. ∗P < 0.001. Error bars represent 95% confidence intervals.
Figure 2.
Mean after-visit clinical documentation time among 7 ophthalmologists for scribed and nonscribed office visits. Statistically significant differences for individual physicians tested using Bonferroni-corrected Welch’s t tests. Overall significance was tested with a mixed-effects linear model. ∗P < 0.001. Error bars represent 95% confidence intervals.
Average time to chart closure was 9.15 ± 27.6 hours (range, 0.017–433 hours) for scribed visits versus 10.5 ± 33.8 hours (range, 0.1–553 hours) for nonscribed visits. Both measurements were highly variable, both between and within physicians, which resulted in the linear model associating a 3.0-hour increase in time to chart closure with scribe use across physicians (P < 0.001; Fig 3). Physicians 5 and 6 showed shorter average chart closure times when using a scribe; however, each showed less than 1-hour differences in mean time to chart closure for scribed and nonscribed visits. The remaining 5 physicians all showed longer mean time to chart closure, ranging from a 28-minute increase to a 14-hour increase for scribed versus nonscribed visits. The increase was significant for 4 of those 5 physicians (P < 0.01).
Figure 3.
Mean time to chart closure by 7 ophthalmologists for scribed and nonscribed office visits. Statistically significant differences for individual physicians were tested with Bonferroni-corrected Welch’s t tests. Overall significance was tested with a mixed-effects linear model. ∗P < 0.001. Error bars represent 95% confidence intervals.
Average office visit length was 76.3 ± 49.6 minutes for scribed visits versus 78.2 ± 41.9 minutes for nonscribed visits. Only 1 of 7 physicians (physician 5) showed a decrease in mean office visit length, which was not significant. The remaining 6 physicians all showed longer office visit lengths with a scribe, 2 of which were statistically significant increases (P < 0.001). A linear mixed-effect model predicted a 2.9-minute/patient increase in office visit length for scribed versus nonscribed visits (P < 0.001).
Notes were significantly longer for scribed visits (a modeled 506-character increase; P < 0.001) with mean note length of 5348 ± 1795 characters versus 3998 ± 2108 characters for nonscribed visits. This increase in note length was significant for 6 of the 7 physicians (P < 0.001).
Overall Note Editing
Full note text for both draft and final note versions, for a total of 25 861 visit notes, 18 356 scribed visits, and 7505 nonscribed visits, was available for comparison. On average, physicians added less to scribed notes than they added to nonscribed notes prepared by the technician (mean ± standard deviation, 14.8 ± 11.4% vs. 21.2 ± 15.3% of the final note text, respectively; P < 0.001; Fig 4). Physicians also deleted less on average from draft scribed notes than nonscribed technician notes (1.9 ± 4.4% vs. 6.0 ± 9.1% of the draft note text, respectively; P < 0.001).
Figure 4.
Mean percent of clinical documentation revised by 7 ophthalmologists for scribed and nonscribed office visits: (A) percent deleted and (B) percent added. Statistically significant differences for individual physicians were tested with Bonferroni-corrected Welch’s t tests. Overall significance was tested with a mixed-effects linear model. ∗P < 0.001. Error bars represent 95% confidence intervals.
Discussion
We hypothesized that the use of scribes during office visits would increase documentation efficiency for ophthalmologists. This study has 3 key findings: (1) scribe presence decreased documentation burden by reducing physician documentation time during the encounter and the amount of note editing, which supports our hypothesis; (2) however, scribe presence potentially increased documentation burden after the encounter because time to chart closure and physician documentation time after the encounter both increased; and (3) differences among physicians in scribe impact on efficiency and documentation editing suggest that scribes are used in different ways with varying effects.
First, scribe use seemed to decrease documentation burden by reducing physician documentation time during the encounter, as well as the amount of note editing. Overall, total physician documentation time per visit was 38% less for scribed than for nonscribed office visits (Fig 1). More specifically, we found that physician documentation time during scribed visits decreased by an average of 52%. This reduction is somewhat higher than in previous studies in emergency departments that found a 33% to 34% reduction in physician documentation time when using scribes.8,13 We suspect that this discrepancy may be the result of differing workflows in clinic versus the emergency department, or perhaps because of more consistency in physician-scribe pairs in our clinic, because many of the scribes in the study had an existing working relationship with the physicians in their role as a technician. This finding also corroborates previous qualitative studies in outpatient settings, which report reduced perceived documentation time by physicians when using scribes.9,10 Reduction in documentation burden with scribe use was also reflected by a decrease in the amount of physician editing (additions to and deletions from) of scribed notes compared with nonscribed, technician-generated notes (Fig 4). Manual review of a small set of notes (data not shown) further showed that most edits by physicians, for both scribe-generated and technician-generated notes, were to the assessment and plan sections, whereas frequently no edits were made to the subjective portions of the note. Scribes, or other paraprofessional support for note writing, may allow physicians to focus their attention on the medical decision-making aspect of clinical documentation. If the goal of implementing scribes for a physician or clinic is to decrease documentation time during an office visit or to decrease the amount of physician oversight in reviewing clinical notes, then our results suggest that scribe use decreases these specific areas of EHR burden.
In contrast, our second key finding is that scribe use in the study clinics increased documentation burden after the encounter based on time to chart closure and physician documentation time after the encounter. Although the number of physician note edits decreased with scribe use, the increased documentation time after scribed visits may be the result of physicians shifting the documentation activities typically performed during the visit to after the visit. This could be to avoid interrupting the scribe’s editing of the patient record during the visit; however, future work is needed to investigate if this is indeed the case. Another sign of documentation burden after the visit was the increased time to chart closure. Although charts for most office visits were closed within 24 hours, scribe use was associated with an increase of 3 hours in time to chart closure across all physicians and as much as a 14-hour increase for 1 study physician. One perceived benefit of scribes is that they can increase note accuracy by encouraging real-time documentation9,19; this is true for the scribe-generated portion of the note during the visit. However, our results suggest that in some cases, physicians may actually finish charts later when working with a scribe, potentially influencing the accuracy of their recall and subsequent edits to a scribe-generated note, because delays in documentation can cause physicians to forget details of clinical encounters.10 This result raises concerns that introducing scribes into documentation workflows has potential unintended consequences on note writing. Additionally, documentation time after the visit was significantly higher for 3 of the 7 physicians studied (Fig 2). This is in contrast to previous studies of self-reported reduced documentation time after hours.10,14 This increase is particularly concerning if the documentation is occurring at home; Gardner et al4 reported 1.9 times higher odds of burnout for physicians reporting EHR use at home versus those without. Because our calculation of after-visit time and note editing did not identify when or where the EHR time took place (e.g., at work vs. home), future studies are needed to identify the reasons for the increased documentation time, as well as when and where it occurs.
We also observed that patient volumes for days when physicians worked with a scribe versus those without were essentially the same (data not shown), so observed increases were not a result of increased patient volume. Because physicians typically need to increase patient volumes to cover the costs of scribes, more studies are needed to determine how to use scribes efficiently to offset their cost. In addition, office visit lengths when using scribes were longer on average than visits without a scribe. Increased visit length with similar clinic volumes and longer after-visit documentation times may present challenges for efficiency as well as overcoming the financial burden of implementing scribes into a clinic workflow.13,20 Previous studies suggest scribes do not decrease physician EHR use outside of documentation, such as time spent reviewing records or placing orders,13 and do not change the time to complete out-of-visit tasks such as prescription refills or patient portal responses.21 In short, merely adding scribes may not address all aspects of documentation or EHR burden.
The final key finding is that large variability in scribe impact among physicians in our sample suggests that differences in efficiency and note editing may be affected by differences in scribe use. This variability is apparent in documentation time after the visit (Fig 2). Four physicians showed increased average after-visit documentation time, one of which more than doubled documentation time after the visit. Significant variability was also noted in time to chart closure (Fig 3)—ranging from a 9-minute decrease to a 14-hour increase for different physicians—and the amount of text revised by physicians in scribed notes (Fig 4). Previous research suggests that the individual physician–scribe duets could influence efficiency in implementation and workflows, and differences in physicians’ preferences for note content, as well as interpersonal characteristics such as expectation setting and communication, may impact variability in editing and quality of scribed notes.22,23 Furthermore, large variability across physicians in documentation and EHR use has been implicated in reduced efficiency and potential compromise of patient safety.24 We believe the differences observed in this study may reflect variability in integration of scribes into the care team or differences in the clinic setup and individual physician workflows. Differences may also be a product of variable scribe experience and training because some of the scribes in this study were trained technicians (licensed), whereas others had no prior ophthalmology training; this study was not designed to assess those differences. Although some of the variability in our data may be explained by differences among individual physicians and scribes, different means of interacting may also have an effect. Currently, not enough evidence is available to direct physicians on how best to work with scribes, nor is research sufficient in evaluating the completeness and accuracy of scribe documentation and their implications for patient safety. Although benefits to scribe use seem to exist, these may be increased if physician–scribe interactions are studied more systematically to define best practices in training, scope of practice, and documentation workflow.
This study has several limitations. First, it was limited to a single academic institution with a small number of ophthalmologists included in the study; more studies are needed to determine the generalizability of the results. Second, the scribes in this study represented 2 different models of implementation, which may impact use and generalizability, but which we did not differentiate. Finally, this study was not designed to evaluate other important factors such as the quality of notes generated, the review of medical records by physicians (and what portions were performed by physicians vs. scribes), or the impact on quality and safety of care.
The adoption of the EHR has changed medical practice and workflows substantially, although it has also been linked to concerns with efficiency, time away from patients, and burnout. Medical scribes have become increasingly popular as an adjunct intervention to reduce EHR burden. This study illustrated some benefit to scribe use for documentation efficiency in ophthalmology clinics that is consistent with the results of prior studies in other outpatient specialties. It also revealed some potential drawbacks to scribe implementation because some physicians showed significantly longer chart closure times and documentation times after the visit with scribes. Overall, this study leaves unanswered questions about the implementation of scribes, best practices for using a scribe, and the quality of documentation scribes produce. More quantitative and qualitative studies are needed to address these questions, because time engaging with the EHR, documentation quality, and the use of scribes have the potential to impact every practicing clinician. Ultimately, we believe that increased efficiency and quality of ophthalmic care will result from developing these best practices to optimize scribe use and to collaboratively refine EHR and clinical workflows.
Manuscript no. D-21-00111.
Footnotes
Disclosure(s):
All authors have completed and submitted the ICMJE disclosures form.
The author(s) have made the following disclosure(s): M.F.C: Consultant – Novartis; Financial support – Genentech; Equity owner – InTeleretina, LLC
M.R.H.: Lecturer – Children's Hospital of Philadelphia
Supported by the National Institutes of Health, Bethesda, Maryland (grant nos.: T15LM007088, R00LM12238, and P30EY10572); and by Research to Prevent Blindness, Inc, New York, New York (unrestricted departmental funding). The funding organizations had no role in the design or conduct of this research.
Dr Dusek is now at the Emergency Medicine Department, Carolinas Medical Center, Charlotte, North Carolina. Dr Rule is now at the Information School, University of Wisconsin-Madison, Madison, Wisconsin.
Michelle R. Hribar, an editorial board member of this journal, was recused from the peer-review process of this article and had no access to information regarding its peer-review.
Presented in part at: Association for Research in Vision and Ophthalmology Annual Meeting, Vancouver, Canada, May 2019; Association for Research in Vision and Ophthalmology Annual Meeting, virtual, May 2020; and the American Medical Informatics Association’s Annual Symposium, virtual, November 2020.
HUMAN SUBJECTS: Human subjects were included in this study. Institutional Review Board approval was obtained for this study at Oregon Health & Science University, including a waiver of informed consent for analysis of EHR records. All research adhered to the tenets of the Declaration of Helsinki.
No animal subjects were included in this study.
Author Contributions:
Conception and design: Dusek, Goldstein, Rule, Chiang, Hribar
Analysis and interpretation: Dusek, Goldstein, Rule, Chiang, Hribar
Data collection: Dusek, Goldstein, Rule, Hribar
Obtained funding: N/A
Overall responsibility: Dusek, Goldstein, Rule, Chiang, Hribar
References
- 1.Read-Brown S., Hribar M.R., Reznick L.G., et al. Time requirements for electronic health record use in an academic ophthalmology center. JAMA Ophthalmol. 2017;135(11):1250–1257. doi: 10.1001/jamaophthalmol.2017.4187. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Bossen C., Chen Y., Pine K.H. The emergence of new data work occupations in healthcare: the case of medical scribes. International Journal of Medical Informatics. 2019;123:76–83. doi: 10.1016/j.ijmedinf.2019.01.001. [DOI] [PubMed] [Google Scholar]
- 3.Bowman S. Impact of electronic health record systems on information integrity: quality and safety implications. Perspect Health Inf Manag. 2013;10(Fall):1c. eCollection 2013. [PMC free article] [PubMed] [Google Scholar]
- 4.Gardner R.L., Cooper E., Haskell J., et al. Physician stress and burnout: the impact of health information technology. J Am Med Inform Assoc. 2019;26(2):106–114. doi: 10.1093/jamia/ocy145. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Koshy S., Feustel P.J., Hong M., Kogan B.A. Scribes in an ambulatory urology practice: patient and physician satisfaction. J Urol. 2010;184(1):258–262. doi: 10.1016/j.juro.2010.03.040. [DOI] [PubMed] [Google Scholar]
- 6.Shultz C.G., Holmstrom H.L. The use of medical scribes in health care settings: a systematic review and future directions. J Am Board Fam Med. 2015;28(3):371–381. doi: 10.3122/jabfm.2015.03.140224. [DOI] [PubMed] [Google Scholar]
- 7.Heaton H.A., Nestler D.M., Jones D.D., et al. Impact of scribes on patient throughput in adult and pediatric academic EDs. Am J Emerg Med. 2016;34(10):1982–1985. doi: 10.1016/j.ajem.2016.07.011. [DOI] [PubMed] [Google Scholar]
- 8.Heaton H.A., Nestler D.M., Barry W.J., et al. A time-driven activity-based costing analysis of emergency department scribes. Mayo Clin Proc. 2019;3(1):30–34. doi: 10.1016/j.mayocpiqo.2018.11.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Gidwani R., Nguyen C., Kofoed A., et al. Impact of scribes on physician satisfaction, patient satisfaction, and charting efficiency: a randomized controlled trial. Ann Fam Med. 2017;15(5):427–433. doi: 10.1370/afm.2122. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Mishra P., Kiang J.C., Grant R.W. Association of medical scribes in primary care with physician workflow and patient experience. JAMA Intern Med. 2018;178(11):1467–1472. doi: 10.1001/jamainternmed.2018.3956. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Bank A., Obetz C., Konrardy A., et al. Impact of scribes on patient interaction, productivity, and revenue in a cardiology clinic: a prospective study. Clinicoecon Outcomes Res. 2013:399–406. doi: 10.2147/CEOR.S49010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Danila M.I., Melnick J.A., Curtis J.R., et al. Use of scribes for documentation assistance in rheumatology and endocrinology clinics: impact on clinic workflow and patient and physician satisfaction. J Clin Rheumatol. 2018;24(3):116–121. doi: 10.1097/RHU.0000000000000620. [DOI] [PubMed] [Google Scholar]
- 13.Heaton H.A., Wang R., Farrell K.J., et al. Time motion analysis: impact of scribes on provider time management. J Emerg Med. 2018;55(1):135–140. doi: 10.1016/j.jemermed.2018.04.018. [DOI] [PubMed] [Google Scholar]
- 14.Martel M.L., Imdieke B.H., Holm K.M., et al. Developing a medical scribe program at an academic hospital: the Hennepin County Medical Center experience. Jt Comm J Qual Patient Saf. 2018;44(5):238–249. doi: 10.1016/j.jcjq.2018.01.001. [DOI] [PubMed] [Google Scholar]
- 15.Goldstein I.H., Hribar M.R., Read-Brown S., Chiang M.F. Association of the presence of trainees with outpatient appointment times in an ophthalmology clinic. JAMA Ophthalmol. 2018;136(1):20–26. doi: 10.1001/jamaophthalmol.2017.4816. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Hribar M.R., Dusek H.L., Goldstein I.H., et al. Methods for large-scale quantitative analysis of scribe impacts on clinical documentation. AMIA Annu Symp Proc. 2021;2020:573–582. eCollection 2020. [PMC free article] [PubMed] [Google Scholar]
- 17.Wrenn J.O., Stein D.M., Bakken S., Stetson P.D. Quantifying clinical narrative redundancy in an electronic health record. J Am Med Inform Assoc. 2010;17(1):49–53. doi: 10.1197/jamia.M3390. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Bates D., Mächler M., Bolker B., Walker S. Fitting linear mixed-effects models using lme4. J Stat Softw. 2015;67(1):1–48. [Google Scholar]
- 19.Woodcock D.V., Robert P., Karess M., et al. The evolving role of medical scribe: variation and implications for organizational effectiveness and safety. Stud Health Technol Inform. 2017:382–388. [PubMed] [Google Scholar]
- 20.Arya R., Salovich D.M., Ohman-Strickland P., Merlin M.A. Impact of scribes on performance indicators in the emergency department: impact of scribes on performance indicators. Acad Emerg Med. 2010;17(5):490–494. doi: 10.1111/j.1553-2712.2010.00718.x. [DOI] [PubMed] [Google Scholar]
- 21.Zallman L., Altman W., Chu L., et al. Do medical scribes help primary care providers respond more quickly to out-of-visit tasks? J Am Board Fam Med. 2021;34(1):70–77. doi: 10.3122/jabfm.2021.01.200330. [DOI] [PubMed] [Google Scholar]
- 22.Corby S., Gold J.A., Mohan V., et al. A sociotechnical multiple perspectives approach to the use of medical scribes: a deeper dive into the scribe-provider interaction. AMIA Annu Symp Proc. 2020;2019:333–342. [PMC free article] [PubMed] [Google Scholar]
- 23.Van Tiem J.M., Stewart Steffensmeier K.R., Wakefield B.J., et al. Taking note: a qualitative study of implementing a scribing practice in team-based primary care clinics. BMC Health Serv Res. 2019;19(1):574. doi: 10.1186/s12913-019-4355-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Cohen G.R., Friedman C.P., Ryan A.M., et al. Variation in physicians’ electronic health record documentation and potential patient harm from that variation. J Gen Intern Med. 2019;34(11):2355–2367. doi: 10.1007/s11606-019-05025-3. [DOI] [PMC free article] [PubMed] [Google Scholar]