Abstract
Background Clinician burnout is a prevalent issue in healthcare, with detrimental implications in healthcare quality and medical costs due to errors. The inefficient use of health information technologies (HIT) is attributed to having a role in burnout.
Objective This paper seeks to review the literature with the following two goals: (1) characterize and extract HIT trends in burnout studies over time, and (2) examine the evidence and synthesize themes of HIT's roles in burnout studies.
Methods A scoping literature review was performed by following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines with two rounds of searches in PubMed, IEEE Xplore, ACM, and Google Scholar. The retrieved papers and their references were screened for eligibility by using developed inclusion and exclusion criteria. Data were extracted from included papers and summarized either statistically or qualitatively to demonstrate patterns.
Results After narrowing down the initial 945 papers, 36 papers were included. All papers were published between 2013 and 2020; nearly half of them focused on primary care ( n = 16; 44.4%). The most commonly studied variable was electronic health record (EHR) practices (e.g., number of clicks). The most common study population was physicians. HIT played multiple roles in burnout studies: it can contribute to burnout; it can be used to measure burnout; or it can intervene and mitigate burnout levels.
Conclusion This scoping review presents trends in HIT-centered burnout studies and synthesizes three roles for HIT in contributing to, measuring, and mitigating burnout. Four recommendations were generated accordingly for future burnout studies: (1) validate and standardize HIT burnout measures; (2) focus on EHR-based solutions to mitigate clinician burnout; (3) expand burnout studies to other specialties and types of healthcare providers, and (4) utilize mobile and tracking technology to study time efficiency.
Keywords: review literature as topic, information technology, electronic health records, burnout, professional
Background and Significance
Burnout has been defined as a “work-related syndrome” commonly characterized by high levels of emotional exhaustion, depersonalization, and dissatisfaction with one's career. 1 2 The prevalence and deleterious effects of clinician burnout, especially physician burnout, have been increasingly studied and documented in recent years. In a 2018 online survey of 15,000 physicians, 42% reported burnout. 3 Concerningly, half of these respondents were between 45 and 54 years of age, an age bracket in which most other professionals find themselves at the apex of their career productivity and experience. 4 5 Although burnout symptom prevalence has decreased between 2014 and 2017, physicians are still at an increased burnout risk compared to other fields. 6 In a 2020 survey to advance practice registered nurses and physician assistants at Vanderbilt University Medical Center, 59.6% of respondents reported that they had experienced or were currently experiencing burnout. 7 As clinicians are key players in the healthcare process, burnout also jeopardizes the well-being of patients, coworkers, and the public health system as a whole. 1 8 Clinicians suffering from burnout risk making medical errors and are more likely to leave their field entirely, which would compromise the quality of patient care, experience and satisfaction with care, increased costs related to errors, and the future of the medical system. 9 10 Therefore, it serves the collective interest to identify the causes of burnout and generate long-term solutions.
Current modalities for mitigating clinician burnout include mindfulness and other individual practices to reduce stress. 11 Moreover, studies have identified a variety of contributing factors to burnout, especially inefficient work processes and poor usability of health information technologies (HIT) that add to clinicians' already considerable workload and hinder the workflow process. 12 13 HIT collect, store, retrieve, and manage clinical data and other related information in healthcare. One primary application of HIT is electronic health records (EHR). While HIT in general, and EHR systems in particular, were designed to facilitate clinical documentation and care routines, HIT can also bring unintended negative consequences to both healthcare providers and organizations. 14 The increased prevalence of EHRs in the workplace has coincided with an uptick in the demand for timely clinical data entry, and the cumbersome and extensive clinical documentation process often leaves clinicians frustrated at the increased workload and time spent documenting rather than providing patient care. 15 16 17 18
An organization-wide survey in Rhode Island by nearly 1,800 physicians further elucidated HIT-related stress as a strong predictor of physician burnout, raising the question of how to best identify HIT-specific factors in order to measure, and remediate burnout. 19 So far, there is limited empirical evidence to demonstrate the extent to which HIT is a contributor. A recent study suggests that while high levels of EHR usage, as recorded in audit trail logs, are associated with self-reported emotional exhaustion, any link between usage and cynicism-related burnout is less clear. 20 In another recent study, however, physician-rated EHR usability was found to be closely associated with the odds of burnout, with a higher usability score correlating to lower odds of reporting burnout. 21 Moreover, attempts to streamline HIT to make it more supportive of workflow processes have yielded mixed results, as EHR-based interventions to improve workflows may increase the likelihood of burnout, while physician-led innovation or improvements can lessen it. 22 23 These largely inconclusive results have led to a greater interest in examining not only the links between HIT implementation and clinician burnout, but also in understanding the efficacy of different intervention methods.
The primary goals of this review are to examine the existing literature regarding the potential roles of HIT in clinician burnout and to categorize the included studies. The review further reveals noticeable trends over time and examines and summarizes the evidence and themes of HIT's roles in the development and progression of burnout. Revealing these patterns will greatly assist in the development of solutions to counteract this insidious problem, hopefully leading to healthier and more beneficial avenues of HIT usage for medical professionals.
Methods
Search Strategy and Screening Process
This paper systematically reviewed the literature following the procedures specified in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) and its extension for scoping reviews. 24 25 Table 1 details the inclusion and exclusion criteria. Search terms were developed based on the above criteria to maximize search yield while maintaining reasonable accuracy of identified publications.
Table 1. Summary of inclusion and exclusion criteria.
Inclusion (n = 3) | Focus on HIT to study clinician burnout |
Written in English | |
Included an abstract and full text | |
Exclusion (n = 7) | Not healthcare related |
Not clinician burnout related | |
Did not use HIT | |
Non-HIT work environment focus | |
Review paper | |
Letters to the editor | |
News articles |
Abbreviation: HIT, health information technology.
The keyword search had two sub-searches to improve its coverage. Both sub-searches were conducted during May 2020. The first sub-search used the keywords “physician burnout” or “clinician burnout” in PubMed, ACM Digital Library, IEEE Xplore, and Google Scholar, whereas the second sub-search was conducted by using the keywords “physician” or “clinician,” and “time pressure” or “time efficiency” in PubMed. Articles were not excluded based on publication year. The search results were combined with a known paper that studied the effect of EHR use on physician fatigue levels by using eye tracking. 26 Two members of the research team (C.X. and A.K.) screened the title and abstract of each paper for eligibility using the inclusion and exclusion criteria. While PubMed yielded many relevant papers, ACM Digital returned three noneligible papers and IEEE Xplore returned no records. To ensure the search coverage, the top 100 records of Google Scholar were screened, and they were either duplicates or not eligible.
The full text of potentially eligible publications was then evaluated for inclusion by the same two team members. Inter-rater reliability of the screening results was calculated by using Cohen's Kappa. Any disagreement between the two reviewers was resolved via discussion. The decisions were also reviewed by the first author as a quality check. References of included papers also followed the same screening and eligibility determination process. All eligible papers were analyzed as described in the following sections.
Data Collection and Management
An online Google spreadsheet was used to track and screen papers for the eligibility determination process and to store information extracted from eligible papers. The online access and simultaneous editing features allowed the research team to coordinate tasks and share information efficiently.
Data Analysis
Two team members extracted relevant information using a framework inspired by the PRISMA Preferred Reporting Items and the Sample, Phenomenon of interest, Design, Evaluation, Research type (SPIDER) tool. 24 27 The information included title, author, year published, location of study, study design, specialty, sample information, description of HIT, measures and variables, results, and limitations from the included papers.
Two phases of data analysis were conducted. Firstly, a statistical summary was generated to characterize all included papers based on the specialty of studied clinicians, publication year, variables of interest, and study design. Study designs were sorted into categories based on existing types of HIT and study design. 28 29 30 Studies were initially placed into experiential (descriptive), observational, and experimental (interventional) categories. Observational studies were further categorized as survey, quantitative data collection, or mixed methods due to special interest in conducting such studies in the future. Studies were also categorized by variable of interest into four groups: physician individual, physician systemic, EHR practices, and patient care ( Table 2 ).
Table 2. Description of study categories and variables of interest.
Study category | Observational | Collected and analyzed burnout data through direct or indirect observations |
Survey | Gathered self-reported data on clinician burnout | |
Quantitative data collection | Time stamp data either from direct observation or from secondary use of EHR data | |
Mixed method | Combination of qualitative and quantitative methods to correlate EHR practices with prevalence of burnout | |
Experiential | Focused on attitudes toward the use of HIT, usually through semi-structured interviews and case studies | |
Experimental | Measured the impact of HIT interventions on the well-being of clinicians before and after its implementation | |
Variable of interest | Clinician individual | Focused on self-reported, individual well-being |
Clinician systemic | Focused on the work environment as it pertains to well-being on a team level | |
EHR practices | Focused on EHR usage, such as number of clicks, messages, and time after hours | |
Patient care | Focused on the relationship between burnout, HIT use, and patient care |
Abbreviations: EHR, electronic health record; HIT, health information technology.
Due to the heterogeneity of the included studies and the exploratory nature of this scoping review, a narrative synthesis was chosen in the second phase of the analysis to identify potential roles of HIT in clinician burnout. Narrative synthesis is an approach that focuses primarily on textual data to summarize and explain the findings of the synthesis. 31 Following the narrative synthesis procedure, the included studies were preliminarily synthesized by a thematic analysis, and their relationships were further explored by using idea webbing. The results of the narrative synthesis on HIT's roles in clinician burnout helped generate implications and inform future directions.
Results
Literature Search Results
The search generated a total of 945 unique articles ( Fig. 1 ). The screening of title and abstract of the articles excluded 894 papers (95.0%). The Cohen's Kappa showed that the inter-rater agreement of the screening process was moderate (53.1%), but the percentage of agreement (both agreed to include or exclude a paper) was substantial (94.7%). The kappa score can be adjusted to reflect the discrepancy between the agreements (adjusted kappa = 92.1%). 32 A total of 51 full texts were assessed for eligibility and 29 of them were included. The references of these 29 articles were scanned and evaluated, resulting in seven more eligible articles. A list of all included articles ( n = 36) and pertinent characteristics is listed in Table 3 .
Table 3. Extracted information from included studies ( N = 36) .
Title | Study | Year | Study location | Data collection method | Study design | Study stage | Study specialty | Type of HIT | Variables of interest | Results |
---|---|---|---|---|---|---|---|---|---|---|
Electronic medical records and physician stress in primary care: results from the MEMO Study | Babbott et al 33 | 2013 | US | Survey | Observational | Examination | Primary care | EHR | Physician individual | Compared to the low EMR cluster, physicians in the moderate EMR cluster reported more stress and lower satisfaction. Physicians in the high EMR cluster indicated lower satisfaction than low EMR cluster physicians. Time pressure was associated with significantly more burnout, dissatisfaction, and intent to leave only within the high EMR cluster. |
Tethered to the EHR: primary care physician workload assessment using EHR event log data and time-motion observations | Arndt et al 46 | 2017 | US | Quantitative data collection | Observational | Identification | Primary care | EHR | EHR practices | Clinicians spent 355 min (5.9 h) of an 11.4-h workday in the EHR per weekday per 1.0 clinical full-time equivalent: 269 min (4.5 h) during clinic hours and 86 min (1.4 h) after clinic hours. Clerical and administrative tasks including documentation, order entry, billing and coding, and system security accounted for nearly one-half of the total EHR time (157 min, 44.2%). Inbox management accounted for another 85 min (23.7%). |
Physician stress and burnout: the impact of health information technology | Gardner et al 60 | 2019 | US | Survey | Observational | Examination | Primary care | EHR | EHR practices | Fewer physicians agreed or strongly agreed that EHRs improve patient care, their clinical workflow, or their job satisfaction. More than a quarter of the sample reported one or more symptoms of burnout. Prevalence of burnout symptoms was 27.2% among physicians with EHRs, compared to 13.6% among those without EHRs. Almost 70% of physicians with EHRs reported HIT-related stress. Agreeing that the EHR adds to daily frustration was associated with more than twice the odds of burnout symptoms and reporting moderately high or excessive time spent on the EHR at home was associated with almost double the odds. Physicians who reported insufficient time for documentation had 2.81 times the odds of burnout symptoms compared to those with sufficient time. |
Electronic health record alert-related workload as a predictor of burnout in primary care providers | Gregory et al 49 | 2017 | US | Survey | Observational | Examination | Primary care | EHR | Physician individual | Subjective alert workload was strongly correlated with the physical fatigue dimension of burnout, whereas objective alert workload was not significantly correlated with any study variables. To reduce workload, PCPs desired color-coded alerts, such that new alerts could be a different color than older alerts. In addition, PCPs wished for workflow enhancing features, such as easy auto-population of test results into templated letters that could be sent to patients, and for better access to their patients' records from other facilities. PCPs also desired additional pieces of information to enhance the system, including how many alerts they had and time-stamps on messages from other providers. |
Physicians' well-being linked to in-basket messages generated by algorithms in electronic health records | Tai-Seale et al 50 | 2019 | US | Mixed method | Observational | Examination | All | EHR | EHR practices | 36% of responding physicians reported burnout symptoms, and 29% expressed an intention to reduce their clinical work time in the coming year. The average number of weekly in-basket messages was 243, of which 114 (47%) were generated by the EHR system. 42% of physicians received above the average number of messages (114 per week). Notably, 45 percent of physicians with burnout symptoms received greater-than-average numbers of weekly system-generated in-basket messages, whereas 29% of physicians with burnout symptoms received only average or less-than-average numbers of the messages. The regression results suggest that factors associated with higher odds of burnout include an above-average number of system-generated in-basket messages and being female. |
Impact of work hours and sleep on well-being and burnout for physicians-in-training: the Resident Activity Tracker Evaluation Study. | Mendelsohn et al 45 | 2018 | Canada | Mixed method | Observational | Examination | All | Consumer | Physician individual | The total hours on-call per week significantly predicted Maslach Burnout Scale scores and the Epworth Sleepiness Scale scores. Nearly two-thirds of participants (61%) scored high burnout on the Maslach depersonalization subscore. Total steps per day and well-being, burnout, and job satisfaction were comparable between groups. Total hours worked, daily sleep, and steps per day did not predict burnout or well-being. |
EHR overtime: an analysis of time spent after hours by family physicians. | Anderson et al 54 | 2020 | US | Quantitative data collection | Observational | Identification | Primary care | EHR | EHR practices | Over the 6-month period reviewed, family medicine trainees and faculty saw between one and 164 patients monthly, spent between 17 and 217 min in the EHR per patient, and spent between 0 and 33 h in the EHR after hours per month. |
Physician anxiety and burnout: symptom correlates and a prospective pilot study of app-delivered mindfulness training | Roy et al 63 | 2020 | US | Mixed method | Experimental | Mitigation | All | Consumer | Physician individual | GAD-7 scores decreased significantly at post-treatment (1 month after treatment initiation, 48% reduction) and at the 3-month follow-up (57% reduction). There was a significant correlation between anxiety and burnout. There was also a significant decrease in cynicism (50% reduction, at posttreatment; 50% reduction, at follow-up) and emotional exhaustion at both time points (20% reduction at posttreatment; 20% reduction, at follow-up). |
Physicians' gender and their use of electronic health records: findings from a mixed-methods usability study | Khairat et al 56 | 2019 | US | Mixed method | Observational | Identification | ICU | EHR | EHR practices | Overall, male physicians reported a significantly higher perceived EHR workload compared with female physicians. Male physicians reported significantly higher levels of frustration with the EHR compared with women. For both men and women, the major complaint focused on time spent looking through patient information, alert fatigue, and documentation. |
Association of electronic health record design and use factors with clinician stress and burnout | Kroth et al 51 | 2019 | US | Survey | Observational | Examination | Primary care | EHR | EHR practices | 210 physicians reported time pressure for documentation, and 170 spent moderately high or excessive time on the EHR at home. The EHR design and use factors significantly associated with high clinician stress were information overload, slow system response times, excessive data entry, inability to navigate the system quickly, note bloat, fear of missing something, interference with the patient-clinician relationship, and notes geared toward billing. |
Practicing clinicians' recommendations to reduce burden from the electronic health record inbox: a mixed-methods study | Murphy et al 58 | 2019 | US | Survey | Observational | Mitigation | Primary care | EHR | EHR practices | PCPs most commonly recommended that inboxes receive only messages with content relevant to the care they provide to their patients. 15% of survey respondents recommended changes to prevent inadvertent loss of important information from the inbox. Seven strategies identified during interviews fit into this theme. Of these, four involved using EHR features to improve message processing efficiency by automating frequently performed actions with macros, templated text, preference lists, and routing lists. |
Electronic health record effects on work-life balance and burnout within the I(3) population collaborative | Robertson et al 34 | 2017 | US | Survey | Observational | Examination | Primary care | EHR | Physician systemic | Only out-of-work hours spent with the EHR was significantly related to satisfaction with work-life balance and EHR effect on work-life balance. A total of 37% of respondents were reported burnout. Of these, 75% attributed their experience of burnout to their EHR to be some or a lot. Like work-life balance, the out-of-work hours spent with the EHR was significantly associated with reported burnout, and reports on the effect of the EHR on burnout. |
Taming the EHR (electronic health record): there is hope | DiAngi et a 59 | 2016 | US | Survey | Experiential | Mitigation | Primary care | EHR | EHR practices | Instead of solely waiting for EHRs to improve their usability and alternative systems to emerge on the market, health systems, and clinicians presently can minimize waste in current EHR usage by investing in ongoing training and tailored technology configurations as well as process redesign and other organizational changes of team based care that support lean workflow. |
Relationship between clerical burden and characteristics of the electronic environment with physician burnout and professional satisfaction | Shanafelt et al 41 | 2016 | US | Survey | Observational | Examination | All | EHRs and CPOE | EHR practices | Physicians who used EHRs and CPOE had lower satisfaction with the amount of time spent on clerical tasks and higher rates of burnout on univariate analysis. On multivariable analysis, physicians who used EHRs or CPOE were less likely to be satisfied with the amount of time spent on clerical tasks after adjusting for age, sex, specialty, practice setting, and hours worked per week. Use of EHRs was not associated with burnout in adjusted models controlling for CPOE and other factors. |
Electronic health record associated stress: a survey study of adult congenital heart disease specialists | Marckini et al 48 | 2019 | US | Survey | Observational | Examination | Cardiology | EHRs | EHR practices | There was strong disagreement that EHRs increased efficiency ( p = 0.006) or the patient portal improved patient care ( p = 0.0215). |
Novel electronic health record education intervention in large healthcare organization improves quality, efficiency, time, and impact on burnout | Robinson et al 64 | 2018 | US | Mixed method | Experimental | Mitigation | All | EHRs | EHR practices | Results from the survey conducted immediately after and 3 mo after training were consistent across all specialties. Most physicians self-reported improved quality, readability, and clinical accuracy of documentation, and also fewer medical errors, and increased efficiency in chart review and data retrieval. The majority of physicians reported an estimated time savings of 4 to 5 min or more per hour; 98% said they would recommend the training to their peers. |
Burnout and EHR use among academic primary care physicians with varied clinical workloads | Tran et al 35 | 2019 | US | Mixed method | Observational | Examination | Primary care | EHRs | EHR practices | Overall, 39% ( n = 41) of providers self-reported burnout on the single item measure. Greater allocated clinical time and patient workloads were closely associated with burnout. Clinicians who self-reported burnout spent more time in the EHR after hours per appointment than their colleagues with the same clinical FTE allocation. Holding clinical FTE constant, reported burnout was associated with an increase of 10 min per appointment after hours compared with no reported burnout. |
The electronic elephant in the room: physicians and the electronic health record | Kroth et al 51 | 2018 | US | Survey | Experiential | Examination | Primary care | EHRs, HICT | Physician systemic | Only 22% of clinicians indicated sufficient time for documentation. Even though 90% reported EHR proficiency, 56% indicated EHR time at home was “excessive” or “moderately high.” Focus group themes included HICT “successes” where all patients' information is accessible from multiple locations. HICT “stressors” included inefficient user interfaces, unpredictable system response times, poor interoperability between systems, and excessive data entry. Suggested “organizational changes” included EHR training, improved HICT usability, and scribes. |
Remote telepsychiatry workforce: a Solution to psychiatry's workforce issues | Gardner et al 60 | 2020 | US | Case study | Experiential | Mitigation | Psychiatry | telehealth | Patient perspective | To provide examples of telepsychiatry being utilized in practice, we discussed three case examples of providers working in differing remote clinical settings. In each of these cases, the providers were able to increase access and to deliver the standard of care. These three providers were able to provide services to a wide range of patients and clinical settings in underserved areas where the psychiatric shortage is especially present, all while optimizing their quality of life and reducing their risk of burnout. |
University of California Technology Wellness Index: a physician-centered framework to assess technologies' impact on physician well-being | Nakagawa et al 61 | 2020 | US | Development of tool | Experiential | Mitigation | Psychiatry | EHR, telehealth | Physician systemic | New EHR: 0 then 4 (negative to positive impact), synchronous: 5, asynchronous: 7, Google glass: 3, VR: 5, remote monitoring; 3 |
Electronic health records and burnout: time spent on the electronic health record after hours and message volume associated with exhaustion but not with cynicism among primary care clinicians | Adler-Milstein et al 20 | 2020 | US | Mixed method | Observational | Identification | Primary care | EHR | EHR practices | One-third (34%) of clinicians had high cynicism and 51% had high emotional exhaustion. Clinicians in the top 2 quartiles of EHR time after hours on scheduled clinic days had 4.78 and 12.52 greater odds of high exhaustion. Clinicians in the top quartile of message had 6.17 greater odds of high exhaustion. No measures were associated with high cynicism. |
Physician burnout in Wisconsin: an alarming trend affecting physician wellness | Hauer et al 37 | 2018 | US | Survey | Observational | Examination | All | EHR | Physician individual | The results indicate that primary causes of physician burnout include utilization and interactions with EHR, lack of a supportive practice environment, the loss of autonomy, and poor work/life balance. 65% of clinicians in the 2017 survey agreed or strongly agreed that the use of an EHR adds frustration to their day. The number of hours physicians spend working on the EHR outside their workday directly correlates with their level of frustration with the EHR. |
Factors associated with provider burnout in the NICU | Tawfik et al 40 | 2017 | US | Survey | Observational | Examination | NICU | EHR | Patient perspective | Overall burnout prevalence was 26.7 ± 9.8%. Highest burnout prevalence was found among NICUs with higher average daily admissions, higher average occupancy, and those with electronic health records. Burnout prevalence showed no association with proportion of high-risk patients, teaching hospital distinction, or in-house attending presence. |
Worklife and wellness in academic general internal medicine: results from a national survey | Linzer et al 42 | 2016 | US | Survey | Observational | Examination | Primary care | EHR | Physician systemic | High stress was present in 67%, with 38% burned out. 60% reported high documentation time pressure, half described too much home EMR time, and most reported very busy or chaotic workplaces. Burnout was associated with high stress, low work control, and low values alignment with leaders. Key themes from the qualitative analysis were short visits, insufficient support staff, a relative value unit mentality, documentation time pressure, and undervaluing education. |
Better usability and technical stability could lead to better work-related well-being among physicians | Vainiomäki et al 47 | 2017 | Finland | Survey | Observational | Examination | All | EHR | EHR practices | Active participation in the development of the EPR system was related to stronger time pressure and stronger job control. In addition, use of several systems daily and the experience of time-consuming documentation of patient information for statistical purposes (billing, national registries, and reporting) were related to higher time pressure, while those with longer experience with the EPR system and those experiencing easy-to-read nursing records reported higher job control. |
To act or not to act: responses to electronic health record prompts by family medicine clinicians | Zazove et al 62 | 2017 | US | Interview | Experiential | Mitigation | Primary care | EHR | EHR practices | Four key direct barriers were identified that impeded alert use: poor standardization and formatting, time pressures in primary care, clinic workflow variations, and mental models of the condition being prompted. One indirect barrier was identified: electronic health record and institution/government regulations. |
Predictors of physicians' stress related to information systems: a nine-year follow-up survey study | Heponiemi et al 52 | 2018 | Finland | Survey | Observational | Examination | All | HIT | Physician systemic | The reciprocity of this association was tested with cross-lagged structural equation model analyses, which showed that the direction of the association was from cognitive workload to SRIS, not from SRIS to cognitive workload. Moreover, increases in time pressure and problems in teamwork were associated with higher levels of SRIS in 2015, whereas job satisfaction increase was associated with lower SRIS. |
The impact of interoperability of electronic health records on ambulatory physician practices: a discrete-event simulation study | Zhou et al 57 | 2013 | US | Quantitative data collection | Observational | Identification | All | EHR | EHR practices | High levels of EHR interoperability were associated with reduced time spent by providers on four tasks: preparing lab reports, requesting lab orders, prescribing medications, and writing referrals. The implementation of an EHR was associated with less time spent by administrators but more time spent by physicians, compared with time spent at paper-based practices. |
Finnish physicians' stress related to information systems keeps increasing: a longitudinal three-wave survey study | Heponiemi et al 36 | 2017 | Finland | Survey | Observational | Examination | All | EHR, IS | Physician individual | SRIS increased during the study period. The estimated marginal mean of SRIS in 2006 was 2.80, and the mean increase was 0.46 points from 2006 to 2010 and 0.25 points from 2010 to 2015. |
Impact of electronic health record technology on the work and workflow of physicians in the intensive care unit | Carayon et al 39 | 2015 | US | Quantitative data collection | Experimental | Examination | ICU | EHR | EHR practices | EHR implementation had an impact on the time distribution of tasks as well as the temporal patterns of tasks. After EHR implementation, both residents and attending physicians spent more of their time on clinical review and documentation. EHR implementation also affected the frequency of switching between tasks, which increased for residents but decreased for attendings, and the temporal flow of tasks, in particular around what tasks occurred before and after clinical review and documentation. |
The impact of electronic health record use on physician productivity | Adler-Milstein et al 53 | 2013 | US | Quantitative data collection | Observational | Identification | Primary care | EHR | EHR practices | Average productivity in our sample was 17.5 RVUs per clinician workday. The average degree of EHR use in our sample was 370 actions per appointment. More intensive EHR use and greater delegation of EHR tasks were independently associated with higher productivity. That is, practices that increased the number of tasks performed in the EHR per visit realized increased clinician productivity and practices that increased the extent to which EHR tasks were performed by clinical support staff as opposed to clinicians saw an independent increase in productivity. |
Association of electronic health record use with physician fatigue and efficiency | Khairat et al 26 | 2020 | US | Quantitative data collection | Experimental | Identification | ICU | EHR | EHR practices | ICU physicians who completed a simulation exercise involving four patient cases recorded a total of 14 h and 27 min of EHR activity. Participants completed the study in a mean of 34:43 min, using 304 mouse clicks, and visiting 85 EHR screens. All physician participants experienced physiological fatigue at least once during the exercise, and 20 of 25 participants experienced physiological fatigue within the first 22 min of EHR use. Physicians who experienced EHR-related fatigue in 1 patient case were less efficient in the subsequent patient case, as demonstrated by longer task completion times, higher numbers of mouse clicks, and more EHR screen visits. |
Estimating the association between burnout and electronic health record-related stress among advanced practice registered nurses | Harris et al 43 | 2018 | US | Survey | Observational | Examination | All | EHR | EHR practices | Of the 371 participants, 73 (19.8%) reported at least one symptom of burnout. Among participants with an EHR ( n = 333), 165 (50.3%) agreed or strongly agreed that the EHR added to their daily frustration and 97 (32.8%) reported an insufficient amount of time for documentation. After adjustment, insufficient time for documentation (AOR = 3.72 [1.78–7.80]) and the EHR adding to daily frustration (AOR = 2.17 [1.02–4.65]) remained predictors of burnout. |
Exploring the association between electronic health record use and burnout among psychiatry residents and faculty: a pilot survey study | Domaney 44 | 2018 | US | Survey | Observational | Examination | Psychiatry | EHR | Physician individual | Residents reported 22 h per week using EHR on average. Mean score of residents surveyed in postgraduate year 1–4 met criteria for high emotional exhaustion associated with burnout. The magnitude of correlation between EHR use and emotional exhaustion was stronger than for other burnout factors including sleep, exercise, and clinical service. |
Physician burnout and timing of electronic health record use | Micek et al 38 | 2020 | US | Mixed method | Observational | Examination | Primary care | EHR | EHR practices | EHR use during in-clinic sessions was related to burnout in both bivariate (OR = 1.04; 95% CI: 1.01–1.06; p = 0.007) and adjusted (OR = 1.07; 95% CI: 1.03–1.1; p = 0.001) analyses. No significant relationships were found between burnout and after-hours EHR use. |
Pajama time: Working after work in the electronic health record | Saag et al 55 | 2019 | US | Quantitative data collection | Observational | Identification | All | EHR | EHR practices | The average physician had scheduled appointments on 3 d/wk, spent 83.3 min in the EHR on days without appointments, and spent 17.3 min in the EHR after work hours on days with appointments. However, time spent working on the EHR on days without appointments increased as the number of appointment days per week increased (15.9 min/unscheduled day for ≤1 d/wk vs. 170.4 min/unscheduled day (2.8 h for >4 d/wk, p < 0.001), as did time spent on the EHR after hours on days with scheduled appointments (17.4 min/appointment-day for ≤1 d/wk vs. 22.0 min/appointment-day for >4 d/wk, p = 0.03) |
Abbreviations: AOR, adjusted odds ratio; CI, confidence interval; EHR, electronic health record; EPR, electronic patient record; EMR, electronic medical records; FTE, full-time equivalent; HIT, health information technology; OR, odds ratio; RVU, relative value units; SRIS, stress related to information systems; US, United States.
Characteristics of Included Studies
Table 4 summarizes the articles included in the analysis. Nearly all were published between 2015 and 2020 ( n = 33, 91.7%). The most common study design was observational ( n = 27, 75%), followed by experiential ( n = 5, 13.9%), and experimental ( n = 4, 11.1%). Nearly half focused on primary care ( n = 16, 44.4%), followed by hospital-wide ( n = 12, 33.3%). The most common variable of interest was EHR practices ( n = 22, 61.1%), followed by one-third of studies being physician individual or systemic ( n = 12, 33.3%). Half of the studies were solely in an academic medical center ( n = 18, 50%). Female clinicians made up more than half of the study participants in nearly half of the included studies ( n = 17, 47.2%). Of note, most papers were attending physician focused ( n = 28, 77.8%), and some included trainees such as residents ( n = 7, 19.4%). Only one study focused solely on nurses or other types of clinicians, although some studies did include nurse practitioners or physician assistants, as well as physicians ( n = 7, 19.4%).
Table 4. Summary of the included health information technology physician burnout papers ( n = 36) .
Attributes | Categories | Count | Percentage |
---|---|---|---|
Year published | 2013–2014 | 3 | 8.3 |
2015–2016 | 4 | 11.1 | |
2017–2018 | 14 | 38.9 | |
2019–2020 | 15 | 41.7 | |
Specialty of studied clinicians | Hospital-wide | 12 | 33.3 |
Primary care | 16 | 44.4 | |
(Neonatal) intensive care unit | 4 | 11.1 | |
Psychiatry | 3 | 8.3 | |
Cardiology | 1 | 2.7 | |
Study design | Experiential | 5 | 13.9 |
Observational | 27 | 75.0 | |
Experimental | 4 | 11.1 | |
Variables of interest | Physician individual | 7 | 19.4 |
Physician systemic | 5 | 13.9 | |
Electronic health record practices | 22 | 61.1 | |
Patient perspective | 2 | 5.5 |
Table 5 shows the characteristics of the observational studies ( n = 27). Primary care was the most common specialty studied ( n = 11), followed by hospital-wide or all specialties ( n = 10), which is consistent with the overall trend. Roughly 60% of the observational studies involved surveys ( n = 16). The rest, in a nearly even split, used either quantitative data collection or mixed methods ( n = 5 and 6, respectively).
Table 5. Summary of the observational health information technology clinician burnout papers ( n = 27) .
Count | Percentage | ||
---|---|---|---|
Specialty of studied clinicians | Primary care | 11 | 40.7 |
Hospital-wide | 10 | 37.0 | |
Other | 4 | 14.8 | |
Data collection method | Survey | 16 | 59.3 |
Quantitative data collection | 5 | 18.5 | |
Mixed method | 6 | 22.2 |
Health Information Technology Trends in the Included Studies
The variables of interest, specialty of studied clinicians, study design, and publication year were compared to reveal trends. Observational studies were published across the years (2013–2020), while experiential studies were published starting in 2016, and experimental studies were mostly published in recent years (2018–2020), with one published in 2015. Experiential studies occurred in primary care ( n = 3) and psychiatry ( n = 2), while experimental studies were focused on intensive care units (ICU, n = 2) or on all specialties ( n = 2), and observational studies were conducted in every field ( n = 12 in primary care, n = 2 in ICU, n = 1 in cardiology, n = 1 in psychiatry, and n = 8 in hospital-wide studies). In terms of the trend in variables of interest by study design, observational studies encompassed all variables of interest themes, with EHR practices being the most common ( n = 17). Experimental studies focused on EHR practices ( n = 3) and physician individual variables of interest ( n = 1). Experiential studies included physician individual, physician systemic, and EHR practices as variables of interest but not patient perspectives. Included studies had a large skew towards EHR as the HIT studied ( n = 32), and other HIT modalities included telehealth ( n = 2) and consumer health information technology applications ( n = 2).
Three Roles of Health Information Technology in Clinician Burnout
The narrative synthesis generated three themes of HIT roles in clinician burnout. First, 21 studies examined HIT as a contributor to clinician burnout, specifically in time pressure, task, and workload. Second, eight studies used HIT to measure burnout levels. Third, seven studies proposed potential HIT-related solutions, but only two of them were focused on developing an intervention to mitigate burnout. The three themes and their relationships are described in the following sections and visualized in Fig. 2 .
Examine Health Information Technology as a Contributor to Burnout ( n = 21)
Studies examining HIT as a contributor to burnout were mostly observational studies and explored whether HIT usage was a factor that contributed to burnout prevalence. A total of 21 such studies were mostly relied on survey data to measure burnout and found that HIT usage and HIT-related time pressures were key factors in causing burnout. Two themes were synthesized from the examination studies: time burden ( n = 15) and task and workload burden ( n = 6).
Time Burden ( n = 15)
Fifteen papers focused on how EHRs impact the amount of time clinicians spend on clerical tasks. Five studies found that clinicians who reported spending more time interacting with the EHR after hours were more likely to report burnout. 33 34 35 36 37 On the other hand, Micek et al found a correlation between burnout and EHR use during in-clinic sessions, but not EHR use after hours. 38 Two papers found that physicians spent more time on documentation after EHR implementation and were at greater risk of burnout. 39 40 Four studies found that feelings of insufficient time for documentation increased the odds for reporting burnout. 19 41 42 43 Domaney found that the correlation between emotional exhaustion and EHR use for psychiatry residents was stronger than for other clinical factors. 44 The remaining study by Mendelsohn et al used consumer activity trackers to collect physicians' work hours and sleep patterns and found that more on-call hours significantly predicted higher scores on the depersonalization aspect of burnout measured by the Maslach Burnout Inventory. 45 Although many studies were cross-sectional and do not necessarily show a causal relationship between EHR usage and burnout, they do emphasize clinicians' large time burden associated with EHR use, with physicians spending an average of 5.9 hours of an 11.4-hour workday in the EHR, 1.4 of those after clinic hours. 46
Task and Workload Burden ( n = 6)
Two papers focused on how the EHR impacts the ability of clinicians to perform clinical tasks. Vainiomäki et al found that technical issues with the EHR were related to high-time pressure and low job control over one's work. 47 In an observational survey study, Marckini et al found that there was high disagreement that EHR improved efficiency and patient care. 48 These studies suggest that EHR implementation has negatively changed clinical workflow. Four papers focused on how the EHR impacts clinicians' workload. Among these four, two papers found that high subjective EHR workload, including alert-related workflow as well as large volumes of in-basket messages, was strongly correlated with exhaustion and increased the odds of burnout. 49 50 The other two papers found that EHR use increased clinicians' workload, leading to more stress. 51 52 These studies elucidate the increased workload possibly caused by EHR implementation and subsequent burnout prevalence.
Use Health Information Technology to Measure Burnout ( n = 8)
Eight papers focused on physician time spent with the EHR and measured different tasks using data from EHR systems, especially time stamps and logs. Adler-Milstein et al identified tasks performed and physician productivity. 53 In a recent study, the authors found that time spent after hours and message volume were associated with exhaustion by using EHR audit trail logs. 20 Arndt et al found that clinicians spent approximately half of their 12-hour workday on the EHR, with clerical and administrative tasks, such as documentation, taking up half of that time (3 hours). 46 Anderson et al 54 found that physicians saw up to 164 patients per month, spending up to 217 minutes per patient interacting with the EHR. 45 Saag et al found that time spent working on the EHR on days without appointments increased as the number of appointment days per week increased, as did time spent on the EHR after hours on days with scheduled appointments. 55 The other three papers identified a time burden on physicians exacerbated by the EHR and mitigated by other factors. Khairat et al found that female physicians used the EHR more efficiently based on the number of clicks and task completion time and observed that female physicians were less likely to report EHR frustrations and to attribute the EHR as a contributor to burnout. 56 Zhou et al found that higher EHR interoperability led to primary care physicians performing tasks more efficiently. 57 Khairat et al also found that EHR use contributed to fatigue in physicians within the first 22 minutes through mouse clicks and screen visits. 26
Developed Health Information Technology-Related Solutions to Mitigate Burnout ( n = 7)
Among the seven papers in this theme, five papers focused on the design and feasibility of HIT implementation to mitigate burnout. Murphy et al found that physicians recommended improvements to the visualization of alerts and streamlining the steps needed to compose a message to reduce burnout. 58 On the other hand, DiAngi et al recommended ongoing EHR training for physicians to improve EHR efficiency rather than waiting for EHRs to become more usable. 59 Gardner et al used three case studies to evaluate telepsychiatry as a viable option to reduce psychiatrists' workloads and risk of burnout without compromising patient care. 60 Nakagawa et al developed a scale with which health care practices could evaluate the impact of the implementation of various HIT on the wellbeing of clinicians. 61 Zazove et al found that poor standardization and formatting of alerts in the EHR, along with time pressure, hindered efficient EHR alert usage. 62 These studies revealed opportunities for the mitigation of burnout using technology-centered improvements.
In addition, two studies in the mitigation stage focused on developing a HIT-related intervention to mitigate physician burnout for all medical specialties. These recent studies (2018–2020) recognized that HIT could be a mitigating factor instead of a propagating factor for physician burnout. In following the idea of using technology to mitigate burnout, Roy et al found that their mindfulness training app reduced anxiety and burnout in physicians. 63 Although having a small sample size, this study showed the efficacy of a consumer-oriented HIT intervention in mitigating burnout. Robinson et al found that most physicians saw improved quality and efficiency of documentation 3 months after receiving in-depth EHR-focused training and education. 64 Although long-term effects were not measured, this study highlighted the need for more comprehensive EHR training in order for physicians to use the EHR more effectively, saving time and mitigating burnout.
Discussion
Key Findings
This paper aims to synthesize the role of HIT in clinician burnout from the published literature. A scoping review was conducted by following the PRISMA guidelines and included 36 papers that placed HIT at the center of their burnout-related studies. All studies were conducted in the past 8 years (2013–2020) and focused on primary care or all specialties. Nearly half of the studies were observational with a focus on EHR practice. Very few were experimental or interventional. The narrative synthesis generated three roles of HIT in contributing to, measuring, and mitigating clinician burnout. Most studies have examined the correlative relationship between the two, especially regarding HIT as a significant contributor to clinician burnout. Other studies utilized HIT as a proxy to study burnout or designed a HIT-related solution to mitigate clinician burnout.
Implications and Recommendations
From this scoping review, four recommendations for further study are generated: validate and standardize HIT burnout measures, focus on EHR-based solutions to mitigate burnout, expand burnout studies to other specialties and types of health care providers, and utilize mobile and tracking technology to study time efficiency. This review also highlights the need for more experimental and interventional studies to evaluate both the accuracy of different proxy methods for measuring time, task, and documentation burden, such as EHR log metrics and tracking technology, to correlate practices with clinician burnout, as well as additional studies to evaluate the efficacy of EHR-based interventions on the prevalence of clinician burnout. Due to the large role EHR usability appears to have on clinician burnout, further experimental studies are also needed to examine which EHR design changes could help reduce its effect on burnout.
Validate and Standardize Health Information Technology Burnout Measures
Although HIT can be generally categorized into three types of study designs based on our findings, the included studies in this paper had a variety of designs and measures. There seems to be no standardized procedure or validated measures to compare the study findings. Hron et al discussed the challenges of using EHR metrics. 65 The study done by Adler-Milstein et al is the first to address this gap by generating EHR-based time measures and correlating them with self-reported burnout levels. 20 However, their use of EHR timestamps was not validated by qualitative or observational or other manually recorded data. Although Sinsky et al, proposed seven core EHR metrics to use to study clinical workflow, more studies are needed to demonstrate the best practices and create a common and validated set of metrics that measure interaction with the EHR to facilitate study design and advance knowledge in this field. 66
Focus on Electronic Health Record-Based Solutions to Mitigate Clinician Burnout
Our findings suggest that EHR-based interventions can be a potential mitigator of clinician burnout. In our search, two recent studies have developed an EHR-related intervention to address burnout, though the studies were on a small scale, measured short term outcomes, and had interventions that were marginally EHR-based (mobile app and training). It is interesting to see that the pervasive use of HIT and EHR systems can contribute to clinician burnout, but at the same time, the EHR may also be the solution. One type of intervention can focus on EHR training and education to bridge the gap between mental models of the EHR designers and users. As Longhurst et al pointed out, adequate EHR training for clinicians is imperative to help mitigate burnout associated with EHR user dissatisfaction. 67 Similarly, Stevens et al created an individualized EHR learning and retraining program for providers and measured burnout as one of the metrics. 68
Another type of intervention can focus on EHR redesign since burnout can be closely correlated with clinicians' interaction with the EHR (e.g., workflow and usability or total duration of exposure to/use of the EHR) as well as the healthcare system demand (e.g., increased workflow and billing). Because the bulk of the papers found focused on clinician time-spent in the EHR, an opportunity for future work is to investigate how to improve clinicians' interactions with the EHR and study how and what types of EHR redesign could help. Studies have shown that acceptable usability can help reduce time pressure and improve work performance as well as quality or care, and may further mitigate clinician burnout. 69 70
Still another direction for future research is to focus on the role of documentation burden. While the 2009 Health Information Technology for Economic and Clinical Health (HITECH) Act proposed the meaningful use of EHRs throughout the United States as a critical national goal, 71 the advent of increased EHR use increased the documentation burden of clinicians, which contributes to their increased time on the EHR both during and after clinic hours, thus contributing to burnout symptoms. 72 Transitioning from one EHR system to another may also cause additional burden on documentation and should be carefully planned to mitigate disruption and its ensuing effect on clinician burnout. 73 Using automatic speech recognition and natural language processing techniques to automate clinical documentation can be a viable solution to reduce documentation burden but requires more research to demonstrate its technical and clinical validity and utility. 74
Expand Burnout Studies to Other Specialties and Types of Health Care Providers
While our search strategy targeted all clinicians, most included papers focused on physicians and did not consider other types of health care providers (e.g., advanced practice providers, nurses, respiratory, occupational, and physical therapists, and social workers). As a result, our paper focused on the burnout of physicians rather than other types of clinicians. Studies have shown that nonphysician clinicians also have a very high level of stress and burnout. 75 76 77 78 These providers' issues have been less studied, so fewer to no solutions have been generated. Additionally, most included studies were conducted in a primary care setting. Other specialties, such as emergency medicine, can have a very high level of burnout due to their time sensitive and high-stress nature. 79 80 81 82 This suggests expanding burnout studies to other specialties and types of providers, as well as examining the differential impact of HIT on burnout in various settings, including but not limited to pediatric versus adult, surgical versus nonsurgical practices, direct versus procedural care, private versus public, and academic versus nonacademic institutions.
Utilize Mobile and Tracking Technology to Study Time Efficiency
Most of the included studies focused on the role of EHR in clinician burnout, and only one paper used mobile-tracking technology as a proxy to study time efficiency with regards to burnout. This EHR bias highlights opportunities to study the efficacy of other types of HIT to study workflow and time efficiency, such as clinical decision support tools, computerized disease registries, and consumer health applications. Among the studies that used HIT as a source or proxy to measure time efficiency, all involved secondary use of EHR data. However, EHR timestamps, whether recorded in the flowsheet or in the audit trail logs, can only capture the activities related to EHR interactions. In order to more fully understand clinician time burden and get a more complete view of clinician workflow, mobile tracking technology used in conjunction with EHR time stamps could be able to capture other activities in the clinical area that cannot be captured by only using EHR timestamps. The recent advances in mobile and tracking technology may provide more opportunities to study clinicians' time efficiency, especially in understanding clinicians' workflow patterns and quantifying bottlenecks and delays. For example, real-time locating systems (RTLS) have become a mature technology and can track the location and movement of staff, patients, and devices in a clinical setting. 83 84 85 86 Smart watches can track people's biometrics in a continuous manner, which can serve as a proxy to clinicians' burnout level, for example, positive association between burnout and heart rate. 87 Future studies should consider utilizing mobile and tracking technologies to collect real-time, individualized data which may be related to burnout and minimizes recollection bias.
Limitations
Our study has several limitations. First, we only searched articles in English and excluded articles written in other languages, although physician burnout may be a universal problem. However, HIT, EHRs, and healthcare systems are often very country or region specific. Correspondingly, burnout may have factors that are regional or culturally specific. Including manuscripts of all languages may obfuscate results by being too generalized. Second, we primarily searched PubMed, which can leave out studies published in other fields. However, our first round of searches yielded a low number of articles in ACM Digital Library and IEEE Xplore and none of them were relevant. While Google Scholar yielded a high number of articles, all of them were either already included through the PubMed search or were not relevant. Third, some articles may have been overlooked due to the design of the search keywords. The relatively low number of papers found and included in this review may have been due to the heterogeneity of the burnout field, making relevant papers hidden under terms and titles as well as abstracts that made them appear irrelevant. While our search strategy was expanded in two directions to include more potential papers that may not use physician burnout as the exact keyword but instead studied time pressure, the additional included papers on physician fatigue revealed an opportunity for a search expansion using “physician fatigue” as a keyword. However, a search of PubMed using “physician fatigue” or “clinician fatigue” as keywords yielded very few relevant results, meaning that our search strategy was comprehensive enough to describe the current literature on physician burnout and HIT while adhering to our study objectives. Screening the references of the included papers also helped expand the search network that may have been limited by the search terms. Additionally, a large portion of our included studies were EHR-based and did not study other types of HIT, which could have impacted the results and recommendations for future study. Last but not least, we did not distinguish adult versus pediatric settings in the information extraction, which might yield different results in the narrative synthesis. However, the number of included papers may not be high enough to show any differences.
Future Work
Because this review has revealed gaps in the literature regarding the use of different types of HIT other than EHRs and the need for more experimental studies, our future work involves applying the four recommendations to develop a set of studies to measure time efficiency and address clinician and provider burnout. We will focus on two specialties, primary care and emergency medicine, and consider the whole clinical team(s). We plan to use mixed-methods and will examine multiple and complementary types of datasets, including but not limited to, self-reported survey data, qualitative interviews, observations, EHR timestamps, and RTLS data, to collect nuanced information about workflow and time efficiency and to track burnout symptoms on a regular basis. The burnout measures will be validated and used across all our studies. This effort can contribute to the standardization of HIT burnout measures and to the design and development of HIT-based interventions to mitigate physician burnout and support long-term experimental studies.
Conclusion
This scoping review suggests that HIT, especially EHRs, can play multiple roles in clinician burnout. While many studies examine HIT's role as a problematic source of burnout, other studies used HIT to measure burnout or developed HIT-based interventions to address this pervasive issue. We encourage researchers to consider HIT's role in contributing to, measuring, and intervening on burnout in order to develop a comprehensive solution to address clinician burnout and improve professional well-being and promote better quality care in the future.
Clinical Relevance Statement
Clinician burnout affects the quality of patient care and clinician productivity. While there are many factors that contribute to clinician burnout, this scoping review focuses on the impact of HIT and the advent of the EHR on clinicians' work experience and prevalence of self-reported burnout with the goal of informing future study on utilizing HIT and EHR interventions to mitigate clinician burnout and thus improve patient care.
Multiple Choice Questions
-
Which of the following was most investigated in the included studies for a correlation to burnout?
After-hours documentation
Volume of in-basket messages
Inefficient EHR
Physician fatigue
Correct Answer: The correct answer is option a. Five included studies investigated the relationship between after-hours documentation and burnout. Two studies investigated the relationship between volume of in-basket messages and burnout, another two studies investigated the relationship between inefficient EHR and burnout, and one study used time spent on EHR to measure physician fatigue.
-
Which of the following was not recommended by the current study?
Utilize mobile and tracking technology
Expand burnout studies to other types of clinicians and specialties
Stop using EHR and go back to paper-based charting
Validate and standardize HIT burnout measures
Correct Answer: The correct answer is option c. While EHR may contribute to clinician burnout, it may also be a potential solution as recommended by the current scoping review. All other three options were part of the implication and recommendations of the current scoping review.
Acknowledgements
This project is partially supported by the service contract that D.T.Y.W. established with the Cincinnati Children Hospital Medical Center Heart Institute to improve the care quality using informatics solutions. The authors would like to thank Dr. Tiffany Grant, an experienced informatician at the UC Health Science Library, for her consultation for developing the search strategy.
Conflict of Interest None declared.
Protection of Human and Animal Subjects
No human or animal subjects were involved in this project.
Authors' Contributions
D.T.Y.W. led this scoping review and coordinated the effort in the research team. D.T.Y.W. designed the study and mentored C.X. to execute the scoping review and write the first draft of the manuscript. A.K. and S.B. helped the paper screening, information extraction and narrative synthesis, and manuscript writing. M.H.E. and K.E.M. reviewed the final draft and contributed significantly to the discussion. All authors reviewed and approved this manuscript before submission.
References
- 1.West C P, Dyrbye L N, Shanafelt T D. Physician burnout: contributors, consequences and solutions. J Intern Med. 2018;283(06):516–529. doi: 10.1111/joim.12752. [DOI] [PubMed] [Google Scholar]
- 2.Rotenstein L S, Torre M, Ramos M A. Prevalence of burnout among physicians: a systematic review. JAMA. 2018;320(11):1131–1150. doi: 10.1001/jama.2018.12777. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Peckham C.Medscape national physician burnout & depression report 2018Accessed June 26, 2020 at:https://www.medscape.com/slideshow/2018-lifestyle-burnout-depression-6009235
- 4.Yates S W. Physician stress and burnout. Am J Med. 2020;133(02):160–164. doi: 10.1016/j.amjmed.2019.08.034. [DOI] [PubMed] [Google Scholar]
- 5.Nicholls M. Cardiologists and the burnout scenario. Eur Heart J. 2019;40(01):5–6. doi: 10.1093/eurheartj/ehy788. [DOI] [PubMed] [Google Scholar]
- 6.Shanafelt T D, West C P, Sinsky C. Changes in burnout and satisfaction with work-life integration in physicians and the general US working population between 2011 and 2017. Mayo Clin Proc. 2019;94(09):1681–1694. doi: 10.1016/j.mayocp.2018.10.023. [DOI] [PubMed] [Google Scholar]
- 7.Kapu A N, Borg Card E, Jackson H. Assessing and addressing practitioner burnout: Results from an advanced practice registered nurse health and well-being study. J Am Assoc Nurse Pract. 2019;33(01):38–48. doi: 10.1097/JXX.0000000000000324. [DOI] [PubMed] [Google Scholar]
- 8.Patel R S, Bachu R, Adikey A, Malik M, Shah M. Factors related to physician burnout and its consequences: a review. Behav Sci (Basel) 2018;8(11):E98. doi: 10.3390/bs8110098. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Stehman C R, Testo Z, Gershaw R S, Kellogg A R. Burnout, drop out, suicide: physician loss in emergency medicine, part I. West J Emerg Med. 2019;20(03):485–494. doi: 10.5811/westjem.2019.4.40970. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Tawfik D S, Profit J, Morgenthaler T I. Physician burnout, well-being, and work unit safety grades in relationship to reported medical errors. Mayo Clin Proc. 2018;93(11):1571–1580. doi: 10.1016/j.mayocp.2018.05.014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Regehr C, Glancy D, Pitts A, LeBlanc V R. Interventions to reduce the consequences of stress in physicians: a review and meta-analysis. J Nerv Ment Dis. 2014;202(05):353–359. doi: 10.1097/NMD.0000000000000130. [DOI] [PubMed] [Google Scholar]
- 12.Mandeville B, Cooper E, Haskell J, Viner-Brown S, Gardner R L. Use of health information technology by rhode island physicians and advanced practice providers, 2019. R I Med J (2013) 2020;103(01):21–24. [PubMed] [Google Scholar]
- 13.Melnick E R, Harry E, Sinsky C A. Perceived electronic health record usability as a predictor of task load and burnout among US physicians: a mediation analysis. J Med Internet Res. 2020;22(12):e23382. doi: 10.2196/23382. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Coiera E, Ash J, Berg M. The unintended consequences of health information technology revisited. Yearb Med Inform. 2016;(01):163–169. doi: 10.15265/IY-2016-014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Colicchio T K, Cimino J J, Del Fiol G. Unintended consequences of nationwide electronic health record adoption: challenges and opportunities in the post-meaningful use era. J Med Internet Res. 2019;21(06):e13313. doi: 10.2196/13313. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Poissant L, Pereira J, Tamblyn R, Kawasumi Y. The impact of electronic health records on time efficiency of physicians and nurses: a systematic review. J Am Med Inform Assoc. 2005;12(05):505–516. doi: 10.1197/jamia.M1700. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Ozeran L, Schreiber R. Reduce burnout by eliminating billing documentation rules to let clinicians be clinicians: a clarion call to informaticists. Appl Clin Inform. 2021;12(01):73–75. doi: 10.1055/s-0041-1722872. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Miyasaki J M, Rheaume C, Gulya L. Qualitative study of burnout, career satisfaction, and well-being among US neurologists in 2016. Neurology. 2017;89(16):1730–1738. doi: 10.1212/WNL.0000000000004526. [DOI] [PubMed] [Google Scholar]
- 19.Gardner R L, Cooper E, Haskell J. Physician stress and burnout: the impact of health information technology. J Am Med Inform Assoc. 2019;26(02):106–114. doi: 10.1093/jamia/ocy145. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Adler-Milstein J, Zhao W, Willard-Grace R, Knox M, Grumbach K. Electronic health records and burnout: time spent on the electronic health record after hours and message volume associated with exhaustion but not with cynicism among primary care clinicians. J Am Med Inform Assoc. 2020;27(04):531–538. doi: 10.1093/jamia/ocz220. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Melnick E R, Dyrbye L N, Sinsky C A. The association between perceived electronic health record usability and professional burnout among US physicians. Mayo Clin Proc. 2020;95(03):476–487. doi: 10.1016/j.mayocp.2019.09.024. [DOI] [PubMed] [Google Scholar]
- 22.DeChant P F, Acs A, Rhee K B. Effect of organization-directed workplace interventions on physician burnout: a systematic review. Mayo Clin Proc Innov Qual Outcomes. 2019;3(04):384–408. doi: 10.1016/j.mayocpiqo.2019.07.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Guo U, Chen L, Mehta P H. Electronic health record innovations: helping physicians - one less click at a time. Health Inf Manag. 2017;46(03):140–144. doi: 10.1177/1833358316689481. [DOI] [PubMed] [Google Scholar]
- 24.PRISMA Group . Moher D, Liberati A, Tetzlaff J, Altman D G. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Int J Surg. 2010;8(05):336–341. doi: 10.1016/j.ijsu.2010.02.007. [DOI] [PubMed] [Google Scholar]
- 25.Tricco A C, Lillie E, Zarin W. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(07):467–473. doi: 10.7326/M18-0850. [DOI] [PubMed] [Google Scholar]
- 26.Khairat S, Coleman C, Ottmar P, Jayachander D I, Bice T, Carson S S. Association of electronic health record use with physician fatigue and efficiency. JAMA Netw Open. 2020;3(06):e207385. doi: 10.1001/jamanetworkopen.2020.7385. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Cooke A, Smith D, Booth A. Beyond PICO: the SPIDER tool for qualitative evidence synthesis. Qual Health Res. 2012;22(10):1435–1443. doi: 10.1177/1049732312452938. [DOI] [PubMed] [Google Scholar]
- 28.Furukawa M F, Raghu T S, Spaulding T J, Vinze A. Adoption of health information technology for medication safety in U.S. Hospitals, 2006. Health Aff (Millwood) 2008;27(03):865–875. doi: 10.1377/hlthaff.27.3.865. [DOI] [PubMed] [Google Scholar]
- 29.Süt N. Study designs in medicine. Balkan Med J. 2014;31(04):273–277. doi: 10.5152/balkanmedj.2014.1408. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.AHRQ Health information technology integrationPublished online February 2013. Accessed April 1, 2021 at:https://www.ahrq.gov/ncepcr/tools/health-it/index.html
- 31.Popay J, Roberts H, Sowden A.Guidance on the conduct of narrative synthesis in systematic reviews: a product from the ESRC methods programme Lancaster University; Accessed 2006 at:https://www.lancaster.ac.uk/media/lancaster-university/content-assets/documents/fhm/dhr/chir/NSsynthesisguidanceVersion1-April2006.pdf
- 32.Chen G, Faris P, Hemmelgarn B, Walker R L, Quan H. Measuring agreement of administrative data with chart data using prevalence unadjusted and adjusted kappa. BMC Med Res Methodol. 2009;9:5. doi: 10.1186/1471-2288-9-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Babbott S, Manwell L B, Brown R.Electronic medical records and physician stress in primary care: results from the MEMO Study J Am Med Inform Assoc 201421(e1):e100–e106. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Robertson S L, Robinson M D, Reid A. Electronic health record effects on work-life balance and burnout within the I 3 population collaborative . J Grad Med Educ. 2017;9(04):479–484. doi: 10.4300/JGME-D-16-00123.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Tran B, Lenhart A, Ross R, Dorr D A. Burnout and EHR use among academic primary care physicians with varied clinical workloads. AMIA Jt Summits Transl Sci Proc. 2019;2019:136–144. [PMC free article] [PubMed] [Google Scholar]
- 36.Heponiemi T, Hyppönen H, Vehko T. Finnish physicians' stress related to information systems keeps increasing: a longitudinal three-wave survey study. BMC Med Inform Decis Mak. 2017;17(01):147. doi: 10.1186/s12911-017-0545-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Hauer A, Waukau H J, Welch P. Physician burnout in Wisconsin: an alarming trend affecting physician wellness. WMJ. 2018;117(05):194–200. [PubMed] [Google Scholar]
- 38.Micek M A, Arndt B, Tuan W-J. Physician burnout and timing of electronic health record use. ACI Open. 2020;04(01):e1–e8. doi: 10.1055/s-0039-3401815. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Carayon P, Wetterneck T B, Alyousef B. Impact of electronic health record technology on the work and workflow of physicians in the intensive care unit. Int J Med Inform. 2015;84(08):578–594. doi: 10.1016/j.ijmedinf.2015.04.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Tawfik D S, Phibbs C S, Sexton J B. Factors associated with provider burnout in the NICU. Pediatrics. 2017;139(05):e20164134. doi: 10.1542/peds.2016-4134. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Shanafelt T D, Dyrbye L N, Sinsky C. Relationship between clerical burden and characteristics of the electronic environment with physician burnout and Professional satisfaction. Mayo Clin Proc. 2016;91(07):836–848. doi: 10.1016/j.mayocp.2016.05.007. [DOI] [PubMed] [Google Scholar]
- 42.Linzer M, Poplau S, Babbott S. Worklife and wellness in academic general internal medicine: results from a national survey. J Gen Intern Med. 2016;31(09):1004–1010. doi: 10.1007/s11606-016-3720-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Harris D A, Haskell J, Cooper E, Crouse N, Gardner R. Estimating the association between burnout and electronic health record-related stress among advanced practice registered nurses. Appl Nurs Res. 2018;43:36–41. doi: 10.1016/j.apnr.2018.06.014. [DOI] [PubMed] [Google Scholar]
- 44.Domaney N M, Torous J, Greenberg W E. Exploring the association between electronic health record use and burnout among psychiatry residents and faculty: a pilot survey study. Acad Psychiatry. 2018;42(05):648–652. doi: 10.1007/s40596-018-0939-x. [DOI] [PubMed] [Google Scholar]
- 45.Mendelsohn D, Despot I, Gooderham P A, Singhal A, Redekop G J, Toyota B D. Impact of work hours and sleep on well-being and burnout for physicians-in-training: the resident activity tracker evaluation study. Med Educ. 2019;53(03):306–315. doi: 10.1111/medu.13757. [DOI] [PubMed] [Google Scholar]
- 46.Arndt B G, Beasley J W, Watkinson M D. Tethered to the EHR: primary care physician workload assessment using EHR event log data and time-motion Observations. Ann Fam Med. 2017;15(05):419–426. doi: 10.1370/afm.2121. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Vainiomäki S, Aalto A-M, Lääveri T. Better usability and technical stability could lead to better work-related well-being among physicians. Appl Clin Inform. 2017;8(04):1057–1067. doi: 10.4338/ACI-2017-06-RA-0094. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.Marckini D N, Samuel B P, Parker J L, Cook S C. Electronic health record associated stress: a survey study of adult congenital heart disease specialists. Congenit Heart Dis. 2019;14(03):356–361. doi: 10.1111/chd.12745. [DOI] [PubMed] [Google Scholar]
- 49.Gregory M E, Russo E, Singh H. Electronic health record alert-related workload as a predictor of burnout in primary care providers. Appl Clin Inform. 2017;8(03):686–697. doi: 10.4338/ACI-2017-01-RA-0003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50.Tai-Seale M, Dillon E C, Yang Y. Physicians' well-being linked to in-basket messages generated by algorithms in electronic health records. Health Aff (Millwood) 2019;38(07):1073–1078. doi: 10.1377/hlthaff.2018.05509. [DOI] [PubMed] [Google Scholar]
- 51.Kroth P J, Morioka-Douglas N, Veres S. Association of electronic health record design and use factors with clinician stress and burnout. JAMA Netw Open. 2019;2(08):e199609. doi: 10.1001/jamanetworkopen.2019.9609. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52.Heponiemi T, Hyppönen H, Kujala S. Predictors of physicians' stress related to information systems: a nine-year follow-up survey study. BMC Health Serv Res. 2018;18(01):284. doi: 10.1186/s12913-018-3094-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53.Adler-Milstein J, Huckman R S.The impact of electronic health record use on physician productivity Am J Manag Care 201319(10 Spec No):SP345–SP352. [PubMed] [Google Scholar]
- 54.Anderson J, Leubner J, Brown S R.EHR Overtime: An Analysis of Time Spent After Hours by Family Physicians Fam Med 2020. Feb5202135–137. [DOI] [PubMed] [Google Scholar]
- 55.Saag H S, Shah K, Jones S A, Testa P A, Horwitz L I. Pajama time: working after work in the electronic health record. J Gen Intern Med. 2019;34(09):1695–1696. doi: 10.1007/s11606-019-05055-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 56.Khairat S, Coleman C, Ottmar P, Bice T, Koppel R, Carson S S. Physicians' gender and their use of electronic health records: findings from a mixed-methods usability study. J Am Med Inform Assoc. 2019;26(12):1505–1514. doi: 10.1093/jamia/ocz126. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 57.Zhou Y, Ancker J S, Upadhye M. The impact of interoperability of electronic health records on ambulatory physician practices: a discrete-event simulation study. Inform Prim Care. 2013;21(01):21–29. doi: 10.14236/jhi.v21i1.36. [DOI] [PubMed] [Google Scholar]
- 58.Murphy D R, Satterly T, Giardina T D, Sittig D F, Singh H. Practicing clinicians' recommendations to reduce burden from the electronic health record inbox: a mixed-methods study. J Gen Intern Med. 2019;34(09):1825–1832. doi: 10.1007/s11606-019-05112-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 59.DiAngi Y T, Longhurst C A, Payne T H. Taming the EHR (electronic health record) - there is hope. J Fam Med. 2016;3(06):1072. [PMC free article] [PubMed] [Google Scholar]
- 60.Gardner J S, Plaven B E, Yellowlees P, Shore J H. Remote telepsychiatry workforce: a solution to psychiatry's workforce issues. Curr Psychiatry Rep. 2020;22(02):8. doi: 10.1007/s11920-020-1128-7. [DOI] [PubMed] [Google Scholar]
- 61.Nakagawa K, Yellowlees P M. University of California Technology Wellness Index: a physician-centered framework to assess technologies' impact on physician well-being. Psychiatr Clin North Am. 2019;42(04):669–681. doi: 10.1016/j.psc.2019.08.005. [DOI] [PubMed] [Google Scholar]
- 62.Zazove P, McKee M, Schleicher L. To act or not to act: responses to electronic health record prompts by family medicine clinicians. J Am Med Inform Assoc. 2017;24(02):275–280. doi: 10.1093/jamia/ocw178. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 63.Roy A, Druker S, Hoge E A, Brewer J A. Physician anxiety and burnout: symptom correlates and a prospective pilot study of app-delivered mindfulness training. JMIR Mhealth Uhealth. 2020;8(04):e15608. doi: 10.2196/15608. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 64.Robinson K E, Kersey J A. Novel electronic health record (EHR) education intervention in large healthcare organization improves quality, efficiency, time, and impact on burnout. Medicine (Baltimore) 2018;97(38):e12319. doi: 10.1097/MD.0000000000012319. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 65.Hron J D, Lourie E. Have you got the time? Challenges using vendor electronic health record metrics of provider efficiency. J Am Med Inform Assoc. 2020;27(04):644–646. doi: 10.1093/jamia/ocz222. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 66.Sinsky C A, Rule A, Cohen G. Metrics for assessing physician activity using electronic health record log data. J Am Med Inform Assoc. 2020;27(04):639–643. doi: 10.1093/jamia/ocz223. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 67.Arch Collaborative . Longhurst C A, Davis T, Maneker A. Local investment in training drives electronic health record user satisfaction. Appl Clin Inform. 2019;10(02):331–335. doi: 10.1055/s-0039-1688753. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 68.Stevens L A, DiAngi Y T, Schremp J D. Designing an individualized EHR learning plan for providers. Appl Clin Inform. 2017;8(03):924–935. doi: 10.4338/ACI-2017-04-0054. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 69.Denton C A, Soni H C, Kannampallil T G. Emergency physicians' perceived influence of ehr use on clinical workflow and performance metrics. Appl Clin Inform. 2018;9(03):725–733. doi: 10.1055/s-0038-1668553. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 70.Pierce R P, Eskridge B R, Rehard L, Ross B, Day M A, Belden J L. The effect of electronic health record usability redesign on annual screening rates in an ambulatory setting. Appl Clin Inform. 2020;11(04):580–588. doi: 10.1055/s-0040-1715828. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 71.Blumenthal D, Tavenner M. The “meaningful use” regulation for electronic health records. N Engl J Med. 2010;363(06):501–504. doi: 10.1056/NEJMp1006114. [DOI] [PubMed] [Google Scholar]
- 72.Gesner E, Gazarian P, Dykes P. The burden and burnout in documenting patient care: an integrative literature review. Stud Health Technol Inform. 2019;264:1194–1198. doi: 10.3233/SHTI190415. [DOI] [PubMed] [Google Scholar]
- 73.Huang C, Koppel R, McGreevey J D, III, Craven C K, Schreiber R. Transitions from one electronic health record to another: challenges, pitfalls, and recommendations. Appl Clin Inform. 2020;11(05):742–754. doi: 10.1055/s-0040-1718535. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 74.van Buchem M M, Boosman H, Bauer M P, Kant I MJ, Cammel S A, Steyerberg E W. The digital scribe in clinical practice: a scoping review and research agenda. NPJ Digit Med. 2021;4(01):57. doi: 10.1038/s41746-021-00432-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 75.Vidotti V, Ribeiro R P, Galdino M JQ, Martins J T. Burnout syndrome and shift work among the nursing staff. Rev Lat Am Enfermagem. 2018;26:e3022. doi: 10.1590/1518-8345.2550.3022. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 76.Molero Jurado M DM, Pérez-Fuentes M DC, Gázquez Linares J JG, Simón Márquez M DM, Martos Martínez Á. Burnout risk and protection factors in certified nursing aides. Int J Environ Res Public Health. 2018;15(06):E1116. doi: 10.3390/ijerph15061116. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 77.Kavalieratos D, Siconolfi D E, Steinhauser K E. “It is like heart failure. it is chronic … and it will kill you”: a qualitative analysis of burnout among hospice and palliative care clinicians. J Pain Symptom Manage. 2017;53(05):901–9100. doi: 10.1016/j.jpainsymman.2016.12.337. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 78.Ben-Zur H, Michael K. Burnout, social support, and coping at work among social workers, psychologists, and nurses: the role of challenge/control appraisals. Soc Work Health Care. 2007;45(04):63–82. doi: 10.1300/J010v45n04_04. [DOI] [PubMed] [Google Scholar]
- 79.Watson A G, McCoy J V, Mathew J, Gundersen D A, Eisenstein R M. Impact of physician workload on burnout in the emergency department. Psychol Health Med. 2019;24(04):414–428. doi: 10.1080/13548506.2018.1539236. [DOI] [PubMed] [Google Scholar]
- 80.Ilić I M, Arandjelović MŽ, Jovanović J M, Nešić M M. Relationships of work-related psychosocial risks, stress, individual factors and burnout - Questionnaire survey among emergency physicians and nurses. Med Pr. 2017;68(02):167–178. doi: 10.13075/mp.5893.00516. [DOI] [PubMed] [Google Scholar]
- 81.Moukarzel A, Michelet P, Durand A-C. Burnout Syndrome among Emergency Department Staff: Prevalence and Associated Factors. BioMed Res Int. 2019;2019:6.462472E6. doi: 10.1155/2019/6462472. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 82.Khairat S, Burke G, Archambault H, Schwartz T, Larson J, Ratwani R M. Perceived burden of EHRs on physicians at different stages of their career. Appl Clin Inform. 2018;9(02):336–347. doi: 10.1055/s-0038-1648222. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 83.Elnahrawy E, Mart R P.Studying the utility of tracking systems in improving healthcare workflow. In: 2010 8th IEEE International Conference on Pervasive Computing and Communications Workshops (PERCOM Workshops) IEEE; Accessed 2010 at:https://www.computer.org/csdl/proceedings-article/percomw/2010/05470651/12OmNrkBwwX
- 84.Chang Y-T, Syed-Abdul S, Tsai C-Y, Li Y-C. A novel method for inferring RFID tag reader recordings into clinical events. Int J Med Inform. 2011;80(12):872–880. doi: 10.1016/j.ijmedinf.2011.09.006. [DOI] [PubMed] [Google Scholar]
- 85.Hung G R, Doan Q. Assessment of the reliability of active radiofrequency identification technology for patient tracking in the pediatric emergency department. Pediatr Emerg Care. 2013;29(02):162–164. doi: 10.1097/PEC.0b013e31828098e1. [DOI] [PubMed] [Google Scholar]
- 86.Berg B, Longley G, Dunitz J. Improving clinic operational efficiency and utilization with RTLS. J Med Syst. 2019;43(03):56. doi: 10.1007/s10916-019-1174-z. [DOI] [PubMed] [Google Scholar]
- 87.Lo E V, Wei Y-H, Hwang B-F. Association between occupational burnout and heart rate variability: A pilot study in a high-tech company in Taiwan. Medicine (Baltimore) 2020;99(02):e18630. doi: 10.1097/MD.0000000000018630. [DOI] [PMC free article] [PubMed] [Google Scholar]