Conceptual considerations for using EHR-based activity logs to measure clinician burnout and its effects

Abstract

Electronic health records (EHR) use is often considered a significant contributor to clinician burnout. Informatics researchers often measure clinical workload using EHR-derived audit logs and use it for quantifying the contribution of EHR use to clinician burnout. However, translating clinician workload measured using EHR-based audit logs into a meaningful burnout metric requires an alignment with the conceptual and theoretical principles of burnout. In this perspective, we describe a systems-oriented conceptual framework to achieve such an alignment and describe the pragmatic realization of this conceptual framework using 3 key dimensions: standardizing the measurement of EHR-based clinical work activities, implementing complementary measurements, and using appropriate instruments to assess burnout and its downstream outcomes. We discuss how careful considerations of such dimensions can help in augmenting EHR-based audit logs to measure factors that contribute to burnout and for meaningfully assessing downstream patient safety outcomes.

INTRODUCTION

Burnout is a work-related syndrome involving 3 dimensions: emotional exhaustion, depersonalization, and a sense of low personal accomplishment.^1–3 Although burnout has been reported to be prevalent in nearly 50% of physicians,¹ a recent systematic review found that prevalence estimates among physicians range from 0%–80.5%, highlighting variations in the definitions of burnout and its assessment methods.⁴ Although the causal contributors of burnout are multifactorial, clinician workload is a major contributor. For example, increased work hours, increased call burden, and dissatisfaction with work–life balance are all associated with an increased risk for developing burnout.^5,6 Furthermore, the ubiquitous use of electronic health record (EHR) systems has been cited as contributing to increased workload, leading to stress and burnout.^7–10 Factors associated with EHR use and its associated clinical workload can include excessive clinical documentation,¹¹ poor usability of the interfaces,¹² and unnecessary need for navigating across pages.¹³

Current informatics methods for evaluating burnout have focused primarily on quantifying clinician workload as 1 of its primary contributors. Traditionally, these efforts have relied on self-reports, participant journals, shadowing and time-motion studies, and focus groups.^14–19 More recently, researchers have leveraged audit logs of EHR-based activities as a source for tracking clinician workload. Studies have used audit logs to measure administrative burden,⁸ cognitive load,^20,21 interruptions,^10,22 medication ordering,¹⁶ interface navigation,^23,24 clinical documentation,^25,26 and out-of-office work.^7,27 However, few studies have explored the relationship of such workload with burnout,^7,28–33 and direct measurement of burnout using such techniques remains limited (see exceptions ^32,34)

Although EHR-based audit logs provide a snapshot of clinician activities, they have limited ability to represent the context of the clinical activities and do not account for personal or situational factors that are significant contributors to burnout. Additionally, even though interventions informed by the analysis of audit logs, such as the provision of contextual clinical decision support tools, afford opportunities for reducing burnout, prior informatics research has not fully evaluated the downstream effects of clinician burnout on quality of care or patient safety outcomes associated with burnout.

Our focus in this perspective is on highlighting the conceptual considerations needed to augment the use of EHR-based audit logs beyond characterizing clinical workload by incorporating personal and situational factors that contribute to burnout. Towards this end, we describe (a) a conceptual framework for utilizing EHR-based audit logs for evaluating burnout within the situated context of a clinical environment, and (b) pragmatic considerations for applying this framework using existing tools and instruments for investigating burnout and its impact on downstream patient safety and quality outcomes.

A CONCEPTUAL FRAMEWORK FOR EVALUATING BURNOUT

Our conceptual framework is based on a systems-oriented approach for a holistic characterization of burnout and its impact.²² A systems-oriented framework helps in characterizing burnout by accounting for humans (clinicians’ personal and emotional characteristics), technology (EHRs), tasks (clinician’s EHR-based work), organizational policies (rules, regulatory considerations), and the clinical environment within a socio-technical system.³⁵ In addition, burnout also impacts the internal (eg, leadership) and external (eg, billing requirements, regulatory compliance) governance structures that comprise the contemporary clinical environment. Informed by such a systems-oriented approach, we conceptualize burnout as being caused by complex interactions of systems-based factors (eg, work-related and individual factors) situated within the context of a healthcare delivery environment.

A recent National Academy of Medicine (NAM) report identified work-related and emotional demands as key contributors to burnout within the socio-technical healthcare system.²² Work-related demands arise from excessive workload, unmanageable schedules and tasks, complex organizational protocols, encroachment into personal time, and patient-based clinical factors.^22,36 The impact of these work-related factors on burnout is affected by individual clinicians’ attributes and inherent characteristics such as their training, coping abilities, resilience, mental health, and personal situations impacting their overall emotional well-being. These inter-related factors manifest as poor physiological and behavioral responses including decreased sleep, reduced activity, and limited engagement with patients.³⁷ Burnout, in turn, affects safety of patients under their care (see Figure 1A, top part of the figure).

Figure 1.

A. A systems-based conceptual model of physician burnout including the work-related factors and individual factors affecting physician burnout (see top half). B. Realization of the systems-oriented conceptual framework; the emotional exhaustion dimension of burnout can be captured through a combination of clinical activity measures, physiological behavioral measures, and personal factors. The list of sub-factors contributing to each of the factors are examples and are not meant to be exhaustive (see bottom half).

Such a systems-oriented approach provides a framework for reconceptualizing the use of EHR-based audit logs—beyond characterizing clinical workload—for studying burnout as a complex, systems-oriented problem. Within this framework, work-related factors can be captured using EHR-based audit logs (see box labeled “EHR-based clinical activity measures” Figure 1B). Audit logs record various types of EHR-based activities (eg, documentation, communication using secure messages, review), information about patients associated with these activities, timing and context of these activities (eg, at the hospital, time of day), and additional technology-driven events (eg, EHR alerts). In other words, these activities can be used to capture work-related factors (ie, clinical workload) contributing to burnout by addressing questions such as: (a) what types of activities were conducted, (b) how much time was spent on these activities, (c) what was the sequence of activities performed, and (d) where the activities were conducted (eg, work, home). Supplementing these activity logs with clinician workflow information (eg, patient complexity, number of patients seen) can provide additional context for EHR-based activities and their relationship(s) to the quality, safety, and outcomes of care delivery.

Similarly, individual clinician factors, such as demographic characteristics, clinical specialty, years of experience, marital status, spousal occupation, coping skills and resilience,³⁸ and number of dependents, have also been shown to be reliable predictors of burnout³⁹ (see box labeled “personal factors” Figure 1B). Furthermore, behavioral and physiological factors, such as fatigue and lack of sleep, have been associated with emotional and physical distress among clinicians (see label “physiological activity measures” Figure 1B).⁴⁰

Considering the work-related and individual factors within a systems-oriented framework offers several advantages in augmenting the study of burnout using audit logs: first, as suggested in the NAM report, it affords the ability to characterize clinical work activities derived from EHR-based audit logs within the context of a specific clinical environment as well as the individual-level factors enumerated earlier. This can guide the design of situated, systems-based interventions to mitigate the effects of burnout.²² Second, it can help in the direct evaluation of the effects of burnout on proximal downstream clinical events such as errors, clinical decision-making, missed clinical events (eg, a missed test or order), and compliance failures. Such evaluation studies are possible only within the context of the considered clinical environment and local organizational policies. Third, it affords the ability to independently consider the effects of each factor (ie, work-related or individual factors) or a combination of these factors on outcomes. For example, the effects of clinical experience, time spent on clinical documentation, and physiological measures (eg, sleep) on burnout can be independently or jointly investigated. Similarly, potential effects of practice differences (eg, face-to-face vs virtual encounters) can be discerned along with other contributors to burnout and their impact on downstream quality and safety outcomes. Finally, it allows for conceptualizing and evaluating prospective studies of targeted interventions for burnout within the real-world context of a clinical environment, while simultaneously accounting for specific work-related and individual characteristics.

CONSIDERATIONS FOR APPLYING THE CONCEPTUAL FRAMEWORK

The framework provided in Figure 1B is a translation of a systems-oriented framework into a set of factors and example subfactors related to burnout and their impact on downstream patient safety events. Such a translation allows for the aforementioned holistic approach to understanding burnout and, further, provides a basis for designing and conducting evaluative studies intended to identify or mitigate sources of burnout as well as associated deleterious clinical outcomes. As such, in the following subsections, we describe 3 directions for aligning the informatics research on burnout with this framework: standardizing measurement of clinical activity and developing aggregate metrics, using complementary measurement strategies and assessing burnout and downstream outcomes related to burnout.

Standardizing measurement and developing aggregate metrics

A recent systematic review highlighted the variations in the use of EHR-based audit logs for research studies, including differences in activity measures, varying techniques used for calculation, and lack of validation of the considered measures.⁴¹ Key to building a strong empirical foundation for informatics-based approaches is the comparison across studies with consistent metrics for assessing EHR-based audit logs. In a recently published paper, Sinsky and colleagues⁴² identified 7 potential core measures for characterizing activities: work outside of work, EHR total time, documentation time, time on orders, clinical inbox time, and teamwork. These metrics provide a preliminary framework for characterizing and comparing a set of structured activities that can be used for evaluating clinical activity measures.

Although these standardized measurements provide a foundation for research, these can only be used for developing descriptive characterizations of activities performed by clinicians (eg, message volume, frequency of out-of-office work). To associate clinical work activities with actual or perceived workload, these measures should be converted to meaningful correlates of workload. One potential approach—especially within the context of clinical workload—is to utilize these measures to compute metrics of cognitive load experienced by clinicians. Cognitive load can be considered as an overall burden that includes sensory, task-based, and psychological factors that are experienced by an individual.⁴³ Such cognitive burden can be determined based on the nature of EHR-based clinical activities, including number of tasks being performed during a time period and order of these tasks; number of task switches within a patient chart, where increased task switching denotes greater expenditure of cognitive resources or switch cost (eg, from documentation to medication ordering); and number of task switches between different patients, with even higher switch costs (ie, multitasking).⁴⁴ Finally, these cognitive load measures can be compared to instruments such as the NASA Task Load Index (NASA-TLX) scale for creating reliable metrics for standardizing and validating EHR-based audit log measures as a proxy for cognitive load.⁴⁵

Adopting complementary measurements

Burnout is a complex phenomenon, with clinical workload only 1 of many contributors; however, to characterize burnout from EHR-based audit log data, additional measurements that account for or otherwise indicate such systems-level features are required. First, as shown in Figure 1B, measurements need to account for the “individual characteristics” that contribute to burnout—including demographic and personal characteristics and physiological and behavioral activities. Towards this end, newer mobile and wearable devices afford the ability to capture behavioral patterns (eg, physical activity, sleep, and fatigue) that are contributors to the emotional demands of clinicians (Figure 1B, box labeled “physiological activity measures”). This is especially important given that some aspects of burnout also manifest as physiological responses including decreased sleep, reduced activity, increased fatigue,⁴⁰ and limited engagement with patients,³⁷ as was noted earlier. Several studies have established the critical role of sleep with regard to exhaustion⁴⁶ and burnout.⁴⁷ These studies were conducted using self-reports and surveys and prior to currently available sophisticated wearable devices; as such, new opportunities exist complementing wearable devices with audit logs for developing a more nuanced understanding of clinical workload.

Studies evaluating burnout using EHR-based audit logs should also consider longitudinal measurements, as there are likely to be dynamic behavioral changes based on workload and other personal factors, providing the ability to model intra-individual differences over time. The role of ecological momentary assessments, an ambulatory data collection technique that queries present-moment experiences, allowing for real-time sampling of thoughts, feelings and behaviors, can provide more stable estimates of situated behaviors (eg, stress, task load).⁴⁸ Physiological measurements from both wearable devices and ecological momentary assessments can be instrumental in developing situated assessments of workload and individual characteristics and are aligned with the systems-oriented approach for evaluating burnout.

Assessing burnout and downstream outcomes

We describe 2 considerations for using EHR-based audit logs for the assessment of burnout and its impact on downstream safety and quality outcomes for informatics research: first, a temporal alignment of assessment of EHR-based metrics (eg, clinical workload) with “ground truths” related to burnout; and second, the use of decision rules for characterizing downstream outcomes.

In selecting burnout survey instruments for measuring, considered instruments should align with the effects of clinical workload being measured. This is because the effects of clinical workload are likely to affect a clinician’s cognitive capabilities in the immediate ensuing period. For example, for evaluating the effects on burnout of residents’ clinical workload during a clinical rotation, we need measurements that have sensitivity for shorter time periods (see eg,⁴⁹) The Maslach Burnout Inventory survey,^50,51 which is often used for burnout measurement, has a sensitivity of 1-year (questions follow the pattern, “in the past one year”). For measuring proximal impacts, such as clinical workload, long-term measurements are unlikely to be true reflections of burnout associated with their immediate workload. Instead, newer scales, such as the Stanford Professional Fulfillment Index (PFI),⁵² which has a 2-week sensitivity, can be a suitable alternative. Other comparable scales such as the Oldenburg burnout inventory⁵³ and Copenhagen burnout inventory⁵⁴ do not provide a time-scale for evaluation.

The 2 most commonly used burnout outcomes, self-reported errors (eg, see,⁵⁵ or using the PFI⁵²) and costs of burnout,^55–57 have important implications for patient safety and workforce retention. Self-reported errors are subject to recall bias and cannot be aligned with a specified period of experienced burnout. Hence, measurement of burnout with EHR-based audit logs should be complemented with similar temporally occurring outcomes. In addition, these outcome measurements should be objectively obtained allowing for broader applicability across healthcare systems. We describe 2 such automated approaches: automated detection of medication errors and clinical decisions.

One such example is wrong-patient medication errors. Such errors can be detected using EHR-based metrics of ordering activity^33,58 and have been previously used to evaluate patient safety outcomes.^59–61 Wrong-patient error, measured using the retract-and-reorder function, is also recommended by National Quality Forum (NQF Measure #2723) and the Office of the National Coordinator for Health Information Technology as a standard patient safety metric.^62,63 Similar decision support rules, such as those for detecting wrong-drug errors,⁶⁴ offer additional opportunities to evaluate the impact of burnout on downstream patient safety events.⁶⁵ This approach of using standardized decision rules for detecting errors offers 2 advantages: first, it allows for the direct measurement of the effect of burnout on individual clinicians; and second, for comparison of periods of burnout with nonburnout periods.

A related direction for evaluating the direct impact of burnout is the comparison of individual clinician performance to quality-of-care standards. For example, in outpatient settings, a lower completion rate of routine health maintenance events such as cancer screening or vaccine administration represents missed opportunities in clinical care. Although both patient- and clinician-specific factors contribute to these missed opportunities, these process metrics can be considered as potential downstream effects of burnout. Although these are preliminary, and further research is needed, such outcomes can be measured directly from EHR-based activities and provide objective measurement strategies.

CONCLUSIONS

Burnout is a significant threat to clinician well-being and consequently to the safety of patients under their care. In this perspective, we described 3 considerations for using EHR-based activity logs for measuring burnout—standardizing measurements, applying complementary measurements, and evaluating outcomes such as errors and clinical decisions that are also temporally aligned to burnout. These considerations are tightly aligned with the systems-oriented framework, highlighting strategies for improving clinical work activity measurement (ie, work-related factors) and accounting for individual characteristics, and consequently, assessing their impact on burnout and patient safety outcomes.

FUNDING

This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.

AUTHOR CONTRIBUTIONS

TK, JA, SL, & PP conceived the idea. All authors contributed to the development, editing, and reviewing of the manuscript and provided approval for publication.

DATA AVAILABILITY

There is no data available to share.

CONFLICT OF INTEREST STATEMENT

None declared.