Impact of HIT on burnout remains unknown – for now

We read your editorial¹ in reference to Gardner et al’s paper² with great interest, as we agree these publications address a critical question: how health information technology (HIT) contributes to clinician wellness or burnout. We found the editorial put Gardner’s study in an appropriately broader context, and we agree with the numerous recommendations Gardner made regarding electronic health record (EHR) optimization. Our own Medical Informatics (MI) division was formed with the motto of delivering IT4US: Information Technology for Usability and Safety. Usability (serving the clinicians’ needs) has always been a core objective of our profession, and the interest in this area is increasing exponentially. However, other core MI domains mandate that we rigorously interpret scientific evidence and communicate the findings in an objective and responsible manner.

From this perspective, we found serious methodological and interpretive concerns in Gardner’s study. Their claim that they measure “the impact of health information technology” rings hollow, since they only measured the correlation between burnout and a proposed set of “HIT-related stressors.” No justifiable argument was provided to actually relate these purported stressors to HIT use in patient care settings nor did they demonstrate a causative relationship between them. We find the claims of having established the impact of HIT and the “related” stress as “predictors” of burnout unsubstantiated and outright dangerous, as secondary publications quote these as facts (eg, claiming “EHRs Contribute to … Burnout”).³ These claims may fuel a self-fulfilling negative attitude toward EHRs among our colleagues at a time when the critically important task of improving EHR usability requires well-informed, well-intended clinicians with a relentless commitment to collaboration.

We applied time-honored validity criteria⁴ to Gartner’s study and found it fails basic requirements to substantiate its findings.

1. “Were the outcomes and exposures measured in the same way in the groups being compared?”

   The answer is no. The exposures of work-related stress factors were only assessed in the EHR-user group and were attributed to the EHR by self-justifying assumptions. The authors acknowledged the study design did not determine whether the most significant “stresses such as insufficient documentation time are related to technology vs the burden of work itself.” However, this is not a mere limitation of their findings, but rather a fundamental rebuke of the claim that the study’s purported “HIT-related stressors” are indeed related to the HIT, let alone any plausible claim for causation.

   Of note, the other 2 purported “HIT-related stressors” are also self-justifying assertions. As the first symptom of burnout is being hypercritical, frustration with the ever-present EHR can obviously be a symptom rather than a cause of burnout. A scientific study can only attribute such “frustration” to the EHR if the same workflows (accessing records, finding lab results, receiving external records, visualizing imaging studies, communicating with patients and colleagues, trending data, etc.) are similarly assessed in the comparison group: the frustration of what used to be called “paperwork.” Such comparison was entirely absent. The same argument applies to the measure of “time spent in the EHR at home.” Additionally, this factor may not necessarily be a cause, but rather a symptom (or a mitigating mechanism) of work-related stress.

2. “Were there clearly identified comparison groups that were similar with respect to important determinants of outcome (other than the one of interest)?”

   The answer is no. Similarity between the groups was not established for important determinants. The comparison groups were not controlled for any known causative factors of burnout even though the paper specifically listed several of these: misaligned values, disorganized workflows, productivity pressures, and loss of autonomy. They also failed to control for, or even mention, the factors that appear to improve usability and EHR adoption: clinician builders and trainers, personalization, and a trusted and agile improvement process.⁵

   While the authors state HIT’s effect on burnout is not well characterized, they provide a list of references regarding how EHRs potentially contribute to burnout. Several of those examples describe the misuse of EHRs rather than its inherent negative effects (poor user interface, delegating clerical work to clinicians, etc.), while other well-known contributors of true HIT-related stress (eg, the transient effect of recent adoption) were not mentioned. Yet, none of these factors were evaluated in the study.

A retrospective case-control study design would not allow establishing causation even if the previously described fatal design flaws had been avoided. However, the fundamental design limitations (subjective and unilateral assessments of the groups) disallow the postulation of a mere “contributory causation.”

Clinician-reported EHR benefits, like quality of documentation and its effect on patient care, were described using misleading dichotomization. Although half of the responders agreed the EMR positively improved their workflow and quality of care, the authors dismissed this important finding by clustering neutral and negative responses and stating that the “majority of physicians feel that EHRs do not improve patient care.” Sadly, this misleading and negative statement is going viral, despite contradictory data from well-implemented EHRs.⁵

In summary, the lack of a legitimately pursued null hypothesis, the unilateral pursuance of self-justifying “HIT-related stressors” and the previously described methodological limitations nullify the study’s claim for establishing association or causation between HIT and burnout.

The widespread deployment of the EHR was arguably the biggest change in the practice of medicine over the past 15 years. While one may argue it was implemented without the rigorous approval process that applies to most health care interventions, we are not absolved from studying its effects with the scientific rigor that the issue deserves. Although we fully support studying and mitigating unintended consequences of EHR implementation, we believe it is critically important that future studies in this domain follow the standards of evidence-based medicine.