To act or not to act: responses to electronic health record prompts by family medicine clinicians

Abstract

Objective: A major focus of health care today is a strong emphasis on improving the health and quality of care for entire patient populations. One common approach utilizes electronic clinical alerts to prompt clinicians when certain interventions are due for individual patients being seen. However, these alerts have not been consistently effective, particularly for less visible (though important) conditions such as hearing loss (HL) screening.

Materials and Methods: We conducted hour-long cognitive task analysis interviews to explore how family medicine clinicians view, perceive, and use electronic clinical alerts, and to utilize this information to design a more effective alert using HL identification and referral as a model diagnosis.

Results: Four key direct barriers were identified that impeded alert use: poor standardization and formatting, time pressures in primary care, clinic workflow variations, and mental models of the condition being prompted (in this case, HL). One indirect barrier was identified: electronic health record and institution/government regulations. We identified that clinicians’ mental model of the condition being prompted was probably the major barrier, though this was often expressed as time pressure. We discuss solutions to each of the 5 identified barriers, such as addressing physicians’ mental models, by focusing on physicians’ expertise rather than knowledge to improve their comfort when caring for patients with the conditions being prompted.

Conclusions: To unleash the potential of electronic clinical alerts, electronic health record and health care institutions need to address some key barriers. We outline these barriers and propose solutions.

BACKGROUND

Electronic health records (EHRs) are improving America’s health care¹ by elevating quality, increasing patient safety, and facilitating finding information and accessing patient medical information for physicians.² Electronic clinical prompts, called best practice advisories (BPAs) in Epic,³ alert clinicians when patient-specific interventions are indicated, including preventive (eg, mammograms), chronic disease management (eg, diabetes), and counseling (eg, advance directives) actions. However, clinicians often do not utilize BPAs. Understanding why could increase utilization, thus improving patient outcomes.

The Early Audiology Referral in Primary Care (EAR-PC) project is developing a “model” BPA using hearing loss as a model due to low HL screening and referral rates by primary care physicians. Moreover, the US Preventive Services Task Force highlights that, although HL has significant impact, studies are needed to understand whether screening for it in community populations identifies it early.⁴ The EAR-PC project used the macrocognition framework^5,6 to guide the “cognitive engineering” of HL alerts into busy clinicians’ workflows. That was accomplished via cognitive task analysis (CTA),^6–8 a set of highly structured and complementary qualitative and quantitative methods with demonstrated effectiveness in eliciting the often invisible, deeply encoded, and highly automatized thought processes of expert decision-making in real-world environments.^8–10

Human thinking can be divided into 2 types, systems 1 and 2.^11,12 System 1 thinking is fast and intuitive, often occurring without much conscious thought, and is common in settings such as primary care¹³ that involve time pressure, limited information, and distracting cues. System 2 thinking is slower, effortful, and involves conscious deliberation. It is used only sparingly in time-pressured settings. Experts often use system 1 thinking and typically have a rich and deeply encoded rule set to depend on, whereas as non-experts more often use system 2 thinking to address situations. A primary care provider encountering a patient with a familiar or frequent clinical issue such as hypertension likely will use system 1 thinking to guide his or her evaluation and management decisions, whereas a much more deliberate system 2 thinking process is needed when working up a patient with new-onset delirium. The thinking type required can impact how a physician responds to clinical alerts (more likely with system 1). Moreover, the thinking type used can cause physicians to arrive at different results given the same inputs.¹²

Research question

How to develop a model BPA that integrates well into system 1 thinking, which family medicine clinicians would use to improve identification of individuals at risk for HL.

METHODS

Setting

The EAR-PC study ran from February 2015 through October 2015 in 2 southeastern Michigan practices 40 miles apart that used the Epic EHR. The University of Michigan Family Medicine (UFM) practice had 23 clinicians: 20 physicians and 3 advanced practice providers. The Beaumont Family Medicine (BFM) practice had 39 clinicians: 13 faculty physicians, 1 advanced practice provider, 4 preceptors, and 21 resident physicians. A model BPA was designed to identify and prompt audiology referrals for patients 55 and older who were at high risk for HL.^14,15 All patients 55 years of age and older who presented at the sites were invited to participate. Those who agreed completed a consent form and a Hearing Handicap Inventory,¹⁶ which served as the gold standard to identify patients likely to have HL. Clinicians saw a BPA with each eligible patient, alerting them to ask whether the patient had HL; they had no access to Hearing Handicap Inventory results.

Procedure

Twenty-three hour-long one-on-one CTA sessions were conducted throughout the study, 11 at the UFM practice and 12 at the BFM practice. These interviews studied how clinicians viewed, perceived, and used BPAs. They also explored views on HL to understand how clinicians perceived the condition and whether it was important and/or addressable in their health care paradigm; we also identified how often the major barriers were mentioned. The information obtained was used to iteratively improve the BPA design. The impact of the BPA on clinicians’ identification and referral of patients at risk for HL is outside the scope of this paper.

The interviewers were trained by a CTA expert,⁶ with mock sessions to hone their interviewing skills. Each clinician interview was conducted by a primary interviewer, with a secondary note-taker present to capture responses and make field notes. The interviewers kept the clinicians grounded in recent patient encounters and had them describe each step of their workflow in addressing patients’ concerns and any related BPAs. The interviewers probed in detail about how the clinicians handled the HL BPA to elicit their decision-making process (eg, ignored, dismissed, or completed the BPA).

The CTA interviews resulted in over 100 pages of field notes, which were first processed in a round of immersion crystallization.¹⁷ Themes were identified and codes developed by the team across multiple meetings, and codes for predetermined issues (eg, mental model, barriers, facilitators) were added. The notes were then coded for emergent and predetermined themes. Each statement was evaluated by at least 2 members of the research team, and coding discrepancies were reconciled by team conference. The final evaluation focused on how often various issues were identified, the root causes of use and non-use of the BPA, sample quotes highlighting major issues, and any potential solutions mentioned.

Analysis/results

Clinician demographics are in Table 1. The average age was 36 (range 28–67 years); 52% were female. The ages of clinicians at both sites were similar. The BFM site had more male clinicians (63% vs 33%) and residents (6 vs 0).

Table 1.

Clinician Demographics

Institution	Age	Gender	Faculty/Resident
Beaumont	30	M	Resident
	34	M	Faculty
	35	M	Faculty
	31	F	Resident
	57	M	Faculty
	28	F	Resident
	28	M	Resident
	41	M	Faculty
	26	M	Resident
	29	F	Resident
	36	F	Faculty
UM	41	M	Faculty
	33	F	Faculty
	36	M	Faculty
	30	F	Faculty
	34	F	Faculty
	39	M	Faculty
	41	F	Faculty
	30	F	Faculty
	67	F	Faculty
	29	F	Faculty
	?	F	Faculty
	?	M	Faculty

Our initial findings in a controlled setting suggest that our BPA caused a 6-fold increase in HL detection (from 1.2% to 7.1%), though the details are out of scope for this paper, which focuses on factors affecting clinician use of BPAs. In addition, our BPA is being reconfigured and tested in real-world settings that may have different outcomes. Five key CTA findings related to BPA use were identified, 4 directly from clinician comments and 1 indirectly from our experience iteratively improving the BPA (Table 2). All issues were similar at both sites and are listed in Table 2, along with pertinent quotes and specific examples of each. Following is a brief discussion of each along with potential solutions.

Table 2.

CTA Findings

CTA finding	Frequency of mentiona	Quote	Example
Direct findings
Poor standardization/ formatting of BPAs	12	“A high school student should be able to do this, but instead they are formatted such that physicians can’t.” “It’s hard to go back and change the situation if more information becomes available.” “I forget how BPAs that I rarely use work. They aren’t intuitive and require too many clicks.”	Documenting that a patient has allergies to eggs and isn’t a flu shot candidate requires clicking multiple unrelated fields. No easy way exists to document that an immunization recommended by a BPA was done elsewhere. Difficult to update health maintenance intervals triggering BPAs such as changing a q 3 year Pap smear requirement to q 5 years.
Time pressure with BPAs/electronic prompts	13	“Tend to not do BPAs unless it’s an HME appointment because otherwise there isn’t time.” “The BPAs at the bottom of the list are less likely to get completed because there isn’t time to do everything.”	Each alert must be signed individually, even related ones. Medical assistants often queue up BPAs inappropriately, causing clinician rework. Each BPA has a different format, requiring clinicians to stop and think how to use many of them.
Clinic workflow variations	16	1. “It’s more likely that a BPA will get addressed if a MA queues it up.”	Physician workflow variations unique to each physician, such as when they do BPAs. Clinical site workflow/patient variations due to site culture, such as how much medical assistants address BPAs before the clinician sees the patient. Institutional workflow variations based on institutional requirements.
Mental model of the condition being prompted	12	“I don’t have a script to address patients with hearing loss.” “It is a messy problem.” “I feel unprepared to answer patient’s questions about hearing loss and treatment options.” “There’s no good answer for what to do if they have a hearing loss.”	Diabetes is a prototype of a system 1 thinking prompt, as clinicians know instinctively what to do based on many lectures and have developed processes due to pay-for-performance encouragement. Hearing loss is a prototype of a system 2 thinking prompt, since most clinicians have no expertise in this area.
Indirect finding
Epic barriers and health system/ government regulations	11	“The colonoscopy BPA form with all the required fields ends up with too many clicks.” “I don’t have time to go to the problem list, type in the diagnosis and do all the clicks.”	Unable to modify color, font, placement of words. Unable to quickly add known diagnosis to problem list. ICD-10 requires listing details even if they do not impact outcomes. Duplication of effort by writing diagnosis in the BPA for specialist physician and then having to write it again for Epic.

^aFrequency of categories identified during our 23 CTAs. Some comments could be placed in more than 1 category, and we selected the 1 we felt most appropriate, eg, a comment that 1 part of the BPA had font so small that the clinician never saw it; although that could be in the “poor formatting” category, we placed in the “Epic barriers” category, since the Epic code did not allow font changes.

Poor standardization/ formatting of BPAs

BPAs differ in multiple ways: screen appearance, types of orders generated, number of clicks required, and options (eg, patient decline buttons). Currently over 40 BPAs exist at the UFM site, and only 1 at the BFM site. No BPAs underwent known usability testing at our institutions, resulting in numerous formats, often nonintuitive and requiring multiple selections (clicks). Clinician comments clarified that using BPAs leads to excessive cognitive demands (ie, requires system 2 thinking). Furthermore, Epic does not inform clinicians up front when BPAs were already addressed at past office visits (eg, mammogram order), resulting in the BPA being inadvertently readdressed. This led to frustration and workarounds to address BPAs during patient encounters.

Possible solutions

EHR companies should conduct usability testing by real-world practicing physicians to develop BPA formats that facilitate, or at least do not disrupt, the system 1 thinking that clinicians depend upon to get through a busy schedule. If this is already being done, companies should figure out why it is not working. Successful commercial Internet sites have been designed to maximize customer efficiency and satisfaction. Though health care differs from commercial sales, by using similar customer efficiency and satisfaction principles, one could design logical, consistent, easy-to-use prompts. Our highly evaluated model BPA made 4 outcomes available with a single click.

Time pressure with BPAs/electronic prompts

Clinicians felt that patient visits were already overloaded, limiting their ability to handle additional BPAs. For example, addressing all recommendations for complex patients requires more than the typical 15-minute office visit.¹⁸ It was felt that BPAs intrude on the doctor-patient relationship, since they rarely address the primary reason for the visit, and the added workload contributes to clinician stress due to falling further behind in the schedule. Having medical assistants address BPAs did increase HL referrals but was perceived as only partially helpful, since some medical assistants’ actions were inappropriate, thus requiring further action to delete the ordered BPA.

Possible solutions

Making BPAs user-friendly will free some time. Allowing clinicians to approve multiple immunizations or laboratory tests by inputting one signature rather than signing individually for alerts that don’t require physician-level decision-making (eg, overdue diabetic eye exams) would help too. Better training of medical assistants to appropriately queue up interventions or address them during “pre-visits” with patients for alerts that do not require physician expertise would help. If BPAs are designed such that a high school student can use them, the above interventions would allow office staff to address many alerts (eg, overdue A1C tests) that are later signed en masse by clinicians, freeing physicians to focus on areas that require their expertise.

Clinic workflow variations

Our CTAs revealed diverse BPA and clinic workflows. Each clinician had a unique approach, ranging from reviewing charts the night before and taking notes, to having scribes do all BPAs, to ignoring all BPAs. Younger participants seemed more receptive to BPAs. Sites had some standardized workflows, with support personnel performing low-level tasks, reducing the system 2 thinking required of physicians. However, even where staff did BPA work, they too had their own workflow variations, further complicating the process. Finally, patient populations varied significantly, impacting workflow. For example, some patients’ distrust of immunizations required more physician involvement.

Possible solutions

BPAs should be designed to support standardization and system 1 thinking. For example, our model BPA had 4 responses that clinicians could click once to complete the action for the following: (1) patient already has HL, which ideally puts HL on the problem list and terminates the BPA forever; (2) patient declines testing, which closes the BPA for 1 year; (3) problem not addressed, thus the BPA appears at the next visit; and (4) referral to audiology, which automatically completes a referral. Our clinicians found this easy to use. Moreover, system-wide efforts to standardize how medical assistants address BPAs improved patient throughput as well as the percentage of BPAs addressed (personal communication, Philip Zazove).

Hearing loss is not easily addressable

Clinicians often felt that they could not address HL, which lessened their perceived need to screen for it. Many commented that there are “more important” diseases to focus on; diabetes was an example mentioned multiple times. When asked why, clinicians mentioned the amount of training and exposure to diabetes, ongoing pay-for-performance for diabetes quality metrics, and perceptions of significant potential complications. Thus they were more likely to address diabetes BPAs. Conversely, they had little HL training, did not know how to advise patients with it, and were unaware of its life-altering sequelae. The fact that HL affects many more people than diabetes exemplifies the complexity here. Family physician experts address diabetes using system 1 thinking and HL (a proxy for important but underdiagnosed conditions) using system 2 thinking, since they have a rich mental model of diabetes and a poor mental model of HL.¹⁹ It should be noted that clinicians did not specifically state that they had a poor mental model of HL, but rather attributed their lack of response to the BPA to other causes, such as time pressure. Our CTA evaluations, by digging deeper into the reasons the clinicians avoided using the BPA, discerned that it was really their mental model of HL, and that the “time pressure” was due to clinicians being uncomfortable and having to take time to think about how to address HL (ie, being forced into slow and effortful system 2 thinking), which they did not have to do with other conditions.

Moreover, clinicians’ mental model of a condition impacts how they respond to prompts. In the time-pressured, stimulus-saturated clinic environment, they avoid BPAs that push them into system 2 thinking in general. HL was described as a quality-of-life issue, not one with serious sequelae. Our CTA probing suggested that this view is similar for other poorly diagnosed conditions, such as falls in elderly persons. Physicians do not classify these in the set of things about which they do, or even should, have expertise. Many labeled these conditions as “messy” problems. For example, HL carries a personal stigma; patients often will not admit it and dislike what doctors offer for it (and hearing aids are costly), and physicians have limited knowledge of urgent medical conditions associated with it (vestibular schwannomas, autoimmune HL, etc.). Our findings suggest that other conditions fall in the range between diabetes and HL in the amount of system 1 and 2 thinking required.

Possible solutions

To improve BPA use for “messy” conditions, rather than improving physicians’ knowledge, we should improve their expertise. For instance, providing detailed education programs (system 2 thinking) about HL could worsen clinicians’ feeling of being overwhelmed. Interventions should focus on a set of rules that can be employed efficiently – a “satisficing strategy”^20,21 triggered by prompts and executed with system 1 thinking. If they are simple enough, solutions could even be incorporated into prompts themselves. Finally, medical schools should dedicate time to teach how to maximize patient care via EHRs, including how to utilize BPAs.

The indirect key finding from our CTA interviews was:

Epic barriers and health system/government regulations

We identified 3 types of “rules” that reduced BPA use.

1. Rigidity of Epic’s structure. We tried to change our BPA font, color, and word placement so clinicians would immediately realize when it was previously addressed and save time. It took months before we could implement just a color change; the other types of changes are not possible in Epic at the present time.

2. Institutional requirements. Though intended to promote patient safety, these require system 2 thinking and can prohibit reasonable requests such as adding HL to the problem list. One institution’s IT leaders require physicians to make multiple clicks and then type the diagnosis. We found that clinicians often declined to do so. This creates a situation where BPAs inappropriately reappear at future patient visits (their algorithms use problem lists), further impacting physician time.

3. External requirements. For example, International Classification of Diseases-10 (ICD-10) requires clinicians to choose whether HL is bilateral, right-sided, or left-sided, when the goal is just having HL on the problem list so clinicians are aware that it exists.

Possible solutions

Some of these rules, inherent in the institutions or legal system, cannot be fixed. EHR companies could provide flexibility to allow word placement, color, and font, which would maximize efficiency and improve patient outcomes. In addition, IT leaders, while maintaining patient care safety, should maximize system 1 thinking for clinicians, especially when the risk is low.

DISCUSSION

Our CTA interviews demonstrated significant barriers to the successful use of a new prompt in an EHR. These findings were used to tailor our pilot BPA to incorporate enhancements addressing some of the issues. First, we configured the HL BPA to only require a single click to generate a referral to audiology, an improvement from the original design of 5 or more clicks plus typing in referral details. Second, once an audiology referral occurs, the BPA now changes from yellow to gray-green, denoting that the action is pending, thus alerting clinicians and reducing multiple reordering of audiology referrals (with other BPAs, the BPA does not change at all until the patient has the intervention done). Third, the simplicity of our BPA could permit training of medical assistants to queue up audiology referrals when appropriate, theoretically freeing clinicians to focus on areas that require their expertise. The potential for this is being evaluated with larger numbers of clinicians. Our clinicians perceived our improvements favorably. However, our improvements did not fully address the perceived time pressure at visits, where addressing all recommended interventions and screenings would take much longer than the standard 15–20 min.¹⁸ Furthermore, patients have multiple complaints per visit, increasing the need for longer visits. Nevertheless, when BPAs were configured such that they could be used by a high school student, as our preliminary version was felt to be, we found that they were more often addressed at visits. This is now being tested in large real-world settings.

We believe BPAs can help address multiple underrecognized conditions that often have significant sequelae and morbidity. For example, HL affects up to 20% of Americans,^14,15 making it America’s second most common disability. Despite its high prevalence and adverse outcomes, including lower income, isolation, poorer mental health, depression, and lower cognitive function, little screening or intervention is done at a primary care level.^19,22,23 Our findings clarify that this is due to clinician discomfort with underrecognized conditions plus the system 2 thinking required to address them. Some clinicians called these “messy” conditions, highlighting their lack of comfort with them.

We do acknowledge that the enormous proliferation of clinical guidelines can increase the time pressure on busy clinicians. As defined by the Institute of Medicine, clinical guidelines are “systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances.”²⁴ They make recommendations such as which diagnostic or screening tests to order and how often to do so. We acknowledge that such guidelines are often seen as impacting clinician autonomy, too inflexible, developed by specialists who do not understand the variability of primary care, and sometimes contradictory.²⁵

However, our insights suggest that part of the reason clinicians do not address BPAs, at least for under-the-radar conditions, is their mental models of those conditions. Thus, rather than reduce the number of guidelines or improve physicians’ knowledge, we should improve their expertise; that should help them be more efficient with such conditions. Medical schools and primary care residencies should consider how to better educate students and residents about important and common under-the-radar medical conditions. However, although that would help, it would not be sufficient to promote system 1 thinking for these conditions. Indeed, we found that at 1 family medicine site with over 40 existing BPAs, more than two-thirds of the time clinicians did not address BPAs. Thus, we propose that teaching clinicians a simple set of “rules” they can use efficiently and are triggered by the BPA – what others have called a “satisficing strategy”^20,21 – would work much better, as it supports system 1 thinking. We did that with 4 1-click options in our BPA, which resulted in a 6-fold increase in HL referrals in our pilot study. Due to institutional limitations, such as the inability to easily add HL to the problem list, we were prevented from making the process totally efficient, which could have generated a greater increase in appropriate referrals. We are currently testing a 10-min video designed to increase clinician HL expertise, and testing our BPA in real-world settings with large numbers of clinicians. Our anticipation is that our BPA will encourage physicians to use system 1 thinking when dealing with HL.

There are limitations to our study. It was conducted in family medicine practices, which may have different receptivity to BPAs and HL than other primary care practices. Our clinicians and patients were predominantly Caucasian/non-Hispanic, and our findings may differ for other ethnic and racial groups. Our clinicians used Epic, and our findings may not be applicable to other EHRs.

SUMMARY

Busy physicians perform real-world decisions using system 1 thinking via a knowledge base with an economy of cognitive effort almost automatically. In cognitive science terms, busy clinicians approach prompts based on how rich their mental models are for specific problems, and this involves a unique mix of problem detection, planning, revision, uncertainty management, and team coordination. Thus, highly monitored conditions (eg, diabetes) are addressed differently from infrequently monitored conditions (eg, HL).

Electronic at-the-visit prompts as currently designed often do not promote system 1 thinking. They also implicitly depend upon clinicians having a detailed mental model that they can activate. Thus, the impact of prompts has been less than anticipated. We identified key barriers to use of prompts and propose solutions to address these. Doing so should improve quality and health outcomes, moving us closer to the Quadruple Aim, which includes physician satisfaction as the fourth aim.²⁶

CONFLICT OF INTEREST

The authors of this manuscript have no conflicts of interest to declare.

FUNDING

This work was supported in part by grant 1R21DC013678‐01 from the National Institutes of Health National Institute on Deafness and Other Communication Disorders.

ROLE OF THE FUNDER/SPONSOR

The funding source had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, and approval of the manuscript;, or decision to submit the manuscript for publication.