Physicians’ use of computerized clinical decision supports to improve medication management in the elderly – the Seniors Medication Alert and Review Technology intervention

Abstract

Background

Elderly people (aged 65 years or more) are at increased risk of polypharmacy (five or more medications), inappropriate medication use, and associated increased health care costs. The use of clinical decision support (CDS) within an electronic medical record (EMR) could improve medication safety.

Methods

Participatory action research methods were applied to preproduction design and development and postproduction optimization of an EMR-embedded CDS implementation of the Beers’ Criteria for medication management and the Cockcroft–Gault formula for estimating glomerular filtration rates (GFR). The “Seniors Medication Alert and Review Technologies” (SMART) intervention was used in primary care and geriatrics specialty clinics. Passive (chart messages) and active (order-entry alerts) prompts exposed potentially inappropriate medications, decreased GFR, and the possible need for medication adjustments. Physician reactions were assessed using surveys, EMR simulations, focus groups, and semi-structured interviews. EMR audit data were used to identify eligible patient encounters, the frequency of CDS events, how alerts were managed, and when evidence links were followed.

Results

Analysis of subjective data revealed that most clinicians agreed that CDS appeared at appropriate times during patient care. Although managing alerts incurred a modest time burden, most also agreed that workflow was not disrupted. Prevalent concerns related to clinician accountability and potential liability. Approximately 36% of eligible encounters triggered at least one SMART alert, with GFR alert, and most frequent medication warnings were with hypnotics and anticholinergics. Approximately 25% of alerts were overridden and ~15% elicited an evidence check.

Conclusion

While most SMART alerts validated clinician choices, they were received as valuable reminders for evidence-informed care and education. Data from this study may aid other attempts to implement Beers’ Criteria in ambulatory care EMRs.

Introduction

Use of five or more medications simultaneously (polypharmacy) is common among adults aged 65 years or more.¹ Polypharmacy increases the risk of adverse drug events (ADEs), potentially inappropriate medications (PIMs), and hospitalizations.^2–6 In the USA alone, health care expenditure related to PIMs was $7.2 billion in 2001.⁷ The first step to reduce polypharmacy is to deprescribe potentially inappropriate and minimally effective medications.⁸ A widely accepted screen for such opportunities is Beers’ Criteria.⁹ The 2012 update was derived from an expert review of best evidence and consists of 53 medications or drug classes that should be avoided or used with caution to prevent ADEs in the elderly. Busy physicians may endorse evidence-based prescribing but make errors of omission when trying to remember guidance about diverse medications, complex drug–drug interactions, and variable drug–disease considerations. Calculations, like the Cockcroft–Gault (CG) formula¹⁰ for quantifying age-related decline in renal function, may not be systematically employed.^11,12

Embedding Beers’ Criteria and CG glomerular filtration rate (GFR) estimates into electronic medical records (EMR) could improve prescribing practices.^13–15 eCLINICIAN is an enterprise EMR (based on Epic Systems’ Epicare) used in the Edmonton, Alberta, area to support outpatient medical care. Medication lists, allergies, and health problems are captured and managed as part of clinical documentation, order entry, and health care planning. Clinical decision support (CDS) can be triggered by unique combinations of provider characteristics, patient demographics, test results, clinical problems, and current and requested medications. Actions can include messages in one or more parts of the chart (passive guidance), or alerts during medication documentation or ordering workflows (active guidance).

We report the use of qualitative and quantitative research methods to support the systematic design, development, implementation, and review of medication management. For seniors attending geriatrics and family practice clinics, an intervention program where CDS was embedded within an ambulatory care EMR to support the application of 2012 modified Beers’ Criteria and CG estimates of GFR was introduced and was named as SMART (Seniors Medication Alert and Review Technologies) intervention. In this study, SMART intervention is assessed for acceptability to frontline prescribers.

Methods

Study aims

The primary objective of this study was to build clinician-acceptable medication management alerts and deploy these in an ambulatory care EMR. The secondary objective was to discover how to deploy SMART tools in a way that minimizes workflow disruption while maximizing clinician attention.

Study design

The study progressed through a preproduction design and development phase followed by a postproduction adaptation and optimization phase. In the build stage, potential SMART interventions were presented in an EMR “sandbox”. This could be used by clinicians to evaluate how the CDS performed using virtual patients with typical medication lists. Feedback was used iteratively during development, with at least three repeat sandbox exposures to enhanced SMART CDS. In the social adaptation and optimization stage, SMART was deployed in busy ambulatory care clinics providing primary and specialty care to older adults. Participatory Action Research (PAR) methods were used to enhance end-user engagement in preproduction and minimize impact on clinic activities with postproduction. PAR is a qualitative research methodology in which individual research participant or subject’s views were revealed and subjects were active in making informed decisions in the research process. From the subjects, data were collected systematically and analyzed for the purpose of taking action and making change in the CDS.¹⁶

Participants

Physicians reporting primary responsibility for older adults (aged 65 years or more) in two clinics were introduced to the study and invited to participate. A referral geriatric clinic included specialists in geriatric medicine (n=5) and family physicians (n=3). A separate primary care clinic (n=7) included family physicians with significant seniors’ caseloads.

Measures and outcomes

A research assistant facilitated online surveys, EMR sandbox focus groups, and postproduction interviews with study participants to capture subjective data about the perceived value, usability, and impact of the SMART CDS. The interview questions were developed based on expert opinion and the interviews were conducted at 3 and 6 weeks. The SMART intervention was iteratively revised to address concerns and heed optimization advice. Post-production objective data were collected from EMR logs. All physician interactions with a patient chart in a single day constituted a single patient encounter. Triggered CDS events were defined as any SMART message (passive or active) appearing at any time during an encounter. Managed events occurred when the clinician accepted guidance or provided a reason for rejecting guidance. Evidence events occurred when the clinician selected a grade of recommendation, evidence, or education link for review. One or more CDS, management, or evidence events may occur during any one encounter.

Data collection and analysis

All participants provided information about their professional training, years in practice, clinic volumes and composition, site of practice, and experience using the EMR. CDS-related perceptions were obtained from online surveys, EMR sandbox focus groups, and postproduction in-person or telephone interviews. Narrative responses, focus group transcripts, and interview records were analyzed for frequency and intensity of common themes, including SMART usability, opportunities for improvement, perceived clinical value, educational impact, and effects on workflows and clinician–patient interactions. At the end of the adaptation and optimization phase, the SMART CDS was used in clinical practice. The first 6 months of EMR audit logs were reviewed for eligible encounters and the frequency of CDS, management, or evidence events. Basic descriptive statistics were used to analyze CDS usage data.

Ethics review and approval was obtained from the Health Research Ethics Board at the University of Alberta. Privacy, confidentiality, and security review were obtained from the Information Stewardship Office of Alberta Health Services.

Results

Eight practitioners consented to participate and attended a project-orientation session before contributing to asynchronous (online survey) and synchronous (EMR sandbox focus groups, postproduction interviews) assessments of evolving SMART interventions. Most participants were female, and there was an even mix of primary care and specialist physicians (Table 1). All had at least 1 year of EMR experience.

Table 1.

Demographic data of practitioners (n=8)

Sex, n (%)	Male: 2 (25%)	Female: 6 (75%)
Study site, n (%)	Multidisciplinary seniors’ clinic: 6 (75%)	Family medicine clinic: 2 (25%)
Health care provider, n (%)	Family physicians: 6 (75%)	Geriatric medicine specialists: 2 (25%)
Years of experience with the electronic medical record, n (%)	1–3 years: 6 (75%)	>3 years: 2 (25%)
Years in practice	Mean: 22 years; range: 14–30 years
Estimated proportion of practice patients over 65 years	Mean: 87%; range: 40%–100%
Number of half-day clinics per week	Mean: 3.5; range: 1–9.5

Four EMR-sandbox focus groups were formed and 16 postproduction semi-structured interviews were conducted. The EMR simulations led to a standard SMART message format with guidance synopses, indicators of strength of recommendation and quality of evidence, and links to evidence summaries and educational resources (Figure 1). CDS management was optimized with quick select of reasons for accepting or rejecting guidance (eg, clinical indication, patient priority, safety established) and for suppressing the CDS for a defined period.

Figure 1.

Snapshot of decision support SMART tool alert.

Notes: Title/type: brief descriptor including the alert type (eg, “warning”) and medication name; Alert summary: short summary of relevant clinical guidance; Strength of recommendation and quality of evidence: describes the relative strength of the recommendation and the quality of supporting evidence using GRADE criteria; Guidance: the category of the alert; Key data: specific patient laboratory data relating to the alert; Manage: quick-select buttons cover the most common reasons that prescribers may give for acknowledging and/or overriding the alert and are supplemented by a more detailed optional pick-list and a tool for optional comments; Actions: follow-up action’s rapid access to EMR areas where management decisions can be implemented. Copyright© 2015 Epic Systems Corporation. Used with permission.

Abbreviations: EMR, electronic medical record; GRADE, Grading of Recommendations Assessment, Development and Evaluation; SMART, Seniors Medication Alert and Review Technologies.

Eight clinicians were interviewed twice at 3 and 6 weeks postproduction (Table 2). Practitioners demonstrated how they used SMART CDS during in-person interviews. Pre-interview exposure to SMART ranged from 2 to 16 days, which was influenced by the frequency of clinics.

Table 2.

Interview questions and response rates

1.	Before today, were you aware of the SMART CDS in the eCLINICIAN EMR?	Yes: 100%
		No: 0%
2.	In your practice, have you seen the SMART CDS when reviewing patient medications?	Yes: 87%
		No: 0%
		No response: 13%
3.	Did the SMART CDS appear where it should not have? Open comments	Yes: 50%
		No: 37%
		No response: 13%
4.	Did the SMART CDS fail to appear at times when it would have been helpful?	Possibly: 87%
	Open comments	No response: 13%
5.	Considering your workflow, were the SMART CDS available at the appropriate time for (a) patient assessment and (b) medication management?	(a) Patient assessment
		Yes: 87%
		No: 0%
		No response: 13%
		(b) Medication management
		Yes: 87%
		No: 0%
		No response: 13%
6.	Since the SMART CDS began appearing, does seeing new patients take more time, less time, or is it unchanged?	Unchanged: 87%
		No response: 13%
7.	Since the SMART CDS began appearing, does it take more or less time to see follow-up patients?	More time: 74%
		Less time: 13%
		Unchanged: 13%
8.	Did the SMART CDS cause disruptions to your practice workflow?	Yes: 50%
		No: 37%
		No response:13%
9.	Was workflow adjusted to incorporate the use of the SMART CDS? In what ways?	No: 87%
		No response: 13%
10.	During patient interviews, how did the SMART CDS inform shared decision-making?	Yes: 37%
		No: 50%
		No response: 13%
11.	Would you continue to use the SMART CDS? Why or why not?	Yes: 87%
		No: 0%
		No response: 13%
12.	Thinking of your practice, did you notice benefits to using the SMART CDS?	Yes: 63%
		No: 24%
		No response: 13%
13.	Thinking of your practice, did you notice risks to using the SMART CDS?	Yes: 37%
		No: 50%
		No response: 13%
14.	Overall, do you think that the SMART CDS improves the quality of patient care?	No response: 13%
	Why or why not?	Advisory was useful and helpful, but too soon to assess impact without a control group: 87%
15.	What could be done to improve the SMART CDS?	Open comments

Abbreviations: CDS, clinical decision support; SMART, Seniors Medication Alert and Review Technologies; EMR, electronic medical record.

The following CDS perception themes were identified (exemplar quotations are presented within quotes):

• SMART alerts presented at the point of care supported the application of knowledge to practice.

  Clinicians generally reported familiarity with the intent Beers’ Criteria, but varying degrees of proficiency with their application during outpatient care. They commonly cited cases where a SMART alert appeared for a drug they did not recognize as relevant to Beers’ Criteria before being reminded of potential concerns. Even when there were no alerts, or no surprises, participants commented that the existence of the SMART system improved drug management by stimulating physicians to

  […] think about the drugs before prescribing and take some time to review drugs that patients are already on. This extra step adds to patient safety. Specialist

• The additive clinical value of CG GFR estimates varied but encouraged drug–renal awareness during prescribing.

  Positive CDS perceptions included the value of reminders to reduce or discontinue drugs in association with CG GFR alerts. Some participants specifically noted that the CG calculation provided a more accurate GFR estimate than the MDRD (Modification of Diet in Renal Disease) formula. Others reported annoyance when a calculation was not performed for lack of current height, weight, or laboratory test information. Clinicians felt that either old data should be used or the prompt should be suppressed. Other participants noted that renal adjustment can be difficult and more specific guidance should be provided about exactly how to adjust or select a safer medication:

  Some drugs are metabolized through the liver, some through the kidneys. It would be useful to identify in the medication CDS which drugs and dosages are influenced as a function of the GFR value. Family Physician

• SMART CDS opened teaching opportunities for resident physicians, patients, and colleagues with limited experience to manage medications for aging adults.

  Participants frequently mentioned using SMART messages as prompts for discussing patient safety when interacting with trainees, colleagues, or referring physicians. Trainees could follow evidence links with modified Beers’ Criteria to review the advantages and disadvantages associated with continuing or discontinuing a particular medication. Some respondents mentioned referencing the alerts during patient education, discouraging specific problem medications (eg, hypnotics), emphasizing the importance of medication compliance, or simply printing linked information for patients to take home. Respondents frequently suggested that the SMART CDS should have the greatest clinical impact for physicians less acquainted with complex care of older adults:

  It has the potential role [as an educational tool] depending on the comfort level of the physician and their knowledge of drug interactions, risks and side effects. I think that it is a supplemental tool for doctors and I think that their use of that tool is dependent on their level of expertise. Family Physician

• SMART CDS messages served as a source of medical evidence.

  Prior to the deployment of the SMART intervention, some physicians reported going online for Beers’ Criteria refreshers. The same physicians reported preference for the embedded Beers’ Criteria postdeployment. They trusted the guidance because they could quickly discern that it had been reviewed and updated by a reputable organization.

• Concerns related to possible future liability for CDS-related actions.

  Many respondents expressed concern about how CDS overrides might be documented or communicated. Physicians described a “balancing act” weighing benefits and risks and noted that a patient may be compromised by drug discontinuation. They lamented lack of specific details about exactly how best to wean or discontinue a drug, monitor for unwanted effects, or choose an acceptable alternative in the face of clinical need. Indeed, whereas physicians expressed concern about overly specific messages at the start of the study, they most commonly requested enhanced guidance toward the end of the study. Considerable concern arose when physicians opted to continue prescribing a medication despite cautionary CDS alerts. Uncertainty about whether CDS-suppression logs could be referenced in any future legal proceedings was troubling some respondents.

• SMART CDS messages and alerts raised questions about clinician accountability.

  Respondents reported concern about discontinuing medications ordered by other physicians. Some clinicians reported confidence treating aging adults and did not hesitate to take action based on the SMART CDS, but at times, a “crossover of physician responsibilities” was cited. Mention was made of debates about the merits of the alerts. Some physicians felt that the management of specific medications should be deferred to the clinician overseeing the related health problem. Although uncertainty was often diplomatically resolved, respondents observed that the management of CDS-related disagreement was inconsistent.

• Iteratively honed CDS proved to be most acceptable but possibly not visible to all.

  Physician workflows, although initially disrupted by early study interventions, were less disrupted by iteratively optimized SMART CDS. Interviewed clinicians described the messages were becoming more user friendly, simple to read, and easy to navigate. Others mentioned being familiar with most of the message content, already committed to medication management, and requiring minimal adjustments to their practices. This group of clinicians reported that it took a “few minutes” when adjusting medication orders to decide the course of action and provide justification when managing alerts. Concern was expressed about physicians who may avoid the EMR during patient interactions, instead dictating and attaching consultation letters to the EMR post-encounter. Their EMR workflows would deny opportunity for CDS at the point of decision-making. Some respondents recognized that suboptimal use of the EMR, particularly lack of medication reconciliation, could lead to false-negative CDS:

  The danger is because we don’t always update our med review and then the [CDS] doesn’t pop up when it should sometimes the one-to-one meds review with the patient or caregiver does not take place. Family Physician

The postproduction implementation of SMART CDS for 25 weeks resulted in 5,442 eligible encounters and a total of 4,067 CDS events. The overall CDS event per encounter burden is plotted in Figure 2, with a trend toward a decreasing burden. Approximately 36% of encounters with the seniors generated at least one SMART alert, with hypnotic and anticholinergic medication warnings presenting most frequently. Approximately 25% of alerts were acknowledged without medication changes (eg, clinically indicated or safety established), and 15% of alerts elicited an action where evidence and/or education resources were checked.

Figure 2.

Alerting rates in the postproduction study period.

Notes: (A) Proportion of eligible clinic visits where one or more alerts was presented of the postproduction study period. (B) Weekly count of eligible encounters during the postproduction study period. Weeks 5 and 6 overlapped a seasonal holiday.

Discussion

The principal observation of this study was that collaboratively developed, evidence-based, and clinically optimized medication CDS was well received by clinicians caring for older adults. Preproduction design, development, and usability testing with virtual patients in an EMR sandbox allowed most clinician concerns to be addressed and workflows to be optimized. Postproduction feedback and fine-tuning further facilitated the promotion of Beers’ Criteria in busy clinical practices.

Use of the SMART CDS intervention exposed medication safety issues that evidence-aware and CDS-accepting study participants easily confirmed, as well as some that would have been missed. The participants reported general satisfaction with and endorsement of a system they helped to shape. Living with that system during busy postproduction exposures revealed additional benefits and harms. Even an optimized decision support system can frustrate clinicians, hamper workflow, or simply add to clinicians’ total informational burden. While typical EMRs have limited ability to measure CDS-triggered change in health processes or outcomes, the ability to measure more subtle social impacts is even more limited.

A recent review of strategies for shaping clinician prescribing behaviors found automated alerts and reminders most effective.¹⁷ Other studies claim computerized CDS interventions improve clinical processes, enhance prescribing quality, and reduce PIMs.^18–20 Even if the cited CDS interventions are transferable, usability challenges could limit impact on health processes and/or outcomes. This study illustrates an approach to improve usability through early physician engagement in iterative design and development. If the resulting clinician acceptance proves durable, then lasting effects on prescribing practices should be associated, in time, with positive impacts on polypharmacy and ADEs.

Complementary strategies for improving physician adoption of CDS relate to education and training. Approximately 75% of our study participants had 1–3 years of experience using EMRs. However, physicians demonstrated varying degrees of EMR use in their daily workflow when managing rapid patient turnover. Previous studies have indicated the importance of continuing training of physicians to achieve meaningful use of digital health records, developing IT skills, and the use of chief residents and peer clinicians in mentoring and training others.^21–23 Although medical residents were not included as participants in this study, clinicians consistently reported benefit of the SMART CDS alerts as teaching tools for residents and referring physicians, yielding added value from time spent on CDS.

The SMART messages and alerts depend upon physician attention where and when the CDS appears in the EMR. Unanticipated differences in physician EMR workflows emerged during the postproduction phase of our study, as highlighted by study participants reflecting on how a broader range of colleagues might benefit from the system. Some colleagues do not routinely populate the chart with medication, vital measurements, or health problems needed by the SMART rules before the point of decision-making. The opportunity for timely alerts may be lost. Even when physicians review medications, document problems, and prescribe in the expected order, multidisciplinary validation can be important in a population where cognitive decline is common, and those inputs may not occur when they would most impact CDS validity. Use of elder-sensitive GFR estimation (CG formulation) is laudable, but if the requisite patient measurements (within the past 12 months) are unavailable or undocumented, then false-negative CDS alerts can undermine system validity. As reported by others, up-to-date, multisource, medication reconciliation is important for any medication-related decision report.^24,25

The use of CDS interventions can have unintended consequences, including negative physician reactions to perceived inappropriate or overly directive guidance.^26,27 In this study, unintended and unanticipated physician frustration focused on repeated prompts for height, weight, and creatinine data needed for CG GFR calculations. The risk of spillover negative reactions to the SMART CDS as a whole led the study team to suppress all GFR alerts until the associated rules could be configured to handle missing data more appropriately.

Other unanticipated concerns related to questions of liability and professionalism. That these issues did not emerge during EMR sandbox simulations may be related to lack of clinical authenticity of virtual patients or to our failure to incorporate shared care in the clinical simulations. Similarly, preproduction surveys and focus groups gave the impression that participating physicians preferred simple SMART warnings, leaving to their clinical judgment how best to adjust medications, whereas postproduction surveys revealed emerging desire for specific and directive guidance about how best to discontinue medications, GFR-adjust dosages, or prescribe safer medications. It is possible that growing familiarity, and comfort, with the SMART CDS increased physician willingness to rely on decision aids when navigating complex medication challenges. Others have reported unexpected shifts in physician expectations associated with unintended consequences of CDS interventions.^28,29

In the present study, we report a slight decrease in the rate of alerts presented to family practice physicians and geriatricians during the 6-month postproduction phase of the study. The respondents acknowledged increased awareness of the hazards associated with the prescribing of inappropriate medications, including during teaching sessions with medical students and residents. The quantitative results may be consistent with physicians avoiding the use of these medications in favor of less harmful alternatives. We cannot rule out the possibility of changes in the underlying patient population or changes in data quality as prescribers became more familiar with the SMART system in general. A much longer period of observation is needed to detect polypharmacy and PIM changes as patients return for follow-up visits.

Strengths of this study include end-user engagement in the iterative design, development, implementation, and assessment of a significant CDS intervention. At the same time, the use of PAR biased participants, and their colleagues, to be well disposed toward the intervention, to understand and anticipate its goals, and to show increased tolerance to its ultimate appearance in their clinics. This dilemma may be unavoidable when complex informational interventions are to be introduced, with end-user adoption best practices, in busy clinical settings. Accordingly, the best candidate for a generalizable finding may be that early clinician engagement is critical to the successful deployment of medication-related computerized decision supports. In settings like ours, the use of evidence-based guidance, educational use of the CDS, and focus on recognized challenges (eg, polypharmacy in the elderly) proved important for attracting clinician interest. While ongoing check-ins with clinicians continued to generate valuable insights, the ability to track EMR-generated data about CDS burden, the relative frequency of different types of guidance, and the “market” for evidence and education links demonstrated the importance of objective postproduction CDS surveillance. This too may be generalizable. Qualitative and quantitative data are necessary, but not independently sufficient, to track the acceptance and impact of CDS interventions.

Limits to the generalizability of our findings include lack of allied health professional involvement in the study population, a limited range of clinical settings and participants, and a short duration of postproduction observations. The possibility of a honeymoon effect is tangible. Future research may include follow-up studies on discontinuation of PIMs and reduced adverse events.

Conclusion

An inclusively developed medication CDS intervention, designed to reduce polypharmacy and inappropriate medication use among the elderly, proved acceptable to primary care and specialist clinicians without significant negative impacts on workflow. The alerts were utilized as a teaching tool for patients, residents, and clinicians and served as an evidence-based support for informed clinical decision-making. Physicians reported that the SMART CDS usually reinforced, and sometimes changed, their clinical judgment. Observations from this study may help others render Beers’ Criteria to actionable and acceptable decision supports embedded within EMRs.