EHR-based Visualization Tool: Adoption Rates, Satisfaction, and Patient Outcomes

Abstract

Background:

Electronic health records (EHRs) have the potential to enhance patient-provider communication and improve patient outcomes. However, in order to impact patient care, clinical decision support (CDS) and communication tools targeting such needs must be integrated into clinical workflow and be flexible with regard to the changing health care landscape.

Design:

The Stroke Prevention in Healthcare Delivery Environments (SPHERE) team developed and implemented the SPHERE tool, an EHR-based CDS visualization, to enhance patient-provider communication around cardiovascular health (CVH) within an outpatient primary care setting of a large academic medical center.

Implementation:

We describe our successful CDS alert implementation strategy and report adoption rates. We also present results of a provider satisfaction survey showing that the SPHERE tool delivers appropriate content in a timely manner. Patient outcomes following implementation of the tool indicate one-year improvements in some CVH metrics, such as body mass index and diabetes.

Discussion:

Clinical decision-making and practices change rapidly and in parallel to simultaneous changes in the health care landscape and EHR usage. Based on these observations and our preliminary results, we have found that an integrated, extensible, and workflow-aware CDS tool is critical to enhancing patient-provider communications and influencing patient outcomes.

Background

Electronic health records (EHRs) hold great promise relative to the ability to improve patient care, especially in the context of modifiable and measurable factors, such as those associated with cardiovascular risk.1 Clinical decision support (CDS) systems and their alerts – which are usually embedded within the EHR – may change provider behavior when implemented in a minimally disruptive manner.2,3 CDS alerts and associated tools have the potential to increase provider adherence to clinical guidelines.2 Evidence exists that such tools prompt the following: counseling and referrals for patient smoking-cessation,4 facilitating goal setting among prediabetics,5 lowering low-density lipoprotein cholesterol,6 and prescribing medications appropriately.7

Specific use cases for the potential for CDS alerts and EHR tools to change provider behavior, as well as to improve patient outcomes, are evidenced in the literature for cardiothoracic surgery,8 HIV,9 and cardiovascular risk management.6 Existing EHR-based tools target individual risk factors (e.g., smoking,4 low-density lipoprotein6) or specific diseases (e.g., diabetes,5 HIV9), but none have integrated multiple, complementary health behaviors and factors for discussion and management at the point of care. For multifactorial chronic conditions such as cardiovascular disease, addressing only one causal factor may be insufficient to improve overall cardiovascular health (CVH). Comprehensive and accessible, point-of-care CVH data allow providers to assess the cardiovascular comorbidity burden of patients in their care and to discuss treatment and referral options for cardiovascular disease risk management.

A barrier to efficiently accessing comprehensive and relevant data at the point of care is the diffuse presentation of CVH data in the EHR.10–12 Vital signs, laboratory results, prescription medications, and health behaviors are captured in multiple discrete fields on distinct screens in the EHR, and are frequently displayed within different activities. We designed the Stroke Prevention in Healthcare Delivery Environments (SPHERE) tool to aggregate and display data relevant to CVH together,13 making it easily interpretable for patients and providers alike.14 The provider- and patient-facing tool was developed and implemented to improve patient-provider communication around CVH at the point of care. We hypothesized that the use of the SPHERE tool would improve patients’ CVH via prevention discussions and subsequent behavior change and risk factor treatment.

The SPHERE tool was designed with a bidirectional research-practice framework in mind.15 Specifically, the investigators built upon the evidence base of CVH as established by the American Heart Association16 to design the SPHERE tool for application in clinical practice. We report on our initial evaluation of the SPHERE tool to inform an evidence base for future tool refinements. Capitalizing on a bidirectional research-practice framework,15 we have the potential to transform patient care with each patient encounter and to have an impact on the health of a population of patients.

We describe the implementation of the SPHERE tool, an EHR-based CVH visualization tool, and subsequent provider satisfaction with the tool as well as patient outcomes after one year. The SPHERE tool can be easily implemented into routine clinical practice, and may be widely disseminated into other clinical practices. In this case study, we highlight challenges to adoption via data tracking and user feedback. We also present generalizable recommendations for future investigations.

Design

We believe that an integrative, workflow-aware, and user-friendly approach to complex risk profiling, data visualization, and joint decision-making ultimately serves to make the EHR an active part of patient-provider dialogues, as opposed to serving as a passive information-capture tool as is commonly the case. Based on user-centered design, we developed the module to have minimum clinical workflow impact by focusing on three features: (1) on-demand availability via a nondisruptive CDS alert; (2) automatic data retrieval for calculations and visualizations; and (3) a nonintrusive view, rendered on a lateral panel that allows concurrent EHR navigation.11,13

When the CDS alert is clicked, it triggers existing EHR application programming interfaces (APIs) that collect current encounter parameters for the patient from the vendor’s live database, and delivers them to the SPHERE tool via a secure POST http request (Figure 1).10 The SPHERE tool then retrieves historical data from the Ohio State University’s (OSU’s) Enterprise Data Warehouse (EDW), containing data from the personal health record (PHR) and EHR, and renders the resulting risk profiling and visualization on an embedded instance of a Web browser engine (Figure 1).13 The SPHERE tool was able to accurately retrieve and render the corresponding risk profile for a given patient during the encounter.

Figure 1.

CDS Alert: Data Sources, Risk Profiling and Visualization Engine, and Point-of-Care Delivery

The SPHERE tool uses the scoring system for CVH for each patient as suggested by the American Heart Association: 2 points for each factor at ideal levels (green), 1 point for each factor classified as intermediate (yellow), and 0 points for each factor indicating poor health.16 The SPHERE inference engine sums these points and divides the result by 10 possible points to display a percentage via the SPHERE visualization engine (Figure 1). A more in-depth methodological discussion has been described previously.10 We provide pseudocode for the risk calculation algorithm and visualization as in Figure 2.

Figure 2.

Risk Calculation Algorithm Components: Summing by Variable Type and Selecting Visualization Color Scheme

To provide a concrete example, the SPHERE tool in Figure 3 displays a composite score of 60 percent. The inference engine calculates this score by summing all ideal (total cholesterol, weight/height), intermediate (blood pressure, smoking status), and poor (hemoglobin A1c) values to a score of 6, and dividing the observed score by the total number of possible points.10 The standard color-coding scheme we employed (green, yellow, and red – which represent ideal, intermediate, and poor health, respectively) is simple and clear for both provider and patient, making it a useful tool for quick understanding and communication (Figure 3). To further guide shared decision-making regarding risk management, the provider can use the SPHERE tool functionality – slider bars and buttons – to show the patient the real-time impact of small changes on their overall CVH, which is realized as changes in the color coding of the factors being manipulated, as well as a recalculation of the CVH score (Figure 3).

Figure 3.

SPHERE Tool (left) and Its Representation Alongside Other EHR Content (right)

Additional parameters for diet and physical activity are accessible by scrolling down in the viewing pane. For providers who update these variables with their patients, the CVH score recalculates to incorporate these CVH behaviors into the composite score (Figure 4).10

Figure 4.

Physical Activity and Diet Parameters: SPHERE Application

We successfully launched the SPHERE tool in two outpatient settings at OSU’s Wexner Medical Center (OSUWMC): (1) a general internal medicine clinic, and (2) a nurse-practitioner-led wellness clinic. These two intervention clinics are part of a network of local general internal medicine and family medicine outpatient clinics at OSUWMC. Providers with access to the tool at the point of care comprise physicians, nurse practitioners, pharmacists, dieticians, and medical residents. Both of the clinics utilizing SPHERE treat underserved, low-income populations within the city of Columbus, Ohio.10

We are currently studying the effect of the tool by comparing data from intervention and control clinics. Intervention clinics have access to the tool, while the control condition does not. The SPHERE tool is operating within Epic, one of the most commonly used EHR systems in the United States.17 A stand-alone Web version is also available. Providers and patients use the tool as a patient-centered decision-making tool to discuss ways to improve CVH; as such the SPHERE tool holds great promise for managing cardiovascular disease risk through patient engagement and shared decision-making.18–20 However, CDS and communication tools, such as the SPHERE tool, must be integrated into clinical workflow to increase their usability.21

Implementation

Prior to the current study, CVH had not been used as a clinical framework for the improvement of cardiovascular disease risk factors or their pharmacologic treatment in the health care setting. The SPHERE tool, targeting CVH, may have a large positive impact on a population of older female patients in the primary care setting, since it is estimated that only 3 percent of older women are in ideal CVH on all health behaviors and factors.22 We expect our findings to inform future investigations of the effect of reducing the risk of coronary heart disease and stroke among older female patients in primary care practices, and will help identify the most effective way to overcome provider-identified barriers to such an intervention.

Thus, we programmed the SPHERE tool to launch for specified users (primary care providers in our intervention clinic) during an encounter with eligible patients (women ages 65 years and older). Specifically, we programmed the CDS alert to launch on eligible patients, regardless of burden or risk factors and regardless of pre-existing coronary heart disease or stroke. As described elsewhere,10 we conducted provider education regarding how to respond to the alert and provide a prevention discussion in response to the CDS alert with the patient at the point of care.

The alert was triggered upon opening an order entry page in the EHR for an eligible patient, instead of upon opening the patient chart. The alert appeared only the first time the provider opened the order entry page during an encounter. This strategy was implemented because prevention discussions would likely not occur at the beginning of the clinical encounter. We took several other steps to decrease the burden of SPHERE on clinical workflow. First, we scheduled the alert to present only once every three months for patients seen by the same provider multiple times within the period in order to minimize “alert fatigue” for those who used the tool with that patient within the three-month period. Second, the provider could select to dismiss the alert and bypass the launch of the SPHERE tool altogether with the click of a button. During the provider training sessions,10 we instructed providers on how to access SPHERE in the case that the provider dismissed the alert unintentionally.10

We leveraged the functionality of our EHR to present the SPHERE tool (Figure 3) as a secure internal Web application that appears in a side panel on the right-hand side of the EHR window, to facilitate an educational discussion between the patient and provider regarding patients’ CVH. The SPHERE tool queries and displays the most recently available CVH values, including those of the current encounter (body mass index and blood pressure), along with other data routinely collected as part of usual patient care (laboratory values, medications, and smoking status). A CVH score is calculated within the application,10 and is updated automatically as providers use SPHERE’s slider bars and buttons to show the patient how CVH improves with changes in modifiable behaviors and factors (Figure 1).

The SPHERE viewing pane remains open until the provider exits the patient’s medical record or closes the side panel. The SPHERE tool is visible to the provider while conducting other activities (placing orders, viewing laboratory values unrelated to SPHERE) within the medical record.

Adoption Metrics

We launched the SPHERE application in October 2013 in the intervention clinic and monitored provider adoption rates using Google Analytics and data from our EDW through the end of September 2014. We were specifically interested in tracking the number of SPHERE uses among eligible patients and the duration of sessions per month, in order to understand the application use case. Since the intervention clinic had recently opened as of the start of the SPHERE study, we did not know how many eligible patients would be seen over the course of the one-year study.

We also measured the proportion of visits in which the application was opened in order to understand utilization. The denominator for utilization was gathered after a year of implementation and included all eligible patient encounters on a monthly basis. Adjustments were made according to the EDW data to account for any women who were seen within three months of a previous encounter – because the application was programmed not to launch with the same provider within this interval.

Google analytics revealed that, on average, SPHERE remained open in the EHR for 2 minutes and 47 seconds during active sessions, ranging from under 1 minute to 11 minutes and 40 seconds. There was no evidence for SPHERE session duration increasing or decreasing over time. Selected results for tool utilization calculated from data available from our EDW are presented in Table 1. One hundred and nine patients had data available at baseline and again one year later. However, the total number of eligible patients seen during the one-year period was 410, and those unique patients had a total of 1,669 encounters during the one-year period during which the CDS alert was presented to the providers. The number of SPHERE sessions shown in Table 1 indicates the number of times the CDS alert for SPHERE was accepted by the provider. The number of SPHERE sessions increased across time, along with the number of eligible patients and patient encounters (Table 1). As of the latter part of our observation period, SPHERE was used with approximately 10 percent of eligible patients. Overall, the average number of eligible patients per month was 113 (standard deviation, 21.5); while the average number of eligible patient encounters was 139 (standard deviation, 29); and the average number of SPHERE sessions was 9 (standard deviation, 5.8).

Table 1.

CDS Alert Tracking and Utilization: SPHERE (October 2013–September 2014)

	OCT 2013	NOV 2013	DEC 2013	JAN 2014	FEB 2014	MAR 2014	APR 2014	MAY 2014	JUNE 2014	JULY 2014	AUG 2014	SEPT 2014	OVER-ALL
ELIGIBLE PATIENTS	75	90	92	107	90	126	130	128	120	134	126	141	410
ELIGIBLE PATIENT ENCOUNTERS	92	102	119	127	103	150	163	153	152	166	165	177	1669
SPHERE SESSIONS	1	1	2	4	10	8	8	12	11	17	14	17	105

Provider Satisfaction Metrics

We asked all providers in the intervention clinic (n=41) to respond to the survey shown in Table 2 regarding their satisfaction with SPHERE training, implementation, and utilization. We adapted these questions from several sources for evaluating electronic medical record systems and provider satisfaction with these systems’ functionality.23–25

Table 2.

Provider Satisfaction Survey Questions and Responses

IN THIS SECTION WE WOULD LIKE TO KNOW YOUR VIEW OF THE SPHERE TOOL BY ASKING ABOUT CENTRAL ASPECT OF USING SUCH SYSTEMS.	MEAN (SD)
1. Content
a. How often does the system provide the precise information you need?	3.6 (1.0)
b. How often does the information content meet your needs?	4.0 (0.8)
c. How often does the system provide reports that seem to be just about exactly what you need?	3.7 (0.7)
d. How often does the system provide sufficient information?	4.0 (0.8)
2. Accuracy
a. How often is the system accurate?	4.4 (0.5)
b. How often are you satisfied with the accuracy of the system?	4.2 (0.6)
3. Format
a. How often do you think the output is presented in a useful format?	4.1 (0.9)
b. How often is the information clear?	4.1 (0.9)
4. Ease of use
a. How often is the system user-friendly?	3.9 (0.7)
b. How often is the system easy to use?	3.9 (0.7)
5. Timeliness
a. How often do you get the information you need in time?	3.8 (0.8)
b. How often does the system provide up-to-date information?	4.1 (0.3)

Note: On a 5-Point Likert Scale with 1=Never/Almost Never and 5= Always/Almost Always

Ten of 41 providers voluntarily completed the survey for a response rate of 24.4 percent. The survey examined five aspects of health information technology satisfaction: Content, Accuracy, Format, Ease of Use and Timeliness. Each question was ranked on a five-point Likert scale as follows: (1) Never/Almost never, (2) Seldom, (3) About half of the time, (4) Most of the time, and (5) Always/Almost always. Means and standard deviations were calculated for the individual 12 items and the five dimensions of usability satisfaction. Results showed positive ratings of usability for content, accuracy, format, ease of use, and timeliness (Table 2).

Overall, providers who completed the survey expressed a high level of satisfaction with the design and function of the SPHERE tool. We defined content as how often the system provided the information needed and included precise detailed information as well as sufficient information. Providers reported that the content met their needs always or most of the time. Additionally, they were satisfied by system accuracy always or most of the time. Providers also felt the system information was clear and presented in a useful format, the system was easy to use and user friendly, and the system provided up-to-date information, with providers receiving the information in a timely manner either always or most of the time. Note, one provider scored the tool low on all domains, which may indicate one individual’s dissatisfaction with technology rather than a critique of SPHERE in particular. Accuracy scored the highest with an average of 4.3+0.6; while content scored lowest at an average of 3.8+0.9.

Patient Outcome Metrics

We quantified the change in CVH from pre- to postimplementation of SPHERE. Specifically, we assessed EHR data before the implementation of the SPHERE tool in the intervention clinic (May 1–July 31, 2013), and compared those CVH data to those collected one year later in the postimplementation and sustained intervention phase (May 1–July 31, 2014). One hundred nine women were seen in the intervention clinic in both the pre- and postimplementation phases. For the purpose of these analyses, we assigned each individual’s data (i.e., body mass index) to the appropriate category of CVH (ideal, intermediate, or poor) and aggregated these individual data points to a clinic-level average, or proportion of body mass index at “ideal,” “intermediate,” and “poor” levels. In order to define categories of CVH, we used published methodology endorsed by the American Heart Association.16 Data from the nurse-practitioner-led clinic are forthcoming, and will be reported on when they are available for analysis.

From pre- to postimplementation (“baseline” to “follow-up”) periods, the ideal levels of some CVH metrics (such as body mass index and diabetes) improved, while others either stayed the same (current smoking, blood pressure) or worsened slightly (total cholesterol) (Figure 5). Among the 109 women seen in the intervention clinic for both the “pre” and “post” periods the proportion of obese women decreased from 47 percent to 43 percent, and the proportion of normal-weight women increased from 15 percent to 19 percent. In addition, the proportion classified as ideal with regard to diabetes status increased from 57 percent to 62 percent (Figure 5). Although perhaps clinically significant, the changes we observed from pre- to post-SPHERE implementation were not statistically significant due to our small sample size. Notably, data challenges uncovered by researchers conducting comparative effectiveness research with EHR data included the following: missing, erroneous, and uninterpretable data; inconsistencies of data collection across providers and time; and data entry in text-only fields.26 Of these data challenges, missing data was the most prevalent for certain CVH behaviors and factors (Figure 5), as all of our data were coded (i.e., nontext), and collected by a group of providers practicing in the same clinic over a one-year period.

Figure 5.

Pre- and Postimplementation CVH Data among 109 Women in Primary Care

Discussion

We successfully implemented the SPHERE tool, and collected follow-up utilization, provider satisfaction, and patient outcomes after one year in one primary-care setting at our medical center. According to our data, the SPHERE tool was used in just over 6 percent of eligible patient encounters (105 of 1,669), and those patient encounters were attributed to 410 unique patients seen over the one-year period. Prior to conducting the study, we did not have a hypothesis for the likely use of the tool. We selected primary care clinics as the setting for the study, since preventative care is often their focus. It remains unknown what constitutes “low” or “high” use of CDS tools, and whether criteria for usage should be based upon use with total number or “unique” counts of patients or encounters. The SPHERE tool should be tested in other clinic settings to facilitate the assessment of clinic-level variables associated with adoption.

A limitation of our study is that the intervention (SPHERE tool) was allocated in the context of a pilot study to determine the feasibility of implementing such a tool in primary care and assessing for patient outcomes. A more robust study design, as a follow-up to this pilot project, could employ group-level randomization and implementation of the SPHERE tool in intervention clinics but not in control clinics. Regardless, our results, lessons learned, and recommendations are generalizable to other research teams preparing to implement novel, EHR-based communication and visualization tools, though we acknowledge that clinical impact might vary depending on the specific tools implemented using this approach, and should be studied individually.

The SPHERE tool’s focus on improving the CVH of patients in primary care is consistent with the prevention services covered by the Affordable Care Act (blood pressure, cholesterol, and diabetes screening; diet counseling; obesity screening and counseling; and tobacco screening and cessation support), and thus has important implications for meaningful use of EHRs. Specifically, stage 2 meaningful use included guidance for managing the preventative care needs of a patient population, such as vaccines and cancer screenings, via EHR-based patient reminders.27,28 From a public health perspective, CVH behaviors and factors are modifiable and amenable to goal setting and self-management. Clinically, however, it is critical to consider that many at-risk populations remain just outside the reach of existing prevention efforts due to socioeconomic resources and access to care.29

During the design, development, and implementation phases of the SPHERE tool, providers often requested additional tool functionality. One of the most common requests was the ability to share a copy of the SPHERE tool with the patient, via print and email. Providers hoped to extend the CVH discussion beyond the care visit, using the SPHERE tool. In addition, providers asked for the ability to export the CVH score into the progress note, or to track and visually present the data longitudinally, in order to monitor changes in CVH from encounter to encounter. These findings point to the need for more extensive employment of user-centered design methods and the conduct of usability studies during the design and development phases of CDS tools, particularly for tools used in shared decision-making between provider and patient. To date, no usability evaluations of this type have been reported. At the completion of provider and patient usability testing, SPHERE may be iterated based on the feedback we receive.

Critical to framing future evaluation of CDS tools is to establish criteria for usage of CDS tools. In the current study, we did not specify usage criteria during provider education sessions. SPHERE’s duration of use varied greatly from encounter to encounter, which likely reflected provider practice patterns. In a study of clinical trial alerts, researchers reported that provider responsiveness to CDS declined gradually over 36 weeks and plateaued at 30–40 percent by the end of the study period.30 However, extant literature fails to identify the appropriate threshold for CDS tool use.

Lessons Learned

After initial deployment of the SPHERE tool, we identified Internet browser compatibility problems. We made modifications to the SPHERE tool so that it was browser agnostic; an important design concept. We collected provider satisfaction surveys and accessed EHR data in order to evaluate the tool’s effectiveness at improving patient outcomes in the intervention clinic. Survey respondents consisted of approximately 25 percent of total users. Perhaps survey completion incentives would have increased the response rate.

Ongoing challenges to presenting data and prevention information via a CDS alert, such as the SPHERE tool, are disruptive to workflow – especially as the number of alerts increases, thus competing for more and more of the providers’ attention.31 Multiple alerts also interfered with the function of the other alerts. For example, when SPHERE was presented along with the flu vaccine reminder alert, selecting “accept” on SPHERE’s alert did not launch the application, even if the other CDS alert was accepted. In that case, providers had to access SPHERE as if they had dismissed the alert by mistake.10

Finally, one of the most important lessons learned through this work is the rapidly changing nature of clinical guidelines and practice. As the health care landscape and EHR usage continue to evolve, we must employ an iterative design and development process to keep SPHERE and other such applications up to date and useful.32 Reimbursement changes, new guidelines, and technological updates are just some of the many factors that can have an impact on provider usage of SPHERE. Continuous evaluation and refinement keep CDS systems relevant and meaningful. Researchers, providers, and health care administrators – using technologies like SPHERE – must work toward the common goal of enabling a learning health care system to improve patient safety and care.12,15

Future Work

We will evaluate the dimensions of usefulness, usability, and satisfaction within the domain of the patient, using focus group and think-aloud protocol methodologies.33 Thus, we will create a more patient-centric evaluation of our application to fully inform the learning health care system.15 The results of our satisfaction assessments among patients and providers will allow us to describe how the technology enables users to successfully perform tasks within the process of the encounter33 via the Task analysis, User analysis, Representational analysis, and Functional analysis (TURF) framework.34,35

Recommendations

• Assess all aspects of usability, from the patient and provider perspective;33–35

• Consider the target audience in order to inform a bidirectional research-practice framework;15 and

• Evaluate CDS tools for such interventions for their ability to remain relevant and actionable for patients and providers alike.