Developing and Evaluating a Clinic-Based Decision Aid Delivery System

Abstract

Background: Despite evidence of their benefits, decision aids (DAs) have not been widely adopted in clinical practice. Quality improvement methods could help embed DA delivery into primary care workflows and facilitate DA delivery and uptake, defined as reading or watching DA materials. Objectives: 1) Work with clinic staff and providers to develop and test multiple processes for DA delivery; 2) implement a systems approach to measuring delivery and uptake; 3) compare uptake and patient satisfaction across delivery models. Methods: We employed a microsystems approach to implement three DA delivery models into primary care processes and workflows: within existing disease management programs, by physician request, and by mail. We developed a database and tracking tools linked to our electronic health record and designed clinic-based processes to measure uptake and satisfaction. Results: A total of 1144 DAs were delivered. Depending on delivery method, 51% to 73% of patients returned to the clinic within 6 months. Nurses asked 67% to 75% of this group follow-up questions, and 65% to 79% recalled receiving the DA. Among them, uptake was 23% to 27%. Satisfaction among patients who recalled receiving the DA was high. Eighty-two to 93% of patients reported that they liked receiving this patient education information, and 82% to 91% reported that receiving patient education information like this is useful to them. Conclusion: Our results demonstrate the realities of clinical practice. One fourth to one third of patients did not return for a follow-up visit. Although nurses were able to assess uptake in the course of their usual duties, the results did not achieve the standards typically expected of clinical research. Despite these limitations, uptake, though modest, was similar across delivery methods, suggesting that there are multiple strategies for implementing DAs in clinical practice.

The goal of implementation research is to integrate evidenced-based medicine into daily clinical practice.¹ Ample evidence demonstrates that decision aids (DAs) can improve patient-centered care.² Use of DAs has been shown to increase patient knowledge and patient’s participation in medical decisions. However, relative to efficacy trials, there has been comparatively little research on implementation of DAs.^3–5

The challenges in facilitating practice change that promotes the implementation of DAs and their use in clinical practice have been well-documented in a recent review.⁶ One approach is to use quality improvement methods, which have been shown to improve care for chronic diseases.⁷ Specifically, the microsystems approach has been shown to be effective by nurturing and empowering teams in the local environment to effect change and improve care.⁸ These programs assure quality of care by using systematic approaches to standardize processes. Measuring progress is a key to their success, as seeing positive changes encourages staff to continue improvement efforts.

Thus, we employed the microsystems approach to implement DAs into our primary care practice processes. Although previous reports have used quality improvement approaches, they have not been implemented in the United States⁹ or have not focused on the microsystem within primary care practices.^4,5

The overarching goal of this study was to compare multiple delivery modalities concurrently. Specifically, our goals were threefold: 1) work with clinic staff to develop and test processes for DA delivery; 2) implement a systems approach to measure DA delivery and uptake; and 3) compare uptake and patient satisfaction across multiple delivery methods. By “uptake” we mean the proportion of patients who reported reading or watching a DA, relative to the number of patients who received a DA. We hypothesized that physician-ordered DAs may result in higher patient uptake, so we wanted to compare a physician-driven model of delivery with those driven primarily by other staff.

Methods

The study design permitted us to receive a waiver of informed consent. We utilized the electronic heath record to assess DA uptake and satisfaction and present only aggregate data. The University of North Carolina Biomedical Institutional Review Board approved this study.

Study Site

The University of North Carolina’s Internal Medicine Clinic in Chapel Hill, North Carolina, has approximately 100 providers, including about 75 residents, and serves about 14,000 patients with approximately 35,000 visits annually. The overall patient population has a mean age of 56 years. Fifty-six percent are female, 30% are African American, 20% are self-pay, and less than 10% are Spanish speaking.

Decision Aids

The DAs used in this study were produced by the Informed Medical Decisions Foundation¹⁰ and were comprised of informational booklets with accompanying DVDs. Each DA included screening, treatment, and/or disease management information on 1 of 17 different topics: Acute Low Back Pain, Advanced Directives, Chronic Low Back Pain, Chronic Pain, Colorectal Cancer (CRC) Screening, Depression, Diabetes, Enlarged Prostate, Growing Older and Staying Well, Herniated Disc, Hip Osteoarthritis, Knee Osteoarthritis, Living with Coronary Heart Disease, Living with Heart Failure, Menopause, PSA (prostate-specific antigen) Screening, and Weight Loss Surgery. The DVDs were 21 to 51 minutes long and were offered in English. The Growing Older and Staying Well DA was available only in booklet form. For all other topics, patients received both the booklet and the DVD and had the option of viewing the DVD in the clinic if they were unable to do so at home. The DAs were developed by the Informed Medical Decisions Foundation and adhered to International Patient Decision Aid Standards.¹¹ They were updated regularly during the time frame of the study.

Planning the Intervention and Evaluation

We utilized a microsystems approach to plan and develop the intervention. A microsystems approach involves a small group of people typically embedded in a larger organization who work together to improve care processes in care delivery that ultimately improves care outcomes.¹²

Consistent with microsystems principles, we worked with providers and staff (key stakeholders) to establish support for the project. We attended staff and physician meetings to explain the project and its goals and then worked individually with clinic staff and providers to develop processes that were acceptable and potentially sustainable. By sustainable, we mean that, where possible, the DA delivery was added to existing workflows and processes already supported by the practice. Physicians in this clinic were already knowledgeable about shared decision making and familiar with DAs, and a research assistant was available to answer questions throughout the project.

Data Collection

In order to implement and evaluate our program, we developed an Access database and tracking tools linked to our electronic health record (EHR) system to 1) identify patients eligible for DAs, 2) create personalized letters to perform outreach to these patients, 3) track DAs as they were signed in and out, and 4) document use of DAs. An EHR programmer worked with us to design and refine these tools, which we found to be reliable and accurate, and to develop a prompt to alert nurses to ask follow-up questions. The EHR was in place prior to the start of this project, and clinic staff were accustomed to receiving alerts and prompts via the EHR.

In order to gauge the success of the implementation strategies, we designed clinic-based processes embedded into clinic workflows to measure DA uptake and satisfaction among patients. All follow-up was completed at the patient’s first clinic visit following receipt of a DA. At that visit, the EHR prompted the nurse to ask the patient a series of follow-up questions before the patient saw the provider. The nurses first asked whether the patient recalled receiving the DA. If so, uptake was assessed by asking whether they watched or read some or all of the DA. Nurses also asked patients two questions regarding their satisfaction with the DA: 1) Did you like getting this patient education information? 2) Is getting patient education information like this useful to you? These questions were asked only of patients who remembered receiving the DA. Nurses entered patient responses in the EHR. However, we were not able to develop a tracking and analysis tool within the EHR, and these data were later downloaded from the EHR to the Access database for tracking and analysis.

We integrated our delivery methods into clinical workflows such that they were supported within usual clinical practice without additional staff costs. The nurses responsible for gathering follow-up data from patients were regular clinic nurses, and the follow-up data were collected in addition to the regular duties. As in any busy practice, the nurses had competing priorities which at times preempted data collection. The nurses, providers, and other clinic staff involved in the study were not incentivized.

Data Analysis

Descriptive statistics were calculated for all patient-level demographic variables using means and standard deviations for continuous variables and frequencies for categorical variables. We assessed the frequency of delivery by method and DA topic. We also assessed follow-up by calculating the frequencies at various stages, including whether the follow-up appointment was attended, whether the nurse completed the follow-up questions about the DA with the patient, whether the patient recalled receiving the DA, whether the DA was read or watched, and patient-rated usefulness of the DA.

Results

Delivery Methods

Working with the practice, we developed and implemented three DA delivery models. The first was embedded in existing disease management programs for several chronic diseases. The second allowed physicians to directly request DAs for patients, and the third continued an existing delivery model used to identify and mail CRC DAs to patients not up-to-date with CRC screening.^13–15 Each model is described in greater detail below.

Chronic Disease Management Program

The internal medicine practice maintains multiple onsite chronic disease management programs designed to serve the needs of patients with one of several chronic conditions. These programs are driven by protocols and administered by care assistants who work with providers to enhance the practice’s ability to provide evidenced-based care. Care assistants typically have a bachelor’s degree. They assist providers by gathering patients’ disease-specific information (e.g., home glucose monitoring), documenting care provided in disease registries, and educating patients on disease-specific care (e.g., insulin injection, nutrition). These programs include Diabetes Enhanced Care, Chronic Pain, and Depression programs. In this setting, DAs were provided as part of the care protocol or at the care assistant’s discretion. All DAs administered through the Chronic Disease Management Program were delivered by care assistants, usually at the end of a regular office visit. Topics administered as part of this model included chronic pain, depression, diabetes, and weight loss surgery.

Physician Request

As we worked with providers, it became apparent that they wanted to have direct access to DAs at the point of care. Initially, we developed a library where physicians could “check out” a DA for their patient. We maintained a library of DAs available on physician request to loan to patients. To facilitate utilization by physicians, we also developed a web-based request system for physicians whereby DAs could be handed out or mailed to patients if they were not in clinic. Our practice already used a similar tool to request referrals to specialists and to order ancillary tests. Therefore, providers used the same processes to request DAs as for these already familiar tasks. Typically, physicians utilized this tool to request DAs during a patient’s office visit and DAs were either delivered to the patient during the visit by a care assistant or mailed to the patient after the visit. The most commonly prescribed DAs included PSA Testing, Knee and Hip Osteoarthritis, and Chronic Low Back Pain. Physicians requested these and other DAs spontaneously based on patients’ health needs; they did not receive a staff or system prompt or reminder from the study team to offer DAs.

Colorectal Cancer Screening Previsit Mailout

Continuing prior work,¹³ this delivery model targeted patients with upcoming clinic visits who were identified by our health information technology system as due for CRC screening. Our definition of “due for screening” included anyone age 50 and older who had never had a screening test or was not up-to-date according to US Preventive Services Task Force guidelines. Two weeks prior to their scheduled clinic visit, research assistants mailed these patients a copy of the CRC DA and a letter signed by the practice director encouraging them to review the DA prior to their visit.

Distribution, Uptake, and Satisfaction

A total of 1144 DAs were delivered using these three methods: 363 via the disease management method, 283 via physician request, and 498 via previsit CRC mailout (Table 1). The DAs were delivered during three time periods totaling 15 months between August 2010 and February 2012: 1 August to 30 November 2010; 1 January to 30 June 2011; and 29 February to 31 May 2012. Between these study periods, DAs were available, but there were no efforts to systematically distribute them or to track use.

Table 1.

Decision Aids by Topic and Delivery Method

Topic	Delivery Method (n)			Total (n)
	Disease Management	Physician Request	Pre-Visit Mailout
Acute Low Back Pain	—	6	—	6
Advanced Directives	—	13	—	13
Chronic Low Back Pain	—	14	—	14
Chronic Pain	180	11	—	191
Colon Cancer Screening	—	18	498	516
Depression	145	49	—	194
Diabetes	29	22	—	51
Enlarged Prostate	—	5	—	5
Growing Older and Staying Well	—	2	—	2
Herniated Disk	—	2	—	2
Hip Osteoarthritis	—	3	—	3
Knee Osteoarthritis	—	16	—	16
Living with Coronary Heart Disease	—	1	—	1
Living with Heart Failure	—	1	—	1
Menopause	—	3	—	3
PSA Screening	—	74	—	74
Weight Loss Surgery	9	43	—	52
Total (n)	363	283	498	1144

Note: PSA, prostate-specific antigen.

The results of the data the nurses collected when patients returned for an office visit after receiving the DA are shown in Table 2. Data loss occurred at each stage of the follow-up process because some patients did not return for an office visit within 6 months, nurses had competing clinical priorities that precluded data collection, and some patients did not recall receiving the DA or did not use the DA. The CRC Previsit mailout method had the highest rate of appointment follow-up at 73%, likely because the mailing of the DA was linked to a preexisting clinic visit. Sixty-three percent of patients in the disease management model and 51% of patients in the physician request model followed-up within 6 months. The mean length of time to follow-up for the CRC delivery method was 49.9 days; the median was 19 days. In the disease management model, the mean time to follow-up was 95.7 days, and the median was 58 days. In the physician request model, the mean time to follow-up was 95.1 days, and the median was 77 days.

Table 2.

Decision Aid (DS) Uptake by Delivery Method

	Delivery Method, n (%)
	Disease Management	Physician Request	Previsit Mailout
Number of DAs delivered	363	283	498
Patient returned for appointment after receipt of DA	228 (63%)	144 (51%)	363 (73%)
Nurse completed uptake questions	165 of 228 (72%)	108 of 144 (75%)	245 of 363 (67%)
Patient recalled receiving DA	102 of 165 (65%)	81 of 108 (75%)	193 of 245 (79%)
Patient read/watched any of the DA	85 of 102 (73%)	66 of 81 (81%)	132 of 193 (68%)
Overall patient uptake (% of all DAs delivered)	85 of 363 (23%)	66 of 283 (23%)	132 of 498 (27%)

Nurses were able to complete the follow-up questions before the provider saw the patient more than two thirds (67% to 75%) of the time. Among those patients who were asked, 65% to 79% recalled receiving the DA. Among those who remembered it, 68% to 81% reported reading or watching it. There was no statistically significant difference in the percentage who reported reading or watching the DA across the delivery modalities. Among those who recalled receiving a DA, a high proportion was satisfied: 82% to 93% reported that they liked getting this patient education information (the DA), and 82% to 91% reported that patient education information like the DA is useful. Overall DA uptake among all patients who received a DA was approximately 23% in the disease management and physician request models and 27% in the CRC mailout model.

We also assessed uptake by DA topic, but the small cell sizes for many of the DA topics precluded meaningful comparison of uptake rates. As shown in Table 1, the number of DAs delivered varied by topic. In the chronic disease model, the most commonly delivered DAs were chronic pain (n = 180), depression (n = 145), and diabetes (n = 29). Of these, only chronic pain and depression had ≥20 patients who completed the follow-up questions and recalled receiving the DA. Uptake for the chronic pain DA was 62%—31 of 50 patients who recalled receiving the DA read or watched at least part of it. Uptake for the depression DA was 58% (36 out of 62 patients). In the physician request model, depression (49), diabetes (22), PSA screening (74), and weight loss surgery (43) were the most frequently used, but none of the DA topics delivered via physician request had a large enough cell size (≥20) to assess uptake.

Discussion

We were able to demonstrate that we could successfully integrate delivery of DAs and measure use and satisfaction from patients within an established, large, academic primary care practice. Although the overall uptake was low, DA delivery was incorporated as a standard component of patient care in the practice, which demonstrates the feasibility of delivering multiple DAs in primary care. We met our goals of developing and testing multiple delivery models, implementing a system of measuring uptake and satisfaction, and comparing uptake and satisfaction across delivery models. The three different delivery methods enabled us to reach patients in multiple ways that were integrated into clinic and practice workflows. Overall, we found that uptake among all patients who received the DAs was modest, despite multiple models of delivery. Furthermore, no single delivery method resulted in superior uptake—all three modalities yielded an uptake of about 25%. The patients who used the DAs reported liking them, but this was a small number among the patients who received DAs.

A recent review published after our project was implemented found that implementing DAs within care pathways using a physician referral model is often difficult, in part due to provider hesitation and concerns about disruptions to existing workflows.⁶ However, our finding that all delivery methods produced similar uptake rates suggests that it is not simply provider reticence that accounts for low uptake.

Although we had previous experience in implementing DAs within our practice, this project is unique in that we implemented several different delivery methods simultaneously. When a similar mailout approach for CRC DAs was utilized previously, we found that the uptake ranged from 7% to 13%.^13,14 However, uptake was determined from data collected via patient-initiated response to the mailing, as opposed to the follow-up initiated by the EHR and conducted by nurses in this study. When we previously had a staff member provide DAs before or during the visit, this approach increased uptake of the PSA DA to 57%. However, unlike comparing delivery models in this study, we were unable to directly compare the results because the studies were not conducted concurrently.

Our hypothesis was that delivery of DA by disease management staff or physicians would increase viewing of DAs, compared to mailout methods. However, there were no differences between delivery modalities. Consequently, in future implementation efforts, it may be sufficient to utilize the most resource-effective modality that is tailored to the clinical setting’s workflows and processes, as additional efforts may not improve uptake. Additionally, determining what is valued most by patients in terms of receiving and viewing DAs, as well as how they would prefer to have it delivered, may be helpful in directing already limited resources to interventions with the greatest potential for benefit.

This project is unique in that we attempted to work within the microsystem of our practice to develop and test delivery systems for DAs and at the same time use these systems to measure progress and determine patient uptake of the DAs. Other health systems have also worked to incorporate DAs into the work flow. Group Health incorporated DA delivery across specialty services for elective surgical procedures.^4,5 The focus of that project, however, was primarily DA distribution and procedure rates. They did not address uptake or satisfaction with the tools. Lin and colleagues performed an observational study of DA distribution in five primary care sites but did not collect information regarding uptake or satisfaction with the information.³ Efforts in the United Kingdom are ongoing and have not yet reported information on uptake of DAs.¹⁶

Integration in clinical practice is complex and challenging. This study took place in a real clinical practice, and the nurses collecting these data were regular clinic nurses with many competing priorities, which led to a substantial amount of missing data. However, our goal was to test implementation in real-world clinical practice. Nurses were able to ask follow-up questions of over two thirds of the patients, which is good for implementation work, although it is not to the standard expected for clinical research.

Importantly, our goal was to determine use of these tools by patients as a first step; we were not trying to measure whether these tools increased shared decision-making. Our rationale was that we must first ensure reliable delivery of DAs before addressing whether these tools influenced patient physician interactions outside of efficacy trials.² The reported uptake of these tools by patients was modest even though nurses were asking about use, which could have resulted in social desirability bias by patients. More recent work suggests that engaging physicians with training could perhaps improve uptake.¹⁷

One challenge moving forward is to not only target patients who could benefit from decision support but also to identify patients who value the information and would use it, so that limited resources can be directed to the right patients. A drawback to this approach, however, is that those patients who could most benefit from the information may not be accessing the information because they do not know they could benefit. Studies have shown that patients with lower socioeconomic status and health knowledge have substantial gains from exposure to DAs.^18,19 Further examination of the utility of DAs for patients with varying preferences and health conditions may enable more tailored and targeted educational materials for patients.

Our findings should be interpreted in the context of several limitations. The project was designed as a quality improvement project to determine the feasibility of DA delivery and to identify measures of use. We are unable to calculate reach because of the difficulty in determining the eligible patient population. That is, we do not know how many patients attended an office visit during the study period and were eligible for a DA but did not receive one. Instead, we focused on measuring uptake of DAs among patients who received them. The comparisons we made across delivery models are subject to selection bias; therefore, the findings should be interpreted within this context. We faced significant data loss at each step of the process, from delivery to reporting DA use. Some patients did not return for a follow-up visit within 6 months or did not recall receiving the DA, and nurses were not always able to collect data from the patients who did return. Additionally, there may be differences in uptake related to the specific DA topics available in addition to the delivery method. Not all DAs were available in each delivery method and the number delivered on each topic was typically low, so it is difficult to compare uptake by topic and delivery method in a meaningful way. These challenges demonstrate the complexity of implementing and evaluating DA delivery clinic-wide.

Furthermore, our results may be subject to social desirability bias. When asked by nurses about DA use and satisfaction, patients may have felt inclined to respond that they had used and liked the DA in an effort to demonstrate cooperation or compliance. However, we think this is less likely given the relatively low reported uptake. It is also possible that the length of time between receipt of the DA and follow-up could lead to recall bias. Some patients who did not return for follow-up for several months may have forgotten receiving or viewing the DA, possibly lowering the actual uptake rate. Additionally, there may be differences between the groups that we did not measure, which could affect DA uptake rates, such as overall health status and demographic variables. Finally, this microsystems project was applied in a single center in an academic primary care setting that is experienced in quality improvement projects and patient-centered care. Thus, these results may not be broadly applicable to other practices. Despite these limitations, this study demonstrates that providing multiple methods of delivery may not prove fruitful in improving actual viewing of DAs by patients.

Conclusion

Using a microsystems approach within our practice, we used three delivery modalities to successfully implement DAs covering a wide variety of topics. We also demonstrated the challenges associated with clinic-wide implementation. We found that nurses were able to follow-up well overall using the EHR tracking system, but not to the standards typically expected of clinical research. Uptake, though modest, was similar across the three methods of delivery, suggesting that there are multiple strategies that can be used, independently or simultaneously, to implement DAs in clinical practice.