Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2015 Aug 26.
Published in final edited form as: Health Promot Pract. 2014 Nov;15(2 0):92S–102S. doi: 10.1177/1524839914545168

Cross-Site Evaluation of the Alliance to Reduce Disparities in Diabetes: Clinical and Patient-Reported Outcomes

Megan A Lewis 1, Carla M Bann 1, Shawn A Karns 1, Connie L Hobbs 1, Sidney Holt 1, Jeff Brenner 2, Neil Fleming 3, Patria Johnson 4, Kathryn Langwell 5, Monica E Peek 6, Joseph A Burton 1, Thomas J Hoerger 1, Noreen M Clark 7, Douglas B Kamerow 8
PMCID: PMC4550084  NIHMSID: NIHMS716463  PMID: 25359255

Abstract

Alliance programs implemented multilevel, multicomponent programs inspired by the chronic care model and aimed at reducing health and health care disparities for program participants. A unique characteristic of the Alliance programs is that they did not use a fixed implementation strategy common to programs using the chronic care model but instead focused on strategies that met local community needs. Using data provided by the five programs involved in the Alliance, this evaluation shows that of the 1,827 participants for which baseline and follow-up data were available, the program participants experienced significant decreases in hemoglobin A1c and blood pressure compared with a comparison group. A significant time by study group interaction was observed for hemoglobin A1c as well. Over time, more program participants met quality indicators for hemoglobin A1c and blood pressure. Those participants who attended self-management classes and experienced more resources and support for self-management attained more benefit. In addition, program participants experienced more diabetes competence, increased quality of life, and improvements in diabetes self-care behaviors. The cost-effectiveness of programs ranged from $23,161 to $61,011 per quality-adjusted life year. In sum, the Alliance programs reduced disparities and health care disparities for program participants.

Keywords: diabetes, health disparities, quantitative evaluation

The Alliance to Reduce Disparities in Diabetes, a 5-year initiative in five programs across the United States, aimed to enhance patient outcomes and reduce disparities for populations that are typically underserved and bear a disproportionate burden of diabetes. The chronic care model (CCM) guided program implementation and the evaluation. This model suggests that health improvement is achieved by targeting multiple contexts such as communities and health care systems, as well as clinician and patient behavior. Program implementation across the five sites (Coleman, Austin, Brach, & Wagner, 2009) used a multicomponent, multilevel approach that garnered community resources, coordinated health systems to deliver improved care, implemented strategies to improve patient and provider communication, and enhanced patient self-management for diabetes.

Although based on the CCM, a unique aspect of the Alliance programs was that they did not follow a prescribed implementation strategy, such as that used by the Breakthrough Collaborative Series (Institute for Healthcare Improvement, 2004). Instead, sites were encouraged to implement programs that addressed the needs of local communities. The programs are described in other articles in this special issue (Collinsworth, Vulimiri, Snead, & Walton, 2014; Johnson et al., 2014; Kaufman, Ali, DeFiglio, Craig, & Brenner, 2014; Langwell, Keene, Zullo, & Ogu, 2014). This article reports on the cross-site evaluation of the Alliance programs, including clinical and patient-reported outcomes

BACKGROUND

Nearly 26 million Americans (or about 8% of the U.S. population) have diabetes (American Diabetes Association, 2013a). Among people with diabetes, racial/ethnic minorities are disproportionally affected by the disease. Compared with non-Hispanic Whites, the risk of diagnosed diabetes in 2010 was 1.2 times higher among Asian Americans, 1.7 times higher among Hispanics, and 1.8 times higher among non-Hispanic Blacks (American Diabetes Association, 2013a). In addition, approximately 16% of American Indian and Alaska Native adults have been diagnosed with diabetes (American Diabetes Association, 2013a).

The prevalence of diabetes is expected to double in the United States over the next 25 years, bringing about related increases service needs and costs (Huang, Basu, O’Grady, & Capretta, 2009). The largest increases in prevalence will be among groups currently experiencing the largest diabetes burden, such as African Americans (Boyle et al., 2001). These increases will require large-scale implementation of evidence-based approaches across the United States to help patients, communities, and health systems manage diabetes and to stem disease progression. Addressing this need will require an approach akin to that described by Glasgow, Vogt, and Boles (1999) that recognizes effectiveness as the byproduct of (evidence-based) efficacy and implementation.

Emerging evidence suggests that multicomponent, multilevel programs can be successful in changing patient outcomes related to diabetes. For example, The Robert Wood Johnson Foundation’s Diabetes Initiative was a 30-month, multisite initiative that promoted diabetes self-management in primary care and community settings (Fisher et al., 2005). Based on the CCM and using an ecological approach, programs focused on (a) integrating the skills and choices of program participants with the services and support they received from the social, physical, and policy environments and (b) enhancing key resources and supports for self-management (RSSM) needed by individuals. An evaluation of this initiative found that hemoglobin A1c (HbA1c) levels decreased from baseline to 12 months by 0.56 point and that patients reporting high levels of RSSM were more likely to achieve HbA1c decreases or maintain baseline controlled levels.

In another large-scale multisite evaluation, Chin et al. (2007) compared measures of care processes and intermediate outcomes at three different time points (1998, 2000, and 2002) at community health centers that focused on diabetes and participated in the CCM Collaborative. This study randomized health centers into a “standard-intensity” or “high-intensity” arm of the study in order to compare effects of additional activities to support implementation of the CCM. The findings showed significant improvements in both care processes and clinical outcomes over the 4-year study period. Other initiatives using multilevel, multicomponent approaches also suggest that measurable outcomes can be achieved via these types of programs (Munoz et al., 2012; Nyman, Murphy, Schryver, Naessens, & Smith, 2000).

Other evidence is mixed regarding the benefit of complex multilevel, multicomponent approaches. Landon et al. (2007), for example, compared clinical outcomes at health centers that were part of the CCM Collaboratives with outcomes at health centers that were not part of the Collaboratives. The study included an analysis of clinical indicators to assess the effectiveness of improving care. Using a 1-year follow-up time, the findings showed that the Collaboratives significantly improved the processes of care (e.g., the number of individuals whose HbA1c levels were assessed) but did not improve the control of HbA1c levels or blood pressure levels.

Although multilevel, multicomponent programs may be effective in changing clinical outcomes, their cost-effectiveness is a concern if wide-scale implementation is required to serve the growing number of patients with diabetes. Researchers sometimes use a cost of $50,000/life year or $50,000/quality-adjusted life year (QALY) as a benchmark or “willingness to pay” for considering an intervention to be cost-effective or not, although the conceptual basis for choosing such a benchmark is controversial (Glick, Doshi, Sonnad, & Polsky, 2007; Grosse, 2008; Woolf, 2009). One study estimated the incremental cost-effectiveness of improving diabetes care with the Health Disparities Collaboratives in community health centers and found an incremental cost-effectiveness ratio of $33,386/QALY (Huang et al., 2007). Other studies for diabetes programs have found similar results (Brownson, Hoerger, Fisher, & Kilpatrick, 2009; Hoerger et al., 2002).

The five programs that comprised the Alliance included Improving Diabetes Care and Outcomes on the South Side of Chicago, University of Chicago, Chicago, Illinois; Camden Citywide Diabetes Collaborative, Camden, New Jersey; Diabetes for Life Program, Memphis, Tennessee; Reducing Diabetes Disparities in American Indian Communities, Wind River Indian Reservation, Wyoming; and The Diabetes Equity Project, Dallas, Texas. These programs focused on reducing disparities in diabetes care and enhancing outcomes through multilevel, multicomponent clinical and community interventions (Clark et al., 2011).

Although the specific content of the interventions varied across the programs, each intervention focused on three core components: patient change, clinician change, and system change. Patient education included community, small group, and individual materials as well as classes and discussions focused on evidence-based topics related to diabetes self-management education. Provider education focused on enhancing cultural competency skills and behavioral change education, as well as communicating effectively with patients to help them make lifestyle improvements. Health care system change included care coordination, use of diabetes registries or electronic medical records, nurse or community health worker participation in care management, enhanced community partnerships, and policy changes. Details of these programs and their specific strategies are described elsewhere (Collinsworth et al., 2014; Goode & Jack, 2014; Johnson et al., 2014; Kaufman et al., 2014; Langwell et al., 2014).

To further contribute to the growing literature focused on the implementation of multicomponent, multilevel chronic disease improvement programs and to expand the knowledge base about these types of programs, this cross-site evaluation investigated four research questions. Guided by the CCM, and using locally adapted and tailored strategies, were the programs able to enhance clinical outcomes (Research Question 1), improve important patient-reported outcomes integral to diabetes management (Research Question 2), implement programs in a cost-effective manner (Research Question 3), and reduce diabetes disparities (Research Question 4)?

METHOD

Data Collection

At the beginning of the 5-year initiative, the grantees and the Alliance National Program Office met with the external evaluation team to decide on data collection procedures and measures to be used across the sites. The programs agreed to provide clinical measures in addition to patient-reported outcomes, but not all were required to provide the measures or outcomes if it was infeasible or conflicted with their local evaluations.

Once data collection started across sites, programs provided clinical data twice yearly over the course of the initiative. Programs used data templates to organize data delivery and management. At each delivery period, data were inspected for quality assurance. New data were integrated into an overall data set that contained both patient-level and site-level identification numbers so that multiple patient data points could be tracked over time. Over the course of the initiative, 2,328 patients participated in the Alliance programs across 47 clinics. To examine change over time, the evaluation reports on the 1,827 patients who were program participants for which both baseline and follow-up data were available. In addition, programs provided data on a comparison cohort of patients seen in their clinics but not participating in their program (N = 586) to help distinguish secular trends from program changes and improve internal validity (Campbell & Stanley, 1966). The data for the comparison cohort focused only on clinical values.

Measures

The clinical measures selected (HbA1c, blood pressure, lipids, and weight) are common clinical outcomes cited as important indicators of quality care related to diabetes. Enhancing these outcomes leads to improvement in the health and well-being of patients with diabetes (American Diabetes Association, 2013b; National Committee for Quality Assurance, 2013). All programs provided clinical outcome data.

The patient-reported outcome measures were selected based on two criteria: (a) outcomes that had been shown to be important for diabetes management and (2) measures that were already in use by sites in their local evaluations. The following patient-reported measures were chosen:

Quality of life: The Veterans/RAND 12-Item Health Survey (VR-12; Kazis et al., 2006) is a 12-item questionnaire that corresponds to eight principal physical and emotional problems, bodily pain, energy fatigue, social functioning, and mental health. The items are summarized into two scores, a Physical Component Score and a Mental Component Score. Previous research has shown the VR-12 to have excellent reliability and validity (Kazis et al., 2006). Cronbach’s alpha in our sample is .87.

Perceived Competence Scale for Diabetes: This measure (Williams, McGregor, Zeldman, Freedman, & Deci, 2004) is a short, four-item questionnaire that assesses feelings of competence to engage in diabetes self-management. Participants respond using a scale from 1 to 7, with 1 representing not at all true and 7 very true. Previous research has shown it to be a valid and reliable measure of diabetes competence (Williams, Freedman, & Deci, 1998). In our sample, Cronbach’s alpha for the measure is .89.

Patient Assessment of Chronic Illness Care (PACIC): The PACIC (Glasgow et al., 2005) consists of 20 items and includes five scales with an overall summary score that measures if patients receive care in line with the CCM. The scale and corresponding scores include (1) none of the time, (2) a little of the time, (3) some of the time, (4) most of the time, and (5) always. The PACIC is reliable (r = .58 for test–rest reliability over a 3-month period) and is correlated (r = .32-.60; p < .001) with measures of primary care and patient activation. Test–retest reliability for the Spanish version is .77 (Glasgow et al., 2005; Shah et al., 2008). In our sample, Cronbach’s alpha is .95.

Resources and Supports for Self-Management: The RSSM (McCormack et al., 2008) is a five-item questionnaire that measures the receipt of self-management support for chronic illness. The items probe patients’ experiences with and support from their health care team. The scale and corresponding scores include (1) never, (2) sometimes, (3) usually, and (4) always. Cronbach’s alpha for the RSSM long form is .70 or greater and is significantly and positively related to diabetes self-management behaviors (McCormack et al., 2008). Cronbach’s alpha for the short form used in this evaluation is .95.

Summary of Diabetes Self-Care Measure (SDSCM): The SDSCM (Toobert, Hampson, & Glasgow, 2000) is a brief self-report questionnaire of diabetes self-management behaviors that includes items assessing general diet, diabetes-specific diet, exercise, blood glucose testing, foot care, and smoking. All items are asked in the context of the participant’s past 7 days (not including days when sick). All items, with the exception of smoking, score from 0 to 7. For the smoking item, 0 equates with no cigarettes smoked and 1 is yes, with an open-ended numeric score for number of cigarettes smoked in the past 7 days. The SDSCM has demonstrated reliability and validity (Toobert & Glasgow, 1994). In our sample, Cronbach’s alpha is .80 across subscales.

All measures were calculated as the mean of the responses, except for the VR-12, for which scores are derived by using an algorithm provided by the measure custodians. The algorithm is designed to normalize the scores to a national population. For all of the scales, higher scores represent better results for the patient-reported outcome measured.

Program costs were collected via a questionnaire that asked programs to identify the activities performed as part of their programs, the type of staff members performing those activities, and the salaries of those staff members to represent an activity-based accounting approach as described by Kaplan and Porter (2011). These data were used to calculate the costs to run the programs and then applied in the cost-effectiveness analysis.

Analytic Approach

Examining Changes in Clinical and Patient-Reported Outcomes

To examine changes in clinical outcomes over time, descriptive statistics (means, standard deviations) and bivariate analyses were computed to compare first and last measurements on each clinical outcome taken during the study period for program participants and comparison group members. Similar analyses were conducted to compare the first and last patient-reported outcome survey measurements for program participants; surveys were administered only to program participants. Paired t tests were conducted for normally distributed continuous outcomes, whereas generalized linear models accounting for repeated measurements and incorporating appropriate distributions were fit for nonnormally distributed variables.

Multilevel regression models were then estimated to test whether the amount of change in the clinical outcomes varied significantly between program participants and the comparison group (i.e., a significant time by study group interaction) while accounting for clustering of participants within sites and controlling for other factors. Each model included study group (program participants vs. comparison group), time (first measurement vs. last measurement), site, age, gender, and a time by study group interaction. Because of collinearity with the program sites, race was not included in the models (e.g., all participants at one site were American Indian).

Finally, for clinical outcomes showing greater improvement among program participants than the comparison group, we examined whether program-related factors served as intervening variables that could explain different program impacts on these outcomes. Specifically, we tested whether level of program participation (e.g., attending all of the required program sessions) and receipt of self-management support (as measured by RSSM) were related to improved clinical outcomes. For each of these factors, we fit multilevel models of the clinical outcomes similar to the models described above (i.e., controlling for site, time, and demographics) and tested for an interaction between time and the variable of interest. For example, a significant interaction between time and program participation would indicate that individuals with greater participation in the program experienced different levels of improvement in their clinical outcomes than individuals with less participation. Additional clinical data were available for program participants, allowing trajectories to be examined across more than two measurements for the analyses with program participants only. All analyses were conducted using SAS Version 9.3.

Estimating Cost-Effectiveness

Another aspect of this evaluation involved estimating future medical costs using the validated Centers for Disease Control and Prevention–RTI diabetes cost-effectiveness model (for details and validation, see Hoerger et al., 2002; Hoerger, Segel, Zhang, & Sorensen, 2009), which allowed us to estimate how the Alliance programs might affect future disease-specific adverse events and total medical costs and utilization over time. The model is a Markov chain simulation of disease progression and cost-effectiveness for type 2 diabetes, simulating the development of diabetes-related complications on five disease paths: nephropathy, neuropathy, retinopathy, coronary heart disease, and stroke. Model outcomes include disease complications, deaths, costs, and QALYs. QALYs account for morbidity and mortality by assigning patient utilities ranging from 1 (perfect health) to 0 (death) to disease states (i.e., diabetes complications); intermediate utility values reflect morbidity that reduces quality of life. QALYs are estimated by summing patient utility over a patient’s remaining lifetime. The model simulates how changes in risk factors that can be observed in short-term studies affect long-term outcomes (complications and death) that will occur years later. In the model, interventions improve risk factors such as HbA1c, lipids, and blood pressure, and these improvements lead to fewer complications, fewer deaths, more QALYs, and lower medical costs in the long term.

Incremental cost-effectiveness ratios (the difference in costs divided by the difference in QALYs) were estimated under two scenarios that make different assumptions about the program’s impact. The first, more optimistic, scenario attributes all of the pre- to postintervention changes in HbA1c, systolic blood pressure, and total cholesterol to the program, as long as these changes are statistically significant. The second, more conservative, scenario is based on the difference in effects between the intervention and comparison groups. For the analyses, we assumed that patients would continue to receive the intervention for the rest of their lives, incurring maintenance costs and maintaining their reductions in HbA1c, blood pressure, and cholesterol. Costs and QALYs are viewed from a health care system perspective and discounted at a 3% annual rate.

RESULTS

The demographic variables for all program participants are reported elsewhere (see Table 3 of Lewis et al., 2014). The 1,827 program participants with baseline and follow-up data included African Americans (37%), Hispanics (48%), Native Americans (5%), Asians/Pacific Islanders (1%), and other/mixed race (1%). Participants were predominantly female (64%) and aged 45 years or older (72%). Age and gender were similar between those with baseline and follow-up measures and those with baseline data only; racial/ethnic distributions differed significantly (p < .001), with African American and American Indian respondents being less likely and Hispanics being more likely to have follow-up data.

Table 1 presents means and standard deviations of first and last clinical measurements among program participants and the comparison group. HbA1c values decreased significantly from first to last measurements in both groups (p < .001), with program participants having mean decreases of 0.66 and the comparison group having mean decreases of 0.35. Program participants experienced significant decreases in both systolic and diastolic blood pressure and the comparison group experienced decreases in diastolic blood pressure only. Both groups had significant decreases in total cholesterol values. While neither group had significant changes in high-density lipoprotein cholesterol or low-density lipoprotein cholesterol, triglyceride values for program participants decreased significantly during the program, with an average decrease of 17.5; the 4.8 decrease in triglycerides among comparison group participants was not statistically significant. There was no statistically significant change in weight between the comparison and program group.

TABLE 1.

Means and Standard Deviations of First and Last Clinical Measurements Among Alliance to Reduce Disparities in Diabetes Program Participants and Comparison Group

Program Participants a
Comparison Group a
Clinical Measure N First, M (SD) Last, M (SD) Difference, M (SD) p N First, M (SD) Last, M (SD) Difference, M (SD) p
Clinical values
 HbA1c 1,515 8.37 (2.15) 7.70 (1.85) −0.66 (1.94) <.001 533 8.36 (2.42) 8.02 (2.14) −0.35 (2.15) <.001
 Blood pressure
  Systolic 1,573 129.24 (18.74) 128.18 (19.27) −1.07 (18.89) .025 574 131.46 (20.52) 130.03 (18.97) −1.33 (21.73) .142
  Diastolic 1,573 78.84 (11.39) 77.95 (11.83) −0.87 (12.49) .006 574 78.75 (13.07) 76.41 (12.37) −2.20 (14.92) <.001
 Weight 1,596 200.97 (53.14) 201.39 (52.52) 0.35 (14.51) .337 54 213.78 (48.94) 208.55 (52.14) −5.23 (23.54) .109
 Lipids
  Total cholesterol 654 181.82 (42.91) 176.86 (41.41) −4.96 (41.25) .002 303 173.80 (42.64) 165.27 (44.22) −8.52 (41.89) <.001
  HDL 501 48.11 (14.52) 47.70 (13.63) −0.42 (10.28) .366 58 49.05 (16.27) 49.97 (16.15) 0.91 (8.58) .421
  LDL 490 101.26 (35.69) 98.84 (33.24) −2.42 (31.95) .094 200 102.05 (39.23) 99.28 (35.10) −2.76 (37.57) .300
  Triglycerides 448 169.82 (117.02) 152.31 (110.04) −17.49 (107.53) <.001 58 160.72 (113.75) 155.93 (107.43) −4.79 (77.63) .640
Meet guidelines for good control
 HbA1c, N (%) 1,514 494 (33) 651 (43) 157 (10) <.001 533 197 (37) 212 (40) 15 (3) .183
 Blood pressure, N (%) 1,572 584 (37) 667 (42) 83 (5) <.001 574 225 (39) 228 (40) 3 (1) .835
 LDL cholesterol, N (%) 489 265 (54) 277 (57) 12 (3) .300 200 103 (52) 113 (57) 10 (5) .197
Open in a new tab

NOTE: HbA1c = hemoglobin A1c; HDL = high-density lipoprotein; LDL = low-density lipoprotein.

a

Analyses include only participants with at least two measurements. Amount of change in clinical values over time differed significantly between program and comparison groups for the following outcomes after controlling for site, age, gender, study group, time of measurement, and interaction between study group and time: HbA1c values, Wald χ2(1) = 11.64, p < .001; good control of HbA1c, Wald χ2(1) = 11.39, p < .001; and good control of blood pressure, Wald χ2(1) = 5.54, p = .019.

After controlling for site, age, gender, study group, and time of measurement, the interaction between time and study group was significant for HbA1c, Wald χ2(1) = 11.64, p < .001), indicating that program participants had significantly greater decreases in HbA1c values than the comparison group.

Over time, programs had increasing numbers of patients that met quality indicators for HbA1c and blood pressure (p < .001; Table 1). There was no significant change in the percentage of the comparison cohort meeting quality indicators for any of the measures. Changes in the percentages of individuals achieving good control varied significantly between the program participants and comparison cohort for HbA1c, Wald χ2(1) = 11.39, p < .001); and blood pressure, Wald χ2(1) = 5.54, p = .019), controlling for site, age, gender, study group, and time of measurement, and the interaction between time and study group.

We also investigated program-related factors that might account for greater decreases in HbA1c values among program participants versus the comparison group. Tests of intervening variables indicated that patients reporting greater RSSM throughout the program had greater decreases in HbA1c values, Wald χ 2(3) = 12.80, p = .005 (Figure 1). In addition, attendance at required program sessions was associated with greater improvements in HbA1c values over time, Wald χ2(3) = 9.93, p = .019 (Figure 2).

FIGURE 1. Model-Adjusted Mean HbA1c, by Measurement Order and RSSM: Alliance to Reduce Disparities in Diabetes Program Participants Only.

FIGURE 1

Open in a new tab

NOTE: HbA1c = hemoglobin A1c; RSSM = resources and support for self-management. Means adjusted for site, age, gender, measurement order, RSSM, and interaction between measurement order and RSSM. Changes in HbA1c values over time differed significantly for patients with different RSSM values: RSSM × measurement order: Wald χ2(3) = 12.80, p = .005.

FIGURE 2. Model-Adjusted Mean HbA1C Values, by Measurement Order and Attending All Program Classes: Alliance to Reduce Disparities in Diabetes Program Participants Only.

FIGURE 2

Open in a new tab

NOTE: HbA1c = hemoglobin A1c .Means adjusted for site, age, gender, measurement order, class attendance, and interaction between measurement order and class attendance. Changes in HbA1c values over time differed significantly across patients, depending on class attendance: class attendance × measurement order: Wald χ2(3) = 9.93, p = .019.

Sites varied in regard to self-reported outcomes measured by surveys administered to program participants. Overall, participants in the program significantly improved across all measures (Table 2). Patients’ quality of life, both physical functioning (p < .02) and mental functioning (p < .001), improved from their first to last measurements. RSSM and practice of self-management behaviors—including diet, exercise, glucose monitoring, and foot care—increased over the course of the program, as did perceived diabetes competence (p < 0.001).

TABLE 2.

Means and Standard Deviations of First and Last Survey Measurements Among Alliance to Reduce Disparities in Diabetes Program Participants

Program Participants a
Patient-Reported Outcomes N First, M (SD) Last, M (SD) Difference, M (SD) p
RSSM 243 2.50 (1.02) 2.79 (0.94) 0.29 (1.01) <.001
DSC
 General diet 546 3.72 (2.04) 4.74 (1.79) 1.02 (2.23) <.001
 Specific diet 550 4.04 (1.50) 4.57 (1.51) 0.53 (1.68) <.001
 Exercise 552 2.63 (2.11) 2.94 (2.16) 0.31 (2.48) .003
 Glucose 552 4.11 (2.27) 5.02 (2.07) 0.90 (2.53) <.001
 Foot care 553 4.14 (2.50) 5.53 (2.02) 1.39 (2.56) <.001
 Smoking status, N (%) 549 80 (15) 87 (16) — .262
PACIC 257 3.47 (0.92) 3.98 (0.83) 0.51 (0.99) <.001
PDC 375 4.92 (1.34) 6.16 (0.92) 1.25 (1.50) <.001
VR-12
 PCS 561 42.10 (10.35) 42.85 (10.29) 0.74 (8.64) .023
 MCS 561 43.32 (10.88) 47.80 (10.89) 4.48 (11.99) <.001
Open in a new tab

NOTE: RSSM = resources and supports for self-management; DSC = diabetes self-care; PACIC = Patient Assessment of Chronic Illness Care; PDC = Perceived Diabetes Competence; VR-12 = Veterans/RAND 12-Item Health Survey, PCS = Physical Component Score, MCS = Mental Component Score. In all measures, higher numbers represent better self-reported outcomes.

a

Analyses include only participants with at least two measurements.

On average, Alliance grantees spent about $975 per patient in the first year to improve general care processes and to monitor their diabetes patients, and an additional $520 per patient in subsequent years. Under the optimistic cost-effectiveness scenario, the program reduces HbA1c by 0.66%, systolic blood pressure by 0.8%, and total cholesterol by 2.7%. With this effect, the program has an incremental cost-effectiveness ratio of $23,161 per QALY. Under the more conservative scenario, the program reduces HbA1c by 0.31% and has insignificant effects on systolic blood pressure and total cholesterol (these effects are set equal to zero). Under this assumption, the incremental cost-effectiveness ratio is $61,011 per QALY.

DISCUSSION

The Alliance programs achieved significant and measureable changes in important clinical and patient-reported outcomes. This finding is encouraging because rigorous programs designed and proven effective in research settings are often adapted and reinvented as they diffuse and are implemented in clinics and communities to meet local conditions (Rogers, 2003). Overall, the findings suggest that the implementation of programs inspired by the CCM that do not require a fixed implementation strategy can result in measurable changes that can improve patient health and well-being. This evaluation suggests that programs were able to do this successfully. For those interested in seeing more diffusion of the CCM without prescribing a specific implementation strategy, the approach of the Alliance is one that may be used more broadly. Although the sites did not participate in coordinated Breakthrough Collaboratives, they did engage in peer-to-peer learning and shared best practices with each other, including how to recruit community partners and establish information systems to coordinate care, among other topics.

The Alliance programs achieved changes comparable to other similar studies but without using a fixed implementation strategy. This is important because such an implementation strategy is likely more realistic in real-world settings. For example, a meta-analysis of 23 diabetes interventions between 1990 and 2000 found that glycemic change was 0.320% better in the intervention group than in the control group (Ellis et al., 2004). Chin et al. (2007) found that between 1998 and 2002, health centers undertaking the standard Health Diabetes Collaborative intervention (which draws from the CCM) lowered HbA1c (−0.45%; 95% CI [−0.72, −0.17]) and low-density lipoprotein cholesterol (−19.7 mg/dL; 95% CI [−25.8, −13.6]). More recently, a study by Tang, Funnell, and Oh (2012) found that following a 2-year diabetes self-management support intervention, patients showed significant improvements for using insulin as recommended (p = .047) and achieving diabetes-specific quality of life (p = .02).

The cost estimates compare the costs and health outcomes associated with the Alliance programs to those that would occur in the absence of the Alliance programs. As discussed above, we observed differences in HbA1c levels, blood pressure, and triglyceride levels among participants in the Alliance programs over time. These changes could have occurred entirely as a result of the Alliance programs because the populations served have had limited access to high-quality diabetes care or have been difficult to engage in care for their diabetes. We used this as one optimistic scenario for the cost-effectiveness analyses. We also compared the clinical changes among Alliance participants to the changes observed for a comparison population and used these relative changes as a second, more conservative, scenario in the cost-effectiveness analyses.

Using the $50,000 per QALY “willingness to pay” benchmark for cost-effectiveness, under the optimistic scenario, the care processes in total delivered by the Alliance programs can be considered cost-effective because the estimate cost-effectiveness ratio is well below the benchmark. Under the conservative scenario, the Alliance programs were not cost-effective relative to the $50,000 per QALY benchmark.

It is difficult to demonstrate cost savings in health care programs, policies, or interventions and far more common to increase total costs and improve health outcomes, which is the case with the Alliance programs. This occurs because the costs to run programs, provide additional medications, and pay for additional medical visits are all incurred for large numbers of people and for several years for each person. When combined, these costs almost always are greater than the costs of the disease-specific adverse events avoided. Adverse events—such as blindness, amputations, and kidney failure—occur several years into the future and only occur for a small proportion of all persons with diabetes. Nevertheless, the improvements may be well worth the increased cost, particularly when disparities between vulnerable populations and more economically advantaged groups are reduced. Under our assumptions, the Alliance programs increase costs and improve outcomes. Consequently, like most health interventions, the programs are not cost saving but do result in improved health at a reasonable price.

Several limitations of the current evaluation may affect the conclusions. First, measurement was not standard across programs. Although clinical and patient-reported outcomes were measured before individuals started and after they completed self-management education, the temporal sequencing of subsequent measurement was not the same across sites. Despite this, we were able to detect significant changes over time in these measures. Second, we did not have comparison data on patient-reported outcome measures; consequently, we cannot rule out selection or secular trend bias. Third, the cost-effectiveness analysis is based on a simulation model and therefore relies on all of the assumptions underlying the model. The model assumes that patients will continue to receive the intervention for the rest of their lives, thereby gaining continuing benefits and incurring continuing costs of intervention. The cost-effectiveness analysis also focuses on the health care system perspective and ignores nonmedical costs incurred by patients and their families and communities. Fourth, collinearity issues prevented us from examining race in the models examining changes over time in clinical and patient-reported outcomes or interactions of race with other variables. Finally, we relied on quality indicators and patient reports of clinician-provided RSSM as proxy measures for changes in health care disparities. Although a potential limitation, this approach allowed us to measure changes in health care disparities consistently across sites.

CONCLUSION

Based on the positive evaluation findings, the Alliance programs as a whole reduced health disparities for underserved program participants. Programs were able to improve important clinical indicators related to diabetes management compared with patients who did not receive the program, and brought clinical indicators closer to those of more privileged populations. Patient outcomes indicative of better quality health care increased over time. These positive findings for clinical and patient-reported outcomes complement the findings of other articles in this special issue that also show the benefit of the Alliance programs in reducing disparities and enhancing the health and well-being of program participants.

Acknowledgments

Authors’ Note: We thank the staff across our organizations who helped implement the Alliance to Reduce Disparities in Diabetes programs and to collect and manage the cross-site evaluation data. We also thank the Alliance National Program Office staff for providing technical assistance and support to the sites.

Footnotes

Supplement Note: This article is part of a journal supplement titled “The Alliance to Reduce Disparities in Diabetes: Infusing Policy and System Change With Local Experience.” The supplement was supported by a grant to the Society for Public Health Education from the Merck Foundation. The Merck Foundation had no role in the development, writing, editing, review, or approval of the content of any of the articles in this issue.

REFERENCES

  1. American Diabetes Association Fast facts: Data and statistics about diabetes. 2013a Retrieved from http://professional.diabetes.org/admin/UserFiles/0%20-%20Sean/FastFacts%20March%202013.pdf.
  2. American Diabetes Association Standards of medical care in diabetes—2013. Diabetes Care. 2013b;36(Suppl. 1):S11–S66. doi: 10.2337/dc13-S011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Boyle JP, Honeycutt AA, Venkat Narayan KM, Hoerger TJ, Geiss LS, Chen H, Thompson TJ. Projection of diabetes burden through 2050: Impact of changing demography and disease prevalence in the U.S. Diabetes Care. 2001;24:1936–1940. doi: 10.2337/diacare.24.11.1936. [DOI] [PubMed] [Google Scholar]
  4. Brownson CA, Hoerger TJ, Fisher EB, Kilpatrick KE. Cost-effectiveness of diabetes self-management programs in community primary care settings. Diabetes Educator. 2009;35:761–769. doi: 10.1177/0145721709340931. [DOI] [PubMed] [Google Scholar]
  5. Campbell DT, Stanley JC. Experimental and quasi-experimental designs for research. Rand McNally; Chicago, IL: 1966. [Google Scholar]
  6. The CDC Diabetes Cost-effectiveness Group Cost-effectiveness of intensive glycemic control, intensified hypertension control, and serum cholesterol level reduction for type 2 diabetes. Journal of the American Medical Association. 2002;287:2542–2551. doi: 10.1001/jama.287.19.2542. [DOI] [PubMed] [Google Scholar]
  7. Chin MH, Drum ML, Guillen M, Rimington A, Levie JR, Kirchhoff AC, Schaefer CT. Improving and sustaining diabetes care in community health centers with the health disparities collaboratives. Medical Care. 2007;45:1135–1143. doi: 10.1097/MLR.0b013e31812da80e. [DOI] [PubMed] [Google Scholar]
  8. Clark NM, Brenner J, Johnson P, Peek M, Spoonhunter H, Walton J, Nelson B. Reducing disparities in diabetes: The Alliance model for health care improvements. Diabetes Spectrum. 2011;24:226–230. [Google Scholar]
  9. Coleman K, Austin BT, Brach C, Wagner EH. Evidence on the Chronic Care Model in the new millennium. Health Affairs. 2009;28:75–85. doi: 10.1377/hlthaff.28.1.75. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Collinsworth A, Vulimiri M, Snead C, Walton J. Community health workers in primary care practice: Redesigning health care delivery systems to extend and improve diabetes care in underserved populations. Health Promotion Practice. 2014;15(Suppl. 2):51S–61S. doi: 10.1177/1524839914539961. [DOI] [PubMed] [Google Scholar]
  11. Ellis SE, Speroff T, Dittus RS, Brown A, Pichert JW, Elasy TA. Diabetes patient education: A meta-analysis and meta-regression. Patient Education and Counseling. 2004;52:97–105. doi: 10.1016/s0738-3991(03)00016-8. [DOI] [PubMed] [Google Scholar]
  12. Fisher EB, Brownson CA, O’Toole ML, Shetty G, Anwuri VV, Glasgow RE. Ecological approaches to self-management: The case of diabetes. American Journal of Public Health. 2005;95:1523–1535. doi: 10.2105/AJPH.2005.066084. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: The RE-AIM framework. American Journal of Public Health. 1999;89:1322–1327. doi: 10.2105/ajph.89.9.1322. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Glasgow RE, Wagner EH, Schaefer J, Mahoney LD, Reid RJ, Greene SM. Development and validation of the Patient Assessment of Chronic Illness Care (PACIC) Medical Care. 2005;43:436–444. doi: 10.1097/01.mlr.0000160375.47920.8c. [DOI] [PubMed] [Google Scholar]
  15. Glick HA, Doshi JA, Sonnad SS, Polsky D. Economic evaluation in clinical trials. Oxford University Press; Oxford, England: 2007. [Google Scholar]
  16. Goode T, Jack L. The Alliance to Reduce Disparities in Diabetes: Infusing policy and system change with local experience. Health Promotion Practice. 2014;15(Suppl. 2):6S–10S. doi: 10.1177/1524839914545784. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Grosse SD. Assessing cost-effectiveness in healthcare: History of the $50,000 per QALY threshold. Expert Review of Pharmacoeconomics & Outcomes Research. 2008;8:165–178. doi: 10.1586/14737167.8.2.165. [DOI] [PubMed] [Google Scholar]
  18. Hoerger TJ, Segel JE, Zhang P, Sorensen SW. Validation of the CDC-RTI diabetes cost-effectiveness model (RTI Press Methods Report) RTI International; Research Triangle Park, NC: 2009. [Google Scholar]
  19. Huang ES, Basu A, O’Grady M, Capretta JC. Projecting the future diabetes population size and related costs for the U.S. Diabetes Care. 2009;32:2225–2229. doi: 10.2337/dc09-0459. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Huang ES, Zhang Q, Brown SE, Drum ML, Meltzer DO, Chin MH. The cost-effectiveness of improving diabetes care in U.S. federally qualified community health centers. Health Services Research. 2007;42(6 Pt. 1):2174–2193. doi: 10.1111/j.1475-6773.2007.00734.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Institute for Healthcare Improvement The Breakthrough Series: IHIs collaborative model for achieving breakthrough improvement. Diabetes Spectrum. 2004;17:97–101. [Google Scholar]
  22. Johnson P, Thorman Hartig M, Frazier R, Clayton M, Oliver G, Nelson BW, Williams-Cleaves BJ. Engaging faith-based resources to initiate and support diabetes self-management among African Americans: A collaboration of informal and formal systems of care. Health Promotion Practice. 2014;15(Suppl. 2):71S–82S. doi: 10.1177/1524839914543012. [DOI] [PubMed] [Google Scholar]
  23. Kaplan RS, Porter ME. How to solve the cost crisis in health care. Harvard Business Review. 2011;89(9):46–64. [PubMed] [Google Scholar]
  24. Kaufman S, Ali N, DeFiglio V, Craig K, Brenner J. Early efforts to target and enroll high-risk diabetic patients into urban community-based programs. Health Promotion Practice. 2014;15(Suppl. 2):62S–70S. doi: 10.1177/1524839914535776. [DOI] [PubMed] [Google Scholar]
  25. Kazis LE, Miller DR, Skinner KM, Lee A, Ren XS, Clark JA, Fincke BG. Applications of methodologies of the Veterans Health Study in the VA health care system: Conclusions and summary. Journal of Ambulatory Care Management. 2006;29:182–188. doi: 10.1097/00004479-200604000-00011. [DOI] [PubMed] [Google Scholar]
  26. Landon BE, Hicks LS, O’Malley AJ, Lieu TA, Keegan T, McNeil BJ, Guadagnoli E. Improving the management of chronic disease at community health centers. New England Journal of Medicine. 2007;356:921–934. doi: 10.1056/NEJMsa062860. [DOI] [PubMed] [Google Scholar]
  27. Langwell K, Keene C, Zullo M, Ogu LC. An American Indian community implements the chronic care model: Evolution and lessons learned. Health Promotion Practice. 2014;15(Suppl. 2):23S–28S. doi: 10.1177/1524839914544171. [DOI] [PubMed] [Google Scholar]
  28. Lewis MA, Williams PA, Fitzgerald TM, Heminger CL, Hobbs CL, Moultrie RR, Kamerow DB. Improving the implementation of diabetes self-management: Findings from the Alliance to Reduce Disparities in Diabetes. Health Promotion Practice. 2014;15(Suppl. 2):83S–91S. doi: 10.1177/1524839914541277. [DOI] [PubMed] [Google Scholar]
  29. McCormack LA, Williams-Piehota PA, Bann CM, Burton J, Kamerow DB, Squire C, Glasgow RE. Development and validation of an instrument to measure resources and support for chronic illness self-management: A model using diabetes. Diabetes Educator. 2008;34:707–718. doi: 10.1177/0145721708321021. [DOI] [PubMed] [Google Scholar]
  30. Munoz M, Pronovost P, Dintzis J, Kemmerer T, Wang N-Y, Chang Y-T, Golden SH. Implementing and evaluating a multicomponent inpatient diabetes management program: Putting research into practice. Joint Commission Journal on Quality and Patient Safety. 2012;38:195–206. doi: 10.1016/s1553-7250(12)38025-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. National Committee for Quality Assurance . The state of health care quality. Author; Washington, DC: 2013. Improving quality and patient experience. [Google Scholar]
  32. Nyman MA, Murphy ME, Schryver PG, Naessens JM, Smith SA. Improving performance in diabetes care: A multicomponent intervention. Effective Clinical Practice. 2000;3:205–212. [PubMed] [Google Scholar]
  33. Rogers EM. Diffusion of innovations. 5th ed Free Press; New York, NY: 2003. [Google Scholar]
  34. Shah NR, Aragones A, Schaefer EW, Stevens D, Gourevitch MN, Glasgow RE. Validation of the Spanish translation of the patient assessment of chronic illness care (PACIC) Preventing Chronic Dis. 2008;5(4):A113. [PMC free article] [PubMed] [Google Scholar]
  35. Tang TS, Funnell MM, Oh M. Lasting effects of a 2-year diabetes self-management support intervention: Outcomes at 1-Year Follow-Up. Preventing Chronic Disease. 2012;9:110313. doi: 10.5888/pcd9.110313. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Toobert DJ, Glasgow RE. Assessing diabetes self-management: The Summary of Diabetes Self-Care Activities Questionnaire. In: Bradley C, editor. Handbook of psychology and diabetes. Harwood Academic; Chur, Switzerland: 1994. pp. 351–375. [Google Scholar]
  37. Toobert DJ, Hampson SE, Glasgow RE. The Summary of Diabetes Self-Care Activities Measure: Results from 7 studies and a revised scale. Diabetes Care. 2000;23:943–950. doi: 10.2337/diacare.23.7.943. [DOI] [PubMed] [Google Scholar]
  38. Williams GC, Freedman ZR, Deci EL. Supporting autonomy to motivate patients with diabetes for glucose control. Diabetes Care. 1998;21:1644–1651. doi: 10.2337/diacare.21.10.1644. [DOI] [PubMed] [Google Scholar]
  39. Williams GC, McGregor HA, Zeldman A, Freedman ZR, Deci EL. Testing a self-determination theory process model for promoting glycemic control through diabetes self-management. Health Psychology. 2004;23:58–66. doi: 10.1037/0278-6133.23.1.58. [DOI] [PubMed] [Google Scholar]
  40. Woolf SH. A closer look at the economic argument for disease prevention. Jamaican Nurse. 2009;301:536–538. doi: 10.1001/jama.2009.51. [DOI] [PubMed] [Google Scholar]

RESOURCES