Clinic-Based Versus Outsourced Implementation of a Diabetes Health Literacy Intervention

ABSTRACT

BACKGROUND

We compared two implementation approaches for a health literacy diabetes intervention designed for community health centers.

METHODS

A quasi-experimental, clinic-randomized evaluation was conducted at six community health centers from rural, suburban, and urban locations in Missouri between August 2008 and January 2010. In all, 486 adult patients with type 2 diabetes mellitus participated. Clinics were set up to implement either: 1) a clinic-based approach that involved practice re-design to routinely provide brief diabetes education and counseling services, set action-plans, and perform follow-up without additional financial resources [CARVE-IN]; or 2) an outsourced approach where clinics referred patients to a telephone-based diabetes educator for the same services [CARVE-OUT]. The fidelity of each intervention was determined by the number of contacts with patients, self-report of services received, and patient satisfaction. Intervention effectiveness was investigated by assessing patient knowledge, self-efficacy, health behaviors, and clinical outcomes.

RESULTS

Carve-out patients received on average 4.3 contacts (SD = 2.2) from the telephone-based diabetes educator versus 1.7 contacts (SD = 2.0) from the clinic nurse in the carve-in arm (p < 0.001). They were also more likely to recall setting action plans and rated the process more positively than carve-in patients (p < 0.001). Few differences in diabetes knowledge, self-efficacy, or health behaviors were found between the two approaches. However, clinical outcomes did vary in multivariable analyses; carve-out patients had a lower HbA1c (β = −0.31, 95 % CI −0.56 to −0.06, p = 0.02), systolic blood pressure (β = −3.65, 95 % CI −6.39 to −0.90, p = 0.01), and low-density lipoprotein (LDL) cholesterol (β = −7.96, 95 % CI −10.08 to −5.83, p < 0.001) at 6 months.

CONCLUSION

An outsourced diabetes education and counseling approach for community health centers appears more feasible than clinic-based models. Patients receiving the carve-out strategy also demonstrated better clinical outcomes compared to those receiving the carve-in approach. Study limitations and unclear causal mechanisms explaining change in patient behavior suggest that further research is needed.

Proper diabetes self-care requires patients to have considerable knowledge, a range of skills, and to sustain multiple health behaviors.^1–3 Not surprisingly, limited literacy has been linked to inadequate treatment knowledge, poorer self-care and glycemic control.^4–8 Interventions that have been designed for use among individuals with lower literacy skills are needed; some approaches have been developed and evaluated, with promising results.^9–11 Yet, questions remain on how best to implement these interventions in the most effective, efficient, and sustainable manner. This is critical for patients receiving primary care in community health centers that have limited resources and disproportionately care for patients with limited literacy.¹²

A national team, led by the American College of Physicians Foundation, developed a patient-centered, low literacy intervention promoting diabetes self-care.^13–15 It includes 1) a Diabetes Guide that uses plain language and descriptive photographs to teach core diabetes concepts and empower patients to initiate behavior change; 2) a brief counseling strategy to assist patients in developing short-term, explicit and attainable goals for behavior change (‘action plans’); 3) a training module for healthcare providers (physicians, nurses, medical assistants) that prepares them to assume educator and counselor roles using the Diabetes Guide as a teaching tool; and 4) electronic tracking and monitoring tools for primary care practices. A prior efficacy trial found that the Diabetes Guide improved patient knowledge, self-efficacy, intent to adopt recommended behaviors, and reduced diabetes-related distress.^14,15

Yet, optimal implementation of this multifaceted education and counseling strategy, particularly as a low-cost intervention for community clinics serving vulnerable populations, is unknown. This is true of many promising diabetes behavioral interventions that unfortunately have had limited effectiveness due to problems with ‘fidelity’.^16–19 Therefore, we sought a better understanding of the benefits and limitations of two potential implementation options for the Diabetes Guide and counseling intervention: 1) a clinic-based approach, wherein an external resource team helps a practice utilize existing resources to create a tailored protocol for targeting eligible patients, introducing and distributing the Diabetes Guide, reviewing content, facilitating action plans and follow-up activities [CARVE-IN]; or 2) an outsourced approach that requires clinics to identify eligible patients and refer them to a centralized, telephone-based diabetes educator, who then contacts patients, reviews the Diabetes Guide with them, and facilitates action plans and follow-up [CARVE-OUT]. Both approaches reflect current models for chronic disease self-management support.^20,21 The former re-deploys existing resources in physicians’ offices without sustained financial support, while the latter contracts with an external entity at a cost to provide patients with services, usually by phone. These strategies are not always exclusive of one another, but are often treated as such in practice. Our investigation specifically addressed whether implementing the Diabetes Guide and counseling intervention via carve-in or carve-out approach might be more feasible for vulnerable populations cared for in community health centers.

METHODS

Study Design

The present study is derived from a larger research investigation comparing carve-in and carve-out approaches to an enhanced usual care arm; a parallel manuscript reports on the effectiveness of each intervention to improve diabetes knowledge, health behaviors, and clinical outcomes compared to an ‘enhanced care’ arm.²² While the original intent of the study was to compare both strategies to usual care, an unprecedented amount of financial resources were made available to control clinics from a state foundation prior to the evaluation. While little guidance was offered to the community health centers on how to use these funds, new services became available to patients in these clinics that included nutritionists, diabetes educators, and group self-management classes. Unlike the carve-in and carve-out strategies that were meant to be standard and sustainable by cost, enhanced care interventions were variable between and within clinics, and cost was less of an issue (as such, these services were eventually discontinued). Overall, the main study results found that both implementation strategies and enhanced care benefitted patients. The specific objective of this study was to compare the two implementation methods and best understand the advantages and challenges of each; analyses were therefore limited to patients in the carve-in and carve-out arms only. As health literacy interventions particularly attempting to reach lower literate patients, we also sought to examine whether either the carve-in or carve-out approach had a disproportionate benefit for these individuals.

Six community health centers in three cities (two per city) in Missouri (St. Louis, Columbia, Kirksville) were enlisted, representing urban, suburban, and rural settings. Kirksville and St. Louis clinics were Federally Qualified Health Centers (FQHCs); in Columbia, university-affiliated family practice clinics serving similar vulnerable groups participated. Within each location, one clinic was randomized to either carve-in or carve-out approach. We refer to our study design as quasi-experimental and disclose two limitations. Due to the diffuse nature of interventions, it was not possible to randomize by patient. Attempts were made to find comparable clinics by size, patient demographics, number of physicians, and clinic staff. Yet, the availability of eligible community health centers was limited, especially in non-urban settings. Second, for exploratory analyses, we relied on routinely collected clinical outcomes data. Due to challenging psychosocial and economic situations of patients, we observed variable follow-up.

Participants

Adults with a chart diagnosis of type 2 diabetes mellitus at participating clinics were potentially eligible for the study. Additional criteria included: 1) ≥ 30 years of age; 2) Hba1c > 6.5 %; 3) no hearing, vision, or cognitive impairment. Recruitment occurred between August 2008 and January 2010; staff at each site worked with research assistants to initiate contact with potential participants and obtain signed consent. Participation rates at all sites were > 90 %. The Northwestern University Institutional Review Board, University of Missouri-Columbia, and an independent review board (Copernicus, Raleigh, NC) approved study procedures. In all, 486 participants enrolled in the two intervention arms.

Interventions

Carve-in and carve-out strategies delivered the same Diabetes Guide and accompanying brief counseling protocol. The Guide is a colorful 48-page booklet written on a fifth grade level that incorporates health literacy practices such as plain language and descriptive photographs to teach diabetes self-care concepts with an emphasis on actionable steps for behavior change.^13,23,24 The brief counseling strategy includes introduction to the Diabetes Guide, review of self-management domains, and facilitation of short-term, highly achievable and concrete action plans. This is an iterative process wherein patients identify behavioral goals they have confidence in completing. The purpose is to engage patients in the behavior change process by having them set personally-defined goals that they perceive as easily attainable, increasing their confidence in making subsequent behavior changes.

Carve-In Strategy

Medical directors, physicians, nurses, and medical assistants at clinics assigned to the carve-in strategy received an extensive orientation to the Diabetes Guide and an approach to: 1) identify patients, 2) introduce and review the Diabetes Guide, 3) briefly counsel patients and establish action plans, and 4) set up a tracking system to follow up with patients via telephone at 2 weeks and 2 months, and via telephone or in-person at 3, 6, and 9 months post-baseline enrollment. During orientation, a diabetes ‘champion’ was identified to work with the research team, deconstruct tasks and assign responsibilities to clinic staff.

A one-day training was conducted in St. Louis with clinic staff (all nurses) responsible for patient engagement activities. Members of the study team (HS, TD, MW) reviewed the Diabetes Guide and taught counseling skills to clinic staff. Counseling techniques incorporated ‘teach back’, positive encouragement, and probing skills to identify goals and coach patients to create action plans. Clinic staff were given a semi-structured script they were to follow, helping to standardize patient counseling encounters. They recorded action plans on electronic forms created using Snap® web survey software (Portsmouth, NH). Interactive educational technique, simulations of counseling sessions, and hands-on training with tracking and action plan systems were performed. None of the carve-in clinics received financial support to sustain these roles.

Carve-Out Strategy

Carve-out clinic staff were briefly oriented to the purpose of the trial. They were asked to distribute the Diabetes Guide to eligible patients, briefly review it and then refer them to a diabetes educator who would contact them by telephone. A trained research staff member in Chicago was provided with scripted text (same as carve-in) to follow for brief counseling. The same training methods and patient engagement activities described for the carve-in arm, including 2-week and 2-, 3-, 6-, and 9-month follow-up contacts, only performed by the diabetes educator via telephone. Feedback from those delivering the brief counseling interventions reported that the average time with patients for the first contact was approximately 10–15 min, while follow-up contacts varied from 5 to 20 min, depending on the individual and nature of set goals.

Structured Interview & Outcomes

In-person interviews were conducted by a trained research assistant at baseline and 12 months, with a follow-up telephone interview at 3 months. The battery included: 1) a diabetes knowledge questionnaire developed and tailored to the general educational content provided in the Diabetes Guide,¹⁴ 2) an 8-item diabetes self-efficacy assessment by Sarkar and colleagues,²⁵ 3) self-report items of recommended physical activity, vegetable and fruit intake from the Behavioral Risk Factors Surveillance System (BRFSS),²⁶ and 4) self-report of medication adherence using the Morisky scale.²⁷ Socio-demographic information was collected, along with the Rapid Estimate of Adult Literacy in Medicine (REALM).²⁸

The primary outcome was intervention fidelity, operationalized as the number of contacts made with patients for counseling and action plan follow-up (six possible contacts). At the end of the 12-month interview, patients were asked four questions about their recall of receiving a contact, if they set action plans, if the intervention was helpful, and if they would like the services to continue. Secondary analyses examined improvement in diabetes knowledge, self-efficacy, and behaviors. For exploratory analysis, hemoglobin A1c (HbA1c), systolic/diastolic blood pressure, and low-density lipoprotein (LDL) cholesterol values were obtained from medical charts within 6 months prior to baseline assessment, and again between 6 and 12 months post-baseline. The average number of days separating baseline and follow-up clinical values did not differ between arms (carve-in: M = 225.1 days (SD = 54.0); carve-out: M = 223.7 days (SD = 58.0), p = 0.85).

Analysis Plan

Statistical analyses were performed using STATA version 10.0 (College Station, TX). Patient-level characteristics and baseline outcomes were compared by arm using chi-square and Student’s t tests to ensure adequate randomization. For intervention fidelity, both the total number of contacts by either the clinic nurse or telephone diabetes educator and rate of successful contacts per follow-up (baseline, 2 weeks, and 2, 3, 6 and 9 months) were similarly investigated. Generalized linear models and clustering by site were used to compare the two interventions, with study arm as the primary covariate of interest. Relevant covariates (age, gender, race) were also entered into models; for diabetes knowledge, self-efficacy, health behavior and clinical outcomes baseline values were also included. Patient literacy was entered in models as a specific variable of interest. An interaction term was generated to determine whether one approach better engaged patients with limited literacy.

RESULTS

Tables 1 and 2 provide baseline sociodemographic and clinical characteristics of the sample. Patients in the carve-out arm were more likely to be African American (27.6 % vs. 12.6 %, p < 0.001) and more recently diagnosed with diabetes (32.8 % vs. 44.4 % living with disease >10 years, p = 0.04). Carve-out patients also scored higher on diabetes knowledge (M = 5.4, SD = 1.9 vs. M = 5.0, SD = 1.9, p = 0.02), and had better glycemic control (M = 7.7 %, SD = 1.8 vs. M = 8.3 %, SD = 1.7, p < 0.001), and lower systolic blood pressure (M = 133.1, SD = 15.4 vs. M = 136.7, SD = 16.6, p = 0.02).

Table 1.

Patient Characteristics by Study Arm

Characteristic	All (N = 486)	Carve-In (n = 214)	Carve-Out (n = 272)	P value
Age, mean (SD)	54.8 (11.0)	54.5 (11.4)	55.0 (10.8)	0.62
Female, %	60.9	59.4	62.1	0.53
African American, %	21.0	12.6	27.6	< 0.001
Limited Literacy, %	31.9	28.5	34.6	0.16
Education, %				0.83
Less than HS	24.9	25.7	24.3
HS Grad/GED	33.9	35.5	32.7
Some College	27.8	26.2	29.0
College grad	13.4	12.6	14.0
Income, %				0.20
< $10,000	27.9	31.4	25.1
$10,000–$14,999	27.5	24.6	29.7
$15,000–$24,999	16.6	14.0	18.6
> $25,000	28.1	30.0	26.6
Work status, %				0.68
Full-time	28.1	26.3	29.5
Part-time	12.2	13.1	11.4
Not working	59.7	60.6	59.1
Years with Diabetes, %				0.04
Less than 1	4.5	5.1	4.1
1–4	29.7	24.8	33.6
5–9	27.8	25.7	29.5
10 or more	37.9	44.4	32.8
Living situation				0.21
Own home	52.1	56.5	48.5
Rent	36.4	32.7	39.3
Live with relative/friend	11.5	10.8	12.1
Number of chronic conditions, mean (SD)	3.3 (1.5)	3.3 (1.5)	3.3 (1.4)	0.70
Lubben Social Network Scale, % isolated	22.9	22.9	22.9	0.99

Table 2.

Baseline Diabetes-Related Outcomes, by Study Arm

Outcome	All (N = 486)	Carve-in (n = 214)	Carve-Out (n = 272)	P Value
Diabetes knowledge (0–8 score), mean (SD)	5.2 (1.9)	5.0 (1.9)	5.4 (1.9)	0.02
Self-efficacy (0–100 score), mean (SD)	82.7 (15.9)	82.2 (16.8)	83.2 (15.2)	0.49
Health behaviors
Recommended physical activity, %	35.6	37.4	34.3	0.48
Recommended fruit intake, %	14.1	14.6	13.7	0.78
Recommended vegetable intake, %	17.3	17.8	17.0	0.82
High medication adherence, %	35.6	35.4	35.7	0.94
HbA1C, mean (SD)	8.0 (1.8)	8.3 (1.7)	7.7 (1.8)	< 0.001
Systolic blood pressure, mean (SD)	134.8 (16.1)	136.7 (16.6)	133.1 (15.4)	0.02
Diastolic blood pressure, mean (SD)	79.1 (9.0)	79.5 (8.1)	78.8 (9.8)	0.40
LDL cholesterol, mean (SD)	96.9 (38.3)	95.2 (35.1)	98.2 (40.5)	0.49

Intervention Fidelity and Satisfaction

A gradient decline in intervention contacts was noted in both arms, from baseline to 9 months (Fig. 1). However, carve-out patients received more than twice as many contacts overall (M = 4.3, SD = 2.2 vs. M = 1.7, SD = 2.0, p < 0.001, Table 3). At 12 months, carve-out patients were more likely to recall having been contacted by the diabetes educator than carve-in patients with their clinic nurse (78.7 % vs. 45.7 %, p < 0.001) and to recall having set action plans (68.4 % vs. 31.7 %, p < 0.001). They also perceived the intervention to be more helpful (M = 6.6, SD = 0.3 vs. M = 3.5, SD = 0.3, p < 0.001), and were more interested in continuing to receive services (75.5 % vs. 62.2 %, p = 0.024). In multivariable analyses, these relationships were maintained.

Figure 1.

Completed intervention contacts by study arm.

Table 3.

Intervention Fidelity and Patient Satisfaction with Carve-In and Carve-Out Approaches

Outcome		Carve-In	Carve-Out	P Value
Number of Total Contacts	M (SD)	1.7 (2.0)	4.3 (2.2)	< 0.001
	β (95 % CI)	—	2.60 (1.11–4.09)	0.001
Over the past 12 months, has a nurse/diabetes educator spoken with you about creating an action plan or setting a goal to improve your diabetes?	%	45.7	78.7	< 0.001
	OR (95 % CI)	—	4.73 (2.37–9.45)	< 0.001
Over the past 12 months, did you set an action plan or a goal with the nurse/diabetes educator who contacted you?	%	31.7	68.4	< 0.001
	OR (95 % CI)	—	5.21 (2.13–12.78)	< 0.001
On a scale of 1 to 10, one being not helpful at all and 10 being extremely helpful, how helpful was this process of setting action plans to improving your health?	M (SD)	3.5 (0.3)	6.6 (0.3)	< 0.001
	β (95 % CI)	—	3.04 (2.20–3.89)	< 0.001
If given the opportunity, would you like to continue to set action plans with the nurse/diabetes educator?	%	62.2	75.5	0.02
	OR (95 % CI)	—	1.72 (1.02–2.89)	0.04

There were significant interactions between study arm and literacy level for all implementation outcomes. Carve-in patients with limited literacy received more nurse contacts (β = −1.02, 95 % CI −1.78 to −0.26, p = .01), better recalled the nurse speaking to them (OR = 0.26, 95 % CI 0.13–0.52, p < 0.001) and setting action plans (OR = 0.27, 95 % CI 0.13 to 0.53, p < 0.001), and rated the process as more helpful than those with adequate literacy (β = 1.32, 95 % CI −2.51 to −.14, p = .03); the reverse was true in the carve-out arm (Table 5). Carve-out patients with limited literacy were more likely than those with adequate literacy to want to continue receiving the intervention (OR = 1.50, 95 % CI 1.25 to 1.79, p < 0.001).

Table 5.

Adjusted Estimates for Outcomes with Significant Literacy Level-Study Arm Interactions

Outcome	Literacy Level	Carve-In	Carve-Out
Intervention Fidelity and Patient Satisfaction
Number of Total Contacts, M (SD)	Limited	2.40 (0.19)	4.29 (0.22)
	Adequate	1.49 (0.22)	4.40 (0.22)
Over the past 12 months, has a nurse/diabetes educator spoken with you about creating an action plan or setting a goal to improve your diabetes?, %	Limited	54.6 %	69.6 %
	Adequate	42.7 %	84.3 %
Over the past 12 months, did you set an action plan or a goal with the nurse/diabetes educator who contacted you?, %	Limited	48.6 %	66.8 %
	Adequate	36.7 %	82.1 %
On a scale of 1 to 10, one being not helpful at all and 10 being extremely helpful, how helpful was this process of setting action plans to improving your health?, M (SD)	Limited	4.08 (0.35)	6.34 (0.38)
	Adequate	3.32 (0.38)	6.80 (0.38)
If given the opportunity, would you like to continue to set action plans with the nurse/diabetes educator?, %	Limited	56.4 %	76.1 %
	Adequate	62.6 %	73.1 %
Knowledge
Diabetes Knowledge at 12 months, M (SD)	Limited	4.26 (0.66)	4.74 (0.87)
	Adequate	5.99 (0.78)	5.89 (0.76)

Knowledge, Self-Efficacy, and Health Behaviors

Carve-in patients improved their knowledge at 3 months compared to those receiving the carve-out intervention (β = −0.17, 95 % Confidence Interval (CI) −0.20 to −0.15, p < 0.001; Table 4), although no differences were noted in self-efficacy, physical activity, nutrition, or medication adherence. By 12 months, carve-out patients had greater diabetes knowledge compared to carve-in patients (β = 0.16, 95 % CI 0.03 to 0.29, p = 0.01), but were less likely to meet physical activity and fruit intake recommendations (Odds Ratio [OR] = 0.74, 95 % CI 0.68 to 0.81, p < 0.001; OR = 0.47, 95 % CI 0.23 to 0.99, p = 0.05).

Table 4.

Effectiveness of Carve-In Versus Carve-Out Implementation Approaches

Outcome			Carve-In	Carve-Out	P Value
Diabetes Knowledge	3 months	M (SD)	5.5 (1.7)	5.5 (1.6)	0.90
		β (95 % CI)	—	−0.17 (−0.20–−0.15)	< 0.001
	12 months	M (SD)	5.4 (1.7)	5.4 (1.5)	0.92
		β (95 % CI)	—	0.16 (0.03–0.29)	0.01
Diabetes Self-Efficacy	3 months	M (SD)	82.2 (13.2)	87.8 (12.8)	0.66
		β (95 % CI)	—	−1.08 (−2.55–0.39)	0.15
	12 months	M (SD)	89.9 (12.2)	87.9 (13.8)	0.17
		β (95 % CI)	—	−3.14 (−6.41–0.12)	0.06
Health Behaviors
Physical Activity	3 months	%	40.2	43.7	0.48
		OR (95 % CI)	—	1.04 (0.54–2.01)	0.90
	12 months	%	50.0	44.9	0.36
		OR (95 % CI)	—	0.74 (0.68–0.81)	< 0.001
Fruit Intake	3 months	%	15.1	14.4	0.84
		OR (95 % CI)	—	1.38 (0.65–2.92)	0.41
	12 months	%	26.0	14.4	0.01
		OR (95 % CI)	—	0.47 (0.23–0.99)	0.05
Vegetable Intake	3 months	%	21.6	19.8	0.66
		OR (95 % CI)	—	0.99 (0.62–1.58)	0.96
	12 months	%	17.7	15.6	0.60
		OR (95 % CI)	—	1.34 (0.78–2.29)	0.29
High Rx Adherence	3 months	%	45.4	46.0	0.92
		OR (95 % CI)	—	0.75 (0.57–0.99)	0.04
	12 months	%	50.0	45.2	0.39
		OR (95 % CI)	—	1.18 (0.86–1.62)	0.30
Intermediary Clinical Outcomes
HbA1c	6 months	M (SD)	8.2 (1.6)	7.4 (1.5)	< 0.001
		β (95 % CI)	—	−0.31 (−0.56–−0.06)	0.02
Systolic Blood Pressure	6 months	M (SD)	136.6 (19.7)	132.6 (18.0)	0.08
		β (95 % CI)	—	−3.65 (−6.39–−0.90)	0.01
Diastolic Blood Pressure	6 months	M (SD)	78.9 (9.1)	78.7 (10.6)	0.86
		β (95 % CI)	—	−1.57 (−7.17–4.02)	0.58
LDL Cholesterol	6 months	M (SD)	95.3 (37.5)	87.0 (31.4)	0.11
		β (95 % CI)	—	−7.96 (−10.08–−5.83)	< 0.001

Bolded values reflect significant associations at p < 0.05

Limited literacy was independently associated with poorer knowledge (β = −0.20, 95 % CI −0.38 to −0.02, p = 0.03) and self-efficacy (β = −0.58, 95 % CI −0.81 to −0.35, p = 0.02) at 3 months, yet a greater likelihood of meeting recommended physical activity goals (OR = 1.33, 95 % CI 1.30 to 1.37, p < 0.001). Among those with limited literacy, carve-out patients had greater diabetes knowledge at 12 months than carve-in patients (M = 4.74, SD = 0.87 vs. M = 4.26, SD = 0.66, p < 0.001, Table 5), while no differences by arm were found among patients with adequate literacy.

Clinical Outcomes

There were no differences found between patients with and without available clinical data by any patient-level characteristic or by study arm. Among those with chart values, carve-out patients had a lower HbA1c (β = −0.31, 95 % CI −0.56 to −0.06, p = 0.02), systolic blood pressure (β = −3.65, 95 % CI −6.39 to −0.90, p = 0.01), and LDL cholesterol (β = −7.96, 95 % CI −10.08 to −5.83, p < 0.001, Table 5) compared to carve-in patients at follow-up. Baseline values were significant independent predictors for all outcomes.

DISCUSSION

The carve-out model was a more reliable approach for contacting patients over time to initiate action plans compared to the carve-in arm. This was backed by patients’ own recall of activities, and their greater satisfaction with and desire to continue to receive services in the carve-out arm. Regardless of literacy level, patients receiving the carve-out intervention had consistently greater recall and ratings of the helpfulness of services. While delivered at a cost, this approach resulted in a more dependable, feasible, and accepted intervention.

Interestingly, this did not translate to greater improvement in diabetes knowledge, self-efficacy, or health behaviors compared to carve-in patients. However, carve-out patients did demonstrate better glycemic control, lower blood pressure and cholesterol values after 6 months; these improvements though could be attributed to differences between study arms at baseline. Despite attempts to recruit comparable community health centers, our clustered study design limits our ability to fully explain these clinical benefits, especially since there were no significant differences between implementation methods across knowledge, self-efficacy or health behaviors that would represent logical causal mechanisms. Our exploratory analyses did provide some evidence to suggest that the carve-out approach better helped patients who had not yet achieved tight glycemic control to reach that goal, and for those who were already controlled to stay well-managed.

The ‘voltage drops’ over time seen in both the carve-in and carve-out arms infer the difficulty of implementing and sustaining either strategy. Prior studies performed in community health centers also reported similar issues.^29,30 This is understandable for the carve-in arm; even with considerable guidance at the onset, no additional financial resources were provided. Lessons learned from the Massachusetts League of Community Health Centers found that to sustain diabetes self-management activities, greater technical support and assistance were needed to help staff assume new roles.³¹ While patients in the carve-out arm received greater follow-up, the annual cost associated with the outsourced diabetes educator (∼$47,000) is not trivial to a community health center. However, given the disproportionate benefits in satisfaction and understanding by those with limited literacy, a network of community health centers might consider sharing the cost in order to better engage those most vulnerable within their practices. With the onset of patient-centered medical homes, newer models of training and embedding diabetes educators are available, and may offer opportunity for delivering self-management support to patients.^32,33

This study has several limitations. Clinics and not patients were randomized and baseline differences were noted between arms that were accounted for in statistical analyses. As we targeted community health centers that serve vulnerable populations, some diversity can be expected as a result of unmeasured differences in quality of care, resources, and patient needs. Further, our analyses of clinical outcomes relied on existing chart values and some patients in both study arms were therefore eliminated from analyses. Quality assurance analyses confirmed there were no differences between patients with or without clinical data. Finally, measurement error might have contributed to non-significant associations in health behaviors. For example, pharmacy claims data were unavailable, and instead we assessed medication adherence via the Morisky scale. Prior studies found self-report to be less accurate.³⁴

Our evaluation underscores the challenge of ‘hardwiring’ health literacy interventions in primary care and the need for robust implementation methods in community health center settings.^35,36 The difficulty demonstrated by clinics in following up patients in the carve-in arm may not be easily overcome without financial support. Yet the greater fidelity of the outsourced intervention has a questionable cost-benefit. More frequent patient engagement and higher receptivity to services could have contributed to clinical improvements, but it is not fully understood how. Recent large-scale evaluations of commercial, outsourced disease management programs found typical telephone counseling services had little impact.^37,38 Yet, several factors may determine effectiveness of an outsourced approach, including who makes the contact, if the service is live or automated, and length, nature and frequency of contacts.^38–41 It is unclear from our evaluation what explicit attributes of the carve-out method were preferable for patients, and whether these features can explain the better clinical outcomes compared to the carve-in approach. These questions are worth further investigation.

The challenge ahead is to find reliable, effective outreach methods that are not cost-prohibitive. Outsourced implementation models, in particular, must address concerns raised by Coleman et al. that current commercial forms of care management run parallel to, but disconnected from, clinical care.⁴² Any self-management support program should be adequately integrated within community health centers to inform clinical decision making. Electronic health records and mobile consumer technologies may aid these efforts.^43,44 From a literacy perspective, using technology for action plans or other engagement activities would increase the proficiency required by patients to assume self-care roles. The result might widen literacy disparities in knowledge, behavior, and outcomes. The acceptability, usability, and satisfaction of any new tools or processes, from a patient perspective, should be investigated among patients with varying literacy levels.^45,46 As new multifaceted interventions are designed, in-depth evaluations need to investigate summative and process outcomes to identify true catalysts to any identified benefits. This will offer health systems and policymakers the evidence needed to endorse and disseminate promising approaches.