A Practical Randomized Trial to Improve Diabetes Care

Abstract

OBJECTIVE

There is a well-documented gap between diabetes care guidelines and the services received by patients in almost all health care settings. This project reports initial results from a computer-assisted, patient-centered intervention to improve the level of recommended services received by patients from a wide variety of primary care providers.

DESIGN AND SETTINGS

Eight hundred eighty-six patients with type 2 diabetes under the care of 52 primary care physicians participated in the Diabetes Priority Program. Physicians were stratified and randomized to intervention or control conditions and evaluated on 2 primary outcomes: number of recommended laboratory screenings and recommended patient-centered care activities completed. Secondary outcomes were evaluated using the Problem Areas in Diabetes scale and the Patient Health Questionnaire (PHQ)-9 depression scale, and the RE-AIM framework was used to evaluate potential for dissemination.

RESULTS

The program was well-implemented and significantly improved both number of recommended laboratory assays (3.4 vs 3.1; P < .001) and patient-centered aspects of diabetes care patients received (3.6 vs 3.2; P < .001) compared to those in randomized control practices. Activities that were increased most were foot exams (follow-up rates of 80% vs 52%; P < .003) and nutrition counseling (76% vs 52%; P < .001).

CONCLUSIONS

Patients are very willing to participate in a brief computer-assisted intervention that is effective in enhancing quality of diabetes care. Staff in primary care offices can consistently deliver an intervention of this nature, but most physicians were unwilling to participate in this translation research study.

There is a well-documented gap between “best practices” demonstrated in randomized trials to improve outcomes and the care delivered in almost all primary care settings for diabetes and other chronic illnesses.^1–4 There have also been numerous well-designed attempts to improve delivery of preventive services, but few have proven to be broadly applicable or successful.^5,6 Major barriers to success are the numerous competing demands placed upon primary care offices and the very limited amount of time available.^6,7 To increase chances of adoption and success, an intervention should be brief, fit into the flow of patient visits, not increase the time demands on physician time, and inform the patient-provider interaction.^8–10

One avenue to help close the gap between research and practice is to conduct and report more “practical clinical trials.”¹¹ Such evaluations select clinically relevant interventions, include a diverse sample of patients recruited from heterogeneous practices, and collect data on a broad range of outcomes.¹¹ The goal of such trials is to provide data on representative samples and on outcomes that are relevant to clinicians and policy makers.

We hypothesize that interactive technology can be used to help both patients and providers to enhance their communication and improve quality of care.^12,13 Interactive technology can consistently collect and provide both parties with immediate feedback and personally tailored information and recommendations.^12–14

The primary purpose of this study was to evaluate a CD ROM-assisted diabetes care enhancement program, the Diabetes Priority Program, relative to a stringent randomized control condition on its effectiveness in improving both laboratory assay and more patient-centered aspects of care recommended by the National Committee on Quality Assurance/American Diabetes Association (NCQA/ADA) Provider Recognition Program.^15,16 Secondary goals were to evaluate the impact of the Diabetes Priority Program on quality of life and depressive symptoms; use the RE-AIM evaluation model^9,17 to assess the rates of reach, effectiveness, adoption, and implementation of the program; and determine whether the program was differentially effective for different types of patients.

METHODS

Setting and Participants

The Diabetes Priority Program was a partnership among our research team, the Copic Insurance Company (which insures over 95% of independent primary care physicians in Colorado), and participating primary practices. We initially sent a brief survey to all 1,258 family physicians and general internists insured by Copic, of whom 84% returned a usable survey. We then sent a letter and fact sheet to 1,059 responding physicians soliciting their participation.

Once a physician agreed to participate, a standard protocol was used to generate diabetes patient lists and recruit patients. The protocol detailed a review of patient billing data for the previous year, specified diagnostic codes, and the need to search all diagnoses for each visit. Adults identified as having diabetes were sent a letter signed by their primary care physician inviting them to participate, a brochure describing the project, and a return postcard to return if they did not want to participate (Fig. 1). Age greater than or equal to 25, ability to read English, and type 2 diabetes (confirmed using the Welborn criteria¹⁸) were the only inclusion criteria in an effort to make the project as broadly applicable as possible. If a reply card was not received, patients were called in approximately 2 weeks, screened for eligibility, invited to participate, and mailed an informed consent form. All procedures were approved by relevant Institutional Review Boards. Patients were recruited during 2001 to 2002.

FIGURE 1.

Modified CONSORT. Figure of participant flow.

Design and Analyses

We employed a two-group, cluster randomized control design. Participating physicians were stratified by size of practice and urban/rural setting as these factors were judged likely to impact results, and then randomized (Table 1). Randomization was conducted by the project statistician, who then notified research staff of condition assignment. To account for clustering of patients within physician, a mixed model was fitted, adjusting for baseline score on the dependent variable with a random physician effect and patients nested within physician. A required sample size of 32 physicians and 774 patients was determined using calculations to have 90% power (α= .05, two-tailed) to detect a moderate effect, assuming an intraclass correlation as large as .05, and allowing for 20% attrition.

Table 1.

Physician and Patient Demographic and Medical Characteristics by Condition

	Intervention		Control
	Mean or %	SD	Mean or %	SD	Significance P Value
Physician characteristics (N = 52)
Single provider office, %	29.2		17.9		0.344
Rural, %	66.7		67.9		0.929
Family practice, %	62.5		53.6		0.525
Female, %	25.0		25.0		1.00
Years since training	15.0	6.1	12.8	8.1	0.330
Patient characteristics (N = 886)
Age, y	61	12.6	65	12.4	< 0.001
Female, %	53.0		50.5		0.457
Comorbid illnesses, n	1.9	1.5	2.2	1.4	0.004
Ethnicity, %					0.167
White/non-Hispanic	83.5		77.9
Black	1.7		2.7
Hispanic	11.3		14.1
Other	3.4		5.4
Education, %					0.893
Less than high school	13.0		14.4
High school graduate	27.1		25.4
College 1 to 3 years	32.0		32.8
College/grad school	27.9		27.4
Annual income, %					0.097
< $10,000	12.3		10.0
$10,000 to $29,999	26.4		33.9
$30,000 to $49,999	28.0		23.9
≥$50,000	33.3		32.1

SD, standard deviation.

Differences between participants and nonparticipants, and between conditions at baseline were conducted using t tests. Outcomes were evaluated using mixed model regression analyses (to account for clustering) and controlling for baseline scores on the dependent variable and any other potential confounding variables. A series of ANCOVA interaction analyses (treatment condition by patient characteristic) was conducted on improvement scores to determine whether the intervention was differentially effective among different patient subgroups in producing improvement on the two primary outcomes.

Interventions

Participants assigned to the Diabetes Priority Program came to their next diabetes-related visit 30 minutes early to complete the computerized Diabetes Priority Program touchscreen assessment and feedback procedure. The first part of the interactive computer program focused on the medical care participants were receiving for their diabetes. Participants were asked to recall how long it had been since they had received each of the 11 items contained in the NCQA/ADA Diabetes Physician Recognition Program (PRP) measures. Seven of these measures involved assays or exams performed or ordered by the physician (e.g., checking blood pressure, cholesterol, feet, microalbumin, having a dilated eye exam). Several of these measures—lipids and A1c assessments—were collected for all participants as part of this project and thus were not eligible for inclusion as outcome measures (because as part of inclusion in the study these were conducted for all participants, there could be no variability in these components). The remaining 4 lab measures shown in Table 2 were summarized to produce a summary score of number of laboratory assessments meeting PRP criteria. The 4 “patient-centered measures” that involved counseling or assistance for the patient with lifestyle aspects of the PRP measures (setting a self-management goal, receiving nutrition therapy, self-monitoring of blood glucose, and patient satisfaction items) were summarized into a patient-centered composite (see Table 2). The second part of the program focused on developing a self-management action plan. Patients answered questions on their dietary, physical activity, and smoking behaviors and were given feedback on each of these. They then were asked to select a behavior change goal in the area of smoking, diet, or exercise. The program then guided them through an interactive session that included selecting specific activities to support the goal area they chose, identifying barriers, and choosing strategies that would help them overcome these barriers. The computer then generated a printout of the patient's personalized action plan that included a summary of medical care procedures they might be due for and a copy of their self-management action plan.

Table 2.

Baseline, 6-Month, and Adjusted 6-Month PRP Measures Completed by Condition

Variable/Condition Primary Outcomes	Baseline Mean or %	6-Month Unadjusted Mean or %	6-Month Adjusted* Mean or %	Significance Level* (P Value)
Lab procedures completed, n
Intervention	3.03 (0.29)	3.40 (0.29)	3.39	0.001
Control	3.00 (0.42)	3.10 (0.42)	3.11
Blood pressure, %
Intervention	98.1	98.2	99.2
Control	98.3	99.6	99.6
Dilated eye exam, %
Intervention	67.7	78.3	76.7
Control	59.5	66.9	68.2
Foot exam, %
Intervention	74.4	87.4	87.8
Control	76.7	79.4	79.1
Microalbumin, %
Intervention	61.9	75.0	76.1
Control	65.3	68.3	67.4
Patient-centered activities completed, n
Intervention	3.00 (0.41)	3.65 (0.21)	3.63	< 0.001
Control	2.90 (0.34)	3.17 (0.45)	3.19
Self-management: goal setting, %
Intervention	59.8	91.6	91.6
Control	58.3	77.0	76.9
Medical nutrition treatment, %
Intervention	55.0	81.4	81.4
Control	49.0	67.6	67.6
Self-monitoring blood glucose, %
Intervention	88.5	92.9	92.0
Control	84.6	84.0	84.9
Patient satisfaction, %
Intervention	96.6	99.0	99.2
Control	96.5	96.4	96.2
Other outcomes, %
Quality of life (PAID-2)†
Intervention	30.28 (4.22)	29.72 (4.90)	29.12 (0.74)	0.747
Control	28.54 (5.02)	26.78 (4.35)	27.30 (0.70)
With major depression, %‡
(10 or higher on PHQ-9)
Intervention	18.6	17.4	15.0	0.717
Control	13.1	11.4	13.4

Standard deviations are in parentheses for T1 and unadjusted T2 scores.

^*Significance levels are P values from ANCOVA results on adjusted 6-month values (adjusted for age and number of comorbid conditions, and baseline values on the dependent variable) and accounting for clustering within physicians.

^†Lower scores indicate better quality of life.

^‡Tests for significance were run on continuous PHQ scores.

PRP, Physician Recognition Program; PAID-2, Problem Areas in Diabetes 2 scale; PHQ-9, Patient Health Questionnaire.

Immediately following the interactive computer session, three printouts were produced: an action plan for the patient, a summary of the patient's needed assessments, and self-management goals for the physician (see Fig. 2), including a prominent notation of areas the patient wished to discuss and a detailed printout to be used by the office's designated “care manager.” The care manager was a clinic staff member (usually a nurse or medical assistant) who conducted a brief counseling session with the patient prior to departure. Care managers were assigned by the clinic and sent a detailed “roles and responsibilities” form that summarized their responsibilities (e.g., meet with patients to review printouts, make follow-up phone calls). (Care managers were not identified or trained in the control condition.) Care managers were trained to use a patient-centered self-management approach^10,19 that included review of the medical care needs and self-care goals that the patient identified, and to brainstorm additional strategies that patients could use to overcome identified barriers to their goals. This took an average of 8 to 10 minutes during the visit. The care manager also arranged two brief follow-up calls between visits to review progress and to reinforce strategies developed during the patient's visit. These procedures were designed to be consistent with recommendations from the Chronic Care Model for self-management support,^2,19,20 yet be feasible to implement during primary care visits.

FIGURE 2.

Diabetes Priority Program printout goals for the physician.

Control Condition

Patients in the control condition completed a touchscreen computer assessment procedure involving the PRP measures as well as general health risk appraisal items (e.g., use of seat belts, cancer-screening assessments). These patients also received a printout, but one that focused on general health risks and ways they could reduce risk that did not include the PRP measures.

Measures

Because the 52 physicians had different medical record formats, very few had an established diabetes registry, and almost none reliably recorded most of these PRP activities, we used patient reports of having received these services as our primary outcome measure. The scales described above were used in two previous studies of patients having almost identical demographic and medical characteristics^3,21 and found to be reliable and to agree well with electronic medical records in a health care system that recorded such information in a diabetes registry.

Patient satisfaction was assessed by a 5-item scale from the PRP.^15,16 These items asked patients to rate their providers in the following areas: 1) answering questions about diabetes; 2) being available during emergencies; 3) explaining lab test results in an understandable way; 4) having a courteous, personal manner; and 5) overall diabetes care. The scale exhibited good internal consistency, α= .88.

Secondary outcomes included:

1. The revised Problem Areas in Diabetes 2 (PAID-2) scale, a newly developed version of the original scale, assessed diabetes-specific quality of life.²² The earlier version has been demonstrated to be reliable and sensitive to change.^22,23 In the present study, the PAID-2 had an internal consistency of α= .93.

2. The Patient Health Questionnaire (PHQ) is a self-administered instrument that has been validated as a diagnostic and depression severity measure.²⁴ The PHQ-9 scores each of the 9 Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) depression criteria on a 0 (not at all) to 3 (nearly every day) scale. A score of 10 has been documented to have a sensitivity of 88% and a specificity of 88% for major depression.²⁴ In the present study the scale exhibited good internal consistency, α= .86.

RESULTS

As detailed in Figure 1, attrition rates were equivalent (5.3% and 4.9%) and minimal across the two conditions at the 6-month follow-up and not due to any consistent reasons. Therefore analyses were conducted on complete cases. Analyses using intent-to-treat procedures (and assuming those lost to follow-up were at baseline levels) produced identical conclusions.

The results are organized using the RE-AIM evaluation framework components^9,17 of reach, effectiveness, adoption, and implementation. Reach refers to the participation rate and representativeness of patients, effectiveness in this case refers to primary outcomes on the PRP items and also impact on quality-of-life and depression measures, adoption refers to the participation rate and representativeness of physicians who participated in the study, and implementation refers to the consistency of delivery of the intervention protocol. Maintenance, the final element in the RE-AIM model, will be covered in later reports; this report presents data on the 6-month follow-up results.

Reach

A total of 886 patients (74.6% of those eligible) participated. Representativeness analyses revealed that nonparticipants were slightly but significantly older than participants (M = 64 vs 63 years), more likely to be Latino (19% vs 13%), have a family income less than $30,000 per year (57% vs 41%), and have less than a 12th grade education (25% vs 14%) (all P values ≤ .05 ≤ .001). There were no differences between participants and nonparticipants on gender or number of comorbid conditions. Participants’ characteristics did, however, match those of a random sample of Colorado diabetes patients (see footnote in Table 1). More detail on recruitment is available elsewhere,²⁵ and patient characteristics are summarized in Table 1.

Effectiveness

Initial analyses revealed baseline differences between conditions on age and number of comorbid conditions but not on other variables. Subsequent analyses revealed that the significance of all outcome analyses was unaffected by whether or not age and comorbid conditions were included. Therefore, only adjusted analyses are reported.

Primary Outcomes

As can be seen in Table 2, patients were receiving high levels of care at baseline, especially for the laboratory assay measures. Sixty percent to 99% of patients were receiving recommended services, a figure substantially higher than in two previous studies of similar samples using this same measure.^3,21 Despite this high initial level of care, patients in intervention practices showed significantly greater improvement on both laboratory assay (F = 9.90; P = .003) and patient-centered (F = 25.2; P < .001) subsets of the PRP measures (Table 2).

To address issues of potential ceiling effects, analyses subsequent to the overall significant effects were conducted using only those patients for each PRP measure who did not meet NCQA/ADA-recommended levels at baseline. In two of the three measures for which there were more than 200 patients, the intervention condition produced superior results on percentage of patients meeting the criterion at 6 months (Table 3). All of the measures showed trends favoring the intervention condition. These intervention patients received rates of care averaging 18% higher than controls (range 10% to 29% higher for the various items). These analyses revealed that the greatest differences between conditions in improvement were on the measures of self-management goal setting, dilated eye exam, foot exams, and medical nutrition therapy. These were also the areas in which performance was the lowest at baseline (Tables 2 and 3).

Table 3.

Results for Individuals Who Did Not Meet PRP Criteria at Baseline

		Patients Who Met PRP Criteria at six Months, %
	Patients Who Did Not Meet PRP Criteria at Baseline, N*	Intervention	Control	P Value
Lab composite measures
Blood pressure	11	100	100	N/A
Dilated eye exam	230	57.7	47.7	.135
Foot exam	141	80.0	51.5	.003
Microalbumin	95	78.3	65.3	.122
Behavioral composite measures
Self-management: goal setting	221	88.7	67.9	< .001
Medical nutrition treatment	266	75.5	52.0	< .001
Self-monitoring blood glucose	99	38.6	29.1	.399
Patient satisfaction	17	70.0	71.4	N/A

^*The N represents the total number of participants who did not meet the recommended criteria at baseline. A given person can be in multiple rows.

N/A, analysis not conducted due to very small N.

PRP, Physician Recognition Program.

Secondary Outcomes

As summarized in Table 2, both conditions improved on measures of quality of life and depressive symptoms, but there was not a significant difference between conditions.

Subgroup Analyses

Dependent variables analyzed in subgroup analyses were: improvement in 1) PRP summary score for laboratory assessments, 2) PRP summary score for patient-centered aspects of care, and 3) quality of life. Independent variables included demographics (gender, age, marital status, insurance status, education, ethnicity, employment, and income) and other patient characteristics (baseline HgA1c values, number of comorbid conditions, patient self-efficacy, and presence of depression), which were each individually analyzed to see whether they interacted with the treatment condition (e.g., was an effect modifier). The only significant interaction was education by treatment (F = 4.29; P < .05). The intervention effect on patient-centered aspects of care was significantly greater among those with less education. This result could be due to chance; to a “ceiling effect,” as many of the more educated patients started out very high in the baseline care they were receiving; or to a true effect of intervention being especially effective for less educated patients who may benefit from such a tailored, self-paced intervention.

Adoption

Fifty-two physicians, consisting of 22 internal medicine and 30 family practice physicians, participated. Based on a prior survey of physician characteristics, the 52 participating physicians (4.9% of the total sample) did not differ from the total sample of 1,059 primary care physicians insured by Copic on age or gender of physician, years in practice, size of practice, or use of any of a series of quality improvement processes for diabetes (e.g., registry, reminder systems, and follow-up calls). Characteristics of participating physicians did not differ between conditions (Table 1). The most common reasons given for declining participations were not enough time, competing demands, and not enough type 2 patients or staff.

Implementation

The project protocol was consistently implemented across the heterogeneous settings. Ninety-nine percent of patients received the computer-based interactive assessment procedure, 92% discussed the printout with their physician, 99.8% met with the care manager to discuss lifestyle goals, and 86.4% received at least one follow-up phone call.

DISCUSSION

The Diabetes Priority Program evaluation provides several lessons about implementing a diabetes care quality improvement program across a wide range of clinical settings and across a wide range of patients. As has recently been pointed out, there are few “practical clinical trials” upon which to base clinical and policy decisions.^9–11 We included several practical trial components to make this study more generalizable than the typical efficacy study. The intervention was delivered by regular health care staff in clinical practice (rather than research staff), and few exclusion criteria were employed (e.g., patients having comorbid conditions including depression were included); the program was conducted during usual medical care visits rather than special research appointments; the touchscreen computer component was designed to be user friendly and usable by low-literacy patients (questions and information were presented aloud as well as on the screen); and the intervention was designed to fit into the flow of usual care. These actions were generally successful in making the intervention practical yet effective. Implications of the study are summarized using the RE-AIM framework.

Reach

The project attracted a broad (75% of eligible patients) sample of type 2 diabetes patients. We think that this high participation rate may be due to the endorsement of the primary care provider in recruitment letters and because the project could be completed largely during regular office visits. The 75% participation rate is among the highest that we have seen,²⁶ and there did not appear to be any consistent reasons for declining. Still, further recruitment progress could be made, because nonparticipants were somewhat more likely to be Latino, less educated, and lower income.

Effectiveness

Overall, intervention effectiveness was moderate compared to the stringent control condition that also received touchscreen computer-assisted assessment and feedback. The magnitude of effect was likely attenuated by the high baseline levels, ceiling effects on some measures, and because intervention was delivered by regular staff members whose time was very limited. Still, the intervention was successful in increasing both laboratory assessments and patient-centered lifestyle counseling—especially on measures in which there was the greatest room for improvement. These improvements were seen across different measures, across a wide variety of primary care practices and types of patients. It was encouraging that the intervention worked as well or better among less educated patients as among those more highly educated, and was equally effective among other less resourced and higher risk subgroups.

Although the program did not enhance quality of life or reduce depression levels more than the control condition, both conditions showed improvement on these outcomes, and intervention patients and providers were dealing with more regimen and guidelines issues without a reduction in quality of life or other apparent adverse consequences. It may have been unrealistic to expect differences on these psychosocial outcomes between conditions given the above issues and short time frame of the study.

Adoption

Participation among physicians was low, in contrast to the encouraging results among patients. Only 5% of family physicians and internists who were invited participated. The primary reasons given for declining were not enough time, too many competing issues, not enough type 2 patients, and not enough staff. If the program had addressed more general care than just diabetes, this might have increased participation. The low adoption rate is probably the largest challenge to generalization of the study results, but adoption rates among a broad sample of clinicians and settings approached is so seldom reported^11,26 that we do not know whether this rate of acceptance (among practicing physicians, most of whom had not participated previously in research), is unusually low. The encouraging results related to adoption are that participating physicians did not differ from the larger sample of over 1,000 Colorado physicians who completed the initial Copic survey on either general characteristics or reported use of diabetes quality improvement processes.

Implementation

Office staff delivered the intervention very consistently despite the competing demands faced by primary care clinics. Almost all patients received the touchscreen computer intervention, and even more staff-intensive aspects of the protocol such as meeting with a care manger and follow-up calls were completed at high levels for an effectiveness study implemented by real-world staff. We think this was due to the specificity of the written guidelines materials provided, the integration of the computer component into usual care, and the operational support our staff provided. This support included a detailed intervention manual and a 3-hour training program. In addition, ongoing fax and phone contacts were made to alert staff regarding scheduled participant appointments, missing paperwork, reminders to do follow-up calls, or to download computer data to transfer to the research center. We conclude that it is feasible to consistently deliver an intervention such as the Diabetes Priority Program in both family practice and general internal medicine primary care settings.

This study has both strengths and weaknesses in terms of drawing implications for practice, policy, and future research.^9,11 Strengths include the demonstrated similarity of participating physicians to the larger population of Colorado primary care physicians, the randomized design, the stringent control condition that also received touchscreen computer assessment and feedback, the breadth of outcomes and patient-centered measures included, analyses that evaluated both potential for translation and accounted for nesting of patients within physicians, and the “practical clinical trial” aspects of the study.^9,11 Limitations include the absence of gold standard registry or medical records data (very few of the participating practices had diabetes registries and other recent reports have documented that medical records in primary care clinics are frequently missing information on recommended preventive services and that patient self-report of diabetes care received is generally accurate).^3,21,27 Other limitations are the relatively short follow-up period, the absence of biological outcomes such as HgA1c or lipids (these measures were not collected at 6 months, but will be included in later reports), and possible ceiling effects on some outcome measures.