Improving Care after Myocardial Infarction Using A 2-year Internet-Delivered Intervention: the Department of Veterans’ Affairs Myocardial Infarction–Plus Cluster-Randomized Trial

Abstract

Context:

Cardiovascular risk reduction in ambulatory myocardial infarction (MI) survivors is effective but underused.

Objective:

To evaluate a provider-directed, Internet-delivered intervention to improve cardiovascular management for outpatients who survived MI.

Design, Setting and Participants:

Cluster-randomized trial involving 168 community-based primary care clinics and 847 providers in 26 states and Puerto Rico, January 2002 to December 2008, with clinic as randomization unit. We collected administrative data for 15,847 post-MI patients and medical record data for 10,452 of these.

Interventions:

Multi-component, Internet-delivered intervention with quarterly educational modules, practice guidelines, monthly literature summaries, and automated email reminders delivered to providers over 27 months.

Main Outcome Measures:

% patients who achieved each of 7 clinical indicators, a composite score of the 7 clinical indicators, and mean LDL and HgbA1c levels.

Results:

Clinics had a median of 3 providers (interquartile range, 2–6), with a median of 50% of providers (33.3–66.7) participating in the study. Patients in intervention clinics had greater improvements (from 85.2% to 88.0%) in the percentage prescribed beta-blockers than patients in control clinics (87% to 89.1%; adjusted improvement gain for intervention versus control, 2.6%; 95% CI, 0.1%−4.1%). We found non-significant differences in improvements favoring patients in intervention clinics for 5 of 6 remaining clinical indicators and levels of LDL and HgbA1c.

Conclusions:

A longitudinal, Internet-delivered intervention improved only 1 of 7 clinical indicators of cardiovascular management in ambulatory MI survivors.

INTRODUCTION

Each year, 1.3 million Americans experience a myocardial infarction (MI) or death from coronary heart disease (CHD).¹ In 2008, almost 8 million US adults were MI survivors.¹ Randomized trials have demonstrated that aspirin, beta-blockers, angiotensin-converting–enzyme (ACE) inhibitors and HMG-CoA-reductase inhibitors (statins) can reduce morbidity and mortality for MI survivors.^2–6 Population-based evidence suggests that better medical treatment and secondary prevention of CHD has led to substantial declines in CHD mortality over the last three decades.^7–10 Despite these advances, providers often fail to adhere to evidence-based guidelines for the management and secondary prevention of CHD.^11–16

Multi-component interventions can increase adherence to clinical practice guidelines.^17,18 In patients with CHD, multi-component interventions have been shown to improve in-hospital care,^19–22 but have yielded mixed results in primary care settings.^23–25 Although guidelines usually focus on CHD in isolation, ambulatory post-MI patients frequently have multiple co-morbidities that may interfere with adherence to CHD guidelines.²⁶ Furthermore, interventions to improve adherence are often limited by their short duration and lack of reinforcement, limiting sustainability over time.^27,28 Reliance on academic detailing and site visits may also make many interventions expensive and impractical.^23,24 Delivering an intervention over longer periods of time using the Internet (e.g., online spaced education) may address some of these limitations.^29,30 Moreover, online spaced education may allow intensive interventions to be widely disseminated with minimal marginal costs.

To test whether a longitudinal, multi-component, Internet-delivered intervention with educational cases, guidelines, monthly updates, and email reminders could improve guideline adherence and reduce CHD risk factors among MI survivors with multiple co-morbidities treated in primary care settings, we undertook a national cluster-randomized trial of community-based outpatient clinics affiliated with the Veterans Health Administration (VHA).

METHODS

Trial Design

We designed a cluster-randomized trial of MI survivors with randomization at the clinic level. We were funded through the VHA Health Services Research and Development office (HSR&D) [IHD 04–387] and by a parallel National Institutes of Health study [R01 HL70786–02].²⁶ The institutional review board at each medical center approved the study.

Participants

Our intervention targeted VHA community-based primary care outpatient clinics.³¹ We targeted all adults ≥18 years discharged from a participating facility with a primary or secondary diagnosis of acute or previous MI (ICD-9-CM codes: 410.xx or 412.xx) and subsequently treated at a facility-affiliated clinic between January 2002 and December 2008. The index visit was the most recent clinic visit during the relevant study period: January 2002 to November 2004 pre-intervention and February 2007 to December 2008 post-intervention.

Study Indicators

A panel of 12 experts generated 7 clinical indicators of post-MI monitoring, treatment, and target goals using nominal group technique (Supplement 1).^32,33 Monitoring indicators included measurement of LDL cholesterol (all patients) and hemoglobin A1c (HgbA_1c) (all patients with diabetes) within 18 months of the index visit. Treatment indicators included prescriptions at the index visit for beta-blockers, statins, and ACE inhibitors or angiotensinogen II receptor blockers (ARBs). Target goals included last LDL cholesterol level <100 mg/dL and last HgbA_1c <8% in patients with diabetes. Although these clinical indicators are process measures, each is a strong independent predictor of clinical outcomes in post-MI patients.^2–6,34

For the primary analysis, we extracted demographic information (age, race, and sex), diagnoses, medications, and laboratory data from over 3 million records using the VHA Decision Support System files (National Patient Care Data, Austin Information Technology Center). Co-morbidities were defined using inpatient or outpatient ICD-9-CM codes during the 12–18 months before the index visit (Supplement 2). Because a growing percentage of post-MI patients have contra-indications to CHD therapies but often receive them,^12,35 and some contra-indications may also be indications (e.g., beta-blockers in post-MI patients with heart failure), we adjusted for patient co-morbidity rather than excluding patients from the denominator for the primary analysis (Statistical Analysis).

We performed secondary analyses to determine receipt of aspirin and to confirm our findings in post-MI patients who were ideal candidates (no contra-indications) for available indicators. We used patient-level data obtained from the VHA Office of Quality and Performance (OQP) (Supplement 3).

Interventions

The intervention included a multi-component website and pushed email cues with educational content.³¹ Eight case-based, interactive educational modules represented the core of the intervention. Each module reviewed the evidence and guidelines for one or more of the 7 clinical indicators developed by our expert panel; presented a clinical scenario followed by a series of questions; and provided tailored feedback to providers based on their responses. The website also included relevant clinical guidelines, monthly summaries of pertinent peer-reviewed manuscripts, downloadable practice tools, and patient educational materials. The website used service-oriented architecture and design principles refined previously.^36,37 Iterative usability sessions refined the content. Supplement 4 provides a site map of the Intervention which was available from November 2004 to January 2007. Proactive emails, demonstrated to increase participation in web-delivered provider interventions,³⁸ notified providers of new updates and materials.

When modules were released, providers in control clinics were sent a link to an existing VHA website that contained links to a wide range of clinical guidelines for various medical conditions (http://www.healthquality.va.gov/). All enrolled intervention and control providers could obtain continuing education credits for reviewing materials on the website and receive a subscription to The Medical Letter.

Outcomes

The primary outcomes were the percentages of patients who achieved each of the 7 clinical indicators. As a secondary outcome, we created a patient-level composite quality indicator score summing and then normalizing to total of 100 those clinical indicators for which the patient was eligible. All patients were eligible for three contributors to the composite score (beta-blocker, ACE/ARB and LDL measurement), while diabetes and hyperlipidemia each added two additional condition-specific indicators. We also calculated mean levels of LDL cholesterol and HgbA_1c. All outcomes were based on the most recent test or clinic visit. User tracing of server logs assessed intervention uptake. Provider participation was measured by randomization group.

Provider Recruitment and Clinic Randomization

Recruitment has been described elsewhere.³¹ Briefly, among 66 eligible medical centers, we recruited 48 (73%) centers affiliated with 219 clinics with 957 providers in 26 states and Puerto Rico; of these, 168 (77%) clinics were randomized (i.e., had at least one of its providers enroll); of the 847 providers in these 168 clinics, 401 (47%) participated (Figure 1). Providers were recruited by e-mail, postal mail, fax, and telephone. A clinic was enrolled and randomized when the first eligible provider at that clinic logged-on to the study website. All providers at a clinic were randomized to the same arm but only those who logged-on were enrolled. Using further email reminders, providers were recruited throughout the 27 months of the intervention. Although we recruited continuously, all clinics had at least 12 months of exposure to the intervention or control website. After website posting, components remained available for the duration of the study. All providers completed an online consent to participate.

Figure 1: CONSORT Diagram.

Sample Size

The planned sample size of 200 clinics provided at least 80% power with two-sided α=0.05 to detect a difference of improvement of 5% in each clinical indicator attributable to the intervention. Our post hoc analysis suggests that we had 80% power to detect a difference of 5% assuming 168 clinics and an average of at least 33 patients per clinic per time period.

Statistical Analyses

We used an intention-to-treat approach for our main analysis, basing our outcome measures on the entire eligible patient population in each of the 168 clinics, regardless of the patient’s provider enrolling on our website. Analyses were at the patient level, but accounted for clustering of patients within clinics (unit of randomization). Baseline clinic characteristics were compared between the intervention and control groups using chi-square tests, t-tests, or Wilcoxon rank sum tests as appropriate.

Performance improvement was calculated as the change (pre-intervention vs. post-intervention) in the proportion of patients receiving each clinical indicator or as the mean change in LDL or HgbA_1c level. The differences in performance improvement for each indicator were analyzed at the patient level by randomization group. Generalized linear models examined the associations between indicator performance, LDL and HgbA1c levels, and the main independent variable, randomized group, accounting for patient clustering within clinic and time of observation (pre- vs. post-intervention), with and without adjustment for patient age, race, sex, and co-morbidity using a summary count of the 12 co-morbidities considered. The P-value for the randomization group-by-time interaction assessed the statistical significance of the intervention effect. Analyses were performed before and after stratification by clinic size (number of providers) and by urban/rural location.

For the secondary analysis, using manually abstracted VHA performance data, we examined receipt of aspirin and performance on “ideal candidates” using the same approach as for the primary analysis except we only adjusted for patient clustering within clinics as other patient-level covariates were unavailable. We conducted another secondary analysis to assess whether there was a dose-response effect manifested as a correlation between degree of participation in the intervention and improved performance (Supplement 5).

RESULTS

The median number of providers/clinic (inter-quartile range) was three (2–6); most clinics were urban (Table 1). Clinic characteristics did not differ by intervention group. Patients (n=15,847) were mostly male with mean age (SD) 66 ± 10 years, 68.5% white and 8.7% black (Table 2). The most common co-morbidities were hypertension and hyperlipidemia.

Table 1:

Characteristics and Participation Measures of Study Clinics by Randomization Group: the VA MI-Plus Study, 2002–2008

Characteristic	Overall (N=168)	Intervention Group (N=84)	Control Group (N=84)	P-value
Parent facility designated as a health services research program, n (%)	39 (23)	19 (23)	20 (24)	0.8
Urban location,* n (%)	139 (84)	70 (84)	69 (84)	0.97
Geographic region, n (%)				0.7
New England/Mid-Atlantic	48 (28)	28 (33)	20 (24)
Midwest	54 (32)	25 (30)	29 (34)
South	49 (29)	23 (27)	26 (31)
West	14 (8)	7 (8)	7 (8)
Puerto Rico/Virgin Islands	3 (2)	1 (1)	2 (2)
Number of providers, n (%)				0.3
1–2	53 (32)	29 (34)	24 (28)
3–4	55 (33)	30 (36)	25 (30)
≥5	60 (36)	25 (30)	35 (42)
Median† (inter-quartile range, IQR)	3 (2–6)	3 (2–5)	4 (2–7)	0.2
Participation Measure†
Providers per clinic who participated (logged on to the website), % (IQR)	50.0 (33.3–75.0)	58.6 (39.2–100)	50.0 (33.3–66.7)	0.03
Number of website visits per provider per clinic, median (IQR)	1.7 (0.6–3.7)	3.7 (2.0–6.8)	0.8 (0.5–1.5)	<0.001
Number of website pages accessed per provider per clinic, median (IQR)	17.9 (4.5–62.4)	57.7 (29.2–99.5)	6.2 (2.2–11.3)	<0.001
Number of months from first logon to study closure, median (IQR)‡	16.5 (12.5–19.1)	17.0 (13.3–19.3)	15.8 (11.0–18.9)	0.2
Number of months between first and last logon (active participation), median (IQR)‡	2.7 (0.0–11.5)	7.6 (0.7–14.2)	0.0 (0.0–3.8)	<0.001
Providers with active participation of ≥12 months, %‡	23.4%	36.1%	10.2%	<0.001

^*Urban/rural location designation was not available for the 3 clinics in Puerto Rico & Virgin Islands.

^†P-values from Wilcoxon rank sum test where medians are given, else from chi-square.

^‡Participation measured among providers who enrolled.

Table 2:

Characteristics of Post-Myocardial Infarction Patients in Study Clinics, by Randomization Group and Intervention Periods: the VA MI-Plus Study, 2002–2008*

Characteristic	Intervention Group-Pre-Intervention (N=4,024)	Control Group-Pre-Intervention (N=3,727)	Intervention Group-Post-Intervention (N=3,080)	Control Group-Post-Intervention (N=2,911)
Male, %	98.6	98.3	98.8	98.7
Age at index visit, years
<55 years	16.9	15.0	10.6	9.0
55–64 years	33.7	35.7	37.9	37.9
65–74 years	25.4	26.1	25.1	26.7
≥75 years	24.0	23.2	26.5	26.4
Race, %
White	68.2	69.4	66.0	70.6
Black	9.7	8.1	8.7	8.0
Other	1.7	2.3	1.5	1.6
Missing/Unknown	20.3	20.3	23.8	19.9
Hypertension, %	86.6	85.0	87.0	84.4
Hyperlipidemia, %	77.5	75.7	84.0	83.6
Diabetes, %	44.0	42.2	46.4	46.0
Heart failure, %	29.0	27.4	30.5	28.8
COPD, %	30.2	26.4	29.5	27.7
Depression, %	23.7	25.1	22.1	20.4
Chronic kidney disease, %	9.3	9.8	17.7	17.5
Stroke, %	12.3	11.5	11.1	11.0
Cancer†, %	12.4	11.6	12.4	13.3
Peripheral vascular disease, %	8.9	8.3	6.0	6.2
Dementia, %	6.0	5.7	7.7	7.3
Asthma, %	5.6	5.3	5.0	3.5
Number of conditions, median (IQR)	3 (2)	3 (2)	3 (2)	3 (2)

Provider Participation and Activity

The median percent of providers from all 168 clinics who logged on to the intervention or control websites was 50%, higher for intervention than control clinics (medians: 58.6% vs. 50.0%; P=0.03). Provider website activity was greater at intervention versus control clinics (medians: website visits/provider, 3.7 vs. 0.8; website pages accessed/provider 57.7 vs. 6.2, P<0.001 for both)(Table 1). Among the intervention group, the median number of completed case modules (maximum 8) was 1.8 (IQR 1–3) modules/provider and 3.3 (IQR 2–5) modules/participating provider. Among providers who enrolled, the median time between first logon to the end of the study (active enrollment) was similar by randomization group, ~16–17 months (P=0.2). However, the median time of active participation (time between first and last logon) and the proportion of providers with ≥12 months of active participation were higher in intervention versus control clinics.

Clinic size was inversely correlated with all measures of provider activity: proportion of providers logged on (r = −0.55); mean website visits/provider (r = −0.30); mean website pages accessed/provider (r = −0.24) (all P<0.001). Correlations were similar across randomization groups.

Changes in Clinical Indicator Performance

Over time, all quality measures trended towards improved performance, and improvement was slightly greater in intervention versus control clinics, for all nine measures (Table 3). After adjustment for patient demographics, co-morbidity score, and clustering, intervention clinic patients had greater improvements than control clinic patients for the prescription for beta-blockers (adjusted improvement gain, 2.6%; 95% CI, 0.1% to 4.1%). Small statistically non-significant differences in improvements favoring patients in intervention clinics were observed after adjustment in 7 of the 8 remaining measures (Figure 2). Results were similar in small or large clinics and in rural or urban clinics (data not shown). The composite quality indicator score tended to improve more for all patients or for patients with diabetes and hyperlipidemia (those at highest cardiovascular risk) in intervention versus control clinics, but these gains were not statistically significant (P=0.12 and P=0.09 for the group-by-time interaction terms, respectively).

Table 3:

Performance of Quality Indicators Pre- and Post-Intervention among Unselected Patients based on Administrative Data and Intention-to-treat Analysis: the VA MI-Plus Study, 2002–2008

Indicator (No. patients with quality indicator/ No. patients eligible for indicator)	Intervention Group-Pre-Intervention	Intervention Group-Post-Intervention	Control Group-Pre-Intervention	Control Group-Post-Intervention	Unadjusted Difference in Improvement†	Adjusted Difference in Improvement (95% CI)‡	P- value
Monitoring
LDL cholesterol measurement in previous 18 months, %	85.2 (3,429/4,024)	88.0 (2,711/3,080)	87.0 (3,241/3,727)	89.1 (2,594/2,911)	0.7%	−0.1% (−8.9 to 4.7)	>0.2
HbA1c measurement in previous 18 months for adults with diabetes, %	75.3 (1,603/2,129)	86.8 (1,357/1,563)	79.1 (1,515/1,916)	88.6 (1,291/1,457)	2.0%	0.4% (−1.4 to 1.9)	>0.2
Treatment
Prescription for beta- blocker, %	70.0 (2,818/4,024)	85.5 (2,633/3,080)	71.9 (2,678/3,727)	84.0 (2,446/2,911)	3.4%	2.6% (0.1 to 4.1)	0.04
Prescription for statin, %	87.4 (2,994/3,424)	95.9 (2,708/2,825)	87.8 (2,773/3,157)	95.3 (2,506/2,630)	1.0%	0.7% (−1.3 to 2.1)	>0.2
Prescription for ACE or ARB, %	69.7 (2,806/4,024)	74.6 (2,299/3,080)	70.6 (2,631/3,727)	73.6 (2,141/2,911)	1.9%	1.8% (−1.4 to 4.8)	>0.2
Achievement of Target Levels
Last LDL cholesterol level <100 mg/dL, %	52.3 (1,738/3,322)	72.2 (1,922/2,663)	55.2 (1,699/3,077)	70.4 (1,766/2,509)	4.7%	4.0% (−0.3 to 7.9)	0.06
Last HbA1c level <8% in adults with diabetes, %	71.3 (1,141/1,600)	77.3 (1,048/1,356)	73.1 (1,107/1,514)	73.8 (953/1,291)	5.3%	4.6% (−1.5 to 9.8)	0.13
Composite Clinical Indicator Score
All patients, %	69.2	79.7	71.1	79.0	2.6	2.3 (−0.6- to 5.1)	0.12
Patients with diabetes and hyperlipidemia, %	75.0	83.8	77.5	83.4	2.9	2.6 (−0.4 to 5.7)	0.09
Physiologic Levels
Last LDL cholesterol level, mg/dL, mean reduction ± SD	102.4 ± 36.1	87.0 ± 33.8	100.5 ± 36.2	88.8 ± 33.2	3.7	3.5 (−0.8 to 7.7)	0.10
Last HbA1c level, %, mean reduction ± SD	7.4 ± 1.8	7.2 ± 1.5	7.4 ± 1.7	7.3 ± 1.6	0.1	0.2 (−0.1 to 0.5)	0.12

^*ACE = angiotensin-converting enzyme; ARB = angiotensin-receptor blocker; HbA_1c = hemoglobin A_1c; LDL = low-density lipoprotein.

^†Difference in improvement was the change (post-intervention minus pre-intervention, except for the physiologic differences, where the signs are reversed because decrease signifies improvement) among intervention clinics minus the change among control clinics. A positive difference reflects greater improvement among intervention clinics.

^‡With use of generalized mixed-effects regression models, these differences and the associated 95% CIs and P-values reflect adjustment for patient clustering within clinics, patient-level covariates (age, sex, race, and co-morbidity score) and random clinic effects (to account for correlation of patient measures within individual clinics). Co-morbidity score was calculated using a composite score based on 12 major health conditions: hypertension, hyperlipidemia, diabetes mellitus, heart failure, emphysema, depression, chronic kidney disease, stroke, cancer (excluding skin, prostate or thyroid), peripheral vascular disease, dementia, and asthma. LDL cholesterol and HgbA_1c levels for eligible patients who had measurement within 18 months of the index visit. Laboratory level or prescription measured at the time of the index visit. The patient-level composite quality indicator score summed the 7 clinical indicators for which the patient was eligible. All members of the cohort were eligible for three contributors to the composite score (beta-blocker, ACE/ARB, and LDL measurement), while diabetes and hyperlipidemia each added two additional condition-specific indicators.

Figure 2: Adjusted Differences in Improvement in the Clinical Indicators, Intervention Minus Control.

Abbreviations: ACE = angiotensin-converting enzyme; ARB = angiotensin-receptor blocker; LDL-C = low-density lipoprotein cholesterol; HgbA1c = hemoglobin A1c.

*Among adults with diabetes.

†Difference in improvement was the change (post-intervention minus pre-intervention, except for the physiologic differences, where the signs are reversed because decrease signifies improvement) among intervention clinics minus the change among control clinics. A positive difference reflects greater improvement among intervention clinics. ‡With use of generalized mixed-effects regression models, these differences and the associated 95% CIs and P-values reflect adjustment for patient clustering within clinics, patient-level covariates (age, sex, race, and co-morbidity score) and random clinic effects (to account for correlation of patient measures within individual clinics). Co-morbidity score was calculated using a composite score based on 12 major health conditions: hypertension, hyperlipidemia, diabetes mellitus, heart failure, emphysema, depression, chronic kidney disease, stroke, cancer (excluding skin, prostate or thyroid), peripheral vascular disease, dementia, and asthma. LDL cholesterol and HgbA1c levels for eligible patients who had measurement within 18 months of the index visit. Laboratory level or prescription measured at the time of the index visit.

Secondary Analyses

Our analysis using chart-abstracted data showed consistent results, again favoring the intervention for LDL, with borderline significance (P = 0.07) (Supplement 3). Our analysis assessing a dose-response effect of provider participation revealed trends all as expected, but only statistically significant for some indicators, particularly when defined by number of webpages accessed/provider; e.g., mean LDL cholesterol <100 mg/dL increased significantly more from pre-intervention to post-intervention as level of provider participation increased, whether the latter was defined as number of website visits/provider (P=0.02) or as number of pages accessed/provider (P=0.01) (Supplement Table 5).

DISCUSSION

Our longitudinal Internet-delivered intervention improved performance on only 1 of 7 quality indicators for care of MI survivors, namely, prescription of beta-blockers. These findings were observed in small and large, rural and urban clinics, and persisted after adjustment for patient demographics and co-morbidity.

Few studies have assessed multi-component interventions to improve CHD management in the primary care setting and results have been mixed.²⁵ Among 20 primary care practices, an intervention with academic detailing, quality improvement (QI) facilitation, and audit and feedback demonstrated non-significant improvements in blood pressure in patients with CHD but no improvements in lipids or beta-blocker prescriptions.²³ An 18-month intervention with tailored care plans and academic detailing did not improve blood pressure or lipids but did reduce hospitalizations in patients with CHD treated in 48 primary care practices.²⁴ Taken together with our results, these studies show the challenges of improving processes of care or clinical outcomes with intensive interventions in primary care.³⁹

Our study has several strengths. Our large-scale intervention was disseminated to 168 community-based outpatient clinics across a wide geographic distribution. Our “light touch” longitudinal intervention combined low intensity and high dissemination ability^40,41 and had low staffing and resource requirements. We used online spaced education with automated reminders to generate knowledge transfer, increase learning efficiency, and change provider behavior. Also, our system included downloadable patient support tools. We used email marketing and continuing medical education credits to enhance engagement. The pushed cues and reminders allowed us to improve on our experience with provider participation and to achieve sustained intervention activity by some intervention providers.^37,42 Although our intervention was multi-component, once designed it was sustained with minimal effort. It is scalable to other settings at a cost that is modest compared to other QI interventions. Although the improvement in beta-blocker prescription attributable to the intervention was modest, it is possible that an additional 3% of CHD deaths may be prevented or postponed if the beta-blocker prescription improvements are maintained for at least 6 months using published estimates.^3,9

Still, our intervention improved only 1 of the studied process measures. Several factors may account for this. Consistent with previous research,⁴³ the high baseline performance, higher than in other reports,^9,23,24 may have limited efforts to further improve performance. Also, as in common in implementation studies, some intervention clinics had low intervention fidelity (e.g., the intervention delivered was never received for providers in the intervention group who did not enroll). Our intervention might have had a much larger effect had baseline performance been lower or had it successfully engaged more providers. Indeed, in a post hoc, as treated analysis, the intensity of provider participation in our intervention was positively associated with clinical indicator performance (Supplement 5). During our study period, there were intense efforts by VHA and non-VHA health systems to improve performance on our indicators. Our findings of large improvements in performance in the control group as well as the intervention group (Table 3) suggest that these secular changes may have minimized our ability to detect the effect of our intervention. Moreover, interventions targeted solely at providers may be less effective than those involving teams or systems re-engineering.^44,45

Several limitations merit discussion. VHA findings may not be generalizable to other clinical settings or women. However, the VHA is the largest integrated health care delivery system in the US with 153 medical centers and over 900 outpatient clinics providing care to about 5.5 million veterans in 2008.⁴⁶ Moreover, the VHA offers unique uniformity of data and geographic diversity for intervention studies. Although our providers may be a select group due to participation in a QI study, their performance on the indicators studied was similar to that of all VHA clinic providers. For example, performance on the lipid control indicator was 59.4% and 63.6% in 2005 (baseline) for the intervention and control groups versus 55% in 2004 for all VHA clinic providers. Similarly, these performances rose to 70.7% and 68.3% in 2007–2008 in our study compared to 68% in 2007 for all VHA clinics. We did not evaluate differences in patient-centered or clinical outcomes such as cardiovascular events because our sample size did not provide sufficient power to detect a clinically meaningful difference. Still, our study indicators are commonly used to assess provider adherence to CHD guidelines and to determine pay for performance in VHA and non-VHA health systems.

Our study has potential implications for implementation research. Disease management programs may improve performance in patients with CHD and/or heart failure but they are resource intensive.⁴⁷ Electronic health records (EHRs) and clinical decision support may help providers manage chronic disease, but their benefits on ambulatory care quality are uncertain and most ambulatory practices do not currently have EHRs.⁴⁸ In addition, most EHRs do not integrate educational systems and patient support tools. The current ubiquity of Internet access positions the Internet as a promising tool for implementation research and QI. Given major initiatives to increase the meaningful use of health information technology and to improve health information technology systems with performance bonuses for tools that ensure patient-centered, appropriate care,⁴⁹ multi-component, Internet-delivered interventions may be feasible and cost-effective strategies to improve ambulatory care quality. Some previous Internet-delivered provider educational interventions have improved single performance measures where baseline performance was low.^36,50 However, consistent with previous research,^37,42 our results suggest that “light touch” educational interventions are less likely to be effective QI strategies when baseline performance is high, societal efforts targeting the same goals are underway, or when targeting a complex set of process measures.

Conclusion

We demonstrated that a longitudinal provider-directed, multi-component, Internet-delivered intervention improved only 1 of 7 clinical indicators of cardiovascular management performance for outpatients who survived MI. Light touch, educational interventions may improve quality less effectively when baseline performance is high or when targeting a complex set of process measures.