Practice-Level Effects of Interventions to Improve Asthma Care in Primary Care Settings: The Pediatric Asthma Care Patient Outcomes Research Team

Abstract

Objective

To assess the practice-level effects of (1) a physician peer leader intervention and (2) peer leaders in combination with the introduction of asthma education nurses to facilitate care improvement. And, to compare findings with previously reported patient-level outcomes of trial enrollees.

Study Setting

Data were included on children 5–17 years old with asthma in 40 primary care practices, affiliated with managed health care plans enrolled in the Pediatric Asthma Care Patient Outcomes Research Team (PORT) randomized trial.

Study Design

Primary care practices were randomly assigned to one of two care improvement arms or to usual care. Automated claims data were analyzed for 12-month periods using a repeated cross-sectional design. The primary outcome was evidence of at least one controller medication dispensed among patients with persistent asthma. Secondary outcomes included controller dispensing among all identified asthmatics, evidence of chronic controller use, and the dispensing of oral steroids. Health service utilization outcomes included numbers of ambulatory visits and hospital-based events.

Principal Findings

The proportion of children with persistent asthma prescribed controllers increased in all study arms. No effect of the interventions on the proportion receiving controllers was detected (peer leader intervention effect 0.01, 95 percent confidence interval [CI]: −0.07, 0.08; planned care intervention effect −0.03, 95 percent CI: −0.09, 0.02). A statistical trend was seen toward an increased number of oral corticosteroid bursts dispensed in intervention practices. Significant adjusted increases in ambulatory visits of 0.08–0.10 visits per child per year were seen in the first intervention year, but only a statistical trend in these outcomes persisted into the second year of follow-up. No differences in hospital-based events were detected.

Conclusions

This analysis showed a slight increase in ambulatory asthma visits as a result of asthma care improvement interventions, using automated data. The absence of detectable impact on medication use at the practice level differs from the positive intervention effect observed in patient self-reported data from trial enrollees. Analysis of automated data on nonenrollees adds information about practice-level impact of care improvement strategies. Benefits of practice-level interventions may accrue disproportionately to the subgroup of trial enrollees. The effect of such interventions may be less apparent at the level of practices or health plans.

Asthma is responsible for substantial morbidity as measured by symptom burden, functional impairment (Fowler, Davenport, and Garg 1992; Maier et al. 1998; Newacheck and Halfon 2000; Annett 2001), and health care utilization (Lozano et al. 1997; Weiss, Sullivan, and Lyttle 2000) among 5.6 percent of U.S. children (Mannino et al. 2002). Treatment guidelines, developed by the National Asthma Education and Prevention Project (NAEPP), were designed to improve and standardize diagnosis and treatment. These were initially promulgated in 1991 (National Asthma Education Program 1991) and revised in 1997 (National Asthma Education and Prevention Program 1997). However, a number of studies have documented suboptimal care for children with asthma, well after their publication (Jatulis et al. 1998; Legoretta et al. 1998; Finkelstein et al. 2000, 2002; Diette et al. 2001). Some of the reasons for slow guideline adoption have been well-documented (Lomas et al. 1989; Cabana et al. 1999), and are not surprising given the limited ability of passive dissemination strategies to change physician behavior (Davis et al. 1995; Soumerai, Mujumdar, and Lipton 2000).

The Pediatric Asthma Care Patient Outcomes Research Team (PAC PORT) designed and fielded a randomized controlled trial to test two strategies for implementation of the NAEPP guidelines in primary care practices belonging to one of three geographically separated health systems. One strategy used practice-based physician peer leader education (PLE) to engage and activate a physician change agent within a practice group. The other involved a more comprehensive intervention that similarly trained peer leaders, but added an asthma nurse educator to implement organizational change (Planned Asthma Care) within the practice, based, in part, on a model for the optimal treatment of chronic disease (Wagner 1998; Bodenheimer, Wagner, and Grumbach 2002). Previously published parent-reported outcomes of the interventions included fewer symptom days, by parent report (using 14-day recall periods), among enrollees in the PLE and Planned Care practices of 6.5 (CI: −16.9, 3.6) and 13.3 (CI: −24.7, 2.1) days per year, respectively, as well as lower rates of steroid bursts in the two intervention arms (Lozano et al. 2004). As both interventions attempted to change physician behavior and asthma management strategies in a practice overall, they might have been expected to improve asthma care for all of the children served by the practice, not only the 638 trial enrollees from whom detailed self-reported longitudinal outcome data were collected.

Because the participating practices were affiliated with managed health care plans, automated data were available for measurement of medication dispensing and health care utilization outcomes on the entire population served by a practice. The specific aims of the current analysis were to determine the effect of each intervention on (1) the rates of appropriate controller medication use (primary outcome); (2) asthma exacerbations as measured by dispensings of oral steroid courses; and (3) medical care utilization including hospital-based and ambulatory visits.

METHODS

The PAC PORT was a cluster-randomized trial designed to test the effectiveness of two interventions for primary care-based asthma care improvement, as compared with usual practice (Lozano et al. 2004; Weiss et al. 2005). Practice groups (here defined as a group of clinicians practicing in common space and sharing coverage of a group of patients) affiliated with three health care systems consented to participation prior to randomization to one of two intervention arms (PLE or Planned Care intervention) or to control (usual care) status. Study practices were affiliated with a participating managed care organization (MCO) in western Washington State, Chicago, or eastern Massachusetts. The 15 Washington state practices were clinics of a single-insurer group-model HMO. The 11 Chicago-area practices were part of staff model MCO or network divisions of a mixed model health plan. The eastern Massachusetts practices were independent group practices affiliated with several health plans; two of the largest regional insurers participated. For consistency, we refer to this group of practices as a third health plan, although data are drawn from both organizations. This study was approved by the Institutional Review Boards at all study sites.

Randomization was performed using a modification of the Finite Selection Model (FSM) (Morris 1979) to help ensure balance among study arms given the relatively small number of units randomized. Practices were characterized according to size and baseline steroid prescribing rate. Then, similar practices were assigned, by a computer program, to a study arm or to “usual care.” The FSM was implemented within health plan, and occurred in stages, as not all eligible practices were identified at the start of the study. Forty-two practices were initially randomized, but automated outcome data was not available for two; the remaining 40 practices are included in this analysis.

Study Interventions and Usual Care

Practices randomized to the PLE intervention selected a physician leader to serve as a “champion” for asthma care improvement. The primary target of care improvement was increased use of antiinflammatory controller agents for children with persistent asthma. Peer leaders attended two 2–3 hour workshops on the evidence supporting the NAEPP guidelines and to learn strategies for physician behavior change. They were provided with a toolkit of materials to facilitate physician behavior change in their practice consisting of the guidelines and supporting materials and practice aids (e.g., pocket cards detailing the NAEPP approach) developed using principles of academic detailing and physician behavior change (Soumerai and Avorn 1990). Peer leaders were supported over the following 2 years by an asthma education coordinator. This experienced asthma nurse educator attempted to contact peer leaders monthly or bimonthly to encourage and support their work. Peer leaders were also invited to participate in a voluntary learning network with other peer leaders on monthly telephone calls. Feedback on antiinflammatory prescribing for each member of a practice was transmitted through its peer leader. Practices randomized to the Planned Care Intervention also selected a peer leader who received all of the training and materials detailed above, but also received the services of an asthma nurse educator. This individual had in-person and telephone contacts with enrolled families to assess symptoms and medication use, support care planning in conjunction with the primary provider, and provide self-management support to families regarding medication adherence and trigger avoidance. Such outreach by the nurse educator was limited to the subset of patients enrolled in person-level data collection for the trial (a small fraction of children included here). Asthma nurses also reviewed panel reports on asthma medication use and emergency department (ED) utilization with physicians. Practices randomized to the control arm received copies of the NAEPP guideline, but no other materials or support from the study.

Subjects

Patients belonging to participating practices were identified in each health plan using automated data. All outcomes were measured in 12-month time periods to eliminate issues of seasonal variation: baseline (12 months before the start date of the intervention), intervention year 1 (the 12 months after the intervention start), and intervention year 2 (12–24 months after the intervention start). Patients were eligible for analysis in one or more of these periods if they met the following inclusion criteria: continuous health plan enrollment and assignment to a study practice for the 12-month period, age 5–17, and claims-based evidence of at least one ER, hospital, or ambulatory encounter for asthma. The intervention period started in each Planned Care or PLE practice with the first workshop for the identified peer leader; asthma nurse educator activities, including individual patient visits, also commenced after this first workshop. The dates of the initial workshops varied by practice and health plan, and were implemented over a 1-year period from September 1997 to October 1998. For the usual care practices, we defined the beginning of the time windows of interest as the median start date of the intervention within a health plan. (In a confirmatory analysis we determined that our results were not sensitive to selection of different start dates for usual care practices.)

Six hundred and thirty-eight individuals were enrolled in the trial with an in-person intake visit and had telephone contacts every 8 weeks to measure asthma symptoms, functional impact, and medication adherence. However, for the current analysis, we utilized automated pharmacy and claims data from the affiliated MCOs to measure outcomes on all asthma patients within a practice, whether or not they were enrolled in the trial. As trial enrollees made up a small minority (≤10 percent) of the patients included in this practice-level analysis, the effect of this intervention on the practice as a whole would have resulted from the activities of the peer leader and asthma nurse to change physician treatment of children not enrolled in the trial.

Data Collection and Analysis

Claims for ambulatory encounters, ED visits and hospital admissions included dates of service and International Classification of Diseases, 9th revision (ICD-9) diagnosis codes. Pharmacy dispensing files included the date and National Drug Code (NDC) of each prescription filled (or refilled) by health plan members. Drugs categorized as controllers included inhaled corticosteroids, cromolyn/nedocromil, theophylline, and leukotriene antagonists, although the last two classes accounted for small numbers of dispensings.

The primary outcome (Fuhlbrigge et al. 2004) was the proportion of children dispensed a controller among those meeting at least one of the criteria for persistent asthma developed for the National Committee on Quality Assurance (NCQA) Health Plan Employer Data and Information Set (HEDIS) measure. The HEDIS criteria for persistent asthma include any of the following measured over a 1-year period: ≥4 asthma medication dispensing events; ≥1 ED visit or hospital admission for asthma (principal diagnosis); ≥4 ambulatory visits with asthma and≥2 asthma medication dispensing events. A secondary pharmacotherapy measure that assessed more chronic controller use was the proportion of such children with three controller dispensings during the 1-year period. We also separately analyzed the change in use of inhaled corticosteroids in particular, as encouraging their use was a particular focus of both interventions. In a secondary analysis, we applied the measures of medication use to all identified asthma patients (not restricted to the HEDIS definition). Other measures were designed to assess the rates of potentially preventable asthma events, including the proportion of patients with either a hospital admission or an ED visit. We also analyzed the proportion of children, among all identified asthmatics, receiving a dispensing of an oral steroid preparation as a proxy for the occurrence of an acute exacerbation. Finally, we measured the rates of ambulatory and hospital encounters (including ED visits and hospital admissions) with an ICD-9 diagnosis of asthma.

For each process or outcome measure, the change in each measure for practices in an intervention arm was compared with the change among the usual care practices. Proportions (e.g., the proportion of patients receiving an antiinflammatory agent) were analyzed assuming binomial distributions. The number of ambulatory asthma visits was analyzed as an ordinal variable. In each case, we conducted bivariate analyses to identify variation among the three health systems and possible imbalances among the treatment arms by logistic regression (accounting for overdispersion) for dichotomous outcomes and analysis of variance (ANOVA) for ordinal variables. We report, for each variable, the unadjusted mean change from baseline of practices in each study arm. We also measured the adjusted estimate of change attributable to the intervention (beyond the change observed in the control practices). For this adjusted analysis, log rate ratios were estimated using optimal estimating equations for loglinear modeling at the patient level, accounting for clustered responses by practice using generalized estimating equations (GEE) (SAS PROC GENMOD). All analyses used the SAS statistical software, version 8 (SAS Institute, Greenborough, NC, USA).

Power

All outcome measures are reported with 95 percent confidence intervals (CIs) which themselves provide information on the effect that is statistically excluded, given the results observed (Goodman and Berlin 1994). Simple power calculations do not account for the effect of clustering in cluster-randomized trials. Therefore, we conducted a series of simulations to further understand the statistical power of the study to detect clinically important differences between intervention and control practices. A logistic model was created to generate binary responses, consistent with an overall specified effect size, using the actual number of patients per practice, and the distribution of practices across health plans included in all arms of the study. Two hundred simulations were conducted and analyzed, using the same methods as the original analysis, with calculation of the fraction of simulations for which a p-value of <.05 was achieved (power).

RESULTS

Automated data were available on patients in 40 practices randomized to one of the three study arms. The number of children enrolled in these practices and insured by the health plan for the full baseline year, with at least one encounter for asthma, was 5,169 (2,492, 1,647, and 1,030, belonging to each of the three health systems, respectively). Of these, 1,796 met the HEDIS criteria for persistent asthma (711, 621, and 464 for the three health systems, respectively). The total number of eligible patients with at least one asthma encounter was 3,843 and 3,440 in the 2 follow-up years, respectively. The number of asthma patients identified in the health plan data per practice ranged from 12 to 441 (median 108), and the number with persistent asthma ranged from 6 to 155 (median 37) (Table 1). All 28 peer leaders attended the first workshop and 24 (86 percent) attended the second. Peer leaders reported conducting a mean±SD of 9.5±6.9 (range 2–27) small group sessions in their practices over the 2-year study period.

Table 1.

Characteristics of 40 Participating Practices in the Pediatric Asthma Care PORT

Participating Practices	Study Arm	MD Providers (N)	PNP and PA Providers (N)	Asthma Patients at Baseline (N)	Patients with Persistent Asthma at Baseline (N)
Health plan 1	PLE	4	0	99	28
		2	0	441	108
		3	1	318	69
		8	0	82	32
		6	0	108	20
	PACI	6	0	171	51
		6	0	214	63
		6	0	188	67
		4	0	65	24
		6	0	133	36
	Usual care	5	0	154	48
		5	0	109	38
		5	1	139	58
		5	0	158	40
		6	1	113	29
	Total	77	3	2,492	711
Health plan 2	PLE	6	4	170	52
		6	0	106	49
		2	3	59	26
		5	0	60	25
		5	2	89	33
		3	2	80	27
	PACI	3	0	85	24
		8	0	176	61
		4	0	73	25
		8	0	159	54
		5	0	70	22
	Usual care	6	0	12	6
		2	0	71	31
		5	2	250	101
		8	0	89	37
		7	5	98	48
	Total	83	18	1,647	621
Health plan 3	PLE	2	0	46	22
		10	1	332	155
		7	0	13	9
	PACI	5	0	148	56
		5	0	47	26
		3	0	106	38
	Usual care	6	0	129	57
		5	1	133	71
		1	1	76	30
	Total	44	3	1,030	464
	Overall total	204	24	5,169	1,796

PNP, pediatric nurse practitioner; PA, physician assistant; PLE, peer leader education; PORT, Patient Outcomes Research Team.

The primary outcome, controller use, was measured both as the proportion of patients receiving at least one controller dispensing and the proportion receiving three or more dispensings to represent chronic use. Table 2 presents baseline values as the mean proportion among practices, summarized by health plan and arm. Assessment of imbalance among treatment arms at baseline, using a criterion of p≤.10, detected imbalance in controller medication use, baseline oral steroid use and ambulatory visits.

Table 2.

Mean (SD) of Baseline Values of Outcome Variables for Primary Care Practices Enrolled in the PAC PORT Trial by Study Arm and Health Plan

		Study Arm
Outcome	Health Plan	PLE	PCI	Control	p
Dispensed ≥1 controller* (y/n)	1 (n =15)	0.64 (0.11)	0.76 (0.12)	0.67 (0.06)
	2(n =16)	0.75 (0.13)	0.77 (0.12)	0.85 (0.13)
	3 (n =9)	0.59 (0.22)	0.79 (0.21)	0.75 (0.17)
	Overall	0.68 (0.15)	0.77 (0.13)	0.76 (0.14)	.10†
Dispensed ≥3 controller* (y/n)	1 (n =15)	0.36 (0.07)	0.41 (0.09)	0.36 (0.07)
	2 (n =16)	0.44 (0.10)	0.42 (0.11)	0.52 (0.14)
	3 (n =9)	0.33 (0.20)	0.54 (0.22)	0.40 (0.21)
	Overall	0.39 (0.12)	0.44 (0.13)	0.43 (0.15)	.16†
Dispensed ≥1 course oral steroid	1 (n =15)	0.24 (0.05)	0.28 (0.06)	0.32 (0.03)
	2 (n =16)	0.35 (0.05)	0.28 (0.06)	0.30 (0.05)
	3 (n =9)	0.43 (0.11)	0.18 (0.09)	0.30 (0.06)
	Overall	0.33 (0.10)	0.26 (0.07)	0.31 (0.04)	.07†
Had ≥1 ED visit or hospitalization	1 (n =15)	0.05 (0.03)	0.06 (0.04)	0.08 (0.03)
	2 (n =16)	0.08 (0.04)	0.07 (0.04)	0.09 (0.06)
	3 (n =9)	0.35 (0.23)	0.09 (0.04)	0.15 (0.08)
	Overall	0.13 (0.15)	0.07 (0.04)	0.10 (0.06)	.22†
Number of ambulatory asthma visits	1 (n =15)	1.64 (0.11)	1.62 (0.14)	1.71 (0.13)
	2 (n =16)	1.99 (0.40)	1.88 (0.17)	1.90 (0.30)
	3 (n =9)	1.46 (0.74)	1.92 (0.21)	2.66 (0.23)
	Overall	1.75 (0.45)	1.79 (0.21)	2.00 (0.44)	.07†

^*Controller medications include inhaled corticosteroids, cromolyn/nedocromil, long-acting β-agonists, and theophylline. Values for controller use shown are for patients who meet HEDIS criteria for persistent asthma.

^†p-value for imbalance among study arms at baseline using robust logistic regression, accommodating overdispersion, with health plan and study arm as main effects.

^‡p-value for imbalance among study arms at baseline using analysis of variance (ANOVA).

PAC PORT, Pediatric Asthma Care Patient Outcomes Research Team; HEDIS, Health Plan Employer Data and Information Set.

Medication Outcomes

The unadjusted mean change in the proportion of patients in each practice receiving medications of interest between the baseline and second intervention year (Table 3) shows substantial increase in controller use in all study arms. Although the magnitude of change is larger for several outcomes in intervention arms, no statistically significant differences between intervention and control arms were detected in the change in the proportion of patients with persistent asthma (using the HEDIS definition) dispensed a single controller medication or dispensed these medicines chronically (≥3 dispensings). We conducted secondary analyses to examine changes in controller use among all identified children with asthma (rather than limiting the analysis to those with persistent asthma). Again, no statistically significant increases beyond that seen in the control arm are detected, although the magnitude of intervention effect appears greater than the secular trend. Finally, we examined the change in the proportion of children dispensed inhaled corticosteroids, as increased use of these medications was a particular target of the intervention. There are apparent increases in the use (at least one dispensing) of these medications across all arms, with from 12 to 19 percent more children with persistent asthma receiving these medications in the intervention years. However, the changes in the PLE and Planned Care arms are, again, not significantly different from controls.

Table 3.

Medication Dispensing Outcomes in Study Arms in Second Follow-Up Year: Unadjusted Mean Change from Baseline and Adjusted Intervention Effect

	Mean Absolute Change from Baseline Proportion in Each Practice (95% CI)			Adjusted Intervention Effect: Absolute Change beyond That Observed in Control Practices (95% CI)*
Outcome	PLE	PCI	Control	PLE	PCI
Among persistent asthmatics†
≥1 controller dispensed	0.09 (0.01, 0.17)	0.04 (−0.02, 0.1)	0.04 (−0.04, 0.12)	0.01 (−0.07, 0.08)	−0.03 (−0.09, 0.02)
≥3 controllers dispensed	0.07 (−0.01, 0.15)	0.11 (0.05, 0.17)	0.01 (−0.09, 0.11)	0.02 (−0.06, 0.10)	0.03 (−0.04, 0.10)
≥1 inhaled corticosteroid	0.19 (0.09, 0.29)	0.17 (0.08, 0.26)	0.12 (−0.01, 0.25)	0.02 (−0.11, 0.16)	−0.02 (−0.13, 0.09)
≥3 inhaled corticosteroid	0.14 (0.06, 0.22)	0.13 (0.08, 0.18)	0.04 (−0.07, 0.15)	0.07 (−0.02, 0.15)	0.03 (−0.04, 0.10)
Among all patients with asthma
≥1 controller dispensed	0.16 (0.08, 0.24)	0.13 (0.07, 0.19)	0.07 (−0.01, 0.15)	0.03 (−0.08, 0.15)	0.04 (−0.06, 0.14)
≥3 controllers dispensed	0.08 (0.02, 0.14)	0.10 (0.06, 0.14)	0.04 (−0.02, 0.10)	0.02 (−0.05, 0.09)	0.04 (−0.02, 0.09)
≥1 inhaled corticosteroid	0.18 (0.1, 0.26)	0.17 (0.11, 0.23)	0.10 (0.00,0.20)	0.05 (−0.08, 0.17)	0.04 (−0.06, 0.14)
≥3 inhaled corticosteroid	0.09 (0.03, 0.15)	0.09 (0.07, 0.11)	0.03 (−0.03, 0.09)	0.04 (−0.02, 0.10)	0.03 (−0.02, 0.07)
≥1 oral steroid dispensed‡	0.05 (0.00, 0.10)	0.04 (0.00, 0.08)	0.02 (−0.01, 0.05)	0.06 (0.00, 0.12)	0.07 (−0.02, 0.15)

^*Additional change in absolute proportion of children beyond that seen in control practices by patient-level analysis using generalized estimating equations accounting for clustering of observations within practice.

^†Persistent asthma defined using HEDIS criteria.

^‡Among all asthma patients, except for those receiving chronic oral steroids.

PLE, peer leader education; HEDIS, Health Plan Employer Data and Information Set; CI, confidence interval.

Changes in medication use in intervention practices beyond those seen in control practices, analyzed at the patient level adjusted for clustering using GEE, are also shown in Table 3. In general, the adjusted increases in dispensing of controller agents were slightly greater in the intervention practices, but the magnitudes of these intervention effects are small, and did not reach statistical significance.

We examined the change in the proportion of children dispensed an oral corticosteroid as a proxy for the number of asthma exacerbations. The GEE analysis showed no intervention effect on oral steroids bursts in year 1, but a statistically borderline trend toward increased oral steroid (absolute increase of 6–7 percent above the change in usual care) use in intervention arms at year 2 compared with baseline (Table 3).

Medical Care Utilization

A goal of both interventions, but particularly Planned Care, was to encourage a more organized, proactive approach to ambulatory preventive care, and to encourage asthma “check-up” visits (focused on patient education and anticipatory management) in particular. The unadjusted data suggest that patients of practices in both intervention groups increased their rates of ambulatory visits. In the first intervention year, patients in PLE increased office visits by 0.26 (CI: 0.13,0.39) visits per year, and those in Planned Care practices increased them by 0.27 (CI: 0.12, 0.42) visits per year. Patients in control practices did not have increased ambulatory visit rates (0, CI: −0.2, 0.2). The adjusted analysis confirmed a statistically significant increase (8–10 percent increase beyond control practices) in the first year. However, by the second intervention year (Table 4), the differences in ambulatory visit rates between intervention and control groups were smaller. Adjusted increases of only 6–8 percent in intervention practices were seen in the second follow-up year compared with baseline, and were not statistically significant. There were no differential changes in rates of hospital services, including both ED and inpatient care, for either of the intervention arms compared with control practices.

Table 4.

Health Care Utilization: Unadjusted Mean (SEM) Change from Baseline in PAC PORT Outcomes by Study Arm and Adjusted Absolute Change beyond That Seen in Usual Care in Follow-Up Year 2

	Mean Absolute Change from Baseline in Each Practice (95% CI)			Adjusted Intervention Effect: Absolute Change Beyond That Observed in Control Practices (95% CI)*
Outcome	PLE	PCI	Control	PLE	PCI
≥1 ED/hospitalization†	−0.01 (−0.05, 0.03)	0 (−0.01, 0.01)	−0.01 (−0.04, 0.02)	0 (−0.06, 0.06)	0.03 (−0.003, 0.06)
Ambulatory visits (N)	0.17 (−0.01, 0.35)	0.21 (0.03, 0.39)	−0.01 (−0.23, 0.21)	0.06 (−0.02, 0.14)	0.08 (−0.01, 0.18)

^*Additional change beyond that seen in control practices by patient-level analysis using generalized estimating equations accounting for clustering of observations within practice.

^†Proportion of children.

PLE, peer leader education; PAC PORT, Pediatric Asthma Care Patient Outcomes Research Team; CI, confidence interval.

Power

The CIs around the result for each outcome provide an indication of the effect sizes statistically incompatible with those observed. The analysis of simulated data (see methods) confirms that the study had reasonable power to detect moderate differences in the processes of care studied. For example, this study had power of 0.8 to detect an absolute increase in children dispensed at least one controller of 7 percent points (in the planned care arm) beyond the change seen in control practices. Among patients with persistent asthma, the power to detect an absolute intervention impact of 7 percent was only 0.66. Similarly, we had power of 0.62 to detect an intervention impact of 3 percent fewer children requiring hospital-based care. This would be a substantial improvement as only 6 percent of children in the control group had ED visits or hospitalizations at baseline.

DISCUSSION

This analysis did not reveal statistically significant differences in medication use in practices in either intervention arm compared with those in the control arm. Therefore, contrary to findings from our analysis of trial enrollees, we find no evidence of change in asthma medication use at the practice level from either of these interventions. We recently reported (Lozano et al. 2004), among 638 study subjects enrolled in this trial, a statistically significant reduction of 13.3 (CI: 2.1, 24.7) asthma symptom days per year for the Planned Care intervention and a 39 percent decrease in the need for oral steroid bursts. Enrolled patients in practices receiving the peer leader intervention had a nonsignificant decrease of 6.5 (CI: −3.6, 16.9) symptom days, and a 36 percent decrease in oral steroid bursts. Both interventions resulted in small improvements in asthma functional status measures among enrolled patients (Lozano et al. 2004). Several explanations exist for the contrasting results between the prior assessment of trial enrollees and the practice level analysis presented here. First, trial enrollees accounted for only 7, 10, and 8 percent of subjects in the three successive years included in this analysis. Enrollees in the PCI arm received direct services from a dedicated nurse educator, which were not made available to nonenrollees, so this practice-level analysis does not measure the full impact of that intervention (only the practice-wide “spill-over” effects of the nurse in conjunction with the activities of the assigned peer leader). Secondly, enrollees represented a subgroup of patients with asthma who met a number of screening criteria and volunteered to participate in the intensive data collection activities of a randomized trial, including in-person intake and exit interviews and bimonthly telephone contacts. They likely differed in disease attributes (e.g., severity) as well as other characteristics that resulted in their volunteering for the trial. Finally, measurement of outcomes using automated data sources may not be as sensitive in detecting certain differences apparent from patient self-reports.

Although this analysis did not detect differences in medication use, both interventions did appear to result in a small increase in the rate of ambulatory visits in the first intervention year. Such visits are recommended by national guidelines (National Asthma Education and Prevention Program 1997), and can be viewed as a necessary first step for other changes in management to occur. In the second study year, the magnitude of the intervention effect on ambulatory visits was roughly similar to the first year, although not statistically significant.

The interventions in this trial were developed on the basis of literature documenting effective methods of physician behavior change (Soumerai, McLaughlin, and Avorn 1989; Soumerai et al. 1998; Soumerai, Mujumdar, and Lipton 2000) and other literature suggesting how structural change in medical care practice can effectively improve care for chronic conditions (Wagner 1998; Bodenheimer, Wagner, and Grumbach 2002). The peer leader intervention was designed to translate the underlying rationale and some of the methods from earlier successes using face-to-face academic detailing (Soumerai and Avorn 1990) and local opinion leader interventions (Lomas et al. 1991; Soumerai et al. 1998) into a form that would be suitable for large scale dissemination in modern managed care practice. It was relatively low cost, as it did not require the addition of clinical staff or major infrastructure, but relied heavily on the volunteerism and extra time and effort of individual clinicians willing to take on a leadership role. The Planned Care intervention was designed to implement aspects of the Chronic Care Model (Wagner 1998) for children with asthma, although it lacked the overall change in practice structure called for by the Model. The patient level results previously reported (Lozano et al. 2004) show substantial benefit to individuals, although these are not confirmed in this practice level analysis. Any practice-wide effect of the Planned Care nurse on the rest of the practice (measured in this analysis) would be indirect through her modeling of care, including an approach to standardized assessment, planned asthma visits, regular follow-up calls, joint doctor/nurse patient discussions, and efforts to enhance patient self-management. As the current analysis includes only a small number of patients who actually had contact with the asthma nurse educator, it does not measure the effect of the major aspect of the Planned Care intervention. We have no data on whether full implementation of the Chronic Care Model, including implementation of systematic quality improvement efforts in a practice could improve the asthma outcomes measured here.

Previous attempts to improve asthma care for children have provided asthma education and self-management tools directly to patients and their families through asthma nurse educators, (Greineder, Loane, and Parks 1999) social workers (Evans et al. 1999), and personnel in specialty clinics (Kelly et al. 2000). Many direct-to-patient strategies have shown positive effects. One provider intervention in 41 practices showed no impact on quality of life and a trend toward a small (3 percent) increase in steroid prescribing (Premaratne et al. 1999). Another showed marked improvements in care with a very intensive intervention (five, 3-hour sessions and additional tutorials) in urban public health clinics, but these practices began with much lower baseline asthma care quality than was found in the delivery systems studied here (Evans et al. 1997). Finally, an intervention by Clark et al. (1998) implemented two seminars for providers and reported improvements in care including increased antiinflammatory prescribing. Clark et al. also showed a reduction in the number of nonemergent ambulatory encounters for asthma, which is opposite to the effect seen here, perhaps because the current intervention encouraged additional asthma planning visits. Our peer leader intervention might be expected to have a smaller magnitude of effect than was seen in some of these studies since we had direct contact with only one provider in each practice, rather than all providers. We encouraged and supported the peer leader to disseminate care improvement strategies to colleagues. Not surprisingly, there was great variability in both the number and content of the activities undertaken.

In measuring the effect of interventions in the setting of a randomized trial, it is useful to have parallel sources of outcomes data. Our intent in the current analysis was to test whether similar intervention effects would be found among all asthma patients in these practices and among trial enrollees. In contrast to a trial of a new drug, interventions to change physician behavior or processes of care are designed to have substantial effects on care for all patients in a practice. This level of generalizability is often assumed, but the availability of automated health plan data in the practice settings in which our trial was conducted allowed us the opportunity to assess outcomes in both trial enrollees and in the population from which they were drawn. The fact that we did not find practice-level differences using automated data suggests either that no practice-level changes occurred beyond those seen in control practices, or that measurement over a longer time period (or with larger populations) would be necessary to detect such changes. The high baseline rate and marked increase in controller use in usual care practices over the 3-year observation period may also have made detection of intervention effects more difficult.

The statistically nonsignificant results for the primary outcome measure of this study raise the question of its power to detect clinically important effects given the sample sizes studied, the variability in the outcome variables, and the nonindependence (clustering) of patients within practices. Assessment of the power of the study show that we had adequate power to detect moderately large differences in medication use, but not smaller differences. While we do not statistically rule them out, such small practice-wide differences in medication use would be unlikely to justify the effort and cost to implement the interventions studied here. The positive intervention effect seen on symptoms and functional measures in the subgroup of enrolled patients (Lozano et al. 2004) might still be justification for such interventions, depending on how these outcomes are valued by parents and health care purchasers (Sullivan et al. 2005).

This analysis is also subject to the limitations of automated data to measure health outcomes. As the claims data analyzed are primarily collected for purposes of payment, some misclassification undoubtedly exists. And, measurement of some elements, such as the use of inhaled medications, is imprecise as we can only identify that a canister was dispensed by a pharmacy—regardless of whether 1 puff or the entire canister was actually administered. On the other hand, health plan data sources have been shown to be valid and useful sources to measure medication and health care utilization across a range of conditions (Platt et al. 2001; Selby 2001).

In summary, while both the PLE and Planned Care interventions demonstrated improvement in self-reported outcomes among trial subjects, we did not detect practice-level effects on controller medication prescribing or hospital or ED visits using automated data. This is the type of data available to most health plans, and the type of data by which health plan performance is frequently judged (e.g., NCQA HEDIS measures). These data do not negate the effect of direct contact with the nurse educator previously reported (Lozano et al. 2004), but suggest that there was no spill-over effect on asthma care in the practices as a whole, either from the nurse or the peer leader. The data from practices receiving only the peer-leader intervention, which do not confirm the benefits reported by trial enrollees, remind us that intervention effects seen in the setting of carefully controlled randomized trials may not directly extend to a wider patient population because of selection bias and related issues. Finally, the finding that, in the first intervention year, these interventions resulted in increased primary care contact is some basis for optimism that improving guideline-adherent asthma care in primary care practice is possible. Additional work is needed to refine the types of intervention strategies that will help practices and health plans realize the benefits of adoption of practice guidelines and innovative models for chronic and preventive care.