Statewide Asthma Learning Collaborative Participation and Asthma-Related Emergency Department Use

Abstract

BACKGROUND:

Quality improvement (QI) efforts can improve guideline-recommended asthma care processes in the pediatric office setting. We sought to assess whether practice participation in an asthma QI collaborative was associated with decreased asthma-related emergency department (ED) visits.

METHODS:

A statewide network of practices participated in a pediatric asthma QI collaborative from 2015 to 2016. We evaluated asthma-related ED visit rates per 100 child-years for children ages 3 to 21 years with asthma, using the state’s all-payer claims database. We used a difference-in-differences approach, with mixed-effects negative binomial regression models to control for practice and patient covariates. Our main analysis measured the outcome before (2014) and after (2017) the QI collaborative at fully participating and control practices. Additional analyses assessed (1) associations during the intervention period (2016) and (2) associations including practices partially participating in QI collaborative activities.

RESULTS:

In the postintervention year (2017), participating practices’ (n = 20) asthma-related ED visit rate decreased by 5.8 per 100 child-years, compared to an increase of 1.8 per 100 child-years for control practices (n = 15; difference in differences = −7.3; P = .002). Within the intervention year (2016), we found no statistically significant differences in asthma-related ED visit rates compared to controls (difference in differences = −4.3; P = .17). The analysis including partially participating practices yielded similar results and inferences to our main analysis.

CONCLUSIONS:

Participation in an asthma-focused QI collaborative was associated with decreased asthma-related ED visit rates. For those considering implementing this type of QI collaborative, our findings indicate that it takes time to see measurable improvements in ED visit rates. Further study is warranted regarding QI elements contributing to success for partial participants.

Asthma leads to substantial health care use for children, accounting for >10 million office visits and 1.6 million visits to emergency departments (EDs) annually.¹ Between 2001 and 2010, the rate of asthma-related ED visits remained high and unchanged.² In 2010, ED visits for asthma cost Medicaid >$272 million.³ National data indicated that children accounted for an increasing percentage of all asthma-related ED visits (from 36% in 2011 to 40% in 2016).⁴

Improving care in pediatric primary care may help improve asthma management and decrease exacerbations, thereby decreasing ED visits. The National Asthma Education and Prevention Program, sponsored by the National Heart, Lung, and Blood Institute (NHLBI), includes evidence-based recommendations to improve practitioner and patient management of asthma.⁵ Unfortunately, significant variability in primary care adherence to NHLBI guidelines persists.^6–8

Quality improvement (QI) in primary care provides mechanisms to address variation in guideline adherence and therefore potentially decrease ED use. Studies of asthma-focused QI in primary care have been focused on process measures like adherence to NHLBI guidelines: tracking asthma scores by using a validated tool,⁹ increasing use of asthma action plans,^9,10 increasing prescription of inhaled corticosteroids,^11,12 and increasing use of spirometry.¹³ Few studies of primary care, asthma-focused QI in which researchers examine ED use exist, and the results of these studies are mixed. In 1 study, researchers reported providing education to primary care practices and implementing clinical decision-making tools led to a decrease in ED visits and hospitalizations for pediatric asthma patients.⁶ This study was limited to 6 practices in 1 city, did not use a learning collaborative approach, and did not have a control group. In a more recent multistate study, researchers using a modified QI collaborative model reported improvements to clinical asthma management measures by primary care practitioners but no significant change in ED or urgent care use.¹⁴ This study did not report statistical tests on ED run chart data and did not have a control group.

The Vermont Child Health Improvement Program (VCHIP), the longest-running improvement partnership nationally,¹⁵ leads statewide QI collaboratives based on the Model for Improvement¹⁶ for pediatric-serving primary care practices.^17,18 A recent VCHIP QI collaborative, focused on asthma care and management in primary care, revealed significant improvement in multiple NHLBI guideline-recommended processes (eg, increased controller medication prescription);¹⁹ however, it was not investigated whether this also led to reduced asthma-related ED use. This presents an opportunity for a longer follow-up study of a primary care asthma-focused QI collaborative examining asthma-related ED use as the outcome. Our overall objective is to test whether participation in an asthma QI collaborative is associated with a decrease in asthma-related ED use over time, compared to controls.

METHODS

Study Setting

In 2012, VCHIP created Child Health Advances Measured in Practice (CHAMP), a voluntary network of pediatric and family medicine primary care practices working collaboratively on QI. Practices in the CHAMP network care for ~60% of children in Vermont. Each year, in collaboration with these practices and state partners, VCHIP selects a focus area for the CHAMP QI learning collaborative. After successful QI collaboratives on improving immunization rates¹⁷ and addressing adolescent depression,¹⁸ from mid-September 2015 through mid-May 2016, CHAMP practices worked collaboratively to improve asthma management.¹⁹ During the CHAMP asthma learning collaborative, VCHIP’s approach followed the Model for Improvement,¹⁶ a QI methodology designed to advance patient care and outcomes through setting an aim, measuring processes, and identifying areas of improvement through iterative Plan-Do-Study-Act cycles.^20,21 Full details of the asthma QI collaborative, clinical measures, and office systems strategies are published,¹⁹ and a brief summary of key elements is included below in the QI collaborative section.

Data Sources

Administrative claims spanning 2014–2017 from Vermont’s all-payer claims database (Vermont Health Care Uniform Reporting and Evaluation System) served as our primary source of data. Medicaid, Medicare, and most commercial insurers in Vermont submitted medical and pharmacy claims to this database for this period. Vermont’s all-payer claims database excluded the uninsured and included an estimated 90% of state residents receiving health care in 2014 and 2015. This percentage decreased to ~75% in 2016 and 2017 because of a Supreme Court ruling,²² which resulted in some commercial self-pay insurance plans opting out of submitting data. VCHIP collected data on practice characteristics (Table 1) from state administrative data and national registries and compiled these data annually as part of VCHIP’s attribution process.²³

TABLE 1.

Characteristics of the Participating and Control Practices

	Participating Practices (n = 20), n (%)	Control Practices (n = 15), n (%)	P
Practice size			.22
<500 patients	2 (10)	3 (20)
500–1499 patients	8 (40)	9 (60)
>1500 patients	10 (50)	3 (20)
Practice specialty			.87
Pediatrics	11 (55)	7 (47)
Family medicine	8 (40)	7 (47)
Mixed practitioners	1 (5)	1 (7)
Organization structure			.04
Hospital owned	12 (60)	3 (20)
Independently owned	8 (40)	12 (80)
HSA			.49
One metropolitan HSA	8 (40)	4 (27)
All other HSAs	12 (60)	11 (73)
FQHC or RHC			.27
FQHC or RHC	4 (20)	7 (47)
Neither FQHC nor RHC	16 (80)	8 (53)

Mixed practitioners include pediatricians and family medicine physicians. Unadjusted analyses were done by using Fisher’s exact tests at the practice level.

QI Collaborative

In brief, practitioners and practice staff in the CHAMP network interested in participating in the QI collaborative attended an all-day in-person learning session in mid-September 2015 covering topics on office systems strategies (eg, asthma assessment, control, and management and patient education). Practices received evidence-based resources, such as NHLBI guidelines and validated tools. Those that decided to participate in the QI collaborative subsequently attended monthly phone calls, including didactics on asthma topics (eg, the use of validated asthma control tests, implementing and documenting asthma action plans, and coding and billing) and knowledge sharing on successes and challenges. During the collaborative, practitioners worked to set practice goals, implement changes, and measure improvements with their practice teams. Participating practitioners submitted monthly convenience samples of patients to monitor progress on clinical asthma measures and received monthly feedback reports with summaries of their performance on the clinical measures, overall collaborative performance, and QI coaching notes. The main findings from this study revealed that practitioners exceeded their 20% improvement goals, aligning with NHLBI guidelines for the assessment of asthma with a validated tool, planned asthma visits, assessment of tobacco exposure, provision of asthma education, and instruction on asthma device use.¹⁹

Practice Participation

The VCHIP approach to QI is inclusive, encouraging primary care practices interested in the annual QI topic to send practitioners and staff to the all-day learning session, without requiring a commitment to participate in the subsequent QI collaborative. This inclusive approach led to the definition of 3 categories of practices for this study on the basis of level of participation: full participants, partial participants, and nonparticipants. Fully participating practices met 3 criteria: (1) attended the in-person learning session, (2) submitted medical record review data for 6 of 7 months, and (3) attended at least 1 all-practice conference call. Of 46 primary care practices in the CHAMP network in 2015, 20 fully participated in the asthma QI collaborative, including 2 practices with practitioners working across both sites and, later, merging. We included 15 nonparticipating CHAMP network practices as controls, and we designated the remaining 11 practices as partial participants because they attended the in-person learning session but did not attend monthly phone calls and submitted <6 months of QI data. For the partial participants, we did not collect information on the extent to which they did or did not implement asthma QI at their practices. Therefore, we excluded partial participants from our main analyses but included them in additional analyses described below.

Patient Population

Patients ages 3 to 21 years, who met criteria for identifiable asthma (Supplemental Table 5)^24,25 and who were attributed to primary care practices (see below), were included. In 2014, the data set included 9883 patients 3 to 21 years old with identifiable asthma, of whom we attributed 4589 patients to CHAMP network practices: 2376 at participating, 1282 at control, and 931 at partially participating practices. In 2017, the data set included 8677 patients 3 to 21 years old with identifiable asthma, of whom we attributed 4186 patients to CHAMP network practices: 2257 at participating, 899 at control, and 1030 at partially participating practices.

Attribution of Children to Practices

Attributing each child to a single practice each year allowed us to measure improvement at the practice level over time. There were 2 steps in our attribution of children to primary care practice, by using a hierarchical approach based on previous work by Christensen et al.²⁶ First, we selected a single primary care practitioner for each child out of all practitioners listed on claims within a year. Second, we linked the child to a practice on the basis of the chosen primary care practitioner from step 1. We describe the full details of our attribution of children to practices using Vermont’s all-payer claims database in a previous publication.²³

Outcome

The outcome measure was the ED visit rate (the number of visits per 100 child-years) for children ages 3 to 21 years with identifiable asthma.^24,25 The measure definition is from the Pediatric Quality Measures Program (PQMP), created under the Children’s Health Insurance Program Reauthorization Act, funded through the Centers for Medicare and Medicaid Services, and overseen by the Agency for Healthcare Research and Quality. The PQMP was established to address gaps in assessing quality in pediatric care and led to the development of numerous pediatric quality measures,²⁷ including the one used in this study.

The denominator and numerator definitions followed the asthma-related ED visit rate measure specifications developed²⁸ and refined through the PQMP.^24,25,27,29 Eligibility was assessed for each month of data. The criteria for the measure were (1) having ≤3 months of consecutive enrollment in the same insurance plan (the measurement month and the 2 months before) and (2) evidence of claims for identifiable asthma (see Supplemental Table 5 for definition) during a look-back period, including the measurement month, all previous months in the measurement year, and the year before the measurement year (see Supplemental Fig 2). The total number of eligible child-months in each year was summed and divided by 1200 to calculate the denominator in units of 100 child-years for the measurement year. The numerator was a count of all ED visits or hospitalizations with a first or second diagnosis of asthma in eligible patients for the measurement month. The inclusion of hospitalizations is in response to measure development work,³⁰ which revealed that the measure was more accurate when including hospitalizations because claims are often not submitted for ED care that leads to hospitalization. The inclusion of visits with an asthma diagnosis in the second position reflects measure development Delphi panel recommendations and exploratory analyses that found that claims with a second diagnosis of asthma often had a primary diagnosis with a related symptom (eg, fever or wheezing) or a known asthma trigger (eg, upper respiratory tract infection, pneumonia, or influenza).

Covariates

For comparing participating and control practices and for inclusion in the difference-in-differences analyses, we used the following practice characteristics (Table 1): size (<500, 500–1499, or >1500 patients), specialty (pediatrics, family medicine, or mixed, including pediatricians and family medicine physicians), organization structure (hospital owned versus not), hospital service area (HSA) location (located in the largest metropolitan HSA versus not), and federally qualified health center (FQHC) or rural health center (RHC) versus neither. We also included covariates for patient characteristics: age categories (3–5, 6–11, 12–17, or 18–21 years), sex (male, female, or unknown), and insurance (Medicaid versus not), for comparing participating and control practices (Table 2).

TABLE 2.

Characteristics of Patients at Participating and Control Practices Before (2014) and After (2017) the Asthma QI Learning Collaborative

	2014			2017
	Participating	Control	P	Participating	Control	P
	n (%)	n (%)		n (%)	n (%)
Total study sample	2376 (100)	1282 (100)	—	2257 (100)	899 (100)	—
Age categories, y			.047			<.001
3–5	453 (19)	206 (16)		430 (19)	160 (18)
6–11	935 (39)	498 (39)		868 (38)	326 (36)
12–17	839 (35)	478 (37)		854 (38)	327 (36)
18–21	149 (6)	100 (8)		105 (5)	86 (10)
Sex			.26			.32
Male	1193 (50)	644 (50)		1125 (50)	442 (49)
Female	1046 (44)	580 (45)		1009 (45)	419 (47)
Unknown	137 (6)	58 (5)		123 (5)	38 (4)
Insurance			.001			<.001
Non-Medicaid	897 (38)	412 (32)		627 (28)	183 (20)
Medicaid	1479 (62)	870 (68)		1630 (72)	716 (80)

P values are from bivariate χ² comparing patient-level percentages within demographic categories across participant and control practices within each year. —, not applicable.

Statistical Analyses

We used Fisher’s exact and χ² analyses, respectively, to compare practice and patient characteristics between participating and control practices.

To assess the relationship between asthma QI collaborative participation and asthma-related ED use, we used a difference-in-differences approach. This approach compared the asthma-related ED visit rate per 100 child-years before (2014) and after (2017) the QI collaborative, at participating versus control practices. The choice of 2017 reflects the assumption that improvements will continue past the end of the QI collaborative. We used a mixed-effects negative binomial multivariable regression model, with random intercepts for patient and practice to account for clustering within patient and practice. To estimate the difference-in-differences effect, models included variables for participation, year, and the interaction term between participation and year. The P value for the interaction term was used to test whether the change in ED use was different between participating and control practices. Practice (Table 1) and patient (Table 2) covariates were included in models as potential confounders. Adjusted ED rates at each time point were obtained by using the postestimation margins command in Stata version 15 (Stata Corp, College Station, TX).³¹ This is used to calculate a marginal (or average) rate by averaging predicted values derived for each observation, assuming it was in each of the intervention groups at each time point (but by using its covariate values for all the other variables to generate the predicted values). We tested for a difference in participating versus nonparticipating preintervention slopes from monthly ED visit rate data (January 2014 to September 2015) to ensure there were no preintervention trend differences. We ran the same model over time in months, including an interaction between participation and time, and we found no difference (P = .684; results not shown), thus supporting the notion that trends in the control practices serve as a reasonable counterfactual for what would have occurred to the participating practices in the absence of the QI learning collaborative.

Additional Analyses

We conducted two additional analyses to test the robustness of our findings and better understand the potential effects of the QI collaborative. We hypothesized that the effects of the asthma improvements may take some time to translate into changes in the distal use outcome. Thus, we repeated our analysis using an earlier time point (2016) for the postintervention period to examine whether immediate effects were apparent, although the QI collaborative was still active during the beginning of 2016. In addition, we hypothesized that the effect of the QI may be different in partially participating practices, so we repeated our analysis, including children with asthma from those partial participating practices.

All analyses were conducted in Stata version 15.1.³² The study was approved by the institutional review board (Committee on Human Research in the Medical Sciences 17-0232).

RESULTS

The 20 participating practices did not have significantly different characteristics compared to the 15 control practices, except there were a larger number owned by hospitals (Table 1). Participating practices had slightly younger children and fewer Medicaid patients in both 2014 and 2017 (Table 2), compared to control practices.

In our main analysis, the asthma-related ED visit rate per 100 child-years for participating practices decreased from 15.3 in 2014 to 9.5 in 2017, for an overall decrease of −5.8 (nearly 40%). Over the same period, the asthma-related ED visit rate per 100 child-years for control practices increased from 17.0 to 18.8, for an overall increase of +1.8. These findings led to a statistically significant adjusted difference in differences of −7.6 visits per 100 child-years associated with participation in the collaborative (Table 3; P = .002). Figure 1 reveals the ED visit rates over time, comparing participants to controls in 2014 and 2017. The dashed lines between our comparison years are data excluded from our analyses, revealing variation in the ED visit rates during that time. For point estimates from the full negative binomial model, including confounders, see Supplemental Table 6.

TABLE 3.

Comparison of Participating and Control Practice Mean Asthma-Related ED Visit Rates From Baseline and Postcollaborative Years (Main Difference-in-Differences Analysis)

Main Analysis	Asthma-Related ED Visits per 100 Child-Years	P	95% Confidence Interval
			Lower	Upper
Control 2014	16.97	—	11.99	21.95
Participating 2014	15.29	—	11.62	18.96
Control 2017	18.80	—	12.89	24.71
Participating 2017	9.50	—	7.07	11.93
Difference in differences	−7.62	.002	−13.45	−1.78

Marginal rates are adjusted for patient (age, sex, and insurance) and practice (specialty, organizational structure, size, geographic region, and FQHC or RHC) characteristics and, also, for patients clustering within practices; SEs are estimated by using the δ method; Coefficients for relative rates are in Supplemental Table 6. —, not applicable.

FIGURE 1.

Quarterly number of ED visits per 100 child-years for participants versus controls from 2014 to 2017, highlighting the QI collaborative (mid-September 2015 to mid-May 2016). Solid lines indicate the data analyzed and presented in the main analysis comparing the calendar year 2017 vs 2014.

Results from additional analyses comparing the change in ED visit rate (1) from 2014 to 2016 and (2) including partial participants are in Table 4. Comparing the slight decrease of −0.8 ED visits per 100 child-years for participating practices to the increase of +3.5 ED visits per 100 child-years for control practices, the overall adjusted difference in differences of −4.3 visits per 100 child-years was not statistically significant (P = .17). The analysis, including the partial with fully participating practices, yielded similar results and inferences as our main analysis, with an adjusted difference in differences of −7.3 visits per 100 child-years (P = .003).

TABLE 4.

Comparison of Participating and Control Practices Change in Mean ED Asthma Visit Rate per 100 Child-Years (Additional Difference-in-Differences Analyses) for (1) Comparison of Baseline (2014) to 2016 (Includes the Last 3 Months of the Learning Collaborative) and (2) Including Partial With Full Participants Compared to Control Practices

Additional Analyses	Asthma-Related ED Visits per 100 Child-Years	P	95% Confidence Interval
			Lower	Upper
2014–2016
Control 2014	17.93	—	13.83	22.04
Participating 2014	14.11	—	11.57	16.65
Control 2016	21.46	—	15.94	26.98
Participating 2016	13.33	—	10.96	15.70
Difference in differences	−4.31	.17	−10.49	1.88
Including partial participants
(2014–2017)
Control 2014	16.43	—	11.01	21.86
Participating and partial 2014	15.90	—	12.07	19.72
Control 2017	18.15	—	11.79	24.52
Participating and partial 2017	10.34	—	7.84	12.84
Difference in differences	−7.27	.003	−12.97	−1.58

Marginal rates are adjusted for patient (age, sex, and insurance) and practice (specialty, organizational structure, size, geographic region, and FQHC or RHC status). Because all participants are included in the marginal effect calculation, the control estimates for the same year across models are not the same (eg, control 2014 estimates across models). Analyses are also adjusted for patients clustering within practices; SEs are estimated by using the δ method. —, not applicable.

DISCUSSION

Our study of a statewide asthma QI collaborative was focused on a use measure, asthma-related ED visit rate, building on our previous findings that the QI collaborative led to improvements in care processes.¹⁹ We found that participation in the QI collaborative was associated with a substantial decrease of nearly 40% in asthma-related ED visit rates >1 year after the end of the collaborative. The large number of nonparticipating practices as a control group strengthens the rigor and validity of our findings by using a difference-in-differences approach. Our findings indicate that successful QI collaboratives, with documented improvements in NHLBI guideline adherent process improvements, are associated with a meaningful decrease in ED visits compared to controls. This is a potential benefit not only to public and commercial payers but also to children and parents, by avoiding missed school and work and, potentially, achieving an improved quality of life.³³

We found that there was a time lag to the association between participation in the QI collaborative and improved asthma-related ED rates. Although there was an improvement by 2016, it was not statistically significant. The improvement by 2017 was nearly twice that by 2016 and represented close to a 40% difference from the control group rate in 2014. Although the confidence intervals for these effect estimates overlap, there are several possible explanations for this lag worth mentioning. Over the course of a collaborative, process measures, like asthma control, take time to improve, and seeing improvement on downstream clinical outcomes, like ED visits, may take additional time. Also, asthma exacerbations are seasonal, with ED visits occurring often in winter months. The collaborative ended early in 2016, so calendar year data from 2016 included 3 winter months when the QI collaborative was still underway, capturing the higher rate of asthma ED visits, before participants had finished improvements. This may explain why the effect of the improvements was not apparent until the next year. Finally, in previous asthma clinical trials revealing that primary care interventions can deliver changes in ED outcomes in a shorter 1-year time frame,³⁴ researchers specifically recruited patients at “high risk” for asthma exacerbations (eg, enrolled only patients with a recent asthma-related ED visit). This focus on higher risk patients can lead to more rapid changes in ED use. In our study, we included all children with asthma (and did not use severity as eligibility criteria), which may also explain the longer observation period before detecting changes. This additional analysis may be useful to payers (or primary care practices participating in value-based accountable care organizations), to forecast potential decreased ED use through a primary care asthma QI collaborative.

We found that including patients from partially participating practices did not change our main results or inferences. One potential implication for those implementing a QI collaborative is that partial participation can still be associated with improvements in ED use. We would add the caveat that the partially participating practices in this study previously engaged in VCHIP QI collaboratives and, therefore, were not naïve to QI implementation. Full participation may be more important in a less experienced group of practices. In a future study, researchers using an implementation science approach (tracking types and extent of participation, previous QI experience, and asthma-related process and outcome measures) may more accurately pinpoint which components of the collaborative implemented in which practice settings most directly affect improvement in asthma health outcomes.

There were several limitations to our current study. Although we demonstrate difference in differences in ED visit rates from 2014 to 2017, during the intervening years, there is variation in the ED visit rates between the participating and control practices. It is possible that unmeasured confounders, such as other concurrent QI or public health efforts that differed across practices, or other unmeasured differences between participating and control practices, such as practice culture and management structure, contributed to the association between participation and decreased ED use and may limit the difference-in-difference approach. However, we accounted for secular trends and state-level efforts in our analysis. Also, the Supreme Court ruling²² resulting in a loss of some self-insured patients could have biased our results, if there was differential loss between participating and control practices. This may be why the percent of non-Medicaid patients decreased in both groups over time but the difference in the proportion of non-Medicaid patients between participating and control practices was stable over time. This study was also limited to children attributed annually to practices in a QI network, limiting the generalizability of our study findings to practices with previous exposure to QI. Finally, children were reattributed to practices each year, and we did not limit our analyses to only children attributed to the same practice over time. This attribution method could have biased our results to find a positive effect, if children with more severe asthma differentially moved away from participating practices to control practices over time, but there are no data to suggest this differential movement of patients with asthma.

CONCLUSIONS

Participation in a 9-month QI collaborative to improve primary care systems and supports for children with asthma led to a substantial decrease in asthma-related ED visits, with improvements continuing over time, reaching statistical significance a year after completion of the QI collaborative. Our study suggests that implementation of an asthma-related QI collaborative may be associated with reductions in asthma-related ED use, although the return on investments may take >1 year to realize after completion. Our network practices that participated partially in the QI collaborative activities still showed improvement in asthma outcomes, implying that practices benefited from partial participation. Future studies may better delineate which collaborative elements result in greatest benefit.

WHAT’S KNOWN ON THIS SUBJECT:

Participation in an asthma quality improvement learning collaborative in primary care is associated with improved processes of care in the office setting for children with asthma.

WHAT THIS STUDY ADDS:

Participation in an asthma quality improvement learning collaborative in primary care is associated with a substantial decrease in asthma-related emergency department visits >1 year after the end of the collaborative.

ABBREVIATIONS

CHAMP

Child Health Advances Measured in Practice

ED

emergency department

FQHC

federally qualified health center

HSA

hospital service area

NHLBI

National Heart, Lung, and Blood Institute

PQMP

Pediatric Quality Measures Program

QI

quality improvement

RHC

rural health center

VCHIP

Vermont Child Health Improvement Program