Respiratory Scores as a Tool to Reduce Bronchodilator Use in Children Hospitalized With Acute Viral Bronchiolitis

Abstract

BACKGROUND AND OBJECTIVES:

Adoption of clinical respiratory scoring as a quality improvement (QI) tool in bronchiolitis has been temporally associated with decreased bronchodilator usage. We sought to determine whether documented use of a clinical respiratory score at the patient level was associated with a decrease in either the physician prescription of any dose of bronchodilator or the number of doses, if prescribed, in a multisite QI collaborative.

METHODS:

We performed a secondary analysis of data from a QI collaborative involving 22 hospitals. The project enrolled patients aged 1 month to 2 years with a primary diagnosis of acute viral bronchiolitis and excluded those with prematurity, other significant comorbid diseases, and those needing intensive care. We assessed for an association between documentation of any respiratory score use during an episode of care, as well as the method in which scores were used, and physician prescribing of any bronchodilator and number of doses. Covariates considered were phase of the collaborative, hospital length of stay, steroid use, and presence of household smokers.

RESULTS:

A total of 1876 subjects were included. There was no association between documentation of a respiratory score and the likelihood of physician prescribing of any bronchodilator. Score use was associated with fewer doses of bronchodilators if one was prescribed (P = .05), but this association disappeared with multivariable analysis (P = .73).

CONCLUSIONS:

We found no clear association between clinical respiratory score use and physician prescribing of bronchodilators in a multicenter QI collaborative.

Acute viral bronchiolitis is 1 of the most common reasons for hospitalization among children.¹ No therapy has proven efficacious for all patients, and use of unnecessary therapy remains common.^2–8

Bronchodilators, in particular, are still substantially overused in bronchiolitis⁹ and consequently are an important target for quality improvement (QI) efforts.¹⁰ Clinical respiratory scores are 1 of the few specific QI tools available for use as an intervention intended to decrease bronchodilator overuse, and care pathways incorporating various respiratory scoring strategies have been reported to have a temporal association with decreased bronchodilator use.¹⁰ Such pathways sometimes set a minimum score as a threshold before allowing for administration of a bronchodilator¹¹ but more commonly attempt to use the score to determine bronchodilator responsiveness by scoring in a pre–post fashion in relation to a trial dose,^12–15 despite the fact that scores are not a validated measure of bronchodilator response in bronchiolitis.¹⁶

High uptake of respiratory score protocols for bronchiolitis was recently described in the Value in Inpatient Pediatrics–sponsored Bronchiolitis Quality Improvement Program (BQIP), a multisite improvement collaborative spanning 2 bronchiolitis seasons.¹⁷ In this collaborative, decreases in bronchodilator utilization metrics were temporally associated with an increase from a mean of 17.2% to 55% in patients evaluated using a respiratory score, suggesting that such improvement may be attributable to respiratory score use. However, this intervention could not be differentiated from other aspects of the QI program through time series analysis.

In this study, we performed a secondary analysis of data from the BQIP project to better understand which elements of the QI change package were associated with improvement. Specifically, we attempted to answer 2 questions: Was documented use of a respiratory score in a patient associated with a decrease in the probability of that patient receiving a bronchodilator, and was use of a respiratory score in a pre–post fashion associated with a decrease in the number of bronchodilator doses used in children who did receive this therapy?

Methods

Study Design and Setting

This study is a secondary analysis of data from a QI collaborative formed by the American Academy of Pediatrics (AAP) Value in Inpatient Pediatrics network titled “A Quality Collaborative for Improving Hospitalist Compliance with the AAP Bronchiolitis Guideline” (BQIP), which is described in detail elsewhere.¹⁷ Specific to bronchodilator use, the intervention included a “change package” with sample order sets and a sample respiratory score based on the WARM score (Supplemental Fig 3). Twenty-two sites began the project. One site provided some data for the preintervention period but dropped out before the intervention period. Twenty-one hospital sites participated in the intervention period and provided monthly data for January through March 2013 and January through March 2014, representing 2 bronchiolitis seasons, before and after an intervention period.

Participants

Hospitalized patients ages 1 month to 2 years with a primary diagnosis of acute viral bronchiolitis were eligible for inclusion. Patients were excluded if they had any of the following conditions: prematurity (<35 weeks’ completed gestational age), chronic illness (congenital heart disease, pulmonary disease including asthma, chromosomal, genetic, congenital, or neuromuscular abnormalities), or admission or transfer to the ICU during the episode under review. Inclusion and exclusion criteria were intended to sample typical cases of mild to moderate bronchiolitis to reflect the population addressed in the AAP Clinical Practice Guideline.¹⁸

Data Source

Data for exposures, outcomes, and covariates were collected by clinicians at each of the 22 participating hospital sites by individual chart review. Data were collected by a designated trained member of the QI team over 6 1-month time periods during 2 separate winters, with months 1, 2, and 3 serving as baseline data before the intervention and months 4, 5, and 6 as performance data after the intervention. Each site reviewed the first 20 charts per month meeting inclusion and exclusion criteria, or all charts if there were <20 per month. Data were then input into the AAP’s Quality Improvement Data Aggregator, a Web-based data repository designed for QI.

Exposure Classification

The primary exposure variables of interest were use of a clinical respiratory score and the method in which the score was used. Use of a respiratory score was defined as any use documented in a patient’s chart during the index hospitalization. Thus, the use of the score had to be confirmed in each chart ≥1 time for the patient to be included in the “scored” population. In the context of the collaborative, sites were free to choose which scoring system to adopt. Ten sites performed some respiratory scoring in the preintervention period; 20 of 22 sites used respiratory scoring systems in the intervention period. Of these sites, 13 adopted the WARM score, 7 sites adopted scores that were unique to their institution, and 1 site did not adopt respiratory scoring as an intervention.

Patients were then classified as having 1 of 3 scoring exposure types: pre–post, alternative, and not scored. In the context of the collaborative, 2 possible methods for applying scoring systems were suggested: use of the score before and after bronchodilator administration to measure response (pre–post method) or an alternative method of score use known as the threshold method, where a certain score was set as a threshold before further intervention was introduced. In addition to choosing which scoring system to implement, sites could choose to implement 1, both, or neither method of scoring. Patients were placed in the pre–post group if “Yes” was indicated in response to the question, “After admission, was a respiratory score used after the bronchodilator dose was administered to document responsiveness?” Because the data collection tool did not specifically ask whether the threshold method for scoring was used, patients who had received a score but had indicated “no” to the above question were placed in the alternative group. Patients who had no scores recorded were placed in the not scored group.

Outcome Variables

The primary outcomes of interest were the prescription of any bronchodilator and number of doses prescribed during the hospitalization, determined by the following questions on the chart review tool: “Was this patient given any dose of bronchodilator AFTER admission?” and “After admission, how many doses of bronchodilator did the patient receive?” Bronchodilators were defined as albuterol, levalbuterol, epinephrine, or racemic epinephrine.

Covariates

Covariates considered in the analysis included site, length of stay, steroid use, and presence of an identified tobacco smoker in the household. Because some sites began this project with respiratory scores in place, we analyzed score use at the patient level regardless of time period with respect to the QI intervention; however, we also used preintervention and postintervention time periods as covariates for adjustment. Time period with respect to the QI intervention was examined categorically as either the preimplementation time period (months 1, 2, and 3) or the postimplementation (months 4, 5, and 6). Length of stay was determined by the reviewer, rounded to the nearest hour, and was examined categorically as ≤24 hours, >24 to ≤48 hours, and >48 hours.

Because the WARM score was the only score type adopted by >1 site, we did not include score type as a covariate because we could not separate score effect from site effect at sites with unique scoring systems. Instead, to account for scoring system, we stratified our analysis to determine whether sites implementing the WARM score performed differently than other sites.

Statistical Analysis

Study variables were presented as frequency counts and percentages. Exposure to any bronchodilator (yes/no) and respiratory score use (yes/no) were examined via χ² test. An unadjusted analysis to examine the number of doses of bronchodilators in relation to respiratory score use was done via Wilcoxon rank-sum test, and results were reported as median and interquartile range. To examine pre–post scoring, we stratified our analysis based on exposure type (pre–post, alternative, not scored) and also administration of bronchodilators because every patient with a pre–post score would have also had a bronchodilator administered. For multivariable analyses, mixed effect models were used. The outcome variable, doses of bronchodilator, was log-transformed. Three models were developed; model 1 was adjusted for cycle and length of stay, model 2 was also adjusted for steroid use, and model 3 was adjusted additionally for presence of an identified smoker in the household or family. The best model was selected based on Bayesian information criteria. Site-level variation was examined as a random factor in mixed models. An unstructured covariance matrix was found to be the best for all models. A stratified analysis by site was also conducted. Sites were ordered in terms of frequency of score use. Based on the distribution, 2 cutoffs were considered and 2 groups for site-level analysis were created: sites with score use <30% and >30% and sites with score use <80% and >80%. To examine the use of ≥2 doses of bronchodilator in comparison with use of a single dose and in relation to score exposure (scored versus not scored), a logistic regression model was used. Results were presented as odds ratios and confidence intervals. The level of significance for all statistical tests was 2-sided, with P < .05. All analyses were conducted in SAS version 9.4 (SAS Institute Inc, Cary, NC).

Results

There were a total of 1876 subjects in this cohort. Of these, 769 had a respiratory score used in any manner, with 203 patients in the pre–post group and 566 in the alternative group, leaving 1107 patients in the not scored group. There were 744 patients who received any bronchodilators, including the 203 patients in the pre–post group, 90 in the alternative group, and 451 in the not scored group. Table 1 provides the summary characteristics of the cohort. Site-specific characteristics with regard to bronchodilator and respiratory score use are depicted in Fig 1.

TABLE 1.

Study Characteristics

	Total	Characteristics by Score Exposure			Characteristics by Bronchodilator Exposure
		Scored (Pre–Post)	Scored (Alternative)	Not Scored	Received Bronchodilator	No Bronchodilator
Patients	1876	203 (10.8)	566 (30.2)	1107 (59.0)	744 (39.7)	1132 (60.3)
Cycle
Preimplementation	1017	68 (33.5)	158 (27.9)	791 (71.5)	474 (63.7)	543 (48)
Postimplementation	859	135 (66.5)	408 (72.1)	316 (28.5)	270 (36.3)	589 (52)
Received bronchodilators	744	203 (100)	90 (15.9)	451 (40.7)	744 (100.0)	n/a
Doses bronchodilator received
0 dose	1132	0 (0)	476 (84.1)	656 (59.3)	0	n/a
1 dose	208	75 (37)	29 (5.1)	104 (9.4)	208 (100.0)	n/a
>1 dose	536	128 (63)	61 (10.8)	347 (31.4)	536 (100.0)	n/a
Received steroids (yes)	153	25 (12.3)	18 (3.2)	110 (9.9)	139 (18.7)	14 (1.2)
Household smoker identified
Yes	407	54 (26.6)	138 (24.4)	215 (19.4)	154 (20.7)	253 (22.4)
No	1049	115 (56.7)	320 (56.5)	614 (55.5)	389 (52.3)	660 (58.3)
Not answered	420	34 (16.7)	108 (19.1)	278 (25.1)	201 (27.0)	219 (19.3)
Received bronchodilator (n = 744)
Patients	744	203 (27.3)	90 (12.1)	451 (60.6)	744	n/a
Cycle
Preimplementation	474	68 (14.3)	44 (9.3)	362 (76.4)	474	n/a
Postimplementation	270	135 (50.0)	46 (17.0)	89 (33.0)	270	n/a
Doses bronchodilator received
1 dose	208	75 (36.1)	29 (13.9)	104 (50.0)	208	n/a
>1 dose	536	128 (23.9)	61 (11.4)	347 (64.7)	536	n/a
Received steroids (yes)	139	25 (18.0)	13 (9.4)	101 (72.7)	139	n/a
Household smoker identified
Yes	154	54 (35.1)	17 (11.0)	83 (53.9)	154	n/a
No	389	115 (29.6)	53 (13.6)	221 (56.8)	389	n/a
Not answered	201	34 (16.9)	20 (10.0)	147 (73.1)	201	n/a

n/a, not applicable.

FIGURE 1.

Change in documented preintervention and postintervention respiratory score use and bronchodilator use by site.

No association between any use of a respiratory score and likelihood of being prescribed a bronchodilator was found by χ² test (P = .25). In the cohort of patients who were prescribed a bronchodilator, the unadjusted analysis showed an association between use of a respiratory score and fewer doses of bronchodilator (Table 2). However, after we adjusted for covariates, respiratory score use was no longer associated with a decrease in the number of bronchodilator doses prescribed (Table 3). This was also true in the preintervention, postintervention, and WARM score–only subcohorts (Table 3) and at the site level (Table 4). The odds ratio (confidence interval) for having received ≥2 doses of bronchodilators was 1.8 (1.33–2.54) for those who were not scored in comparison with those receiving some score. Stratification of the multivariable analysis by receipt of bronchodilators and respiratory score exposure type demonstrated no significant difference between exposure types and number of bronchodilator doses (Fig 2).

TABLE 2.

Unadjusted Associations Between Respiratory Score Use and Number of Bronchodilator Doses

	n	Median Doses (Interquartile Range)	P*
Total cohort (n = 1876)
Score used (pre–post or alternative)	769	0 (0–1)	.05
Not scored	1107	0 (0–3)
Patients receiving bronchodilators (n = 744)
Pre–post group	203	2 (1–6)	.001
Alternative group	90	3 (1–11)
Not scored	451	4 (2–9)

^*P according to Wilcoxon rank sum test.

TABLE 3.

Adjusted Mean Estimates and CIs for Score Used Versus Not-Used in Association With Number of Bronchodilator Doses (Stratified Analysis, Bronchodilator Subcohort)

	Total, n	Score Used			P*
		Score Used/Not Used, n	Pre–Post or Alternative, Mean (CI)	Not Scored, Mean (CI)
All patients receiving bronchodilatorsa
(reference: not scored)	744	293/451	5.48 (4.69–6.40)	5.33 (4.60–6.19)	.73
Pre-BQIPb	474	112/362	5.77 (4.53–7.36)	6.20 (5.21–7.37)	.58
Post-BQIPb	270	181/89	5.36 (4.38–6.56)	4.50 (3.61–5.61)	.11
WARM score subgroupa	474	185/289	5.10 (4.14–6.28)	5.08 (4.10–6.29)	.96

Geometric means presented; Site modeled as random factor. CI, confidence interval.

^aAdjusted for implementation cycle (post/pre), length of stay (≤24 h, >24 to ≤48 h, >48 h), steroid use (no/yes), and household or family smoker identified (no, yes, unanswered).

^bAdjusted for length of stay (≤24 h, >24 to ≤48 h, >48 h), steroid use (no/yes), and household or family smoker identified (no, yes, unanswered).

^*Examined using linear mixed modeling.

TABLE 4.

Adjusted Mean Estimates and CIs for Score Used Versus Not-Used in Association With Number of Bronchodilator Doses (Stratified by Site)

	Total, n	Score Used/Not Used, n	Scored, Mean (CI)	Not Scored, Mean (CI)	P*
Sites with score use <30% (n = 12)	452	73/379	5.14 (4.14–6.37)	5.83 (5.22–6.52)	.30
Sites with score use >30% (n = 10)	292	220/72	5.08 (4.48–5.77)	4.86 (4.06–5.83)	.65
Sites with score use <80% (n = 16)	582	139/443	6.06 (5.18–7.09)	5.60 (5.07–6.18)	.40
Sites with score use >80% (n = 6)	162	154/8	4.35 (3.70–5.13)	3.97 (2.43–6.49)	.70

Adjusted for implementation cycle(post/pre), length of stay (≤24 h, >24 to ≤48 h, >48 h), steroid use (no/yes), and household/family smoke identified (no, yes, unanswered). CI, confidence interval.

FIGURE 2.

Log-scale difference in bronchodilator doses received between patients grouped by method of respiratory score use.

Discussion

In this QI cohort we found no association between documented use of a respiratory score and any decrease in likelihood of a bronchodilator being prescribed overall. We did find an association between use of a score and fewer doses of bronchodilators received when these medications were prescribed; however, this association disappeared after adjustment for available cofactors. Comparison by scoring method (pre–post, alternative, and not scored) also revealed no differences in doses of bronchodilators prescribed between the groups after adjustments.

Patients who were not scored but did receive ≥1 bronchodilator dose were more likely to have received ≥2 or than those who were scored. It may be that a subgroup of patients with bronchiolitis who were more likely to receive multiple doses of bronchodilators may have benefitted from a formalized scoring strategy. It may also be that providers who wanted to use bronchodilators for their patients actively chose not to use a respiratory score; however, analysis at the site level, as depicted in Fig 1 and Table 4, showed no statistical relationship between use of a respiratory score and overall number of bronchodilators prescribed. Interestingly, it appears that whereas some sites demonstrating successful scoring system implementation (such as sites 1 and 18) also demonstrated large decreases in bronchodilator use, other sites that were already high performing in the preintervention period (such as sites 3 and 7) showed little change or even an increase in the proportion of patients receiving bronchodilators after increases in scoring system use.

It is important to note that this collaborative used clinical respiratory scores as a tool to modify physician behavior, with a group of patients selected for mild to moderate illness, thus appropriate for a supportive approach. Our intervention and data collection were designed to assess physician behavior and not predictive of construct validity of clinical respiratory scores as they relate to patient severity of illness. To the authors’ knowledge, this is the first examination of patient-level associations between respiratory scoring as a QI tool and bronchodilator prescription in bronchiolitis. Primary strengths of our study include the multicenter nature of the cohort and the diversity of the sites. As a national collaborative of both academic and community hospitals, the patient population is probably representative of typical physician treatment of mild to moderate bronchiolitis.

Comparing our results with previous QI initiatives that used a respiratory score for bronchiolitis is somewhat difficult. Bronchodilator usage rates in our cohort were low compared with those described in previous QI reports, with only 33% of patients in our cohort receiving bronchodilators in the postintervention time period.¹⁷ Temporal trends in bronchodilator use according to various respiratory scoring algorithms have been previously described,^{11–15,17,19} but this body of literature does not examine whether use of a clinical respiratory score in the individual patient is directly associated with bronchodilator prescribing. In addition, these studies vary in the bronchodilator outcome reported, including reporting >1 dose,^13–15 >2 doses,²⁰ and number of days treated.¹⁹

Not all QI reports describe decreases in bronchodilator use after implementation of respiratory scoring strategies. Perlstein et al¹³ reported an initial decrease in doses of bronchodilators administered after implementation of scoring but a subsequent increase after inclusion of an algorithm that included use of a score to assess efficacy of a bronchodilator trial. After elimination of the algorithm, an associated fall in use of >1 dose of bronchodilator per patient was seen. The authors suggested 2 separate explanations for their observations: poor uptake of the respiratory scoring tool they implemented (<10% of included patients in their study actually had a respiratory assessment form documented) and implicit encouragement of bronchodilator use by the application of the pre–post scoring method. Another possibility is the degree to which these scores are subject to a biased interpretation. Unlike asthma, the validity of clinical respiratory scores in patients with bronchiolitis has not been clearly established, although their use is widespread.²¹ Todd et al¹⁹ also reported a slight increase in first-day bronchodilator use after implementation of a guideline where a respiratory score was used to determine clinical improvement after bronchodilator use.

Our study has several limitations. BQIP was a QI intervention, and the study was designed to assess physician behavior in a cohort of typical hospitalized patients with bronchiolitis. Thus, in-depth assessment of patient severity of illness was not part of the design, and because there is no evidence basis around which to standardize scores, score use in the collaborative was not standardized. Although we were able to control for length of stay to begin to address disease severity, other clinical information such as length of illness before hospitalization, patient age, use of oxygen, and oxygen saturation information were not collected. However, patients who were born prematurely, had significant comorbidities, or needed ICU care were excluded from the cohort, making it likely that most patients in the cohort had mild to moderate disease severity. Because score use was not standardized, it may also be that some methods of applying clinical respiratory scores, with different treatment thresholds, for example, would have been associated with decreased bronchodilator prescribing.

A difference in the prevalence of atopic phenotype might be expected between older and younger patients in the cohort²²; it may be that physicians treated the older patients in the cohort more like asthma patients (although asthma was an exclusion criterion) and therefore used respiratory scores to titrate rather than prevent future bronchodilator doses. We attempted to mitigate this limitation by controlling for steroid use; however, it may be that physicians treating patients with previous wheeze or atopic phenotype are more likely to alter bronchodilator prescribing patterns in these patients than in patients without these clinical features.

Another consideration is heterogeneity in the method of respiratory score use at the different participating sites, because the specific method of implementation was left to the discretion of individual sites. Information on how sites undertook implementation or which sites chose to use scores in the pre–post method and which sites chose to implement clinical respiratory scores as a threshold was not available for analysis. Another limitation to our analysis is that in stratifying by method of scoring and receipt of any bronchodilator, our sample size declines significantly, and we may have missed a small but clinically meaningful effect because of inadequate statistical power. Nevertheless, our sample size of 769 patients scored is more than adequate to detect a 10% absolute difference in rates of bronchodilator prescribing under standard assumptions (α = .05, 80% power).

Despite the generally reported success of clinical respiratory scores as a QI tool for decreasing bronchodilator overuse in bronchiolitis, no optimal method of applying a score has been established. Although our findings should be interpreted as exploratory in light of the limitations of our data, our analysis does suggest that factors beyond the application of a score itself may account for the improvement seen in the QI literature. Our findings do not preclude the possibility that alternative methods of applying a clinical respiratory score, such as use of a scoring threshold for bronchodilator treatment,¹¹ may be effective to reduce bronchodilator overuse. It may also be that alternative scoring strategies have different effects for patients in different age ranges. Given the difficulty of establishing construct validity for clinical respiratory scores, their application seems most reasonable in the QI setting, where the results of implementation can be examined and the intervention modified or discarded should improvement in project targets not be demonstrated. Future work should attempt to clarify utilization and outcome differences associated with alternative scoring strategies.

Conclusions

In a multicenter QI cohort, we found no association between documented use of a clinical respiratory score and a decrease in the likelihood of physician prescribing of any bronchodilator or a decrease in the number of bronchodilators received after adjustment for covariates. More investigation is needed to determine whether there is a similar lack of association in other settings when covariates not available for this analysis are considered, or whether the particular method of scoring can decrease use of bronchodilators in bronchiolitis.