Risk Stratification for Hospitalization in Acute Asthma: The CHOP Classification Tree

Abstract

Objective

Simple risk stratification rules are limited in acute asthma. We developed and externally validated a classification tree for asthma hospitalization.

Methods

Data were obtained from two large, multicenter studies on acute asthma, the National Emergency Department Safety Study (NEDSS) and the Multicenter Airway Research Collaboration (MARC) cohorts. Both studies involved emergency department (ED) patients aged 18–54 years, presenting to the ED with acute asthma. Clinical information was obtained from medical record review. The classification and regression tree (CART) method was used to generate a simple decision tree. The tree was derived in the NEDSS cohort and then was validated in the MARC cohort.

Results

There were 1,825 patients in the derivation cohort and 1,335 in the validation cohort. Admission rates were 18% and 21% in the derivation and validation cohorts, respectively. The CART method identified four important variables (CHOP): change [C] in peak expiratory flow (PEF) severity category, ever hospitalization [H] for asthma, oxygen [O] saturation on room air, and initial PEF [P]. In a simple 3-step process, the decision rule risk-stratified patients into 7 groups, with a risk of admission ranging from 9% to 48%. The classification tree performed satisfactorily on discrimination in both the derivation and validation cohorts, with an area under the receiver operating characteristic curve of 0.72 and 0.65, respectively.

Conclusions

We developed and externally validated a novel classification tree for hospitalization among ED patients with acute asthma. Use of this explicit risk stratification rule may aid decision-making in the emergency care of acute asthma.

INTRODUCTION

Acute asthma is a common presentation to the emergency department (ED), accounting for approximately 2 million ED visits and 500,000 hospitalizations in the United States each year [1]. Despite the significant morbidity associated with acute asthma, there has been a paucity of simple, practical, and validated tools for risk stratification in adult patients with acute asthma. Previous studies have used multivariable modeling and identified certain characteristics that are associated with asthma admissions [2, 3]. These multivariable models, however, may have limited utility in clinical practice. They often involve mathematical equations or scoring systems and require access to a calculator or even a computer to convert point scores to risk estimates. Furthermore, these rules were derived from single-center studies and may be less generalizable to other EDs.

Unlike multivariable regression models, the classification and regression tree (CART) method produces a simple decision tree that is intuitive and easy to apply in clinical practice [4–6]. The structure of the tree is clinically appealing and congruent with physicians’ decision-making processes. As a result, the decision trees generated by CART have been used in a variety of clinical fields, such as emergency medicine [7], pulmonology [8], cardiology [9, 10], neurosurgery [11], and oncology [12]. In asthma, we are only aware of two studies that have used this method [13, 14]. These studies, however, have identified the factors associated with health services utilization in chronic asthma, which is not directly applicable to the acute care setting.

The objective of the present analysis was to develop and validate a practical and user-friendly classification tree for hospitalization among ED patients with acute asthma.

METHODS

Study Design and Setting

The present analysis used data from two large cohort studies on acute asthma, the National Emergency Department Safety Study (NEDSS) and the Multicenter Airway Research Collaboration (MARC).

Study Population

Derivation Cohort: Asthma Component of the NEDSS

The NEDSS is a large, multicenter study designed to characterize factors associated with the occurrence of errors in EDs. Details of the study design and data collection have been published previously [15]. Three clinical conditions were examined in the NEDSS, including acute myocardial infarction, dislocations, and acute asthma. The current analysis utilized the asthma component [16]. Using a standardized data abstraction tool, trained research personnel at 63 EDs in 23 US states abstracted data from randomly-selected ED visits for acute asthma during 2003–2006. The visits were identified by using International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes 493.xx. Inclusion criteria were age 14 to 54 years and a history of asthma before the index visit. The following visits were excluded: repeat visits; transfer visits; patient visits with a history of chronic obstructive pulmonary disease, emphysema, or chronic bronchitis; or visits not prompted, in large part, by asthma exacerbation. Repeat visits were excluded to avoid within-person correlations between repeat visits. Transferred patients were excluded because they may have received treatments at other EDs prior to the index visit, which may alter their acuity at presentation. Data abstraction included baseline patient characteristics, past asthma history, ED presentation, management, and disposition. The Institutional Review Board at all participating hospitals approved the study.

Validation Cohort: MARC

The MARC is a division of the Emergency Medicine Network (EMNet, www.emnet-usa.org). Details of the study design and data collection have been published previously [17]. The MARC database used for the present analysis combines data from 2 observational cohort studies performed during 1996–1998. Using a standardized protocol, investigators at 36 EDs across the US provided 24-hour per day coverage for a median of two weeks. All patients were managed at the discretion of the treating physician. Inclusion criteria were physician diagnosis of acute asthma, age 18–54, and the ability to give informed consent. Repeat visits by individual subjects were excluded. Patients’ demographics, asthma history, and details of their current exacerbation were obtained by ED interview. Data on ED management and disposition were obtained using medical chart review. For those who did not complete the ED interview (missed by investigators, refused, or other reasons), their medical records were reviewed to capture full data on demographics, ED presentation, ED course, as well as limited information on asthma history. Because each of the interviewed subjects also had data collected from their medical records, the MARC database represents all eligible patients presenting to the ED during the study periods. For the current analysis, we focused on the variables taken from medical records.

Measurements

Peak expiratory flow (PEF) was recorded in liters per minute and expressed as the absolute value; no percent predicted values are presented due to infrequent recording of the patient’s height in ED charts. Instead, severity of acute asthma was classified according to the initial PEF as follows: mild, 300+ for women, 400+ for men; moderate, 200–299 for women, 250–399 for men; severe, 120–199 for women, 150–249 for men; and very severe, <120 for women, <150 for men. These absolute PEF cut-offs represent approximately 70%, 40%, and 25% predicted, respectively, for a typical adult woman and man [18, 19]. Changes in PEF are expressed as improved, unchanged, and worsened, based on the change in severity category. For example, changes from severe or very severe to moderate would be categorized as improved.

Outcome Measure

The primary outcome measure was hospital admission, which was defined as admission to an inpatient unit, observation unit, or intensive care unit. Although these three response categories represent different levels of severity, we were not able to model this nominal outcome because only binary outcomes are allowed in the CART analysis and that sufficient numbers of outcomes are required for building a stable classification tree. For the purpose of the current analysis, we further excluded few patients who left against medical advice (2%).

Statistical Analysis

Analyses were conducted using CART version 6.0 (Salford Systems, San Diego, CA) and Stata 10.0 (StataCorp, College Station, TX). Summary statistics are presented as proportions (with 95% confidence intervals [CI]), means (with standard deviations), or medians (with interquartile ranges). All P values are two-sided, with P<0.05 considered statistically significant.

Derivation of the Classification Tree

The CART method is a non-parametric statistical technique based on recursive partitioning analysis [4–6]. It involves the segregation of data by different values of classification variables through a decision tree composed of progressive binary splits. Each parent node in the decision tree produces two child nodes, which in turn can become parent nodes producing additional child nodes. The splitting process is repeated until the degree of outcome homogeneity in each subgroup cannot be improved by further stratification, or until the size of a subgroup is smaller than a pre-defined value. Candidate variables for building the tree were selected a priori based on review of the medical literature [2, 3, 20–26] and clinical importance. The domains and variables chosen are as follows: demographics (age, sex), chronic asthma-related factors (ever hospitalized for asthma), and ED presentation and severity (duration of symptoms, initial oxygen saturation on room air, initial respiratory rate, initial PEF severity category, and change in PEF severity category). For continuous variables, optimal cutoff points were chosen by CART algorithm. Gini index was used as the impurity measure for binary outcomes. At each possible split, the splitting factor was chosen to maximize the change in Gini index between parent and child nodes. Nodes in the CART tree were constrained to have a minimum size of 100 patients in parent nodes and 50 patients in child nodes. Finally, the large tree was pruned according to cost-complexity index, and the final tree was determined as the one with the lowest misclassification in the validation set to avoid model overfitting. Equal cost was assigned to false positives and false negatives. The probability of admission was calculated for each of the terminal nodes in the CART tree.

When a missing variable was encountered at a splitting point in the decision tree, CART analysis used a different variable most closely resembling the missing variable in its ability to make a similar split in the data (i.e., a “surrogate” variable). This process requires computer algorithms and would be impractical at the bedside. To minimize the use of surrogate variables, we restricted the analysis to patients with available initial PEF.

Validation of the Classification Tree

The patients in the validation cohort were classified into risk groups based on the CART tree from the derivation cohort. Chi-square tests for trend were used to assess where the risk of admission was increasing across the risk groups generated by the classification tree. The discriminatory performance of the tree was quantified by determining the area under the receiver operating characteristic (ROC) curve in the derivation and validation cohorts.

RESULTS

There were 1,825 patients with acute asthma in the derivation cohort and 1,335 in the validation cohort. The patients in the derivation and validation cohorts were similar with respect to age, sex, and initial oxygen saturation (Table 1). Patients in the validation cohort were more likely to be admitted for asthma in the past and present to the ED sooner, were more severe based on initial respiratory rate and PEF, and appeared to be more responsive to treatments, as suggested by more patients in the improved PEF category. Admission rates were 18% and 21% in the derivation and validation cohorts, respectively.

Table 1.

Patient characteristics in the derivation and validation cohorts.

Variable	Derivation Cohort (n=1,825)	Validation Cohort (n=1,335)
Age, median (IQR), years	33 (24–43)	34 (23–42)
Female (%)	62	65
Ever admitted for asthma (%)	29	45
Duration of symptoms <24 hours (%)	33	48
Initial O2 saturation on room air, median (IQR)*	97 (95–99)	97 (95–98)
Initial respiratory rate, median (IQR), breaths per minute	20 (18–24)	24 (20–26)
Severity based on initial PEF (%)
Mild	23	15
Moderate	39	36
Severe	28	35
Very severe	10	14
Change in PEF severity category (%)†
Improved	49	66
Unchanged	48	33
Worsened	2	1
Admission	18	21

Abbreviations: IQR, interquartile range; PEF, peak expiratory flow.

^*Available for 1,427 patients in the derivation cohort, and 1,119 in the validation cohort.

^†Available for 1,159 patients in the derivation cohort, and 1,192 in the validation cohort.

Figure 1 depicts the final tree generated by the CART analysis along with the admission rate for each terminal node of this tree. Overall, the CART method used 4 variables (CHOP: change [C] in PEF, ever hospitalization [H] for asthma, initial oxygen [O] saturation on room air, and initial PEF [P]) to stratify patients into 7 terminal nodes. “Ever hospitalization for asthma” was the best single discriminator, followed by room-air oxygen saturation (≥95% vs. <95%) and PEF severity category (severe/very severe vs. mild/moderate) as the second-level discriminators. For the node of lower oxygen saturation, changes in PEF (improved vs. unchanged/worsened) provided additional prognostic value. Overall, the tree resulted in 3 groups with an admission rate that was below the average (light gray boxes), and 4 groups with an admission rate that was above the average (dark gray boxes).

Figure 1.

Classification tree for the derivation cohort. Shaded boxes represent the terminal groups. Groups with an admission rate that is above the average are shaded dark gray. Groups with an admission rate that is below the average are shaded light gray.

Abbreviations: ADM, admission; O2 Sat, oxygen saturation; PEF, peak expiratory flow; Mod, moderate; S, severe; VS, very severe.

Among the final 7 groups, the risk of admission ranged from 9% to 48% (Table 2). There was a significant increasing trend in risk of admission across the risk groups in the derivation cohort (P<0.001).

Table 2.

Admission rates by risk group in the derivation and validation cohorts, based on the classification tree.

Risk Group	Derivation Cohort (n=1,825)	Validation Cohort (n=1,335)
	Admission no./Total no. (%)	Admission no./Total no. (%)
Group 1	84/957 (9)	55/488 (11)
Group 2	11/97 (11)	5/75 (7)
Group 3	12/92 (12)	27/117 (23)
Group 4	50/195 (26)	49/223 (22)
Group 5	44/171 (26)	30/70 (43)
Group 6	20/73 (27)	14/65 (22)
Group 7	114/240 (48)	96/297 (32)
Ptrend	<0.001	<0.001
Area under the ROC curve	0.72	0.65

Abbreviations: ROC, receiver operating characteristic.

The decision tree generated by CART analysis of the derivation cohort was then tested for its ability to risk-stratify patients in the validation cohort (Table 2). Similarly, there was a significant increasing trend in risk of admission across the risk groups in the validation cohort (P<0.001). The classification tree performed satisfactorily on discrimination both in the derivation and validation cohorts, with an area under the ROC curve of 0.72 and 0.65, respectively.

DISCUSSION

Using the data from two large multicenter cohorts, we developed and externally validated a simple classification tree for hospital admission in acute asthma. Given its internal and external validity, we believe that this classification tree is a potentially useful tool for risk stratification and may aid decision-making in the emergency care of acute asthma.

The classification tree includes three important variables (i.e., hospitalization history, oxygen saturation, and initial PEF) that can be readily assessed at the time of ED presentation, plus one variable (change in PEF) that updates the risk of admission during the ED course. The risk stratification scheme is similar to what is outlined in the asthma guidelines – a brief history-taking, measurements of PEF and oxygen saturation, plus repeated assessments of PEF [19, 27], but extends the guideline algorithm to an explicit, relatively simple, bed-side tool for risk stratification. For example, CART identified 95% as the cutoff point for room-air oxygen saturation in the classification tree. This finding is of potential importance as the CART algorithm considered this as the optimal cutoff for the continuous predictor of oxygen saturation by an exhaustive search of all possible splits.

The similarity between the guideline and CART algorithms suggests that emergency physicians have followed the guidelines to risk-stratify ED patients with acute asthma; however, the validation process indicates there are modest variations in admission practices over time. For example, compared with the NEDSS physicians, the MARC physicians seemed to put more weight on “non-improved PEF” when making admission decisions, as reflected by higher admission rates in Group 5. This variation also explained the somewhat decreased discriminatory ability of the decision tree in the validation cohort. Nonetheless, this is expected because applying a previously derived model to external data is the true test of a predictive model [28].

The CART algorithm has several strengths. First, in a simple 3-step process using 4 variables, the tree-structured decision rule can identify high-risk and low-risk patients, with a variation of more than a 5-fold difference in risk of admission. Compared with multivariable-generated decision rules, the processes and the number of variables involved in obtaining risk estimates are significantly reduced. Second, the CART method uses asymmetric stratification, i.e., different binary splitters at the same level. This feature not only is congruent with physicians’ decision-making processes but also greatly enhances the efficiency of the risk-stratification process.

This study has some potential limitations. First, we did not have information on follow-up outcomes for both cohorts, such as post-ED relapse. Inclusion of follow-up outcomes would have allowed us to examine the need for admission. Rather, we applied the CART method to understand admission practices in large ED samples over time. Second, there is no precise risk threshold that can uniformly dictate which patients should be admitted to the hospital. Thus, the CHOP tree should be used to aid decision making, not to replace physician’s judgment. Third, the areas under the ROC curve in CART analyses are not as high as those seen in multivariable modeling, as CART usually uses fewer variables and generates risk estimates for groups, not individuals [4, 10]. Fourth, we were not able to derive the percent predicted PEF due to lack of height. Future studies that prospectively measure this item may improve the classification tree. Finally, there are some important variables (e.g., physical findings) that have been shown to predict asthma hospitalization [2, 3] but are infrequently documented in the medical record. Inclusion of these variables might better risk-stratify patients but would add complexity to the rule.

In summary, we developed and externally validated a novel CHOP classification tree for hospitalization among ED patients with acute asthma. By elaborating on the algorithm outlined in the asthma guidelines [19, 27], this CART-based algorithm risk-stratifies patients using three variables at ED presentation and one variable after initial management. Use of this explicit risk stratification rule may aid decision-making in the emergency care of acute asthma.