Development and Validation of the Composite Asthma Severity Index – An Outcome Measure for use in Children and Adolescents

Abstract

Background

Asthma severity is reflected in many aspects of the disease, including impairment and future risks, particularly for exacerbations. According to the EPR-3, however, to assess more comprehensively the severity of asthma the level of current treatment needed to maintain a level of control should be included.

Objective

Development and validation of a new instrument, the Composite Asthma Severity Index (CASI), which can quantify disease severity by taking into account impairment, risk and the amount of medication needed to maintain control. At present, there is no instrument available to measure and assess the multidimensional nature of asthma.

Methods

Twenty-six established asthma investigators, who are part of the NIH-supported Inner City Asthma Consortium (ICAC), participated in a modified Delphi consensus process to identify and weight the dimensions of asthma. Factor analysis was performed to identify independent domains of asthma using the Asthma Control Evaluation (ACE) trial. CASI was validated using the Inner City Anti-IgE Therapy for Asthma (ICATA) trial.

Results

CASI scores include five domains: day symptoms and albuterol use, night symptoms and albuterol use, controller treatment, lung function measures, and exacerbations. At ACE enrollment, CASI ranged from 0 to 17 with a mean of 6.2. CASI was stable, with minimal change in variance after 1 year of treatment. In external validation, CASI detected a 32% larger improvement than symptoms alone.

Conclusion

CASI retained its discriminatory ability even with low levels of symptoms reported after months of guidelines-directed care. Thus, CASI has the ability to determine the level of asthma severity, and provide a composite clinical characterization of asthma.

Introduction

Asthma severity and its components, i.e. impairment and risks, are important for treatment decisions, patient characterization, and population studies. In EPR-3¹, changes were made to the definition of severity, which, in prior versions of the guidelines, was considered to represent the “intrinsic intensity of the disease process.” In addition, the previous characterization of asthma severity was based upon features of disease prior to initiation of treatment, and thus became a directive for the step-care at which to initiate treatment. In revisiting this definition of severity, EPR-3 recommended that severity include impairment, risk (i.e. exacerbations), and the amount of medication required to achieve the noted level of control.

At present, there are no generally accepted instruments to assess and rank asthma severity under this new composite concept. This unmet need was also noted by the NIH Asthma Outcome Workshop in 2010, and its pending report indicates that a composite measure of severity, which includes treatment, along with measures of impairment and risk, is a necessary instrument for future use in clinical research in asthma.

Based on the potential importance and unmet need for a comprehensive score for severity, the NIH-supported Inner City Asthma Consortium undertook the initiative to develop a composite asthma score that included impairment, risk and the amount of medication needed to maintain control. We have designed the Composite Asthma Severity Index (CASI) to combine the EPR-3 proposed facets of asthma, i.e. impairment, risk and treatment, which reflect disease severity. The following manuscript describes the development, validation, and parametric properties of the CASI, which we have designed and evaluated to address this unmet need.

Methods

Overview

There were four distinct steps in the development of the CASI (Figure 1). First, the independent outcome domains of asthma were determined via factor analysis using data from more than 500 children and adolescents in the Asthma Control Evaluation (ACE) trial. Second, 26 asthma experts made individual and group decisions as to how the components of these domains should be weighted and combined into a final CASI score. Third, the scale properties of the CASI were evaluated. Finally, the CASI was externally validated against data from more than 400 children and adolescents in the Inner City Anti-IgE Therapy for Asthma (ICATA) trial. These four steps are described in detail below.

Figure 1.

CASI Development Overview

Determining Independent Dimensions of Asthma Severity

ACE was a double-blind, parallel-group clinical trial which randomized 546 children to either 46 weeks of guidelines-based care or to guidelines-based care supplemented by measurement of exhaled nitric oxide.² In addition to the close monitoring of medication use and guideline-based treatment adjustment, the ACE study repeatedly assessed symptoms (wheeze, sleep and activity disruption), lung function and exacerbations. Factor analysis with orthogonal rotation was applied to the eleven asthma outcomes in the ACE study to identify the major domains of asthma and assess their independence.

Clinical Weighting of the Dimensions of Asthma Severity: The Delphi Method

Following the statistical determination of asthma outcome domains, the components of asthma were evaluated and weighted by 26 Inner City Asthma Consortium (ICAC) clinical investigators and a Delphi consensus process was used to finalize the scoring algorithm for CASI.³ The Delphi method is a structured group decision-making process designed to create consensus. In this procedure, each clinician determined the importance of the various asthma outcomes by weighting them independently. The investigators distributed 100 points amongst the five domains based upon their relative clinical importance in determining asthma activity. Following those individual weightings, the investigators met by conference call to discuss the weighting scores and then adjusted their weights as a result of this discussion. A final call was held where the new weightings were presented and a consensus was determined for the final CASI scoring system.

Scale Properties of the Composite Asthma Severity Index

Although criterion validity cannot be assessed since no gold standard measure for asthma severity exists, we examined the properties of the CASI with regard to construct validity, internal consistency and test-retest reliability. For construct validity, we evaluated the CASI’s variability at different time points throughout the ACE trial and compared it to the variability of other asthma outcomes. We tested for a change in variance between enrollment and the end of the study using the Fligner–Killeen test for homogeneity of variance. We also calculated Pearson and Spearman correlation for the CASI and other asthma outcomes, such as the ACT and the direct measure of symptom days. Internal consistency reliability and test-retest reliability were evaluated using Cronbach’s alpha and intra class correlation respectively.

External Validation against an Independent Sample

Having developed the CASI model based upon the ACE study, we then proceeded to establish external validation by determining how the CASI performed in the ICATA population. ICATA was a double-blind, placebo-controlled, parallel-group, multi-center trial where 419 participants were randomized to receive omalizumab versus placebo added to guidelines-based therapy.⁴ A small percentage of the participants (n=39) were enrolled in both ACE and ICATA. To establish reproducibility and responsiveness of the CASI within this independent sample, the ability of CASI to detect differences between treatment arms was analyzed. We compared the CASI to traditional symptom measures in primary efficacy and subgroup analyses. Analyses were conducted using SAS version 9.2 and R version 2.13.1.

Results

Determining Independent Dimensions of Asthma Severity

The factor analysis revealed five independent domains of asthma among the eleven commonly employed asthma outcomes. Those domains were: symptoms (combining days and nights of symptoms and days and nights of albuterol usage), controller medication usage (inhaled corticosteroids prescribed and long acting β–agonists prescribed), lung function (FEV₁ percent predicted and FEV₁/FVC), oral corticosteroid bursts and unscheduled visits (including Emergency Department visits) and hospitalizations. Those domains combine to account for 76% of the variance in the original data. The same 5 domains were identified whether we used data from the initial randomization visit or from end of the study assessments (data not shown). To maximize comparability with the NAEPP guidelines, the domains from the factor analysis were reorganized in two ways. First, symptoms were split into separate domains for day and night and albuterol use. Second, hospitalizations were combined with oral corticosteroid bursts; unscheduled visits without corticosteroid bursts were removed in order to create an exacerbation domain that is compatible with recommendations from the Asthma Outcomes Workshop and the ATS/ERS Report.⁵ As a result, the final domains of the CASI are: days of symptoms and albuterol use, nights of symptoms and albuterol use, controller treatment, lung function measures and exacerbations (Figure 2).

Figure 2.

Determining Independent Domains of Asthma Severity: The Five Dimensions of the CASI

Clinical Weighting of the Dimensions of Asthma Severity: The Delphi Method

All 26 ICAC investigators participated in the Delphi process. After the initial rankings, the median response for relative importance between the domains was calculated; exacerbations had the highest relative-importance score (median score of 25%) while lung function had the smallest (15%). The other domains all had median scores of 20%. During the consensus phase of the modified Delphi process, the 26 investigators used the median responses as a baseline and then adjusted their relative importance rankings (Table 1). During the consensus discussions, the domains for days and nights of symptoms were reduced by 5 percentage points (from 20% to 15%), and 5 percentage points were added to controller treatment (20% to 25%) and exacerbations domains (25% to 30%).

Table 1.

Clinical Weighting of the Dimensions of Asthma Severity: CASI Score Card

	Final Score*
Day Symptoms and Albuterol in the last 2 weeks (15% of Total Score)
0–3	0
4–9	1
10–13	2
14	3
Night Symptoms and Albuterol in the last 2 weeks (15%)
0–1	0
2	1
3–4	2
5–14	3
Lung Function Measures (15%)
FEV1 % > 85	0
FEV1 % - 80–84	1
FEV1 % – 70–79	2
FEV1 % < 70	3
Controller Treatment (25%)
No treatment	0
Albuterol as needed	1
Low Dose ICS†	2
Low ICS + LABA or Medium Dose ICS	3
Medium Dose ICS + LABA	4
High Dose ICS	5
Exacerbations (30%)
Prednisone Burst	2
Prednisone Burst + Hospitalization	4
Max Possible Score	20

^*Scoring Example: A participant taking albuterol as needed with 2 nights of symptoms, 5 days of symptoms, an FEV₁ value 87% of predicted and one prednisone burst (without hospitalization) in the last two months would have a final CASI score of 5 (1 points for day symptoms, 1 points for night symptoms, 1 point for controller treatment, 0 points for lung function measures, and 2 points for exacerbations).

^†Singulair should be considered equivalent to low dose ICS and scored as 2 points.

A simple scoring mechanism for the CASI was derived based upon the investigators weightings and the statistical determination of the 5 domains. First, the five domain scores are calculated using the CASI score card (Table 1). The score calculated for each domain (except exacerbations) is summed to a single overall score which is then added to the exacerbation scores. For the exacerbations domain, participants receive 2 points for each prednisone burst within the past two months and an additional 2 points (4 points total) for a hospitalization. Since the maximum score for each domain corresponds to the relative importance weightings from the modified Delphi process, exacerbation domain scores are capped at 6 points so that they will comprise no more than 30% of the 20-point maximum score. Finally, to calculate CASI, the five domain scores are summed to determine a final CASI score which can range from 0 to 20.

Scale Properties of the Composite Asthma Severity Index

At ACE enrollment, CASI ranged from 1 to 17 and had a mean value of 6.2 (SD = 3.0, Table 2). The CASI score decreased by 23% by the time participants were randomized after 3 weeks of standard NAEPP directed-treatment, due largely to a decrease in symptoms (Table 2). After randomization, the CASI score stabilized with no significant differences between the distributions at randomization and the final ACE visit nearly one year later. As expected, however, the domain scores shifted to reflect the rebalancing of the independent domains; primarily a reduction in symptoms which corresponded to an increase in medication use.

Table 2.

Scale Properties of CASI: Distribution in ACE

ACE Visit	Total CASI		Domain Scores – Mean
	Mean	SD	Day Symptoms & Albuterol	Night Symptoms & Albuterol	Lung Function	Controller Treatment	Exacerbations
Visit 1 – Enrollment	6.2	3.0	0.9	1.2	0.6	2.8	0.7
Visit 2 – Randomization	4.8	2.3	0.3	0.3	0.5	3.5	0.2
Visit 3–8 – Double Blind	4.7	2.5	0.2	0.2	0.5	3.5	0.2
Visit 3	4.8	2.3	0.2	0.2	0.4	3.7	0.2
Visit 4	4.9	2.5	0.3	0.3	0.5	3.6	0.2
Visit 5	4.8	2.5	0.3	0.3	0.4	3.5	0.3
Visit 6	4.6	2.5	0.2	0.2	0.4	3.4	0.3
Visit 7	4.5	2.5	0.2	0.2	0.5	3.3	0.3
Visit 8	4.4	2.5	0.2	0.2	0.5	3.3	0.2
Max Possible Value	20	--	3	3	3	5	6

CASI was correlated with traditional measures of control. At randomization in ACE, CASI correlated both with symptom days (r=0.47, p<0.01) and ACT (r=−0.38, p<0.01). While significant, these correlations are lower than the correlation between ACT and symptom days (r=−0.55, p<0.01), both of which reflect only the single domain of asthma symptoms. Unidimensional measures will typically demonstrate high item-whole correlations, of 0.70 or greater, since the total score is reflecting a single dimension. As expected, item-whole correlations for the CASI reflect the multidimensional structure of this measure (Cronbach’s Alpha=0.33).

When looking across multiple dimensions, asthma activity is a relatively stable phenomenon and correspondingly CASI was significantly (p<0.001) more stable between visits than symptom days or the ACT score, with an across-visit intra-class correlation of 0.53 for CASI as compared to 0.25 and 0.12 respectively for symptom days and ACT. At the same time, CASI maintained within visit variability as the study progressed and was not highly skewed at the end of the study. While the variance for ACT and symptom days both decreased significantly between enrollment and the final study visit (24% and 47% reductions respectively, p<0.001), there was smaller change in the variance of CASI during that time (10% reduction, p=0.08, Figure 3). These findings suggest the CASI will maintain its ability to discriminate between asthma levels even in well-controlled groups receiving standard care, i.e. Guideline-directed, for extended periods.

Figure 3. Scale Properties of CASI: ACT, Symptom Days and CASI Distribution.

CASI, symptom days and ACT at recruitment (Visit 1) and at the end of the study for subjects who attended each of eight ACE study visits. The dashed line and shaded area show the mean ± one-half standard deviation. Annotated values are means (SD). The variances of ACT and Symptom Days are lower at Visit 8 (p<0.001) while the change in variance for CASI is minimal (p=0.08).

External Validation against an Independent Sample

To further understand and appreciate the properties of CASI, we analyzed the efficacy results of the ICATA trial using CASI and then compared the results to those seen using symptom days (the study’s primary outcome). The ICATA intervention, treatment with omalizumab, was highly effective in reducing symptoms, controller treatment levels, and asthma exacerbations.⁴ Compared to the placebo treatment group, participants treated with omalizumab showed significant improvement in both symptom days (0.48 day improvement, p=0.001) and CASI (0.67 points improvement, both p<0.001, Figure 4), but CASI demonstrated a 32% greater magnitude of improvement (standardized effect size of 0.25 for CASI vs. 0.17 for symptom days, Table 3). Furthermore, larger CASI effect sizes were consistent across a variety of phenotypic subgroups (Table 3). For example, CASI showed a 65% larger effect (0.68, p=0.010) on the important subgroup of participants who were both sensitized and exposed to cockroach than symptom days (0.41, p=0.056). A sensitivity analysis was performed removing the participants that were enrolled in both ACE and ICATA and results were similar (data not shown).

Figure 4. External Validation Against an Independent Sample - Effect of the ICATA Intervention on Symptom Days and Composite Asthma Severity.

* - Values are adjusted for omalizumab dosing group, season and study site. The first 12 weeks of the double-blind phase served as a wash-in period and were not included in analyses.

Table 3.

External Validation Against an Independent Sample: Comparison of Efficacy Results across Asthma Outcomes in ICATA

Population	CASI		Symptom Days
	Effect*	p	Effect*	p
All participants	0.25	<0.001	0.17	0.001
Per protocol	0.20	0.019	0.18	0.003
Males	0.24	0.038	0.17	0.014
Females	0.27	0.006	0.17	0.037
Cockroach sensitive	0.29	0.538	0.20	0.585
Cockroach sensitive & exposed	0.68	0.010	0.41	0.056
Mite sensitive	0.26	0.937	0.20	0.611
Mite sensitive & exposed	0.52	0.125	0.29	0.395
BMI > 95th percentile	0.14	0.322	0.17	0.988
Unscheduled visit in last 6 months	0.35	0.016	0.20	0.302
IgE > 100 kU/L	0.34	0.018	0.15	0.391

^*“Effect” shows the change due to omalizumab, measured in standard deviations, for the given subgroups. Effects are standardized and comparable between outcomes. Tests are for with-in group effect sizes for lines 1–4 and for between-group interaction otherwise. Statistically significant cells (p<0.10) are in bold. Sensitivity is indicated by a skin-test wheal size of greater than 3 mm. Significant exposure is indicated by a dust-based allergen level greater than 8 U/g for cockroach (bla g 1) and 2 ug/g for dust mite (maximum of der f 1 and der p 1). Omalizumab, the study drug, is not included in the calculation of CASI.

Discussion

Unlike many instruments which measure only asthma control, CASI quantifies asthma severity using multiple dimensions. By incorporating symptoms, exacerbations, lung function and treatment requirements into a single index, the CASI combines domains of impairment and future risk with the level of treatment needed to reach the current clinical state, and in doing so, addresses the need for a comprehensive asthma index that reflects disease severity. This measure of asthma severity provides a means to discriminate among individuals who would otherwise appear well controlled and equally severe. We propose that this method will more comprehensively assess asthma severity, allowing it to play a key role in determining responses to new asthma therapies, such as immunomodulators. These effects are not always apparent in populations whose symptoms have been greatly ameliorated by receiving a high standard of asthma care and by a reduction in concomitant medication previously needed to maintain asthma control. Similarly, the CASI should be especially sensitive to changes in environmental interventions where the reduction of exposures should lead to both reduced symptoms and medications need.⁶

The EPR-3 asthma guidelines specify that assessment of disease severity after treatment has begun must include both impairment and risk domains, in addition to the treatment required by the individual to achieve that state of overall control.⁷ The CASI addresses the need for an instrument to measure this cross-dimensional impact of asthma. The results of our factor analysis verified that the risk, impairment and treatment are important and independent components of asthma. The CASI includes the major domains of asthma as identified in the EPR-3, namely, impairment, as measured by day and night symptoms, along with albuterol use; and risk measured by FEV₁ percent predicted and past asthma exacerbations, both of which are important predictors of future exacerbations.^8–10

Since the EPR-3 does not specify how the various domains should be combined into an assessment of severity, we drew upon the experience and expertise of a large number of asthma specialists and employed a consensus process to properly balance the dimensions of asthma. This consensus procedure, the Delphi method, provided a structure within which experts could iteratively rank the domains, discuss the rankings, and adjust the rankings based on the discussion. The process was repeated until a general consensus on the final algorithm was reached. This method has been used in asthma research to assemble indicators of emergency care for asthma¹¹ and to establish minimally important differences in asthma clinical trials.^{3, 12}

The multidimensional nature of asthma presents unique challenges when assessing asthma outcomes^13–15. Not only does asthma manifest itself differently from person to person, but it is highly variable across time; and symptoms can be dramatically influenced by medication use.^{16, 17} With guidelines-based care, the majority of individuals with asthma can quickly achieve excellent control,^{18, 19} which results in control scores becoming more uniform. In studies that test the impact of an intervention in addition to standard care, this uniformity makes it difficult to evaluate the efficacy for outcomes measuring control. The CASI retains the ability to detect differences between treatment groups even in a well-controlled population. For example, two groups with minimal symptoms could require very different intensities of treatment to reach this control status, thus indicating different levels of disease activity and severity. This improved discriminatory ability of the CASI will provide more statistical power and allow for smaller sample sizes for typical environmental or immunological interventions.

External validation, as demonstrated by its use in the ICATA study (Figure 3), clearly indicates the ability of the CASI to better quantify the efficacy of an immune-based therapy when compared to a single control parameter, such as symptom frequency. In addition to added discriminatory ability, CASI has significantly higher within subject stability than standard control measurements. This suggests that CASI is truly measuring long-term disease activity and that a treatment that does not modify disease activity will not influence the CASI.

As a quantitative measure of asthma severity, the CASI meets a need specified by the NIH Asthma Outcome Initiative, but several limitations remain. Since CASI is the first attempt to quantify disease severity, there is no gold-standard measure available for comparison. Without a gold standard, it is difficult to calculate the minimally important clinical difference for CASI or to determine precisely what constitutes an “elevated” CASI score. Further research must address these issues. The CASI would also benefit from validation in broader populations such as suburban and adult populations where the distribution of its components, especially pulmonary function, may differ. Further, while CASI shows promise in a research setting, its role in clinical practice is currently unclear. The Inner City Asthma Consortium has a mandate from its sponsor, the NIAID, to continue developing the CASI and to address these limitations. Finally, despite its multiple components the CASI does not measure every aspect of asthma, such as quality of life or the economic cost of asthma. While these are important aspects of asthma, they are not included in the EPR-3 definition of severity and add little to the measure’s discriminatory power.

Instruments such as CASI which go beyond simply measuring control are necessary to assess the effectiveness of an intervention or drug in the context of guidelines-based care. As innovative immune-based, environmental, or health care delivery interventions are developed, they must be tested in the context of excellent guidelines-based “standard care” treatment. CASI is designed for such clinical trials. The increased effect sizes that CASI provides could allow for smaller sample sizes in targeted studies where recruitment is difficult.²⁰ It could also have utility in birth cohort studies where it offers an objective measurement of asthma activity which will better define outcomes and allow measurement of asthma activity over time. CASI may also play a role in helping patients and caregivers to better understand their asthma, and thus, could play a role in asthma education as well as providing a tool to improve adherence in patients with moderate or severe disease.

In summary, we believe that CASI provides an instrument to measure asthma severity and will be helpful in many aspects of research and care of this disease. In earlier guideline documents, asthma severity was considered as the intrinsic features of disease but was assessed prior to beginning treatment and designed to serve as a reference for initial level of step-care. As now proposed, asthma severity should be a composite measure of control, i.e. impairment and risk, and level of treatment required to reach this level of control.¹ CASI captures and reflects these features of asthma severity. As argued by Taylor et al.,²¹ there is now broad support for the concept that “severity” ought to indicate the intensity of treatment required to treat a patient’s asthma and there is little evidence to justify or retain a definition of severity as “disease activity off-treatment.” Finally, the CASI measure will prove useful to distinguish clearly the several phenotypes of asthma that are becoming more salient to asthma researchers and clinicians. As a consequence, CASI would support the recommendations regarding methods to define severe asthma as proposed by the ATS Workshop²² and the experience of the Severe Asthma Research Program.²³ We feel, therefore, that not only does CASI fill a clearly identified need to assess asthma severity in a comprehensive fashion, but this measure has the potential to be an integral instrument in asthma care, research, and disease classification.

Clinical Implications.

In well-treated asthma populations, symptom scores lose their discriminatory ability. With the addition of controller treatment, lung function measures and exacerbations to symptom scores, the CASI allows for the measurement of asthma activity even in well-treated populations.

Abbreviations

CASI

Composite Asthma Severity Index

ACE

Asthma Control Evaluation

ACQ

Asthma Control Questionnaire

ACT

Asthma Control Test

EPR-3

Expert Panel Report 3: Guidelines for the Diagnosis and Management of Asthma

FEV₁

Forced Expiratory Volume in One Second

FEV₁/FVC

Ratio of FEV₁ to Forced Vital Capacity

ICAC

Inner City Asthma Consortium

ICATA

Inner City Anti-IgE Therapy for Asthma

ICS

Inhaled Corticosteroids

LABA

Long-acting Beta Agonists

NAEPP

National Asthma Education and Prevention Program