Development and Initial Validation of the Asthma Severity Scoring System (ASSESS)

Abstract

Background

Tools for quantification of asthma severity are limited.

Objective

To develop a continuous measure of asthma severity, the Asthma Severity Scoring System (ASSESS), for adolescents and adults incorporating domains of asthma control, lung function, medications, and exacerbations.

Methods

Baseline and 36-month longitudinal data from participants in Phase 3 of the Severe Asthma Research Program (SARP, ) were utilized. Scale properties, responsiveness, and a minimal important difference (MID) were determined. External replication was performed in participants enrolled in SARP Phase 1/2. Utility of ASSESS for detecting treatment response was explored in participants undergoing corticosteroid responsiveness testing with intramuscular triamcinolone and participants receiving biologics.

Results

ASSESS scores ranged from 0 to 20 (8.78 ± 3.9; higher scores reflect worse severity) and differed between 5 phenotypic groups. Measurement properties were acceptable. ASSESS was responsive to changes in quality of life with a MID of 2, with good specificity for outcomes of asthma improvement and worsening, but poor sensitivity. Replication analyses yielded similar results, with a 2-point decrease (improvement) associated with improvements in quality of life. Participants with ≥2 point decrease (improvement) in ASSESS scores also had greater improvement in lung function and asthma control after triamcinolone, but these differences were limited to phenotypic clusters 3, 4 and 5. Participants treated with biologics also had ≥2 point decrease (improvement) in ASSESS scores overall.

Conclusions

The ASSESS tool is an objective measure that may be useful in epidemiologic and clinical research studies for quantification of treatment response in individual patients and in phenotypic groups. However, validation studies are warranted.

Graphical Abstract

Capsule summary

Objective tools for quantifying asthma severity are limited. The Asthma Severity Scoring System (ASSESS) assesses 4 domains of asthma and may be useful for quantifying treatment responses in research settings.

Introduction

Asthma severity is a unique construct, independent of asthma control, that encompasses a range of asthma patients from mild to severe.^1,2 Whereas asthma “control” refers to the extent to which symptoms and other features of asthma are present in patients, asthma “severity” reflects the level of treatment required to control symptoms and exacerbations.² Like asthma control, asthma severity is not a static feature and can change over the course of months or years in response to seasonal variations, infections and other exposures.^3–5 However, while there are many standardized tools available for the assessment of asthma control in both clinical and research settings, tools for quantification of asthma severity are not widely available.

The National Institute of Allergy and Infectious Disease-sponsored Inner-City Asthma Consortium recently developed a Composite Asthma Severity Index (CASI) which quantifies asthma severity through domains of impairment, risk, and medication requirements, but that tool was developed in a population of children (and adolescents) and lacks details on systemic corticosteroids, emerging controller medications such as tiotropium, and biologic therapies that are currently utilized for the treatment of severe asthma.² Recognizing that severe asthma is a multidimensional construct, we aimed to develop a continuous measure of asthma severity for adolescents and adults that: 1) incorporates fundamental aspects of asthma control, lung function, current controller medications, and recent exacerbations in the assessment of the construct without extensive questioning, 2) is sensitive to short-term temporal changes in asthma and its management, 3) does not require prior knowledge of the patient, and 4) could be utilized by future investigators as well as clinicians in real-world practice settings. The development, measurement properties, and validation of this tool, named the “ASthma Severity Scoring System” (ASSESS), are reviewed below.

Methods

Participants age 12 and older enrolled in Phase 3 of the National Heart, Lung and Blood Institute’s Severe Asthma Research Program (SARP) between November 2012 and February 2015 with at least one year of follow-up data were included in the analysis (Figure E1). Eligibility criteria for SARP3 included a physician diagnosis of asthma and either ≥ 12% reversibility in the forced expiratory volume in one second (FEV₁) after bronchodilator administration or airway hyperresponsiveness to methacholine, evidenced by a provocative concentration of methacholine ≤ 16 mg/mL. Thirteen participants with FEV₁ values below 50% of predicted who could not undergo methacholine challenge due to concerns for safety were also enrolled at the discretion of the principal investigator. Current smoking, smoking history >10 pack years if ≥30 years of age or >5 pack years if <30 years of age, premature birth before 35 weeks gestation, and other chronic airway disorders such as aspiration or vocal cord dysfunction were criteria for exclusion. Permission to proceed with SARP studies was granted by the Institutional Review Board of each institution and an independent Data Safety and Monitoring Board. The study was also registered at ClinicalTrials.gov (). Informed written consent and assent (if less than 18 years) were obtained from all participants prior to enrollment in these studies.

SARP design and procedures.

Details of the study design have been reported previously.^{6, 7} Participants completed a baseline characterization visit during which systemic triamcinolone acetonide was administered, a follow-up visit at 18 ± 3 days to assess triamcinolone response, and follow-up characterization visits at 12, 24 and 36 months (±90 days). Visits were postponed if an asthma exacerbation treated with systemic corticosteroids or a respiratory infection treated with antibiotics was reported within the preceding four or two weeks, respectively. Telephone calls were completed every 6 months (±60 days) to assess for asthma control and healthcare utilization. Exacerbations were defined according to a consensus report⁸ as a worsening of asthma necessitating treatment with systemic corticosteroids. Severe asthma was defined using a modification of the European Respiratory Society (ERS)/American Thoracic Society (ATS) definition⁹ that required the presence of: 1) asthma requiring Global Initiative for Asthma (GINA) step 4-5 treatment with high doses of inhaled corticosteroids (ICS) and other controller medications, or 2) systemic corticosteroid treatment for ≥50% of the previous year to achieve or maintain asthma control. Management by an asthma specialist was not a criterion for severe asthma in SARP3. Other SARP3 characterization procedures including assignment of participants to 5 phenotypic asthma clusters previously identified in Phase 1 and Phase 2 of SARP¹⁰ are detailed in the online repository.

Tool development.

ASSESS is an adaptation of the CASI,¹¹ which was developed using the Delphi method and tested with data from children enrolled in a clinical trials network (Table 1). Details of the ASSESS tool development are provided in the online repository and in Tables E1–E3 and Figure E2.

Table 1.

Asthma Severity Scoring System (ASSESS) for age 12 and older.

ACT = Asthma Control Test, CASI = Composite Asthma Severity Index, ICS = Inhaled corticosteroid, LABA = Long-acting-beta agonist, LTRA = leukotriene receptor antagonist

¹For the baseline SARP visit, exacerbations were assessed over the preceding 12 months. Follow-up SARP visits (at 12, 24 and 36 months) assessed exacerbations over the preceding 6 months

Scale properties of the Asthma Severity Scoring System

Internal consistency of the ASSESS tool was evaluated with Cronbach’s alpha. Test-retest reliability was evaluated with intraclass correlation using two-way random models with the following format:

$$({\text{Mean square}}_{\text{rows}}-{\text{Mean square}}_{\text{error}})/{\text{Mean square}}_{\text{rows}},$$

which is the preferred approach for evaluation of rater-based clinical assessments that are designed for routine use by any clinician.¹² Given that there is no available gold-standard measure for asthma severity in adults, validity could not be determined with traditional approaches. Instead, baseline ASSESS scores were compared in participants treated with ICS who reported 3 or more exacerbations in the previous 12 months, and participants with a hospitalization or intensive care unit admission in the previous year using t-tests. ASSESS scores were also compared in participants who did and did not satisfy the modified dichotomous ERS/ATS definition of severe asthma using Point-Biserial correlation analysis.

Responsiveness of the Asthma Severity Scoring System.

Changes in ASSESS scores were compared to changes in Asthma Control Test (ACT) score, absolute change in FEV₁ percent predicted value (calculated as FEV₁ % predicted_{follow-up visit} - FEV₁ % predicted_{prior visit}), and the change in the number of asthma controller medications using Pearson correlational analyses. Responsiveness of the ASSESS tool to changes in the Asthma Quality of Life Questionnaire (AQLQ) total score and individual AQLQ domains (symptoms, activities, emotions, environment) was assessed using Pearson correlational analyses. For comparison purposes, the ability of the modified dichotomous ERS/ATS definition of severe asthma to discriminate these same outcomes was also assessed with t-tests.

Mean changes and Minimal Important Difference.

Mean changes in ASSESS scores were further calculated for groups of participants meeting pre-specified cut-points as follows: 1) ACT score, 3 point increase¹³; 2) FEV₁ absolute change, 10% increase;^{7, 14, 15} 3) controller medications, 2 medication decrease; and 4) AQLQ, 0.5 point increase.¹⁶ Mean differences between groups were compared with t-tests.

The Minimal Important Difference (MID) of the ASSESS tool was determined at baseline and 12, 24 and 36 months with a distribution-based approach, which utilizes the statistical properties of the scale.¹⁷ Data from each corresponding time point were used to determine the standard deviation (SD) and standard error of measurement (SEM). The MID was defined by both 0.5*SD¹⁸ and 1*SEM.¹⁹ The SEM was calculated as follows:

$$\mathit{SEM}={\mathit{SD}}_{\mathit{ASSESS}}\times \sqrt{1-{\mathit{Reliability}}_{\mathit{ASSESS}}},$$

Where reliability corresponds to the Cronbach α value. To aid in clinical interpretation of MID results, receiver operating characteristic (ROC) analyses were performed with one-year differences in ASSESS scores as the predictor and outcomes of asthma improvement (≥1 fewer exacerbations, ≥1 fewer controller medications, ≥0.5 point increase in AQLQ¹⁶) and asthma worsening (≥1 more exacerbation, ≥1 more controller medication, ≥0.5 point decrease in AQLQ¹⁶, hospitalization). Analyses were performed in a merged dataset of repeated measures from 12, 24 and 36 months (N=1686 observations).

External replication.

External replication was performed on a sample of 100 participants age ≥12 years enrolled in phase 1 and 2 of SARP program between 2003 and 2010 with baseline and 6 month follow-up data. None of these participants were enrolled in SARP3. Methods for participant characterization in the SARP 1 and 2 program have been published previously.²⁰

Utility for detection of treatment response.

Utility of the ASSESS tool in detecting treatment responses was assessed in participants in SARP3 who underwent corticosteroid response testing with intramuscular triamcinolone. Changes in Asthma Control Questionnaire (ACQ) scores and FEV₁ % predicted values before and after triamcinolone receipt were compared in participants with a ≥2 point improvement (i.e., decrease) on the ASSESS tool versus participants with ≤2 point improvement. Exploratory analyses also compared ASSESS scores in participants in whom biologic therapy was initiated or discontinued.

Data analysis.

Data were analyzed with IBM SPSS Statistics (Version 24.0). For all analyses, an alpha level of 0.05 was considered the threshold for statistical significance. Given the exploratory nature of the analyses, no adjustments were made for multiple comparisons.

Results

Five hundred ninety eight participants were initially selected for inclusion; however, 562 participants provided at least 12 months of follow up data and were included in the analysis. Features of these participants are shown in Table 2.

Table 2.

Features of the participants, shown as mean ± SEM or number of participants (%). Groups are not significantly different.

Feature	N = 562
Age (years)	44.1 ± 0.7
Age < 18 years	65 (11.6)
Age > 18 years	497 (88.4)
Asthma duration (years)	28.5 ± 0.7
Male	202 (35.9)
Hispanic ethnicity	24 (4.3)
Race
White	350 (62.3)
Black	142 (25.3)
More than one race	49 (8.7)
Other	21 (3.7)
Severe asthma (modified dichotomous ERS/ATS definition)	328 (58.4)
Highest household educational attainment
Did not complete high school	10 (1.8)
High school diploma	57 (10.1)
Some college or technical training	115 (20.5)
Associate degree	87 (15.5)
Bachelor’s degree	289 (51.4)
Decline to answer	4 (0.7)
Highest annual household income
Less than $25,000	104 (18.5)
$25,000 to $49,999	114 (20.3)
$50,000 to $99,999	159 (28.3)
$100,000 or more	109 (19.4)
Decline to answer	76 (13.5)
Saw an asthma specialist (previous year)1	306 (54.4)
Asthma healthcare utilization (previous year)
Unscheduled physician visit	250 (44.5)
Emergency department	143 (25.4)
Hospitalization	70 (12.5)
Intubation for asthma (ever in lifetime)	35 (6.2)
Daily asthma medications
Inhaled corticosteroid	489 (87.0)
Long-acting beta-agonist	439 (78.1)
Long-acting anti-muscarinic	52 (9.3)
Leukotriene receptor antagonist	226 (40.2)
Theophylline	23 (4.1)
Systemic corticosteroids	91 (16.2)
Biologic	43 (7.7)
Asthma Control Test Score	17.2 ± 0.2
23-25	66 (11.8)
20-22	153 (27.3)
17-19	116 (20.7)
14-16	98 (17.5)
11-13	65 (11.6)
8-10	42 (7.5)
5-7	20 (3.6)
Baseline lung function (% predicted value)
FVC	86.7 ± 0.8
FEV1	74.2 ± 0.9

¹Severe asthma (modified dichotomous ERS/ATS definition), n = 236 (72.0%), non-severe asthma, n = 70 (29.9%)

ASSESS score distribution.

In the total sample of SARP3 participants, ASSESS scores ranged from 0 to 20 (mean ± standard deviation, 8.78 ± 3.98; 95% confidence interval (CI) for the mean, 8.45 – 9.11). Responses to the individual items of the ASSESS tool were varied, but some clustering was noted in the pattern of responses (Figure E3). For example, one group of participants with high ASSESS scores had poor asthma control yet normal lung function and few exacerbations, whereas another group of participants with high ASSESS scores had poor asthma control with significant airflow obstruction and frequent exacerbations, consistent with the heterogeneity of asthma previously reported in the SARP3 program.²¹

The distribution of ASSESS scores according to the modified dichotomous ERS/ATS definition of severe asthma and SARP Phase 1 and 2 phenotypic cluster assignment (Cluster 1: well controlled early-onset atopic asthma; Cluster 2: early-onset atopic asthma with increased medication requirements; Cluster 3: late-onset, non-atopic obese asthma with moderately reduced lung function and frequent exacerbations; Cluster 4: severe airflow obstruction with bronchodilator reversibility and frequent exacerbations despite multiple controller medications; Cluster 5: severe airflow obstruction with less bronchodilator reversibility and frequent exacerbations despite multiple controller medications) is shown in Figure 1. There were no significant differences in ASSESS scores between adolescents 12-17 years (8.1 ± 3.4) and adults ≥18 years (8.9 ± 4.0).

Figure 1.

Distribution of baseline and longitudinal Asthma Severity Scoring System (ASSESS) scores in all participants, participants according to the modified dichotomous European Respiratory Society/American Thoracic Society definition of severe asthma, and participants according to phenotypic cluster assignment. Data in panels A and B reflect the mean ± SEM.

Reliability measures.

Given the multi-dimensional (versus unidimensional) nature of the ASSESS tool, internal consistency of the item questions was expected to be less than 0.7.²² The Cronbach α value was 0.639 for the entire sample, 0.468 for adolescents 12-17 years, and 0.662 for adults ≥18 years, which reflects an expected lack of concordance between symptoms, lung function, medication use and exacerbations in all asthma patients. However, the intraclass correlation coefficients between baseline and 12 months, 12-24 months, and 24-36 months were 0.764, 0.768, and 0.813 for the entire sample, respectively (Figure E4) and suggested “good” test-retest reliability according to the conventions of Koo et al.¹² Intraclass correlation coefficients were also similar between age groups (age 12-17: 0.717, 0.841, 0.732; age ≥18 years: 0.768, 0.766, 0.816). Therefore, further analyses were not stratified by age.

Construct validity.

At baseline, ASSESS scores were higher in participants who reported 3 or more exacerbations, hospitalization or intensive care unit admission for asthma in the previous 12 months (Figure 2). Agreement between ASSESS scores and the modified dichotomous ERS/ATS definition of severe asthma was 0.639 at baseline and 0.635, 0.533, and 0.532 at 12, 24, and 36 months, respectively.

Figure 2.

Asthma Severity Scoring System (ASSESS) scores in participants with 3 or more exacerbations hospitalization, or intensive care unit admission in the previous year. Boxplot whiskers correspond to the 5^th and 95^th percentiles. **p<0.001

Responsiveness.

By design, the ASSESS tool was sensitive to changes in ACT score, FEV₁ % predicted, and the number of asthma controller medications between 0-12 months, 12-24 months, and 24-36 months (Figure E5). The ASSESS tool was also responsive to changes in the AQLQ score (total score and each domain), as shown in Figure 3. In contrast, the modified dichotomous ERS/ATS definition of severe asthma (also assessed at baseline, 12 months and 24 months) did not discriminate changes in these same outcomes (Figure E6).

Figure 3.

Responsiveness of the Asthma Severity Scoring System (ASSESS) score to changes in the Asthma Quality of Life Questionnaire (AQLQ) score between baseline and 12 months, 12 and 24 months, and 24 and 36 months. *p<0.001

Mean differences and Minimal Important Difference.

Mean differences in ASSESS scores between pre-specified participant groups are shown in Table 3 and were approximately 2 overall. Calculation of the ASSESS MID, defined as the smallest difference in the ASSESS score that represents a meaningful change, produced an MID of approximately 2 using a distribution-based approach (for the 0.5*SD calculation, range 1.83-1.96; for the 1*SEM calculation, range 2.39 – 2.56) (Table E4). Clinical interpretation of the calculated ASSESS score MID was further guided by ROC analyses with ASSESS scores as the predictor and outcomes of asthma improvement and worsening reflected by changes in the number of exacerbations, number of controller medications, and AQLQ score and hospitalization occurrence. A MID of 2 had high specificity for outcome detection (80.3 – 85.5% for asthma improvement and 88.0 – 91.4% for asthma worsening), but poor sensitivity (45.8 – 46.5% and 30.0 – 37.3% for improvement and worsening, respectively) (Table E5). A MID of 1 increased sensitivity for asthma improvement (~62%), but only marginally increased sensitivity for asthma worsening (~46%).

Table 3.

Mean difference in ASSESS scores between visits. Negative values indicate a lower ASSESS score at the follow up visit.

Interval	Asthma outcome	n (%) with outcome	Change in ASSESS score		Mean difference in ASSESS score (95% CI)	p-value
			Participants without outcome mean (SD)	Participants with outcome mean (SD)
Baseline – 12 months	ACT score, ≥3 point increase	169 (28.3)	−0.20 ± 2.23	−3.15 ± 2.34	−2.95 (−3.36, −2.54)	< 0.001
	FEV1, ≥10% absolute increase	84 (15.1)	−0.77 ± 2.49	−2.86 ± 2.77	−2.08 (−2.67, −1.50)	< 0.001
	Medication, decrease of ≥2	35 (6.3)	−0.90 ± 2.50	−3.80 ± 3.11	−2.90 (−3.77, −2.02)	< 0.001
	AQLQ score, ≥0.5 point increase	159 (26.6)	−0.44 ± 2.36	−2.73 ± 2.62	−2.29 (−2.74, −1.84)	< 0.001
12-24 months	ACT score, ≥3 point increase	111 (18.6)	0.41 ± 2.12	−2.46 ± 2.72	−2.88 (−3.35, −2.40)	< 0.001
	FEV1, ≥10% absolute increase	60 (11.8)	−0.01 ± 2.45	−1.72 ± 2.79	−1.71 (−2.38, −1.04)	< 0.001
	Medication, decrease of ≥2	5 (1.0)	−0.17 ± 2.52	−4.00 ± 3.16	−3.82 (−6.06, −1.60)	0.001
	AQLQ score, ≥0.5 point increase	117 (19.6)	0.21 ± 2.30	−1.62 ± 2.85	−1.83 (−2.33, −1.32)	< 0.001
24-36 months	ACT score, ≥3 point increase	97 (16.2)	0.21 ± 2.00	−2.25 ± 2.17	−2.46 (−2.92, −2.00)	< 0.001
	FEV1, ≥10% absolute increase	59 (12.7)	−0.02 ± 2.08	−2.18 ± 2.60	−2.16 (−2.75, −1.57)	< 0.001
	Medication, decrease of ≥2	11 (2.3)	−0.27 ± 2.23	−1.45 ± 3.35	−1.18 (−2.53, 0.17)	0.087
	AQLQ score, ≥0.5 point increase	101 (16.9)	0.12 ± 2.12	−1.48 ± 2.41	−1.50 (−1.98, −1.01)	< 0.001

External replication.

Features of the external replication sample are shown in Table E6 and were similar to those of participants in SARP3. Mean ASSESS scores at baseline and the follow-up visit were 8.04 ± 4.61 (95% CI, 6.63 – 8.36) and 7.00 ± 4.42 (95% CI, 5.54 – 7.12). Changes in ASSESS scores were also linearly associated with changes in AQLQ scores (Figure 4A–E). Patients who achieved a clinically meaningful improvement in quality of life (≥0.5 point improvement) also had significant reductions in ASSESS score, although not all participants met the MID of 2 for the ASSESS instrument (Figure 4F–J).

Figure 4.

Associations between changes in Asthma Severity Scoring System (ASSESS) and Asthma Quality of Life Questionnaire (AQLQ) scores in the replication cohort. Figures on the right are stratified by changes in AQLQ scores. Boxplot whiskers and dashed lines correspond to the 5^thand 95^th percentiles and the minimal important difference of −2 for ASSESS improvement. *p < 0.05, **p<0.01

Utility for detection of treatment response.

Overall, ASSESS scores were significantly lower after the triamcinolone intervention (p < 0.001, Figure 5A). Participants with ≥2 point improvement (i.e., decrease) in ASSESS scores after triamcinolone also had greater mean improvements in FEV₁ % predicted and ACQ scores at 18 ± 3 days after triamcinolone administration (Figure 5B–D). However, these differences were limited to participants with a modified dichotomous ERS/ATS definition of severe asthma (Figure E7, Table E7) and participants assigned to phenotypic clusters 3, 4 and 5 (Figure E8). Further exploratory analyses of patients with severe asthma in whom biologic therapy was initiated (N = 22) also demonstrated higher baseline ASSESS scores than severe patients not treated with biologics and improvement by approximately 2 points (−1.95 ± 3.27) after biologic initiation (Figure E9).

Figure 5.

Radar plots depicting changes in individual domains of the Asthma Severity Scoring System (ASSESS) at baseline and after intramuscular triamcinolone. Spikes correspond to individual participants responses. Panels B-D depict changes (mean ± SEM) in FEVi, ACQ7, and ACQ6 scores after triamcinolone in participants in whom ASSESS scores improved (≥2 point decrease), did not change, and worsened (≥2 point increase). **p<0.01

Discussion

Objective measures of asthma severity are limited, despite the emphasis of asthma severity by treatment guidelines for over a decade.^{2, 23} The relative explosion of new product development for asthma in recent years now necessitates objective measures for distinguishing patients who might benefit from new treatments such as biologics as well as patients who do not respond to those therapies.²⁴ Advantages of objective versus subjective patient assessments are clearly demonstrated in the asthma literature. For example, other studies of asthma control and asthma functional status have shown that concordance between objective and judgment-based assessments is often poor and may even worsen in patients receiving higher Step-based therapy.^25–28 Asthma severity determination is also discordant between physicians in both primary care²⁹ and specialist³⁰ settings. Objective measures such as ASSESS have potential to mitigate the rising costs of asthma severity both in the United States^{31, 32} and elsewhere^{33, 34} through comprehensive assessment of treatment responses and ultimately, elimination of unnecessary treatments in patients who do not benefit.

ASSESS was developed in response to the unmet need for a severity scoring system in adolescent and adult patients with asthma.³⁵ Despite the multidimensional nature of the ASSESS tool, it has acceptable measurement properties including test-retest reliability and responsiveness to changes in asthma-related quality of life in heterogeneous asthma populations. Our exploratory analyses also suggest that a reduction (i.e., improvement) of ASSESS scores by at least 2 points may also aid with quantification of response to severe asthma treatments in individual patients and in some phenotypic subgroups. While single outcome variables (such as FEV₁ or exacerbation rate) have been used previously in many clinical trials, such narrowly focused analyses may miss the complexity of this disease manifestation and the effects of selected outcome variables on each other. We feel that ASSESS can overcome the limitation of single response variable by offering an integrated numerical score that is sensitive to changes in individual patients. The ASSESS tool is also the first to include biologics and anticholinergics in the assignment of medication treatment step.

Other tools have been proposed for determining asthma severity, but these have limitations in adult asthma populations. The CASI tool was derived from a population of children enrolled in clinical trials¹¹ and the distribution of its components (i.e., lung function measures) differ in adults. The CASI tool also does not address medications that are commonly used in adult asthma populations.¹¹ A separate continuous score of asthma severity developed for adults has also been proposed for epidemiological studies,³⁶ but this score does not adequately capture the constructs of severity as recognized by treatment guidelines and focuses only on symptom frequency and pharmacologic treatment. The coding of the variables in that study (i.e., “yes” or “no” for most symptoms and “none,” “GINA step 1,” or “≥GINA step 2” for medications) also does not distinguish more severe patients with multiple controller medications.³⁶

Nonetheless, there are limitations to our approach. Most importantly, there is no gold standard measure of asthma severity for the purpose of comparison. We instead evaluated performance of the ASSESS score against its own component parts, so it is not surprising that associations were seen. We also were unable to directly compare ASSESS scores to CASI scores. The CASI tool requires that daytime and nighttime symptoms be assessed by self-report using a 2-week recall period prior to the patient encounter, and this information was not available in SARP. However, given the extensive utilization of the ACT tool in clinical practices across the world, our intent was to develop a tool that could utilize information captured by the ACT. An advantage of this approach is that the ACT tool uses a 4-week recall period and collects symptom information as well as information on self-rated asthma control and impact of asthma on functioning. Nonetheless, it is also recognized that despite the 4-week recall window, ACT scores may more strongly reflect symptoms in the immediate 2 weeks.³⁷ We were also limited in our assessment of exacerbation rates at the baseline visit, which impacted tool development, and assessment of treatment responsiveness given the design of the SARP3 study. For example, post-triamcinolone ASSESS scores were calculated from data obtained at the 6-month phone call, with the exception of FEV₁, which was obtained at the post-triamcinolone follow-up visit 18 ± 3 days after triamcinolone administration. This long interval between triamcinolone administration and follow-up may not have been optimal for detection of asthma control responses. Indeed, other studies have shown that peak symptom effects with systemic triamcinolone are evident at 7-10 days.³⁸

Although the Cronbach alpha value associated with the ASSESS tool are lower than those associated with other tools, asthma “severity” is not a unidimensional construct and therefore struct cut-points of 0.70 for “acceptability” are not appropriate. Indeed, Brunner et al.²² argue that is worth re-thinking recommended benchmark values for Cronbach alpha in the context of multi-dimensional measures, because scale scores below these values may still contain valuable information and may still have predictive power. That fact that the Cronbach alpha was less than 0.7 was therefore expected and reflects a lack of concordance between symptoms, lung function, medication use and exacerbations in all patients.

The ASSESS score also does not take into account medication adherence and could be limited if prescribing practices are not aligned with treatment recommendations. Indeed, over-prescription of systemic corticosteroids and under-prescription (or under-utilization) of ICS and other asthma controller medications have been reported in the literature and may be particularly problematic in certain patient populations.^39–41 The ASSESS tool also focuses on current medications and does not account for treatment duration or the timing of medication initiation (for example, the previous week or 12 months earlier). Generalization to other asthma populations is also cautioned since the ethnic distribution of the SARP3 population was limited and participant recruitment occurred predominately at large academic medical centers across the United States.

The lack of a gold standard measure of asthma severity also poses a challenge for calculation of the MID of the ASSESS tool. The two broad methods of estimating MID, the distribution method and the anchor-based methods, are conceptually different.¹⁷ Whereas the distribution method uses the statistical properties of the score distribution, the anchor-based method uses a direct response from the patient on another tool that measures the same construct to evaluate the meaning of a particular degree of change.¹⁷ Anchor-based methods often yield more conservative estimates of MID compared with the distribution method⁴² and thus the clinical meaningfulness of our calculated MID should be interpreted with caution until further studies are performed. We did compare changes in ASSESS scores to changes in patient-reported quality of life, and found the ASSESS tool to be responsive in this regard. Patients achieving a MID of 0.5 point improvement¹⁶ on the AQLQ instrument also had improvement (i.e., decrease) of ~2 points on the ASSESS tool, which was similar to our calculated MID value of ~2. The specificity of a 2-point MID was also good, but the sensitivity for outcome detection (either asthma improvement or asthma worsening) was poor. Although a separate analysis of the CASI tool found that a MID of 1 point or greater may be appropriate for future studies,⁴³ a one-point change in ASSESS could occur spuriously (for example, from a 1-point increase in FEV₁ from 69 to 70% predicted, or a 1-point increase in ACT score from 10 to 11) and have little clinical meaningfulness. Recognizing that asthma severity is a multi-dimensional construct and not a unidimensional measure, an MID of 2 for ASSESS is also more likely to capture changes in more than one severity dimension.

Despite these limitations, there are a number of strengths. A major strength of the present study is the comprehensive characterization of enrolled participants and enrichment of the study population with severe asthma as defined by ERS/ATS criteria, which has a prevalence rate of less than 5% in the entire asthma population.⁴⁴ Another strength is the availability of longitudinal data of the majority of the participants, allowing for testing this tool in the same group of subjects over time. The multi-center design of SARP is another strength since asthma features and related outcomes can be subject to wide geographic variability.^{45, 46} The SARP study also utilized crude measures of socioeconomic status such as education and income and had adequate representation across socioeconomic brackets.

In conclusion, the ASSESS tool, which assesses 4 domains of asthma severity, has acceptable measurement properties and is responsive to changes in asthma quality of life with a MID of approximately 2. ASSESS explains significant variance in selected asthma outcomes and may be useful in epidemiologic and research studies to quantify treatment response in individual patients and in phenotypic subgroups. However, additional validation studies are needed. We caution against the use of ASSESS in clinical practice until such studies are undertaken, to avoid inappropriate changes in asthma treatment.

Clinical implications.

The Asthma Severity Scoring System (ASSESS) incorporates four asthma domains and may be useful in research settings for the quantification of treatment response in individual patients and in phenotypic groups.

Abbreviations

ACQ

Asthma Control Questionnaire

ACT

Asthma Control Test

ASSESS

Asthma Severity Scoring System

ATS

American Thoracic Society

AQLQ

Asthma Quality of Life Questionnaire

CASI

Composite Asthma Severity Index

ERS

European Respiratory Society

FEV₁

Forced expiratory volume in one second

FVC

Forced vital capacity

GINA

Global Initiative for Asthma

ICS

Inhaled corticosteroid

LABA

Long-acting beta-agonist

MID

Minimally important difference

ROC

Receiver operating curve

SARP

Severe Asthma Research Program