Abstract
Objectives:
To estimate meaningful score differences (MSDs) and meaningful score regions (MSRs) for the Patient-Reported Outcomes Measurement Information System® (PROMIS®) Pediatric Asthma Impact scale to enhance score interpretability.
Methods:
Secondary analysis included 106 children with asthma (8–17 years of age) who completed weekly surveys for 4 weeks. Repeated measures correlations examined the magnitude of association of the PROMIS Pediatric Asthma Impact scale with other asthma measures. MSDs were calculated using mixed models with the Global Impact of Change (GIC) on Asthma and Health scores as anchors. MSRs were calculated using the receiver operating characteristics analysis with Self-Reported Asthma Symptom Rating (ASR), Global Initiative for Asthma (GINA) control criteria, and Asthma Control Test (ACT) or Childhood Asthma Control Test (cACT) as anchors.
Results:
Changes in PROMIS Pediatric Asthma Impact T-scores were correlated with GIC-Asthma (r=0.45) and GIC-Health (r=0.34). MSDs were 2.3–2.5 points for improvement and 3.5–3.6 points for deterioration. PROMIS Pediatric Asthma Impact T-scores were correlated with GINA (r=−0.22), ACT (r=−0.42), cACT (r=−0.41), and ASR (r=−0.47). The MSR cutoff T-scores for GINA were 38.7 and 49.2 between controlled, partly controlled, and uncontrolled asthma; for ACT/cACT, 45.6 between controlled and uncontrolled; and for Self-Reported ASR, 47.9, 50.1, and 55.8 between very good, good, a little good, and bad.
Conclusions:
Following recommendations from the FDA’s patient-focused drug development guidance on clinical outcome assessments, estimated MSDs and MSRs aid the interpretation of scores and changes in scores observed in clinical research studies to reflect the meaningful impact of asthma on children.
Keywords: Meaningful score differences, Meaningful score regions, PROMIS, Pediatric, Asthma
Precis:
Following FDA guidance, meaningful score differences and regions are defined for the PROMIS® Pediatric Asthma Impact scale to enhance clinical interpretation of pediatric asthma outcomes.
INTRODUCTION
Pediatric asthma is a prevalent chronic respiratory condition affecting millions of children worldwide, with 1 in 10 children having asthma.1 Characterized by coughing, wheezing, shortness of breath, and chest tightness, this chronic condition significantly burdens children’s daily lives.2 It limits their physical activities, causes absenteeism from school, disrupts sleep, and leads to anxiety and social exclusion, thereby diminishing their overall quality of life.3 Chronic asthma can also result in hospitalizations and increase the psychological burden on both children and their families.4 Understanding and comprehensively measuring the impact of asthma on children’s lives is crucial for developing effective interventions and improving clinical outcomes.
In pediatric asthma research studies and healthcare practice settings, health-related quality of life (HRQOL) serves as a critical clinical outcome, providing insights into functional status and well-being beyond traditional asthma control measures.5–8 Among the HRQOL measures for pediatric asthma, the Patient-Reported Outcomes Measurement Information System® (PROMIS®) Pediatric Asthma Impact Scale was designed based on the application of rigorous qualitative and psychometric methods.9–12 It has garnered evidence supporting its validity,11 responsiveness,13,14 and reliability15 in the context of children living with asthma. However, the meaningful interpretation of PROMIS Pediatric Asthma Impact Scale’s T-scores remains unclear, limiting its application to clinical care and clinical research. This study aims to establish meaningful score differences (MSDs) and meaningful score regions (MSRs) for the PROMIS Pediatric Asthma Impact Scale using anchor-based approaches.
MSD and MSR represent two different frameworks for interpreting scores on clinical outcome assessments, which are increasingly being used in the specification of endpoints in regulatory decision-making of clinical trials, as outlined in the U.S. Food and Drug Administration (FDA)’s Patient-Focused Drug Development (PFDD) draft Guidance 4.16 MSD represents the within-person change or difference score that patients perceive as meaningful. MSD for a clinical outcome assessment is typically estimated using anchor-based or distribution-based approaches. Anchor-based approaches compare the change in the clinical outcome assessment with the change in another related clinical outcome assessment (i.e., anchors)17–19; Distribution-based methods involve statistical criteria from the clinical outcome assessment scores, such as fractions of the standard deviation of the scores, the standard error of measurement, and the effect size.20,21 Anchor-based approaches are recommended by the FDA over the distribution-based approaches in the context of score interpretation due to their direct consideration of patients’ perspectives. To date, the MSD of the PROMIS Pediatric Asthma Impact Scale has been examined only using distribution-based approaches13,14 without incorporating patients’ input on meaningfulness.
MSRs reflect distinct ordinal regions or levels of a construct (e.g., controlled vs uncontrolled asthma, feeling bad vs good) across the score range and serve as references for interpreting absolute scores of a clinical outcome assessment. MSRs are estimated using methods such as bookmarking22 or applying related anchors to delineate the distribution of PRO scores at each ordinal level.16 When treatment effects from different clinical trial arms fall into different score regions that are predefined to be meaningful by patients, it indicates a higher likelihood of treatment efficacy. In clinical practice settings, clear-cut thresholds are essential for interpreting an individual child’s scores reflecting different levels of asthma control and quality-of-life impact. These thresholds between score regions, traditionally referred to as “cut points” or cutoff scores, are typically derived from established anchors, such as the Patient Global Impression of Severity (PGIS). Thus far, only one brief report23 estimated MSRs for the PROMIS Pediatric Asthma Impact Scale using two established asthma control measures: Global Initiative for Asthma (GINA) and the Asthma Control Test (ACT).
Our study aims to estimate MSDs and MSRs for the PROMIS Pediatric Asthma Impact Scale using anchor-based approaches. These estimated MSDs and MSRs will enhance the interpretability and clinical utility of the PROMIS Pediatric Asthma Impact Scale in future clinical trials and clinical practice, illustrating the FDA’s PFDD guidance.
METHODS
Study design
This study is based on a secondary analysis of longitudinal observational data collected at two academic medical centers from children 8 to 17 years of age with asthma.24 The study involved initial and final in-person visits, spaced 28 days apart, with 3 weekly online surveys conducted every 7 days. This study was registered on ClinicalTrials.gov (NCT03933540) and was approved by the institutional review boards, with Duke University serving as the coordinating center.
Participants
One hundred six children with uncontrolled or partly controlled asthma were recruited from pediatric hospitals in Chapel Hill, NC and Boston, MA between December 2018 and July 2019. Children and adolescents aged 8 to 17 were included if they had a Forced Expiratory Volume In 1s (FEV1) less than 80% predicted, a history of exacerbations, or a diagnosis of partly controlled or uncontrolled asthma. Participants also needed to read, speak, and understand English and have access to a smartphone, computer, tablet, or other device with internet access capable of supporting electronic survey completion. Exclusions applied to those with significant cognitive impairments, state custody, or comorbid conditions that could affect results (see ClinicalTrials.gov: NCT03933540 for more details).
Measures
Target measure: PROMIS Pediatric Asthma Impact Scale
The PROMIS Pediatric Asthma Impact 8-item Short Form is a patient-reported questionnaire designed to assess the symptoms and asthma-related impacts on daily functioning and well-being of pediatric patients aged 8 to 17 over the past 7 days.12 Responses are captured on a 5-point Likert scale ranging from Never to Almost Always, with higher scores indicating worse quality of life and higher asthma impact. The PROMIS standardized T-scores have a mean of 50 and a standard deviation of 10, with the PROMIS Pediatric Asthma Impact Scale specifically calibrated based on a diverse clinical sample of children aged 8 to 17 years with asthma.25
The 8-item short form comes from the larger PROMIS Pediatric Asthma Impact Scale item bank that has demonstrated good reliability and validity in the context of pediatric asthma, including significant convergent and divergent validity with other asthma-related and non–asthma-related scales,11 significant predictive validity on school functioning and daytime sleepiness,26 significant responsiveness to asthma control status change,13,14 and test-retest reliability of 0.82 in 2 weeks and internal consistency of 0.89.15 PROMIS Pediatric Asthma Impact Scale was completed weekly through an online survey on Days 7, 14, and 21, and at the in-person clinic follow-up visit (Day 28).
MSD anchors: Global Impact of Change (GIC) questions
Two GIC questions were used to assess children’s perceptions of changes in their asthma and overall health since the last survey (which was 1 week in the current study): “Since the last time I completed this survey, my asthma [or ‘health’] has been ….” The options included: “Much better,” “A little better,” “The same,” “A little worse,” and “Much worse.” The GIC questions were completed weekly through an online survey on Days 14 and 21 from the home setting and on Day 28 during an in-person follow-up visit in the clinic.
MSR anchors:
Global Initiative for Asthma (GINA) criteria
GINA2 is a 4-item clinician-reported tool designed to assess asthma control status over the past 4 weeks. It evaluates daytime symptoms, nighttime symptoms/awakenings, the need for reliever/rescue medications, and activity limitations with a binary response option: yes or no. Depending on the number of items that meet predefined criteria, the control status is categorized into uncontrolled (3–4 items with yes), partly controlled (1–2 items with yes), and well controlled (no item with yes). The GINA was completed at in-person clinic visits at baseline (Day 0) and follow-up (Day 28).
Asthma Control Test (ACT) and Childhood Asthma Control Test (cACT)
The ACT is a 5-item patient-reported questionnaire designed to evaluate asthma control of individuals aged 12 and older in the past 4 weeks.27 It assesses the impact on daily functioning, shortness of breath, nighttime symptoms, use of rescue inhalers, and patients’ perception of asthma control. Responses are scored on a 5-point Likert scale. The total score ranges from 5 to 25, with higher scores indicating better asthma control. A score of 20 or above suggests well-controlled asthma. The ACT was completed by adolescents aged 12 and older at in-person clinic visits at baseline (Day 0) and follow-up (Day 28).
The cACT is a 7-item questionnaire used to assess asthma control in children aged 4 to 11.28 It includes 4 child-reported items on symptoms and activity limitations, and 3 observer-reported (e.g., parent) items on the frequency of asthma symptoms. The total score ranges from 0 to 27, with higher scores indicating better control. A score of 20 or above suggests well-controlled asthma. The cACT was completed by children aged 8 to 11 years old and their caregivers at in-person clinic visits at baseline (Day 0) and follow-up (Day 28).
Self-Reported Asthma Symptom Rating
The Self-Reported Asthma Symptom Rating (ASR) assesses children’s current perception of their asthma symptoms. It includes a single item: “How would you rate your asthma today?” with response options of “Very bad,” “Bad,” “A little bad,” “A little good,” “Good,” and “Very good.” The Self-Reported ASR was completed weekly by all patients through an online survey on Days 7, 14, and 21, and at the in-person clinic follow-up visit (Day 28).
Statistical analysis
We used anchor-based approaches to calculate MSDs and MSR cutoff points for the PROMIS Pediatric Asthma Impact Scale. Anchors included in this study were required to have a minimum correlation with the PROMIS T-scores (or the changes in the PROMIS T-scores) of 0.30.29 Repeated measures correlations30 were used to estimate the intra-individual association between two measures on repeated assessments. Using analysis of covariance (ANCOVA), the repeated measures correlation accounts for non-independence among observations. It removes measured variance between participants and assumes parallel slopes of linear regression of the within-subject paired measures.
For the calculation of MSDs, data from Days 14, 21, and 28 were used for both GIC asthma and GIC health. One important consideration while generating MSDs, as emphasized in FDA’s PFDD draft Guidance 4,16 is to generate MSDs separately for deterioration and improvement because these directions may reflect distinct patient experiences and thresholds for clinical significance. This differentiation could ensure that the clinical outcome assessments appropriately capture both positive and negative changes or differences, aligning with the context of use and enabling a more precise interpretation of the scores in the clinical contexts. The GIC scores were categorized into three levels: “worse” for deterioration (combining “A little worse” and “Much worse”), “Same” for no change, and “Better” for improvement (combining “A little better” and “Much better”). The combination was necessitated due to the small sample size for some response options. Data from Days 7, 14, 21, and 28 were used for PROMIS Pediatric Asthma Impact Scale T-scores to calculate its change scores in the past 7 days for Days 14, 21, and 28. Mixed-regression models were employed for repeated measures to predict changes in PROMIS Pediatric Asthma Impact T-scores with the anchor measure as the categorical predictor (three levels: “better”, “same”, and “worse”). MSDs for each anchor were quantified through least squares mean estimates in PROMIS T-scores change for the “better” group (improvement) and the “worse” group (deterioration).
In terms of MSRs, based on data availability, in-person visit data from Day 0 and Day 28 were used for GINA and ACT/cACT, and data from Day 7 and Day 28 were used for PROMIS Pediatric Asthma Impact T-scores correspondingly. Additionally, data from Days 7, 14, 21, and 28 were used for the self-reported asthma rating question and for PROMIS Pediatric Asthma Impact T-scores correspondingly. For the self-reported asthma rating, “Very bad,” “Bad,” and “A little bad” were combined into a single level—“bad”—because of the low frequency in each individual category, resulting in these four levels: “Bad,” “A little good,” “Good,” and “Very good.” Receiver operating characteristic (ROC) curves and area under the ROC curve (AUC) calculations were employed to establish the cutoff scores between each level of the anchor measure, using a multiclass nominal logistic regression model. The accuracy criterion, distance to the point (0,1) on the ROC curve, sensitivity-specificity difference, and Youden Index were used to determine the optimal cutoff points. No additional analysis was conducted for GINA and ACT scores, as referenced in the brief report.23 The results from the report were displayed here for comparisons. SAS version 9.4 was used for all analyses.
RESULTS
Demographics
In the baseline, we recruited 106 participants, with one participant missing the demographic data in the baseline assessment. The remaining sample with 105 participants consisted of 47% females, with a mean age of 11.5 years (SD = 2.6). The racial/ethnic composition was 33% White, 45% Black, 16% Hispanic, 2% Asian, and 14% mixed race. Compared to the baseline, the attrition rates on the 7th, 14th, 21st, and 28th day of online assessments were 0%, 11%, 10%, and 12%, respectively. Reasons for attrition included failure to comply with study procedures or loss to follow-up despite reminders to complete the online survey before the survey window closed. The asthma medications or treatments reported at baseline included inhaled corticosteroids: Flovent HFA (fluticasone; n=45, 43%), Pulmicort Flexhaler (budesonide; n=1, 1%), Pulmicort Neb (budesonide; n=1, 1%), Asmanex HFA (mometasone; n=1, 1%), and combined inhaled corticosteroid and long-acting beta agonist treatments: Advair HFA (fluticasone/salmeterol; n=34, 32%). Two participants reported using both Flovent HFA and Advair HFA. Table 1 presents the descriptive statistics of the target measure and anchor measures of each time point.
Table 1.
Descriptive statistics of key outcomes
| Day 0 | Day 7 | Day 14 | Day 21 | Day 28 | |
|---|---|---|---|---|---|
| Sample Sizea | 105 | 105 | 94 | 95 | 93 |
| Key Outcomes | |||||
| PROMIS Pediatric Asthma Impact Scale Short Form, M (SD), n | / | 45.01 (10.84), | 87 45.43 (10.78),82 | 46.40 (11.18), 84 | 43.42 (9.65), 78 |
| Global Impact of Change of Asthma, n (%) | |||||
| Worse | / | / | 17 (18.09%) | 20 (21.05%) | 12 (13.64%) |
| Same | / | / | 45 (47.87%) | 40 (42.11%) | 33 (37.50%) |
| Better | / | / | 32 (34.04%) | 35 (36.84%) | 43 (48.86%) |
| Global Impact of Change of Health, n (%) | |||||
| Worse | / | / | 13 (13.83%) | 17 (17.89%) | 6 (6.82%) |
| Same | / | / | 55 (58.51%) | 40 (42.11%) | 41 (46.59%) |
| Better | / | / | 26 (27.66%) | 38 (40.00%) | 41 (46.59%) |
| Global Initiative for Asthma (GINA) criteria, n (%) | |||||
| Uncontrolled | 36 (34.29%) | / | / | / | 18 (19.35%) |
| Partly controlled | 52 (54.29%) | / | / | / | 39 (41.94%) |
| Well controlled | 12 (11.43%) | / | / | / | 36 (38.71%) |
| Asthma Control Test (ACT), M (SD), n | 15.00 (4.33), 49 | / | / | / | 15.74 (3.57), 42 |
| Childhood Asthma Control Test (cACT), M (SD), n | 20.46 (4.37), 56 | / | / | / | 21.72 (3.95), 47 |
| Self-Reported Asthma Symptom Rating, n (%) | |||||
| Badb | / | 21 (21.65%) | 9 (9.57%) | 21 (22.11%) | 5 (5.69%) |
| A little good | / | 6 (6.19%) | 16 (17.02%) | 10 (10.53%) | 9 (10.23%) |
| Good | / | 32 (32.99%) | 29 (30.85%) | 26 (27.37%) | 33 (37.5%) |
| Very good | / | 38 (39.18%) | 40 (42.55%) | 38 (40.00%) | 41 (46.59%) |
The sample size was from the demographic information and did not represent the sample size for each outcome below.
“Bad” represents the combined categories of “very bad,” “bad,” and “a little bad.”
Meaningful Score Differences
The correlation of the change in PROMIS Pediatric Asthma Impact Scale T-scores with the GIC-Asthma item was r=0.45 (95% CI, 0.32 to 0.56) and with the GIC-Health was r=0.34 (95% CI, 0.20 to 0.47). The MSDs for deterioration (increased asthma impact) were 3.5 and 3.6, and the MSDs for improvement (reduced asthma impact) were 2.3 and 2.5 for GIC-Asthma and GIC-Health, respectively. Detailed results of the mixed models are presented in Table 2.
Table 2.
Mixed models results with GIC score predicting PROMIS change score
| Variable Level | Least Square Means | Standard Error | DF | t | p | Lower 95% CI | Higher 95% CI |
|---|---|---|---|---|---|---|---|
| GIC-Asthma | |||||||
| Better | -2.46 | 0.59 | 63 | -4.18 | <.001 | -3.64 | -1.28 |
| Same | -0.01 | 0.58 | 63 | -0.01 | 0.990 | -1.19 | 1.16 |
| Worse | 3.64 | 1.01 | 63 | 3.60 | <.001 | 1.62 | 5.67 |
| GIC-Health | |||||||
| Better | -2.31 | 0.61 | 52 | -3.79 | <.001 | -3.53 | -1.09 |
| Same | 0.10 | 0.53 | 52 | 0.19 | 0.8538 | -0.97 | 1.16 |
| Worse | 3.53 | 1.19 | 52 | 2.98 | 0.004 | 1.15 | 5.92 |
CI = confidence interval; DF = degree of freedom; GIC = global impact of change
Meaningful Score Regions (Cutoff Points)
The PROMIS Pediatric Asthma Impact Scale T-scores were correlated with the level of asthma control assessed through GINA (r=−0.22, 95% CI, −.01 to −0.42), ACT (r=−0.42, 95% CI, −0.13 to −0.65), cACT (r=−0.41, 95% CI, −0.12 to −0.63), and the Self-Reported ASR (r=−0.47, 95% CI, −0.37 to −0.56). In Figure 1, the box-and-whiskers plots illustrate these correlations, showing the distribution of the PROMIS Pediatric Asthma Impact Scale T-scores according to each level of the anchors (i.e., MSRs). In order to find the optimum cutoff scores between the overlapped MSRs, ROC analysis was conducted. As shown in Table 3 and Figure 2, results suggested that, using GINA as the reference, the cutoff scores of the PROMIS Pediatric Asthma Impact Scale T-scores were 38.7 and 49.2, distinguishing between controlled, partly controlled, and uncontrolled asthma. For ACT/cACT, the cutoff score was 45.6, differentiating between controlled and uncontrolled asthma. For the Self-Reported ASR, the cutoff scores were 47.9, 50.1, and 55.8, distinguishing between very good, good, a little good, and bad asthma symptom ratings.
Figure 1.
Meaningful score regions of PROMIS Pediatric Asthma Impact Scale’s T-scores corresponding to the anchors. Note: The box represents the interquartile range (IQR) between the first quartile (Q1, 25th percentile) and the third quartile (Q3, 75th percentile), with a line inside indicating the median (50th percentile). A circle within the box marks the mean. Whiskers extend to the smallest and largest data points within 1.5×IQR from Q1 and Q3.
Table 3.
PROMIS Asthma Impact cutoff scores
| Response | Levels | AUC | PROMIS Asthma Impact Cutoff Score | Sensitivity | Specificity | |
|---|---|---|---|---|---|---|
| ACT/cACT | Uncontrolled | Well controlled | 0.950 | 45.6 | 0.932 | 0.850 |
| GINA | Partly controlled | Uncontrolled | 0.709 | 49.2 | 0.827 | 0.843 |
| Partly controlled | Well controlled | 0.661 | 38.7 | 0.708 | 0.727 | |
| Uncontrolled | Well controlled | 0.909 | ||||
| aMulti-level Accumulative | 0.760 | |||||
| ASR | Bad | A little Good | 0.543 | 55.8 | 0.829 | 0.875 |
| A little Good | Good | 0.632 | 50.1 | 0.908 | 0.878 | |
| Good | Very Good | 0.763 | 47.9 | 0.916 | 0.847 | |
| Bad | Good | 0.769 | ||||
| Bad | Very Good | 0.953 | ||||
| A little Good | Very Good | 0.928 | ||||
| Multi-level Accumulativea | 0.765 | |||||
ACT = Asthma Control Test; ASR = Self-Reported Asthma Symptom Rating; cACT = childhood Asthma Control Test; AUC = area under ROC curve; GINA = Global Initiative for Asthma; PROMIS = Patient-Reported Outcomes Measurement Information System
Multi-level Accumulative AUC is the average of the pairwise AUCs.
Figure 2.
PROMIS Pediatric Asthma Impact Scale cutoff T-scores with three anchors
DISCUSSION
The primary objective of this study was to calculate meaningful score differences (MSDs) and meaningful score regions (MSRs) for the PROMIS Pediatric Asthma Impact scale. Our results revealed MSDs ranging from 2.3 to 2.5 for improvement (decrease in the T-score and asthma impact) and 3.5 to 3.6 for deterioration (increase in the T-score and asthma impact). Further, we identified specific cutoff scores based on the Global Initiative for Asthma (GINA), the Asthma Control Test (ACT)/Childhood Asthma Control Test (cACT), and the Self-Reported Asthma Symptom Rating (ASR), providing a comprehensive framework for interpreting changes and scores in pediatric asthma (as shown in Figure 2).
Our findings of MSDs align with previously published results about MSDs in the PROMIS Pediatric Asthma Impact Scale and other PROMIS Pediatric measures.14,31 For instance, Nelson et al (2022) found the MSD for PROMIS Pediatric Asthma Impact Scale to be around 3 using a distribution-based approach (Standard Error of Measurement).14 For other PROMIS Pediatric measures, Thissen et al (2016) similarly found the “minimally important difference” to be around 2 (by clinicians) or 3 (by patients or caregivers) using the scale-judgment approach, but did not include the PROMIS Asthma Impact measures in their study.31 In addition, neither study differentiated between MSD for improvement versus deterioration in health. Although the magnitude of the MSDs found in this study aligned with the scores derived from other approaches in the literature, one thing to note is that we collapsed the categories for “worse” (i.e., “A little worse” and “Much worse”) and “better” (i.e., “A little better” and “Much better”) due to the relatively small sample sizes in some levels. Thus, the MSDs we found cannot be considered the “smallest” magnitude of meaningful change.
We also observed comparable MSDs in both change directions for GIC in asthma and in health, which underscores result consistency and may also suggest the strong association between the two constructs. Given that asthma significantly influences patients’ health status and quality of life,32 patients’ ratings on health may be tied closely to their underlying asthma condition, especially when responding to the single-item global rating questions. Further, the close assessment timing of these two questions may introduce anchoring effects,33 where responses to the latter question are influenced by prior responses.
One interesting observation worth noting is that the magnitude of MSDs for improvement was approximately one point smaller in magnitude than that for deterioration. This pattern has been reported in some previous studies34; however, there are also other studies that have found comparable magnitudes for improvement and worsening MSDs, or even the opposite pattern35. This asymmetry could be attributed to several factors. One possible explanation is the influence of baseline scores. For instance, a study examining MSDs for the PROMIS Pain Interference measure in rheumatoid arthritis adult patients found that MSDs were larger for improvement than for worsening among adult patients with more severe conditions or higher pain levels, were comparable for those with moderate pain, and showed the opposite pattern in those with lower pain36. A similar effect may be present in our study, given that our sample’s baseline scores were better than those of the reference clinical asthma population (i.e., the mean PROMIS T-score in our sample was approximately five points lower than the reference sample mean of 50). It is likely that when a sample’s baseline condition is not as severe, the potential for improvement is more limited, leading to a smaller MSD for improvement than for worsening. Future studies should further investigate the impact of the sample’s baseline T-scores on MSD estimates for the PROMIS Pediatric Asthma Impact Scale with larger samples to confirm these patterns.
Another potential explanation is the presence of response bias,37 in the form of underreporting deteriorations and overreporting of positive or socially desirable changes. This means that for people with response bias, when they experience the same amount of positive and negative changes in their disease experience, they may be more likely to acknowledge the meaningfulness of the positive changes (e.g., I am doing better now) rather than the negative ones (e.g., I am doing worse now). For example, one study showed that higher response bias tendencies were correlated with higher self-reported quality of life in pediatric cancer patients.38 For patients who are aware of the deterioration, cognitive dissonance39 may still influence their perception—patients may minimize perceived deterioration to avoid psychological distress, necessitating a larger change for acknowledgment. For example, in clinical practice, it is common to observe adolescents with asthma to underplay their asthma symptoms and impacts, which may be related to their needs to resolve the dissonance between what they are feeling and external influences or expectations (e.g., peer pressures).
Regarding MSRs or cutoff scores, ACT/cACT as the anchor had the stronger discriminative ability for distinguishing different regions of PROMIS Pediatric Asthma Impact Scale T-scores, with an AUC exceeding 0.9.40 Further, these measures exhibited impressive sensitivity and specificity, both near or above 0.9. In contrast, the GINA and asthma symptom rating question exhibited fair to poor AUC,1 despite demonstrating good-to-excellent sensitivity and specificity. It is noteworthy that, unlike ACT/cACT, which are child/caregiver-reported, GINA is a clinician-reported measure. The relatively lower AUC for GINA could be attributed to the broad spectrum of event definitions for “partly controlled” asthma, making it challenging to identify an optimal cutoff score that effectively discriminates between partly controlled asthma and uncontrolled or well-controlled asthma. Regarding the asthma symptom rating anchor, while it appears to enhance the granularity of score interpretation, its discriminative ability was found to be only fair to poor. The only cutoff score deemed acceptable was identified between the “good” and “very good” levels.
Clinical recommendations
Clinicians are encouraged to apply the identified MSDs and MSRs in routine practice to better monitor and manage asthma in pediatric patients. These metrics can serve as valuable tools for detecting significant changes in asthma control or status, guiding treatment decisions, evaluating the effectiveness of interventions, and tailoring care plans to individual needs. Further, MSDs and MSRs could facilitate clearer communication with patients and their families by providing tangible measures of progress or decline, enabling timely and precise interventions that align with personalized health goals. Researchers should also consider these findings when designing studies involving the PROMIS Pediatric Asthma Impact Scale. Additionally, the integration of these benchmarks into clinical guidelines and policy-making can enhance asthma control strategies and ultimately improve patient outcomes.
Specifically, in the context of screening disease status (e.g., asthma control), it is often important to prioritize sensitivity to ensure that all potential cases are identified. For example, the ACT/cACT test with a cutoff T-score on PROMIS of 45.6 has a high AUC of 0.95 and a high sensitivity of 0.932 when distinguishing between uncontrolled and well-controlled levels. This stringent cutoff score could be a good choice for clinical practice screening where the goal is to ensure early intervention and rigorous monitoring.
In the context of clinical trials, both sensitivity and specificity of the chosen cutoff point might be equally important to ensure that the effects of an intervention are accurately judged. Here, a balance between sensitivity and specificity is needed. For instance, the Self-Reported ASR test with a cutoff PROMIS T-score of 47.9 provides a good balance between sensitivity (0.916) and specificity (0.847) when distinguishing between good and very good levels. This score would be a suitable reference if the endpoint of the clinical trial valued patients’ subjective experience and wanted more granularity in addition to the control status. Note that this threshold should only be used as a reference for score interpretation and not to determine responders.
Strengths and limitations
This is the first study that uses anchor-based approaches to identify MSDs and MSRs for PROMIS Pediatric Asthma Impact Scale. These estimates apply to all short forms and the computerized-adaptive testing versions derived from the PROMIS Pediatric Asthma Impact Scale’s item bank; as all the items were calibrated by an item response theory model41–43. The study’s robustness is underscored by its demographic diversity and its longitudinal design that includes five assessment points over a 4-week period and allows for the capture of within-person changes over multiple time points. Further, the use of multiple anchors provided different perspectives of score interpretation.
Despite its strengths, the study has some limitations. First, although we mainly recruited participants with partly controlled or uncontrolled asthma using GINA criteria, the sample size of participants with severe or uncontrolled asthma remained small, which may influence the generalizability of the results to this population. This is also the reason for combining the “very bad,” “bad,” and “a little bad” levels. The lack of granularity to differentiate between the severely impacted levels may potentially mask nuanced differences in the experiences of the participants. Future studies could benefit from a sample with a more diverse range of participants and a larger sample size that allows for more detailed categorization and analysis.
Additionally, due to the smaller sample size, we were unable to differentiate MSDs across different score ranges as suggested by the FDA. For instance, the MSDs for a person improving or deteriorating from a score of 35 may vary from those of a person with a score of 65 due to the anchoring effects. This limitation could potentially affect the interpretation and application of the study’s findings. Future studies should consider stratifying the analysis based on score ranges to account for potential anchoring effects.
Finally, while we used the PGICs as the anchors for MSDs due to more stable subgroup sizes, we acknowledge that severity-rating-based change scores, such as using Patient Global Impression of Severity (PGIS), are often preferred for estimating meaningful change, as they avoid potential recall bias44. Nonetheless, the PGIC approach aligns more directly with the patient-centered focus of the FDA’s Patient-Focused Drug Development guidance, as it reflects the patient’s own perception of change, which may sometimes be more sensitive than researcher-calculated differences in severity scores. Unfortunately, small subgroup sizes limited our ability to use severity-rating-based change scores in this analysis. Further justification for using the PGIC is that the assessments were collected weekly, reducing concerns about the child’s memory for their health status a week earlier, compared with longer time gaps in assessments. Future studies should aim to incorporate both PGIS- and PGIC-based approaches to strengthen interpretability.
Conclusion
This study provides valuable insights into the interpretation of the PROMIS Pediatric Asthma Impact Scale T-scores. The estimated MSDs and MSRs can aid clinicians and researchers in interpreting meaningful changes and scores in patients with pediatric asthma. This study also serves as an example of estimating MSDs and MSRs in accordance with the FDA’s Patient-Focused Drug Development guidance for clinical outcome assessments. Despite the limitations, the findings contribute to the growing body of evidence supporting the use of the PROMIS Pediatric Asthma Impact Scale in clinical research and healthcare delivery settings. We hope that our findings will inspire further research in this area and contribute to improved patient care.
Supplementary Material
Highlights:
To date, there has been no evidence available on meaningful score differences and meaningful score regions for the PROMIS® Pediatric Asthma Impact scale. This study provides estimates of both.
The research establishes MSDs for improvement and deterioration for asthma impact and defines MSRs using patient-reported, observer-reported, and clinician-reported anchors, providing a comprehensive framework for interpreting pediatric asthma scores.
This study exemplifies the estimation of MSDs and MSRs according to the FDA’s Patient-Focused Drug Development guidance on clinical outcome assessments.
Funding/Support:
This work was supported by the National Institute of Arthritis and Musculoskeletal and Skin Diseases of the National Institutes of Health under Award Numbers U19AR069525, U19AR069522, and U19AR069526.
Role of the Funder/Sponsor:
The funder had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Authorship Confirmation: All authors certify that they meet the ICMJE criteria for authorship.
REFERENCES
- 1.The Global Asthma Report 2022. Int J Tuberc Lung Dis 2022;26(Supp 1):1–104. [DOI] [PubMed] [Google Scholar]
- 2.Global Initiative for Asthma. Global Strategy for Asthma Management and Prevention. 2024. Available at: https://ginasthma.org/wp-content/uploads/2024/05/GINA-2024-Strategy-Report-24_05_22_WMS.pdf. Accessed January 16, 2025. [Google Scholar]
- 3.Everhart RS, Fiese BH. Asthma severity and child quality of life in pediatric asthma: a systematic review. Patient Educ Couns 2009;75(2):162–168. [DOI] [PubMed] [Google Scholar]
- 4.Ferrante G, La Grutta S. The Burden of Pediatric Asthma. Front Pediatr 2018;6:186. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Banjari M, Kano Y, Almadani S, Basakran A, Al-Hindi M, Alahmadi T. The Relation between Asthma Control and Quality of Life in Children. Int J Pediatr 2018;2018:6517329. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Juniper EF. How important is quality of life in pediatric asthma? Pediatr Pulmonol Suppl 1997;15:17–21. [PubMed] [Google Scholar]
- 7.Schatz M, Mosen DM, Kosinski M, et al. The relationship between asthma-specific quality of life and asthma control. J Asthma. 2007;44(5):391–395. [DOI] [PubMed] [Google Scholar]
- 8.Wilson SR, Rand CS, Cabana MD, et al. Asthma outcomes: quality of life. J Allergy Clin Immunol 2012;129(3 Suppl):S88–123. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.DeWalt DA, Rothrock N, Yount S, Stone AA. Evaluation of item candidates: the PROMIS qualitative item review. Med Care. 2007;45(5 Suppl 1):S12–21. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Reeve BB, Hays RD, Bjorner JB, et al. Psychometric evaluation and calibration of health-related quality of life item banks: plans for the Patient-Reported Outcomes Measurement Information System (PROMIS). Med Care. 2007;45(5 Suppl 1):S22–31. [DOI] [PubMed] [Google Scholar]
- 11.Thissen D, Varni JW, Stucky BD, Liu Y, Irwin DE, Dewalt DA. Using the PedsQL™ 3.0 asthma module to obtain scores comparable with those of the PROMIS pediatric asthma impact scale (PAIS). Qual Life Res 2011;20(9):1497–1505. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Yeatts KB, Stucky B, Thissen D, et al. Construction of the Pediatric Asthma Impact Scale (PAIS) for the Patient-Reported Outcomes Measurement Information System (PROMIS). J Asthma. 2010;47(3):295–302. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Howell CR, Thompson LA, Gross HE, Reeve BB, DeWalt DA, Huang IC. Responsiveness to Change in PROMIS(®) Measures among Children with Asthma: A Report from the PROMIS(®) Pediatric Asthma Study. Value Health. 2016;19(2):192–201. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Nelson A, Singh A, Dasgupta M, et al. Detection of changes of functioning over time after asthma exacerbation in children with the use of PROMIS domains. J Asthma. 2022;59(10):1981–1988. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Varni JW, Magnus B, Stucky BD, et al. Psychometric properties of the PROMIS ® pediatric scales: precision, stability, and comparison of different scoring and administration options. Qual Life Res 2014;23(4):1233–1243. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.U.S. Food and Drug Administration. Patient-Focused Drug Development: Incorporating Clinical Outcome Assessments Into Endpoints for Regulatory Decision-Making (Draft). FDA; 2023. Available at: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/patient-focused-drug-development-incorporating-clinical-outcome-assessments-endpoints-regulatory. Accessed January 16, 2025. [Google Scholar]
- 17.Fulk GD, Ludwig M, Dunning K, Golden S, Boyne P, West T. How much change in the stroke impact scale-16 is important to people who have experienced a stroke? Top Stroke Rehabil 2010;17(6):477–483. [DOI] [PubMed] [Google Scholar]
- 18.Lansdall CJ, McDougall F, Butler LM, et al. Establishing Clinically Meaningful Change on Outcome Assessments Frequently Used in Trials of Mild Cognitive Impairment Due to Alzheimer’s Disease. J Prev Alzheimers Dis 2023;10(1):9–18. [DOI] [PubMed] [Google Scholar]
- 19.Solberg T, Johnsen LG, Nygaard ØP, Grotle M. Can we define success criteria for lumbar disc surgery? : estimates for a substantial amount of improvement in core outcome measures. Acta Orthop 2013;84(2):196–201. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Mouelhi Y, Jouve E, Castelli C, Gentile S. How is the minimal clinically important difference established in health-related quality of life instruments? Review of anchors and methods. Health Qual Life Outcomes. 2020;18(1):136. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Watt JA, Veroniki AA, Tricco AC, Straus SE. Using a distribution-based approach and systematic review methods to derive minimum clinically important differences. BMC Med Res Methodol 2021;21(1):41. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Cook KF, Cella D, Reeve BB. PRO-Bookmarking to Estimate Clinical Thresholds for Patient-reported Symptoms and Function. Med Care. 2019;57 Suppl 5 Suppl 1:S13–s17. [DOI] [PubMed] [Google Scholar]
- 23.Burbank AJ, Lucas N, Lin L, Hernandez ML, Reeve BB. Measuring PROMIS Asthma Impact cut point scores for identifying uncontrolled asthma in children. J Allergy Clin Immunol Pract 2023;11(12):3783–3785. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Hernandez ML, Lucas N, Mann C, et al. Association of step count with PROMIS pediatric health-related quality of life measures in children and adolescents with persistent asthma. J Allergy Clin Immunol Pract 2021;9(6):2492–2494. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Irwin DE, Stucky BD, Thissen D, et al. Sampling plan and patient characteristics of the PROMIS pediatrics large-scale survey. Qual Life Res 2010;19(4):585–594. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Jones CM, DeWalt DA, Huang IC. Impaired Patient-Reported Outcomes Predict Poor School Functioning and Daytime Sleepiness: The PROMIS Pediatric Asthma Study. Acad Pediatr 2017;17(8):850–854. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Nathan RA, Sorkness CA, Kosinski M, et al. Development of the asthma control test: a survey for assessing asthma control. J Allergy Clin Immunol 2004;113(1):59–65. [DOI] [PubMed] [Google Scholar]
- 28.Liu AH, Zeiger R, Sorkness C, et al. Development and cross-sectional validation of the Childhood Asthma Control Test. J Allergy Clin Immunol 2007;119(4):817–825. [DOI] [PubMed] [Google Scholar]
- 29.Revicki D, Hays RD, Cella D, Sloan J. Recommended methods for determining responsiveness and minimally important differences for patient-reported outcomes. J Clin Epidemiol 2008;61(2):102–109. [DOI] [PubMed] [Google Scholar]
- 30.Bakdash JZ, Marusich LR. Repeated Measures Correlation. Front Psychol 2017;8:456. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Thissen D, Liu Y, Magnus B, et al. Estimating minimally important difference (MID) in PROMIS pediatric measures using the scale-judgment method. Qual Life Res 2016;25(1):13–23. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Bateman ED, Bousquet J, Keech ML, Busse WW, Clark TJ, Pedersen SE. The correlation between asthma control and health status: the GOAL study. Eur Respir J 2007;29(1):56–62. [DOI] [PubMed] [Google Scholar]
- 33.Furnham A, Boo HC. A literature review of the anchoring effect. The Journal of Socio-Economics. 2011;40(1):35–42. [Google Scholar]
- 34.Cella D, Hahn EA, Dineen K. Meaningful change in cancer-specific quality of life scores: differences between improvement and worsening. Qual Life Res 2002;11:207–221. [DOI] [PubMed] [Google Scholar]
- 35.Terwee CB, Peipert JD, Chapman R, Lai JS, Terluin B, Cella D, et al. Minimal important change (MIC): a conceptual clarification and systematic review of MIC estimates of PROMIS measures. Qual Life Res 2021;30:2729–2754. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Katz P, Kannowski CL, Sun L, Michaud K. Estimation of minimally important differences and patient acceptable symptom state scores for the patient‐reported outcomes measurement information system pain interference short form in rheumatoid arthritis. ACR Open Rheumatol 2020;2(6):320–329. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Paulhus DL. Measurement and control of response bias. In: Measures of personality and social psychological attitudes. San Diego, CA, US: Academic Press; 1991:17–59. [Google Scholar]
- 38.O’Leary TE, Diller L, Recklitis CJ. The effects of response bias on self-reported quality of life among childhood cancer survivors. Qual Life Res 2007;16(7):1211–1220. [DOI] [PubMed] [Google Scholar]
- 39.Festinger L A Theory of Cognitive Dissonance. Stanford University Press; 1957:xi, 291. [Google Scholar]
- 40.Hosmer DW Jr, Lemeshow S, Sturdivant RX. Applied Logistic Regression. John Wiley & Sons; 2013. [Google Scholar]
- 41.Thissen D, Reeve BB, Bjorner JB, Chang CH. Methodological issues for building item banks and computerized adaptive scales. Qual Life Res 2007;16(1):109–119. [DOI] [PubMed] [Google Scholar]
- 42.Bjorner JB, Chang CH, Thissen D, Reeve BB. Developing tailored instruments: item banking and computerized adaptive assessment. Qual Life Res 2007;16(Suppl 1):95–108. [DOI] [PubMed] [Google Scholar]
- 43.Reeve BB, Hays RD, Bjorner JB, et al. Psychometric evaluation and calibration of health-related quality of life item banks: plans for the Patient-Reported Outcomes Measurement Information System (PROMIS). Med Care. 2007;45(5 Suppl 1):S22–S31. [DOI] [PubMed] [Google Scholar]
- 44.Eremenco S, Chen WH, Blum SI, et al. Comparing patient global impression of severity and patient global impression of change to evaluate test–retest reliability of depression, non-small cell lung cancer, and asthma measures. Qual Life Res 2022;31(12):3501–3512. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.