Meaningful changes for efficacy outcomes in patients with chronic rhinosinusitis with nasal polyps

Abstract

Objective

Nasal Polyp Score (NPS) and Nasal Congestion Score (NCS) are commonly used clinical trial endpoints to determine improvements in response to treatment in patients with chronic rhinosinusitis with nasal polyps (CRSwNP). However, limited information is available on within-patient meaningful change thresholds (MCTs) and between-group minimal important differences (MIDs) for NPS and NCS, which would aid interpretation of results.

Methods

Data from phase 3 placebo-controlled trials of omalizumab in patients with CRSwNP (POLYP 1 and POLYP 2) were used to estimate MCTs and MIDs for both NPS and NCS using anchor-based methods. Sino-Nasal Outcome Test-22 (SNOT-22) and SNOT-22 Sino-Nasal Symptoms Subscale (SNSS) scores were used as anchors (≥0.35 correlation with NPS and NCS). Within- and between-group differences in NPS and NCS change scores were used to estimate MCTs and MIDs, respectively. Identified MCTs were used in unblinded responder analyses to compare the proportions of patients per treatment group achieving a meaningful improvement.

Results

MCTs and MIDs were estimated at −1.0 and −0.5 for NPS and −0.50 and −0.35 for NCS, respectively, and were consistent across studies. Overall, 57.0% of patients achieved the MCT in NPS with omalizumab vs 29.9% with placebo (p < 0.0001). Similarly, 58.9% of patients achieved the MCT in NCS with omalizumab vs 30.7% with placebo (p < 0.0001). Group differences in mean change were statistically significant and exceeded the estimated MIDs.

Conclusions

Meaningful change estimates for NPS and NCS could be used to assess response to treatment for patients with chronic rhinosinusitis with nasal polyps.

Trial registration: POLYP1: clinicaltrails.gov NCT03280550; registered September 12, 2017; https://clinicaltrials.gov/ct2/show/NCT03280550). POLYP2 (clinicaltrials.gov NCT03280537; registered September 12, 2017; https://clinicaltrials.gov/ct2/show/NCT03280537).

Introduction

Chronic rhinosinusitis with nasal polyps (CRSwNP) is a chronic inflammatory condition in which benign bilateral polyps develop in the paranasal sinuses.¹ CRSwNP is common, affecting ∼2%–4% of the general population, and is associated with significant health-related quality of life (HRQOL) impacts² due to symptoms such as facial pressure, nasal obstruction, runny nose, postnasal drip, and loss of sense of smell.

Clinical assessment of CRSwNP is typically based on the endoscopically determined clinician-reported outcome, Nasal Polyp Score (NPS),³ and change in NPS is frequently used in clinical trials to evaluate treatment efficacy.^3,4 Given the significant correlation between NPS and symptom severity experienced by the patient,⁵ patient-reported outcomes (PROs) and objective measures of CRSwNP severity are also considered important. For example, the Nasal Congestion Score (NCS) evaluates patient-rated nasal blockage,³ and the 22-item Sino-Nasal Outcome Test (SNOT-22) evaluates patient-reported symptoms and health-related impacts.

However, despite widespread use of NPS and NCS in clinical trials and clinical practice, to date only 1 published study has reported what changes in these scores may be considered clinically relevant.⁶ To establish relevancy of changes regardless of treatment, values of clinically meaningful improvement for an individual patient (meaningful change threshold; MCT) and values of the smallest difference between treatment arms that can be considered clinically meaningful (minimal important difference; MID) determined from other studies are needed. In the analysis reported here, we used data from 2 replicate phase 3 clinical trials of omalizumab, an approved treatment for patients with CRSwNP, to estimate MCT and MID for both NPS and NCS.

Materials and methods

Analysis overview

Calculation of MCT and MID estimates was based on change from baseline to week 24 in NPS and NCS from blinded data from pooled and individual POLYP 1 and POLYP 2 studies, using anchor-based methods (primary analysis). Suitable anchors were identified as those outcome measures examining the construct of interest that were sufficiently correlated with NPS and NCS at baseline and change from baseline. Results from anchor-based analyses were supported by distribution-based analyses (an overview of the analysis is provided in Supplementary Methods).

Primary objectives for this analysis were to use anchor- and distribution-based approaches to determine within-patient MCTs and between-group MIDs for 2 coprimary endpoints (NPS, NCS) in POLYP 1 and POLYP 2, and to then apply derived estimates in responder analyses using POLYP 1 and POLYP 2 data.

POLYP 1 and POLYP 2

POLYP 1 (clinicaltrials.gov NCT03280550) and POLYP 2 (clinicaltrials.gov NCT03280537) were phase 3, randomized, multicenter, double-blind, placebo-controlled clinical trials conducted in parallel as replicate studies investigating omalizumab for patients with CRSwNP.³ Study protocols were approved by respective institutional review boards and ethics committees, and were conducted in accordance with International Conference on Harmonisation Good Clinical Practice, Declaration of Helsinki, and all applicable laws and regulations. All patients provided written informed consent. Methodologies for POLYP 1 and POLYP 2 have been reported.³ In total, 138 patients were randomized in POLYP 1 (omalizumab, n = 72; placebo, n = 66) and 127 patients were randomized in POLYP 2 (omalizumab, n = 62; placebo, n = 65).³ In brief, patients had severe nasal polyps, loss of smell, and low health-related quality of life; the most common comorbidity was asthma; about one-fifth of patients had used systemic corticosteroids in the past year and half had undergone prior sinonasal surgery; see Gevaert et al³ for detailed characteristics.

Outcomes

NPS is a clinician-reported outcome measure scored after endoscopic evaluation of the nasal cavities. Each nostril is scored from 0 to 4, with 0 indicating no visible nasal polyps and 4 indicating complete obstruction of the nasal cavity by nasal polyps.⁷ Combined left and right scores give a total possible score range of 0–8, with higher scores indicating larger nasal polyps and greater disease severity. NCS is a single-item PRO measure of whether a patient feels their nose is blocked. The score ranges from 0 (“not at all”) to 3 (“severe”) and is reported as average daily score over the previous 7 days.

SNOT-22⁸ and EuroQol 5-Dimension 5-Level (EQ-5D-5L)9, 10, 11 were used as anchors to determine meaningful change in accordance with US Food and Drug Administration guidance that recommends using multiple such variables to provide “an accumulation of evidence to help interpret meaningful within-patient score change.”¹² SNOT-22 uses a two-week recall period to measure common symptoms of CRS with and without nasal polyps.⁸ The 22 items are scored using a six-point severity rating scale (0 = “no problem”; 5 = “problem as bad as it can be”) and are summed to provide a total score, ranging from 0 to 110, with a lower score indicating lesser symptoms and better HRQOL. A previous factor analysis of SNOT-22 demonstrated that it measures 5 different underlying domains: 3 sinus-specific symptom domains (rhinologic, extra-rhinologic, ear/facial symptoms) and 2 general HRQOL domains (psychological, sleep dysfunction).^13,14 Of the 22 items in SNOT-22, 8 measure symptoms that seem directly associated with nasal polyps (ie, need to blow nose, nasal blockage/congestion, sneezing, runny nose, cough, posterior rhinorrhea, thick nasal discharge, loss or decreased sense of smell/taste). Exploratory factor analysis and Rasch modeling (details in Supplementary Methods) supported a nasal polyp–specific 8-item subscale of SNOT-22 (SNOT-22 Sino-Nasal Symptoms Subscale [SNSS]), which was developed for use as a potential anchor.

Analytic procedures

Anchor-based approach

Candidate anchors were chosen based on their established clinical relevance and included SNOT-22 (total score), SNOT-22 SNSS, and EQ-5D-5L scores. Baseline and change from baseline correlation between the proposed anchors and NPS and NCS were explored. Only those anchors with a correlation of ≥0.35 with NPS and NCS were used.

Patients were grouped by change in disease severity from baseline at Week 24 on the identified anchors based on improvements of at least the established MID of −8.9 points for SNOT-22^8,13 and, calculated proportionately, −3.2 points for SNOT-22 SNSS. Stratification was based on both uncollapsed and collapsed change. Uncollapsed change stratified those patients improving by at least the MID (eg, improved, ≥8.9-point decrease; no change/worse, <8.9-point decrease) into 2 approximately equally sized groups of “improved” and “much improved” by dividing at the median score (decrease of 22 for SNOT-22; decrease of 9 for SNOT-22 SNSS), and those who declined by at least the MID into “worsened” and “much worsened” by dividing at the median score (increase of 12 for SNOT-22; increase of 5 for SNOT-22 SNSS). Collapsed change simply stratified patients based on the MID for that anchor at week 24 (eg, improved, ≥8.9-point decrease; no change/worse, <8.9-point decrease). Patients who did not achieve the established MIDs for SNOT-22 or SNOT-22 SNSS were classified as experiencing “no change.” Mean change within each uncollapsed and collapsed anchor group was calculated to determine within-group MCT.

Cumulative distribution function (CDF) plots were used to assess the degree to which the anchor categories could distinguish between degree of change on NPS and NCS. Probability distribution function (PDF) plots assessed central tendency and variability of NPS and NCS within each anchor category. Receiver operating characteristic (ROC) analysis was used to identify the value of NPS and NCS change (referred to as the best cut point) equivalent to a specific change in anchor scores (improved vs stable or worsened; much improved vs improved, stable, or worsened).

To determine between-group meaningful difference, differences in mean change scores in the improved, worsened, and no change groups were calculated. Observed differences between groups with (improved) and without (no change) a meaningful change reflect the MID.

Distribution-based approach

Distribution-based estimates provided boundary information and ruled out the possibility of patients being classified as a responder by chance. Single timepoint assessment NPS and weekly average NCS were used to derive distribution-based estimates at baseline, calculated as 0.5 times standard deviation (SD) of NPS and NCS baseline scores.

Responder analyses

An unblinded responder analysis by treatment group was performed, where NPS and NCS changes in scores from baseline to week 24 were classified according to MCT status: improved (change in score was greater than/equal to MCT), or stable/declined (change in score was less than MCT). Associations between treatment groups and MCT status were assessed using chi-square or Fisher's exact test, as appropriate. The two-sided p-value was reported alongside the proportions of patients improving by treatment group.

Finally, CDFs of the change at week 24 in NPS and NCS from baseline were produced by treatment group, with the overall MCT indicated on each figure. Absolute change from baseline was expressed on the x-axis and the cumulative number of patients who had a given score change was expressed on the y-axis.

Between-group differences in mean change

An analysis by treatment group was performed, where adjusted mean changes in NPS and NCS from baseline to week 24 were compared by calculating between-group mean differences.

Statistical analysis

All analyses were conducted on the cross-sectional analysis population (all randomized patients who had data on NPS or NCS at the relevant timepoint) and longitudinal analysis population (patients who had clinician-reported responses to NPS or who had patient-reported responses for weekly average NCS at baseline and at the relevant assessment timepoint) and on the POLYP 1 and POLYP 2 populations both separately and combined (POLYP 1/2). All analyses were conducted using SAS version 9.4.

Results

SNOT-22 factor analysis

Ordinal exploratory factor analysis supported by Velicer's minimum average partial test revealed a 4-factor solution for SNOT-22, with factor loadings shown in Supplementary Table S1.15, 16, 17, 18, 19, 20 These results indicate SNOT-22 SNSS should be created from all 6 items loading on 1 factor (need to blow nose, nasal blockage, runny nose, postnasal discharge, thick nasal discharge, decreased sense of smell/taste), in addition to 2 items (sneezing, coughing) also included for conceptual reasons. This proposed factor structure was further confirmed with Rasch modeling (details in Supplementary Results).

Anchor correlations

For POLYP 1/2, baseline correlations between SNOT-22 and SNOT-22 SNSS and NPS were small (<0.10); correlations with NCS were larger (>0.30). However, for change from baseline to week 24, correlations for both NPS and NCS with SNOT-22 and SNOT-22 SNSS were ≥0.35 (NP, 0.354 and 0.390, respectively; NCS, 0.517 and 0.605, respectively). Correlations between NPS and NCS and EQ-5D-5L index score and VAS were <0.35 at both baseline and for change from baseline to week 24. Therefore, only SNOT-22 and SNOT-22 SNSS were used as anchors in the meaningful change analysis.

CDF plots for uncollapsed SNOT-22 groups for NPS change showed that the “improved” group had a similar distribution to the “no change” group, whereas separation between curves was most evident for the “much improved” group (Supplementary Figure S1). Similar results were observed for SNOT-22 SNSS and for PDF plots for both anchors (not shown). Therefore, mean changes for the “much improved” group were used to estimate NPS meaningful change. Overall, across both studies, patients with “much improved” SNOT-22 scores had mean NPS changes of −1.0. In contrast, while CDF plots for uncollapsed SNOT-22 groups for NCS change showed distinct separation between the “much improved” and “no change” groups, differentiation between the “improved” and “no change” groups was also observed (Supplementary Figure S2), with similar results for SNOT-22 SNSS and PDF plots (not shown). Therefore, mean changes for the “improved” group were used to estimate NCS meaningful change. Overall, across both studies, patients with “improved” SNOT-22 scores had mean NCS changes of −0.5. Both within-group meaningful change and between-group differences were highly consistent between SNOT-22 and SNOT-22 SNSS scores (Table 1).

Table 1.

Differences from baseline to week 24 in NPS and NCS (pooled POLYP 1/2 data).

Parameter	NPS		NCS
	Based on SNOT-22 anchor	Based on SNOT-22 SNSS anchor	Based on SNOT-22 anchor	Based on SNOT-22 SNSS anchor
Within-group meaningful change, mean (SD), [n]
Collapsed anchor
“Improvement”a	−0.8 (1.35), [149]	−0.8 (1.34), [157]	−0.9 (0.95), [145]	−0.9 (0.95), [154]
Uncollapsed anchor
“Much improvement”b	−1.1 (1.42), [81]	−1.1 (1.42), [81]	−1.2 (0.95), [78]	−1.2 (1.03), [87]
“Improvement”c	−0.5 (1.17), [68]	−0.4 (1.03), [63]	−0.5 (0.85), [67]	−0.5 (0.69), [67]
Between-group meaningful change, mean (SD)
Collapsed anchor
Difference between “improved” and “not improved”	−0.6 (0.17)	−0.7 (0.17)	−0.7 (0.11)	−0.9 (0.11)
Uncollapsed anchor
Difference between “much improved” and “improved”	−0.7 (0.21)	−0.8 (0.21)	−0.6 (0.14)	−0.7 (0.13)
Difference between “improved” and “no change”	−0.2 (0.21)	−0.1 (0.21)	−0.3 (0.14)	−0.5 (0.13)

NCS, Nasal Congestion Score; NPS, Nasal Polyp Score; SD, standard deviation; SNOT-22, Sino-Nasal Outcome Test-22; SNSS, Sino-Nasal Symptoms Subscale.

^aDefined as ≥8.9-point decrease in SNOT-22, and ≥3.2-point decrease in SNOT-22 SNSS.

^bDefined as ≥22-point decrease in SNOT-22, and ≥9-point decrease in SNOT-22 SNSS.

^cDefined as 8.9- to <22-point decrease in SNOT-22, and ≥3.2- to 9-point decrease in SNOT-22 SNSS

ROC analysis with collapsed improvement groups with either SNOT-22 or SNOT-22 SNSS as the anchor showed the best cut point for NPS to be −1.0, and the best cut point for NCS to be approximately −0.32.

Distribution-based approach

Given that a value of 0.5 SD has been shown to be equivalent to a MID, change scores for this degree of change provided supportive estimates of meaningful change for NPS and NCS. In the combined analysis, these values were 0.48 for NPS and 0.33 for NCS.

MCT and MID estimates

Results were consistent across the above anchor-based approaches (and across studies), triangulating on MCT estimates of −1.0 for NPS (0–8 rating scale) and −0.5 for weekly average NCS (0–3 rating scale). MIDs were estimated as −0.50 for NPS and −0.35 for NCS (Table 2).

Table 2.

Final estimates of MCT and MID for the NPS and NCS.

Measure	MCT (individual patient)	MID (group difference)	0.5 SD from pooled POLYP 1/2
NPS (0–8)	−1.0	−0.50	−0.48
NCS (0–3 weekly average)	−0.5	−0.35	−0.33

MCT, meaningful change threshold; MID, minimal important difference; NCS, Nasal Congestion Score; NPS, Nasal Polyp Score; SD, standard deviation

For NPS, MCT of −1.0 means that a change in 1 category of the NPS 9-point scale is meaningful for an individual patient. For NCS, MCT of −0.5 means that a change of −0.5 in weekly average NCS 4-point scale is meaningful for an individual patient.

For NPS and NCS, MIDs of −0.50 and −0.35 mean that a group-based difference (e.g., between treated and placebo patients) of −0.50 and −0.35, respectively, is clinically meaningful. This is supported by distribution-based estimates of 0.5 SD, which were slightly smaller than each anchor-based estimate.

POLYP 1/2 responder analysis

The estimated MCTs for NPS and NCS were applied to determine proportions of patients achieving a meaningful improvement in each measure.

For NPS, proportions of patients who achieved MCT of a −1.0-point improvement from baseline to week 24 in the omalizumab group vs the placebo group were 55.9% vs 33.3% for POLYP 1 (p = 0.0120), 58.0% vs 26.6% for POLYP 2 (p = 0.0003), and 57.0% vs 29.9% (p < 0.0001) for pooled data from POLYP 1/2, respectively (Fig. 1).

Fig. 1.

Proportion of patients from POLYP 1 and POLYP 2 achieving the estimated MCT for NPS at week 24. MCT, meaningful change threshold; NPS, Nasal Polyp Score

For NCS, the proportions of patients who achieved the MCT of a −0.5-point improvement from baseline to week 24 in the omalizumab group vs the placebo group were 57.6% vs 25.4% for POLYP 1 (p = 0.0003), 60.0% vs 35.9% for POLYP 2 (p = 0.0054), and 58.9% vs 30.7% (p < 0.0001) for pooled data from POLYP 1/2, respectively (Fig. 2).

Fig. 2.

Proportion of patients from POLYP 1 and POLYP 2 achieving the estimated MCT for NCS at week 24. MCT, meaningful change threshold; NCS, Nasal Congestion Score

Between-group differences in mean change

Overall, omalizumab patients achieved greater adjusted mean differences in NPS and NCS from baseline through week 24 vs placebo (Fig. 3). Between-group difference at week 24 in NPS (−0.86) and NCS (−0.52) exceeded estimated MIDs of −0.50 for NPS and −0.35 for NCS, giving confidence that these observed differences in pooled POLYP 1/2 data are clinically meaningful. Pooled POLYP 1/2 results were additionally found to be consistent across each study individually.

Fig. 3.

Adjusted mean difference from baseline over time in (A) NPS and (B) NCS (pooled POLYP 1/2 patients). MID, minimal important difference; NCS, Nasal Congestion Score; NPS, Nasal Polyp Score; SE, standard error

Safety

Safety findings from POLYP 1 and POLYP 2 have been previously reported.³

Discussion

To aid clinicians treating patients with CRSwNP, our analysis has formally identified thresholds for meaningful change in the efficacy measures NPS and NCS. The use of a nasal polyp-specific patient-reported measure as an anchor supports relevancy, and the estimated MCTs and MIDs were consistent across clinical trials and different methodological approaches, including supportive use of distribution-based methods to account for measurement error). All these factors combine to increase the likelihood that these estimates indicate a true meaningful change. Furthermore, when applied to clinical trial data in responder analyses, the proportion of patients reaching NPS and NPC MCT estimates was significantly greater for omalizumab than placebo, confirming the applicability of our findings.

Our findings are similar to a recent publication that estimated meaningful change of efficacy outcomes for patients with CRSwNP treated with another biologic, dupilumab.⁶ Han et al also used the SNOT-22 SNSS anchor to estimate MCTs of NPS and NCS as −1.0 and −1.0, which are similar to our estimated MCTs of −1.0 and −0.5, respectively. Taken together, these analogous studies provide confidence that these improvements are both relevant to the patient and applicable to a range of biologic treatments.

Of note, anchor-based methods to determine MCT require sufficiently correlated anchors with established clinical relevance. Given the post hoc nature of these analyses, anchors were selected from outcome measures included in the clinical trials. Nevertheless, the patient-reported SNOT-22 and SNOT-22 SNSS both measure the construct of relevance and, in terms of change from baseline, were sufficiently correlated with change in NPS and NCS. These outcome measures also have published values of meaningful change that could be used specifically to anchor change in NPS and NCS.

Limitations of this analysis included a lack of a specifically designed anchor, such as a global impression of severity or change. In addition, the analyses were based on 2 identical phase 3 trials, with enrollment criteria specifying a patient population with relatively severe disease (NPS ≥5) and inadequate response to nasal corticosteroids: confirmation of results in patients with less severe disease or greater responsiveness to nasal corticosteroids would be important for broader generalizability of these results. Further, as the data is sourced from clinical trials, the external validity of the results are somewhat limited due to the highly controlled and specific way in which clinical trials are conducted, and the positive outcome of the trial may have affected our findings; values for MCT applied in responder analyses and values of MID in group-based analyses should ideally be defined a priori, rather than using trial data themselves. Nonetheless, MCT and MID estimates were derived from blinded trial data, increasing confidence in their validity. Finally, an observational, real-world study would be important to confirm the validity of our results, and qualitative research would be valuable in ascertaining what these meaningful change estimates mean to patients in terms of symptoms, daily activities, and function.

Conclusion

In conclusion, our study defines clinically meaningful changes in both clinician-reported and PROs for patients with CRSwNP. In combination with meaningful change estimates from other studies, our findings will help clinicians and researchers set prespecified goals for treatment of patients with CRSwNP.

Abbreviations

CRSwNP, chronic rhinosinusitis with nasal polyps; CDF, cumulative distribution function; EQ-5D-5L, EuroQol 5-Dimension 5-Level; HRQOL, health-related quality of life; MCT, meaningful change threshold; MID, minimal important differences; NCS, Nasal Congestion Score; NPS, Nasal Polyp Score; PDF, probability distribution function; PRO, patient-reported outcomes; ROC, receiver operating characteristic; SD, standard deviation; SNOT-22, Sino-Nasal Outcome Test-22; SNSS, Sino-Nasal Symptoms Subscale.

Acknowledgments

Medical writing assistance was provided by Nicole Tom, PhD, of Envision Pharma Group, and funded by Genentech, Inc., a member of the Roche Group.

Funding

This study was funded by Genentech, Inc., a member of the Roche Group, and Novartis Pharma AG. Genentech, Inc., was involved in the study design, the analysis of data, and the decision to submit the manuscript for publication.

Availability of data and material

Qualified researchers may request access to individual patient level data through the clinical study data request platform (https://vivli.org/). Further details on Roche's criteria for eligible studies are available here (https://vivli.org/members/ourmembers/). For further details on Roche's Global Policy on the Sharing of Clinical Information and how to request access to related clinical study documents, see here (https://www.roche.com/research_and_development/who_we_are_how_we_work/clinical_trials/our_commitment_to_data_sharing.htm).

Author contributions

JB, LI, and TAO designed the study, interpreted the results, and prepared the manuscript. CG and HD analyzed the data and interpreted the results. All authors critically reviewed the manuscript and approved the final draft.

Ethics approval and consent to participate

Study protocols were approved by respective institutional review boards and ethics committees, and were conducted in accordance with International Conference on Harmonisation Good Clinical Practice, Declaration of Helsinki, and all applicable laws and regulations. All patients provided written informed consent. Trial registration: POLYP1: clinicaltrails.gov NCT03280550; registered September 12, 2017; https://clinicaltrials.gov/ct2/show/NCT03280550). POLYP2 (clinicaltrials.gov NCT03280537; registered September 12, 2017; https://clinicaltrials.gov/ct2/show/NCT03280537).

Consent for publication

All authors have provided consent for publication.

Declaration of competing interest

JB and LI are employees of and shareholders in Roche. HD is an employee of Clinical Outcomes Solutions. CG is a former employee of Clinical Outcomes Solutions. TAO is an employee of and shareholder in Genentech, Inc.

Appendix A. Supplementary data

The following is the Supplementary data to this article: