Validation of the Nasal Mucus Index, a novel measurement of acute respiratory infection severity

Abstract

Background:

To assess the concurrent and predictive validity of the Nasal Mucus Index (NMI), a novel measurement of acute respiratory infection (ARI) severity.

Objective:

ARI, including the common cold and influenza, imposes a great burden on individuals and society. Previous research has attempted to assess the severity of ARI with self-reported and laboratory-based measurements. Self-reported measurements may introduce bias. Laboratory-based metrics are often expensive. Therefore, there is a need for non–self-reported, affordable, and validated ARI severity tests.

Methods:

Participants (N = 719) with an ARI episode underwent nasal lavage on days 1 and 3. The samples were visually assessed for the amount of mucus present in the sample and were given a subsequent NMI score. Collected samples were further assessed for interleukin (IL) 8 values (in pg/mL) and polymorphonuclear neutrophils (PMN) per high-power field. The participants rated episode severity and nasal symptoms daily by using the validated Wisconsin Upper Respiratory Symptom Survey-21 (WURSS-21). A subset of nasal symptoms was used as an additional comparator. NMI scores were compared with same-day IL-8 level, PMN count, and WURSS-21 scores for concurrent validation purposes by using the Spearman ρ as the index of correlation. NMI scores were correlated with overall episode severity measurements to assess predictive validity. Overall episode severity was measured as the WURSS-21 area under the curve, nasal symptoms area under the curve, and episode duration.

Results:

The NMI score correlated significantly with the same-day IL-8 level (ρ = 0.443, p < 0.001), PMN count (ρ = 0.498, p < 0.001), WURSS-21 score (ρ = 0.098, p = 0.004), and nasal symptom score (ρ = 0.162, p < 0.001). No significant predictive correlations were found.

Conclusion:

Associations with inflammatory biomarkers and self-reported severity measurements provided evidence of concurrent validity for the novel NMI score. The NMI can be used in future research as a simple, inexpensive, non–self-reported indicator of ARI severity.

Acute respiratory infection (ARI), including the common cold and influenza, imposes a great burden on individuals and society.¹ Previous studies used a number of subjective (self-reported) and objective (laboratory-based) assessment methods when attempting to quantify ARI severity. Self-reported measurements can assess outcomes that are important to patients but that may be biased because they potentially reflect differences in assessment and reporting behaviors as well as actual symptomatology.² Use of objective measurements, such as nasal interleukin (IL) 8 level or nasal polymorphonuclear neutrophils (PMN) counts, helps avoid reporting bias. The downsides of existing objective measurements are that they can be costly and do not always reflect outcomes important to patients. Therefore, there is a need for non–self-reported, affordable, and validated ARI severity assessment methods.

In a response to this need, researchers at the Wisconsin State Laboratory of Hygiene developed a relatively inexpensive and easy-to-administer scale, known as the Nasal Mucus Index (NMI). Although previously used in a moderately large randomized controlled trial,³ the NMI measurement has yet to be validated. The present study tested the NMI for concurrent and predictive validity. For this purpose, both objective and subjective measurements were chosen as the criterion tests. The objective criterion tests are IL-8 and PMN measurements, both obtained from the same nasal wash samples assessed for the NMI. The nasal IL-8 level and PMN count are commonly used indicators of nasal inflammation.⁴

IL-8 is an established marker of inflammation in community-acquired ARI episodes and correlates reasonably well with symptoms.⁵ Counting PMN in nasal mucus is also a relatively well-established indicator of severity of ARI episodes.⁶ The subjective criterion test used is the Wisconsin Upper Respiratory Symptom Scale-21 (WURSS-21). The WURSS-21 is a validated, illness-specific, self-reported instrument developed and validated for ARI assessment and monitoring.⁷ These comparative measurements were chosen because they are widely accepted, reliable, and valid measurements of ARI severity. By using these comparative measurements, this study assessed the concurrent and predictive validity of the NMI. No predictive validity for assessing ARI severity was found. The assessments showed that the NMI had substantial concurrent validity for assessing ARI severity.

METHODS

Participants and Study Design

The data for this study came from a moderately large (N = 719), randomized controlled ARI trial that tested the effects of echinacea, placebo, and physician-patient interaction.^3,8–10 This study was approved by the University of Wisconsin-Madison Health Sciences Institutional Review Board, and informed consent was obtained from all the subjects. All human research guidelines of the United States Department of Health and Human Services and of the University of Wisconsin were followed. Community-recruited participants presented with a new onset ARI, with symptoms that existed no longer than 36 hours. ARI episodes were confirmed by a Jackson cold index score of ≥2,¹¹ which was calculated by summing eight symptom scores, in which 0 = absent, 1 = mild, 2 = moderate, and 3 = severe. The eight symptoms were the following: sneezing, headache, malaise, chilliness, nasal discharge, nasal obstruction, sore throat, and cough. When using the WURSS-21, participants self-rated symptoms and functional impact twice daily until their colds had resolved, to a maximum of 14 days. Each WURSS-21 item assesses severity on a seven-point Likert scale. Detailed methods and results are available elsewhere.^3,8–10

Objective ARI Monitoring

Nasal lavage was collected twice during the study, once at enrollment (day 1), and then again 2 days later (day 3). The NMI test is conducted by instilling a 5-mL aliquot of sterile isotonic medium into each nostril, followed by expression into a sterile container. The collected sample is transferred into a 15-mL conical centrifuge tube and allowed to settle for 10 minutes, which allows the mucus to settle to the bottom of the tube. The collected nasal sample is then visually inspected and given a NMI score according to the following scale: 0, a few gobs of mucus; 1, a third of the sample consists of mucus; 2, two-thirds of the sample consists of mucus; or 3, the entire sample is full of mucus. After being given a visual NMI score, the samples were analyzed for neutrophil counts and sent for IL-8 assay. IL-8 concentrations were measured in pg/mL by enzyme-linked immunosorbent assay by using previously described methods.¹² For the PMN count, the nasal wash samples were stained with acridine orange and then visually assessed within 2 hours of sample acquisition, with cell counts reported per high-power field, equivalent to 4 mm².

Subjective ARI Monitoring

Self-reported ARI severity outcomes for this trial included averaged daily WURSS-21 scores and episode duration. Day 1 and day 3 WURSS-21 scores were used to test concurrent validity of the NMI. Day 1 and day 3 nasal symptom scores (runny nose, nasal obstruction, and sneezing) were also extracted and combined to create the nasal symptoms variable, which was also assessed for same-day NMI validity.

Besides these concurrent daily measurements, self-reported overall episode severity measurements were assessed for predictive validity. Overall episode severity was calculated as area under the curve (AUC), with daily WURSS-21 scores as the y-axis and episode duration as the x-axis; trapezoidal approximation was used while creating the WURSS-21 AUC variable. Similarly, the nasal symptom AUC variable was calculated by using the sum of the WURSS-21 nasal symptoms scores. Nasal symptom AUC was used because we hypothesized that it would correlate more closely with nasal inflammation biomarkers than would the total WURSS-21 score, which includes throat and chest symptoms as well as self-assessments of quality of life and function.⁷

Duration began at enrollment and continued through the last time the participant answered “yes” to “Do you think you still have a cold?” The date and time of filling out questionnaires was recorded, which allowed duration to be quantified as a continuous measurement. To confirm that the illness had ended, this last “yes” had to be followed by “no” for 2 consecutive days. We limited monitoring to a maximum of 14 days to reduce potential bias from extended length illnesses.

Statistical Analyses

To make the data more normally distributed, the highly skewed IL-8 and PMN variables were log transformed. Concurrent validity was assessed by Spearman correlations on both day 1 and day 3. Specifically, the relationships between the NMI scores of that particular day were compared with the corresponding same-day criterion tests (IL-8 level, PMN count, WURSS-21 score, nasal symptoms). The predictive validity was assessed by Spearman correlations by assessing the correlation between day 1 NMI scores and subsequent measurements of ARI episode severity: episode duration, the WURSS-21 AUC, and nasal symptom AUC. One-way analysis of variance models were used to test whether the explained variance was significant.

RESULTS

Descriptive Statistics

Of the 719 total participants enrolled, nasal lavages were collected and analyzed from 710 participants. Missing data were excluded from the analyses. The participants were 64% female participants, 88% white, with 84% reporting at least some college education. Age ranged from 12 to 80 years (mean [standard deviation], 33.7 ± 14.4 years). Descriptive statistics of the various ARI severity variables are provided in Table 1.

Table 1.

Descriptive statistics of the various ARI severity variables^*

ARI = Acute respiratory infection; NMI = nasal mucus index; IL = interleukin; PMN = polymorphonuclear neutrophil; WURSS-21 = Wisconsin Upper Respiratory Symptom Survey, 21 item; AUC = area under the curve.

^*Day 3 measurements were similar to day 1 measurements with the exception that day 3 measurements reported slightly higher acute respiratory infection severity. Day 3 measurements had slightly higher ARI severity.

ARI Severity Findings

ARI severity findings are portrayed in Figs. 1 and 2. The distribution across the four categories of the NMI are portrayed in Fig. 1, which illustrates the skewness of the NMI versus IL-8 and PMN samples. The NMI data are skewed right (skewness = 2.4), with most participants scoring in the zero category. The central tendency and variability across the four NMI categories are portrayed in Fig. 2. Increases in the NMI score correlate with the same-day increases in IL-8 level, PMN count, WURSS-21 score, and nasal symptom (Fig. 2, A–D). How day 1 NMI scores correlated with subsequent ARI episode severity, viz. the WURSS-21 AUC, illness episode duration, and nasal symptom AUC are shown in Fig. 2, E–G. The skewness of the data and increase in the NMI score that correlated with increases in same-day criterion test measurements, shown in Figs. 1 and 2.

Figure 1.

The Nasal Mucus Index (NMI) compared with nasal interleukin (IL) 8 and polymorphonuclear neutrophils (PMN) allocation. Categories and units: IL-8, each category represents 1000 pg/mL; PMN, each category represents 100 neutrophils counted per 4 mm²; NMI, each category represents the categorical severity classification. Data represents day 3 measurements. Day 3 measurements were similar to day 1 measurements. Day 3 measurements had slightly higher acute respiratory infection (ARI) severity and, therefore, were chosen to display in this visual representation.

Figure 2.

Central tendency and variability of comparators across the four Nasal Mucus Index (NMI) categories. (A–D) Represent the concurrent relationship between day 3 NMI and day 3 interleukin 8 (IL-8) (A), polymorphonuclear neutrophils (PMN) (B), Wisconsin Upper Respiratory Symptom Survey-21 (WURSS-21) (C), and nasal symptoms (D). Day 3 measurements were similar to day 1 measurements. Day 3 measurements reported slightly higher acute respiratory infection (ARI) severity and, therefore, were chosen to be displayed in the box-and-whisker plots. (E–G) Show the predictive relationship between day 1 NMI score and overall episode severity measurements of the following: (E) WURSS-21 area under the curve (AUC), (F) nasal symptoms AUC, (G) episode duration. For all the boxes, the third quartile represents the upper bound, the first quartile represents the lower bound, the middle line represents the median, and the whiskers extend to the most extreme data point, which is no more than 1.5*interquartile range from the box. Units: NMI, each category represents the categorical severity classification; IL-8, pg/mL; PMN, neutrophils counted per 4 mm²; duration, days.

Concurrent Validity

NMI scores were correlated with same-day IL-8 values, neutrophils count, WURSS-21 scores, and nasal symptoms for concurrent validation purposes. Assessment of concurrent validity with objective criterion tests revealed significant positive Spearman correlations between NMI scores and same-day IL-8 level (day 1, p = 0.000, ρ = 0.443; day 3, p = 0.000, ρ = 0.458) and PMN scores (day 1, p = 0.000, ρ = 0.498; day 3, p = 0.000, ρ = 0.534) (Table 2). We also found significant positive Spearman correlations between the NMI score and same-day, self-reported scores for the daily WURSS-21 (day 1, p = 0.004, ρ = 0.098; day 3, p = 0.000, ρ = 0.093) and nasal symptoms (day 1, p = 0.000; ρ = 0.162; day 3, p = 0.000, ρ = 0.141) (Table 2). One-way analysis of variance showed that the explained variance was significant (p = 0.000). These assessments showed that the NMI has substantial concurrent validity for assessing ARI severity.

Table 2.

NMI correlations with criterion tests

NMI = Nasal mucus index; IL = interleukin; PMN = polymorphonuclear neutrophil; WURSS-21 = Wisconsin Upper Respiratory Symptom Survey, 21 item; AUC = area under the curve.

*Concurrent validity assesses same-day measurements; predictive validity assesses day 1 NMI with overall acute respiratory infection episode severity.

#Based on the Student's t-test for testing whether specific regression coefficient equals zero.

Predictive Validity

Day 1 NMI scores were correlated with overall episode severity measurements for predictive validity purposes. One-way analysis of variance showed that the explained variance was not due to chance. However, none of these predictive severity measurements correlated significantly. Predictive validity was tested by correlating the day 1 NMI scores with overall episode severity outcomes. The Spearman correlations for the predictive validity were as follows: WURSS-21 AUC (p = 0.276, ρ = 0.022), total nasal symptom AUC (p = 0.147, ρ = 0.038) as well as with duration (p = 0.192, ρ = 0.033) (Table 2). The NMI, therefore, has no predictive validity for assessing ARI severity.

DISCUSSION

We assessed and validated the NMI as a novel, non–self-reported severity measurement during ARI. Analysis of our results indicated that the NMI had substantive concurrent validity when assessed with criterion measurements IL-8, PMN, WURSS-21 scores, and nasal symptoms. By using the measurements at hand, the NMI did not seem to have predictive validity. We concluded that the NMI was a simple, inexpensive, non–self-reported indicator of ARI severity that can be used as a measurement of concurrent ARI severity.

One limitation of the NMI was that it was limited to four categories and provided a skewed data distribution, with nearly 80% of all the subjects scored in the lowest category, visually rated as a “few gobs of mucus” (Fig. 1). The few allocations to category 3 of the NMI score may explain the weaker correlation with IL-8 level, as illustrated in Fig. 2. To some extent, the skewness of the NMI scores could represent actual skewness in ARI severity because IL-8 level and PMN count are also skewed right (Table 1). The NMI skewness may provide similar limitations in quantifying ARI severity as do the accepted IL-8 and PMN measurements. Future modifications of the NMI might expand the scale to include more categories, which could yield a more even spread across the categories. For example, it might be useful to test a scale with categories of “no mucus at all” and “very few gobs of mucus,” which extends the scale to five ordinal categories.

The applicability of the NMI might be most useful in research settings in which cost and laboratory facilities are limited. The NMI is easy to administer and relatively inexpensive, and is not subject to participant self-reported bias. It seems to correlate reasonably well with both nasal symptoms and laboratory-assessed markers of inflammation. Covariates such as sex, age, and psychosocial factors may influence reporting behavior.^13,14 These self-reported differences may or may not translate to parallel objective findings. Whether these differences reflect biased reporting behavior or real differences in symptomatology experienced is difficult to assess.

Given these uncertainties, it is advisable to use non–self-reported as well as self-reported severity measurements when assessing ARI. The NMI could provide a simple, quick, low-cost measurement for this purpose. The cost reductions that would result from implementing the NMI instead of other more costly laboratory measurements, might allow for larger ARI studies that require non–self-reported ARI severity measurements. Future studies could assess the reliability and responsiveness of the NMI as an ARI severity measurement by studying interobserver or NMI performance over time. Future research could also potentially focus on the applicability of the NMI with diseases other than ARI, e.g., allergic rhinitis.

CONCLUSION

Participants (N = 719) with a new onset ARI underwent nasal lavages on two different days. Assessments showed that the novel NMI correlated significantly with same-day inflammatory biomarkers (IL-8, PMN) and with self-reported severity by using a validated outcome assessment tool (WURSS-21, and component nasal symptoms). No significant predictive correlations were found. We concluded that there was evidence of concurrent validity for the NMI as an inexpensive and easy-to-administer tool for assessing ARI severity. The NMI can be used in future research as an indicator of ARI severity.