Use of the Modified SNOT-16 in Primary Care Patients with Clinically Diagnosed Acute Rhinosinusitis

Abstract

Objective

Determine the reliability, validity, responsiveness and the minimal important difference (MID) for the SNOT-16 in the measurement of disease-specific quality of life in adults with acute rhinosinusitis

Study Design

Randomized controlled trial to evaluate antibiotic treatment for acute rhinosinusitis

Setting

Ten community practices in St Louis, MO

Subjects and Methods

The modified SNOT-16 was completed at baseline (by both face-to-face and telephone interviews) and by telephone interview at 3, 7 and 10 days by 166 adults with acute rhinosinusitis diagnosed clinically using standardized criteria (36% male, 78% Caucasian). Considering severity and frequency, subjects rated how much they were bothered by each item using a 4-point scale. The mean SNOT-16 score (ranging from 0, no problem to 3, large problem) was compared to the subjects’ global assessment of change to evaluate responsiveness and the MID.

Results

The instrument was easy to use and took < 5 minutes to complete. The SNOT-16 score identified statistically significant differences in the hypothesized direction for those reporting more or less severe symptoms (p=0.0161) and more or less bother (p<0.0001) demonstrating construct-related validity. Cronbach’s alpha ranged from 0.82 to 0.91 demonstrating high internal consistency. There was a statistically significant decrease in scores with time (MANOVA, p<0.0001). The effect sizes at days 3, 7, and 10 were 1.45, 2.34 and 2.90 respectively, indicating high sensitivity to clinical change. The Minimum Important Difference (MID) was 0.5 units.

Conclusion

The modified SNOT-16 is a valid instrument to assess effectiveness of interventions to improve disease-specific QOL in adults with acute rhinosinusitis.

INTRODUCTION

Acute rhinosinusitis is an important cause of morbidity, anxiety, lost time from work and school, and treatment costs.¹ Treatment is directed at reducing symptoms (such as headache, cough, and nasal obstruction) and limiting functional impairment (such as fatigue, difficulty sleeping and concentrating). As there are no clinical objective measures of disease resolution for use in clinical trials, tools to assess outcomes that are meaningful for patients are needed. Improvement in quality of life (QOL) is important to patients. Several instruments to measure disease-specific quality of life (QOL) have been developed and evaluated in patients with chronic rhinosinusitis. These instruments measure the impact of the illness on physical and psychosocial functioning and general well being.² No disease-specific QOL instrument has been evaluated for use in studies to assess the effectiveness of interventions for patients with acute rhinosinusitis

We report the validation of the SNOT-16, a disease-specific QOL instrument, for use in acute rhinosinusitis. The SNOT-16 is one of several Sinonasal Outcome Tests derived from the Rhinosinusitis Outcome Measure (RSOM-31).³ These instruments were developed using established psychometric methods³ including input from patients via focus groups to ensure inclusion of items they reported as important. These included functional limitations, physical problems and emotional consequences. The SNOT-20, SNOT-16, and most recently the SNOT-22 have been evaluated for use in patients with chronic rhinosinusitis and found to be reliable, valid and responsive to meaningful clinical change.^4–6 We evaluated the reliability, concurrent validity, and responsiveness of the SNOT-16 when used in the primary care setting. This evaluation was conducted during a randomized, placebo-controlled trial to evaluate antibiotic treatment for adult patients with acute rhinosinusitis.

METHODS

Study Population

We recruited subjects from offices of primary care physicians in St Louis, MO. Adult patients (18 to 70 years old) were eligible if they were diagnosed with acute rhinosinusitis by their physician. Diagnostic criteria were standardized and included a report of maxillary pain or tenderness in the face or teeth and purulent nasal secretions; rhinosinusitis symptoms for ≥ 7 days and ≤ 28 days that were not improving or worsening; or rhinosinusitis symptoms for < 7 days that had worsened after initial improvement. Patients rated the severity of their current sinus symptoms on a 5-point categorical global rating scale (very mild, mild, moderate, severe and very severe). Those who reported their symptoms as moderate, severe or very severe were eligible to participate.

Patients were excluded if they had: allergy to penicillin or amoxicillin, had received antibiotic therapy within the past 4 weeks, had complications of sinusitis, were pregnant, had a comorbidity that could impair their immune response, had cystic fibrosis, or required an antibiotic for a concurrent condition.

All participants provided written consent. The study was approved by the Washington University Human Research Protection Committee.

The Intervention

Participants were randomized to receive a 10-day course of either amoxicillin (500mg orally three times daily) or placebo similar in appearance and taste and dispensed in the same fashion. Unless their physician felt it was contraindicated, all patients received a 5 to 7-day supply of the symptomatic treatments to be used as needed, including acetaminophen, guaifenesin, dextromethorphan hydrobromide, pseudoephedrine sustained action, and saline spray.

Measurement

Data were collected using structured questionnaires administered by trained research assistants (RA) blinded to group assignment. At study enrollment (Day 0), the subject completed a brief face-to-face interview with the on-site RA. The SNOT-16 was completed during this face-to-face interview and again later that day during a telephone interview. Subsequent interviews were conducted by telephone at Days 3, 7, and 10.

SNOT-16

The patient’s disease-specific quality of life was measured using a modified version of the SNOT-16, an instrument that gathers information on 16 sinus-related symptoms.⁵ To simplify telephone administration, we used a version of the SNOT-16 with a 4-point response scale rather than the 5-point scale used in previous studies of chronic disease.⁵ For our tool, respondents were instructed to consider both the severity and frequency for that item (eg, headache) and report how much they had been bothered by each item in the last few days. Response options included: 0, no problem; 1, mild or slight problem; 2, moderate problem; and 3, severe problem. This modified tool was used for all study interviews. At baseline, the subjects selected the items they felt were most important (up to five) from the list. The SNOT-16 score was calculated as the mean of all responses and ranged from 0 to 3.

Global rating of change in sinus symptoms was assessed using an anchor-based scale. Participants assessed status of their current sinus symptoms relative to their status at enrollment at each follow-up interview using a 6-point categorical scale (a lot worse, a little worse, the same, a little better, a lot better, or no symptoms).

Data Analysis

For this study, subjects from the intervention and controls groups were analyzed as one group. We used the Day-0 telephone-administered SNOT-16 score for all change analyses, except for 4 participants with this score missing, for whom we used the face-to-face score.

Construct-related validity

To assess construct-related validity (the degree to which the instrument provides results that are correlated with related measures), we examined SNOT-16 scores in subgroups expected to have higher scores. We compared Day-0 scores in subjects reporting more severe symptoms and those who were more bothered by their symptoms with those from subjects with less severe or bothersome symptoms. We also examined the Day-0 score in subjects with asthma, with allergies, and who smoked.⁴ Statistical significance of the differences across subgroup scores was assessed using Student’s t tests or ANOVA as appropriate.

Reproducibility

We evaluated internal consistency using Cronbach’s alpha. The minimum acceptable score for this statistic for group comparison is >0.7.⁷ We assessed test-retest reproducibility comparing the test administered at baseline during a face-to-face interview with that administered later that same day by telephone. We calculated Pearson’s correlation statistic, compared the baseline SNOT-16 scores using a paired t-test, and calculated the intra-class correlation coefficient (ICC).

Responsiveness

We defined responsiveness as the ability of the instrument to detect small but important clinical changes over time.⁸

We calculated the mean change in SNOT-16 scores from baseline to each time point (3-, 7- and 10- days) and compared scores at baseline with those at later time points using paired t-tests. We evaluated the trend in change scores using MANOVA.

We calculated the effect size as a measure of responsiveness using the formula: (U-V)/C where U= the mean baseline score, V= the mean follow-up score and C= the standard deviation of the baseline score.⁸ By convention, an effect size of >0.2 is considered to be a small improvement in health-related quality of life, >0.5 is a moderate change and >0.8 is a large change.^{7, 9} Larger effect sizes indicate an instrument that is highly sensitive to change.⁹

Clinical interpretability

To aid in the interpretation of study results that show a change in the SNOT-16 score, we calculated the “Minimally Important Difference” (MID),⁷ the smallest difference in score that is clinically significant. We defined this value as the change in SNOT-16 score for those who reported a small change in their symptoms on the global rating scale. Using the subject’s global assessment of change in symptoms since baseline, we categorized “ a lot worse” and “a lot better” and “no symptoms” as a large change, “a little worse” and ”a little better” as a small change, and “the same” as no change.

RESULTS

Study subjects

Between November 1^st, 2006 and May 1^st, 2009, the SNOT-16 instrument was administered to 166 subjects randomized in the intervention study and recruited from 10 primary care practices in St Louis, MO (Table 1). Follow-up interviews at Days 3, 7, and 10 were completed by 94%, 94%, and 92% participants respectively with no difference by study group.

Table 1.

Baseline Characteristics Of 166 Subjects With Clinically Diagnosed Acute Sinusitis

Variable
Sociodemographic characteristics
Age in years median (range)	32 (18 to 69)
Male gender n (%)	60 (36%)
Racial group n (%)
White	130 (78%)
Black	26 (16%)
Other	10 (6%)
Hispanic n (%)	5 (3%)
Educational level n (%)
High school or less	113 (68%)
Bachelors degree	32 (19%)
Post graduate/professional degree	21 (13%)
Health insurance n (%)
Employment-based/Private	112 (67%)
Government	15 (9%)
No insurance	7 (4%)
Private	32 (19%)
Lives alone n (%)	29 (17%)
Currently employed	108 (65%)
Family income/year
< $10,000	9 (5%)
$10,000 – $24,999	16 (10%)
$25,000 – $49,999	28 (17%)
$50,000 – $99,999	51 (31%)
≥ $100,000	36 (22%)
Declined to answer	26 (16%)
Medical History n (%)
Usual health excellent or very good	91 (55%)
History of sinus disease	122 (79%)
Allergic rhinitis	54 (33%)
Nasal polyps	3 (2%)
Positive test to mold, dust, pollen, or animal dander	69 (50%)
Asthma	18 (11%)
Smoker	32 (19%)
Sinus Symptoms
Symptom severity
Moderate	80 (48%)
Severe	71 (43%)
Very severe	15 (9%)
Days of symptoms (Mean, SD)	11.2 (5.7)
< 7 days of symptoms	23 (14%)
Days missed from work before visit (Mean, SD)	1.4 (3.2)

SD=standard deviation

Subject Characteristics

The median age of the study population was 32 years (range 18 to 69 years), 36% were male, 78% were Caucasian and 16% were African American (Table 1). All participants reported purulent nasal discharge and maxillary pain or tenderness in the face or teeth (94 bilateral, 56 unilateral, 14 laterality unknown). The mean symptom duration prior to the enrollment visit was 11.2 days (standard deviation, sd 5.7 days). Participants described their symptoms as moderate (48%), severe (43%) or very severe (9%).

Performance of the SNOT-16

Respondent burden

The instrument was easy to use and took less than 5 minutes to complete. It was well accepted by subjects and did not require special training for the RA to administer. For each completed interview, there were no missing data.

Construct-related validity

The mean SNOT-16 scores assessed at Day-0 for various subgroups are presented in Table 2. The score was able to identify statistically significant differences in the hypothesized direction for those reporting more or less severe symptoms (p=0.0161) and more or less bother (p<0.0001). Significant differences were also found between those with and without allergies (p=0.0022).

Table 2.

SNOT-16 scores by reported symptom severity and bother at Day-0.

	SNOT-16 score
	Mean	Standard deviation	P-value
Symptom severity
Moderate	1.59	0.51	0.0161
Severe	1.81	0.52
Very severe	1.85	0.47
Reported bother
Bothered a little	1.25	0.54	<0.0001
Bothered a lot	1.64	0.47
Extremely bothered	2.03	0.45
History of allergies
Yes	1.86	0.50	0.0022
No	1.59	0.49
Asthma
Yes	1.93	0.48	0.058
No	1.68	0.52
Smoker
Yes	1.72	0.54	0.87
No	1.71	0.51
Prior sinus infection
Yes	1.69	0.53	0.45
No	1.77	0.50

Internal consistency

Cronbach alpha scores for the baseline face-to-face interview and the Day 0 telephone interview and at days 3, 7, and 10 were 0.82, 0.82, 0.87, 0.91, and 0.91 respectively. This indicates high internal consistency for this measure in this study population.

Reproducibility

We compared the SNOT-16 scores at baseline administered by face-to-face and telephone interviews in 162 subjects for whom both measures were available. The mean baseline scores differed significantly, 1.84 (sd 0.51) for the face-to-face score, and 1.70 (sd 0.52) for the telephone measure, p<0.0001. On average, the SNOT-16 score was 0.13 units (sd 0.36) lower when measured by telephone interview compared with the face-to-face interview. Pearson’s correlation coefficient was 0.75, and the ICC was 0.73.

Responsiveness

There was a statistically significant decrease in scores with time (MANOVA, p<0.0001)(Table 3). The effect sizes at days 3, 7, and 10 were 1.45, 2.34 and 2.90 respectively, indicating a highly responsive instrument.

Table 3.

Responsiveness of the SNOT-16 score over time.

	N	SNOT-16 score		P-value*
		Mean (sd)	95% Confidence Interval
Day 0	166	1.71 (0.52)	1.63 to 1.79
Day 3	155	1.13 (0.54)	1.04 to 1.21	< 0.0001
Day 7	155	0.74 (0.54)	0.66 to 0.83	< 0.0001
Day 10	152	0.49 (0.44)	0.42 to 0.56	< 0.0001

^*P-value calculated using paired t-test compared to Day 0

MANOVA. P<0.0001

Clinical Interpretability

The mean 3-. 7- and 10-day change in SNOT-16 scores are shown in Tables 4 and 5. In Table 4, data from subjects who reported the same level of change (eg, a little better and a little worse) are combined, regardless of the direction of that change. For each day, the numbers reporting “no change” on the global rating scale were small, and the calculated mean change scores did not differ from 0. For those reporting a small change in symptoms on the global rating score, the mean change in SNOT-16 score ranged from 0.48 at day 3 to 0.80 at day 10. For those reporting a large change in symptoms, the mean change in SNOT-16 score increased, ranging from 0.8 at day 3 to 1.3 units at day 10. From these data we estimate the MID for the SNOT-16 to be 0.5 points. When improvement and deterioration were examined separately (Table 5), a similar pattern was seen for those who improved.

Table 4.

Changes in SNOT-16 scores versus global rating of change – better and worse data combined

	Day 3		Day 7		Day 10
	N	Change in SNOT-16 score Mean (SD) 95% CI	n	Change in SNOT-16 score Mean (SD) 95% CI	n	Change in SNOT-16 score Mean (SD) 95%CI
Same	14	−0.098 (0.33) −0.29 to 0.09	7	−0.52 (0.60) −1.07 to 0.03	4	−0.22 (0.33) −0.74 to 0.30
Small change	83	−0.48 (0.52) −0.60 to −0.37	46	−0.60 (0.50) −0.75 to −0.45	26	−0.80 (0.65) −1.07 to −0.54
Large change	58	−0.83 (0.51) −0.96 to −0.69	102	−1.19 (0.57) −1.31 to −1.08	122	−1.30 (0.60) −1.41 to −1.19

SD=Standard deviation

Table 5.

Change in SNOT-16 scores versus global rating of change over time

Global rating	Day 3		Day 7		Day 10
	N	Change in SNOT-16 score (mean, sd)	N	Change in SNOT-16 score (mean, sd)	N	Change in SNOT-16 score (mean, sd)
No symptoms	0		6	−1.39 (0.52)	16	−1.40 (0.68)
A lot better	55	−0.86 (0.49)	95	−1.19 (0.58)	104	−1.31 (0.54)
A little better	75	−0.53 (0.49)	44	−0.61 (0.48)	24	−0.92 (0.51)
The same	14	−0.10 (0.33)	7	−0.52 (0.60)	4	−0.22 (0.33)
A little worse	8	0.03 (0.59)	2	0.47 (1.10)	2	−0.66 (0.04)
A lot worse	3	0.19 (0.47)	1	0.63	2	−0.31 (1.24)

We repeated the analyses for responsiveness and cllnical interpretability using only the 5 items from the SNOT-16 considered to be most important by each subject. Again, there was a statistically significant decrease in scores over time (p<0.0001), with larger effect sizes (Day 3, 1.45: Day 7, 2.34; Day 10, 2.90). The minimally important difference was 0.6. For those reporting a small change in symptoms (better or worse combined), the changes in SNOT-16 scores at days 3, 7 and 10 were −0.63 (−0.81 to −0.45), −0.92 (−1.11 to −0.73) and −1.10 (−1.45 to −0.75) respectively.

DISCUSSION

This is the first study to evaluate a disease-specific QOL tool for use as an outcome measure for patients with clinically diagnosed acute rhinosinusitis. We have demonstrated internal consistency, construct-related validity, and responsiveness to change of the modified SNOT-16 in this patient population and calculated the minimally important difference in score. The SNOT instruments were developed for use in patients with chronic rhinosinusitis and have been used extensively in studies to evaluate surgical and non-surgical management for these patients.^{4, 10} The four domains included in the SNOT-16 (rhinologic symptoms, ear/facial symptoms, sleep symptoms, and psychological function) were identified from extensive consultations with patients with chronic disease ^{3, 10} Our findings suggest that the conceptual basis used to develop the tool for patients with chronic disease is relevant for patients with acute rhinosinusitis. The instrument performed well in this patient group and can be used to assess change in disease-specific QOL in studies evaluating management of adults with acute rhinosinusitis.

There are many aspects to assessing the validity of an instrument. For the SNOT-16, criterion validity cannot be assessed, as there is no “gold standard” measure for the QOL to use as a comparator. Content-related validity has been established during the development of the tool from the RSOM-31 and the SNOT-20.⁶ We assessed construct-related validity and determined that the score behaved as it would be expected to do in subjects with acute rhinosinusitis who reported more or less severe symptoms and overall bother from their illness.

The internal consistency of the modified SNOT-16 used in this study when assessed with Cronbach’s alpha was high (≥0.82) and similar to other SNOT instruments used in patients with chronic sinusitis.^4–6 This suggests that items included in the SNOT-16 instrument are homogeneous, and the instrument is reliable for use in intervention studies in patients with acute sinusitis. We hypothesized that baseline SNOT-16 scores assessed in the office during a face-to-face interview would not differ significantly from scores generated when the instrument was administered during a telephone interview later the same day. Unfortunately, this was not the case. We do not know if the 0.13 unit decrease in these two scores assessed within 12-hours of each other was due to the different mode of administration, resolution of this acute illness in the short time period involved, or regression to the mean. However, we have confirmed that if telephone follow-up is planned for outcome assessment, then the initial SNOT-16 measurement must also be completed by telephone.

Responsiveness is an important characteristic of an outcome measure for use in a clinical trial. We compared the change in the SNOT-16 score to a global rating of change provided by the subject, a metric commonly used in clinical practice and clinical research.¹¹ We demonstrated that the modified SNOT-16 was highly responsive to change over time for patients with acute disease. The measured effect size increased with time, but was consistently >1.45. This finding compares favorably with the value of 0.69 and 0.81 calculated for other SNOT instruments for patients with chronic rhinosinusitis.^{4, 5} We were unable to use the approach suggested by Guyatt to assess responsiveness (dividing the difference in mean scores by the standard deviation of the change scores in stable subjects) to detect responsiveness above the normal variations in score⁸ as very few subjects remained stable in the study population.

The SNOT-16 allows subjects to identify the 5 items most important to them at baseline, regardless of their magnitude. Change scores for these most important items were significantly associated with treatment response in patients with chronic sinusitis.⁶ The instrument developers suggest that this feature is what distinguishes the SNOT instrument as a QOL life measure from a measure of health status, and can be used to track response to therapies and guide clinical management.⁶ Others have argued that this feature simply adds respondent burden without providing additional information.⁴ We found that the additional respondent burden was minimal, the effect size assessed using the SNOT-16 was larger, and the MID was about the same (0.6 vs 0.5). Investigators planning to use this tool for outcome assessment will need to determine if this additional feature of the instrument is needed in their study.

The clinical meaning of a change in QOL score is difficult to assess. One approach is to calculate the smallest change that would be detected as an improvement (the MID). We calculated the MID for the modified SNOT-16 in patients with acute sinusitis to be 0.5 units. While this is the group average and may not represent meaningful change at the level of the individual, it suggests that a patient with a change in SNOT-16 score of <0.5 is unlikely to perceive any benefit from treatment. One way an individual could achieve a clinically meaningful difference (change of 0.5 units) would be a change of 8 units across the 16 items in the SNOT-16. The MID can also be used to compare outcomes in effectiveness trials when a difference between groups of < 0.5 units would be considered to be not clinically significant. This magnitude for the MID is comparable to that for other QOL instruments that use a 7-point scale.¹²

Recently, the SNOT-20 was modified for use in studies of chronic rhinosinusitis. On the recommendation of a focus group of experienced endoscopic sinus surgeons from the United Kingdom, two items were added due concerns about the validity of the tool for patients with chronic disease (nasal blockage and altered taste or small).⁴ Our findings suggest that these additions to the instrument are unnecessary for patients with acute rhinosinusitis, as the 16-item score was responsive and easy to use.

There are several limitations to our work. We did not repeat previous work that was part of the development of the SNOT instruments. For example, we did not conduct focus groups with patients with acute rhinosinusitis to ensure that the conceptual model developed for use in the assessment of chronic sinusitis still applied. Nor did we evaluate content-related or discriminant validity for the SNOT-16 in patients with acute sinusitis. These metrics of instrument performance have been evaluated for the SNOT-16, SNOT-20 and SNOT-22 in previous studies in subjects with chronic sinusitis,^4–6 and we did not plan to use the instrument as a diagnostic tool for acute rhinosinusitis. As our interest is in an acute disease, it is difficult to assess test-test reliability and identify a group expected to remain stable as a comparator to evaluate responsiveness.

Conclusions

We found the modified SNOT-16 to be valid, responsive and easy to use, and recommend its use to assess change in disease-related QOL in interventions targeting patients with acute rhinosinusitis. It is important to note that if the planned mode of administration is by telephone interview, then the baseline assessment must also be by telephone interview.