Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2018 Aug 1.
Published in final edited form as: Int Forum Allergy Rhinol. 2017 May 26;7(8):821–826. doi: 10.1002/alr.21964

Defining the Minimal Clinically Important Difference for Olfactory Outcomes in the Surgical Treatment of Chronic Rhinosinusitis

Joshua M Levy 1, Jess C Mace 2, Todd E Bodner 3, Jeremiah A Alt 4, Timothy L Smith 2
PMCID: PMC5544549  NIHMSID: NIHMS874861  PMID: 28556611

Abstract

BACKGROUND

Olfactory dysfunction is a common and defining symptom of chronic rhinosinusitis (CRS). Many measures of olfactory dysfunction in CRS are limited by scoring criteria defined within general populations with interpretations of statistical significance to infer clinically meaningful improvement. This investigation defines a minimal clinically important difference (MCID) for the Brief Smell Identification Test (BSIT) in CRS patients electing endoscopic sinus surgery (ESS).

METHODS

A multi-center cohort of 290 adult patients electing ESS for medically recalcitrant CRS were prospectively enrolled between March, 2011 and June, 2015 and completed BSIT evaluations before and after ESS. Distribution and anchor-based analytic approaches were utilized to define MCID values of the BSIT across patient cofactors.

RESULTS

A total of 92 (~32%) patients were found to have preoperative olfactory dysfunction (BSIT<9); significantly associated with nasal polyposis (χ2=35.0; p<0.001). The effect size distribution-based approach identified 1.0 as a MCID criterion value between “small” and “medium” effect (range: 0.61–1.52) overall. Significant mean postoperative change (∆M) was reported for patients with olfactory dysfunction (∆M=2.28; p<0.001), both with (n=54; ∆M=2.52; p<0.001) and without (n=38; ∆M=1.95; p<0.001) nasal polyposis, significantly exceeding the MCID criterion. Anchor-based approaches with regression modeling confirmed associations between MCID values and postoperative changes to olfactory-specific, survey responses (p<0.001).

CONCLUSIONS

Clinically meaningful change in BSIT scores may be defined as an absolute value difference of at least 1.0 unit for heterogeneous patients electing ESS for CRS. Significantly exceeding that criterion may be restricted to CRS patients with baseline olfactory dysfunction, regardless of nasal polyposis.

Keywords: Sinusitis, chronic disease, patient outcome assessment, olfaction, olfactory test, quality-of-life, sinus surgery

INTRODUCTION

Olfactory dysfunction is a common, defining feature of chronic rhinosinusitis (CRS), with a reported prevalence of 30–78% among this patient population.1–3 Extent of olfactory dysfunction is associated with quality-of-life (QoL) and disease severity among patients with CRS,4–7 and improves following both medical and surgical intervention.8, 9

The assessment of olfactory dysfunction is important in evaluating patient symptoms and treatment monitoring, and is aided by commercially available tools with the ability to characterize olfactory dysfunction in the ambulatory setting.10 While multiple diagnostic tests are available, their utilization in routine clinical practice is limited by significant resource and time requirements.10 Commonly cited instruments for the evaluation of olfactory dysfunction in CRS include Sniffin’ Sticks11 (Burghart Messtechnik GmbH, Wedel, Germany), the 40-item Smell Identification Test (SIT-40)12 (Sensonics, Haddon Heights, NJ) and the 12-item, cross-cultural Brief Smell Identification Test (BSIT; Sensonics Inc., Haddon Heights, NJ).13 While these resources allow for the quantification of changes in olfactory dysfunction over time, a threshold value for a minimal clinically important difference (MCID) has not been previously defined, with current studies limited to average measures and statistical significance to assume clinical effect.4–6, 8, 9

This study sought to define the MCID for olfactory outcomes among a heterogeneous patient populations undergoing endoscopic sinus surgery (ESS) for CRS. First described by Jaeschke et al., the MCID seeks to place a patient-centered focus on outcomes research, with determination of a discernible threshold value for clinically-apparent change.14 Determination of an olfactory-specific MCID has the potential to improve the specificity of outcomes evaluation, with exclusion of patients that demonstrate statistically significant, but clinically-silent change in BSIT scores following ESS. This value also improves the utility of the BSIT instrument in clinical practice, and may be used to advance outcomes assessment in future study of olfactory dysfunction in CRS.15

MATERIALS and METHODS

Inclusion Criteria and Treatment Modality

Adult (≥18 years of age) participants with a clinical practice guideline diagnosis of medically refractory CRS were included in study evaluation,3, 16 with initial appropriate medical therapies consisting of broad spectrum or culture-directed systemic antibiotics (≥14 days), daily saline irrigation (240ml.), and either a trial of oral corticosteroid therapy (≥5 days) or topic nasal corticosteroid sprays (≥21 days). Study participants were offered enrollment from three academic, tertiary referral centers located within the Departments of Otolaryngology-Head and Neck Surgery at the Oregon Health & Science University (OHSU, Portland, OR), Stanford University (Palo Alto, CA), and the Medical University of South Carolina (Charleston, SC). Study participants provided voluntary, written, informed consent before prospective enrollment into this observational, continuing cohort study to evaluate treatment outcomes of CRS without deviation from the standard of care. The Institutional Review Board (IRB) at each enrollment center provided annual study review and regulatory oversight for all study protocols. Additional findings from this investigation has been previously described.17–20

Treatment modality was not randomized or assigned, in any way, for investigational purposes. Study participants selected ESS as the subsequent treatment option for alleviation of symptoms associated with CRS. Surgical planning was directed by the intraoperative discretion of each enrolling physician based on the extent and location of mucosal disease and completed under general anesthesia. Study participants were either primary or revision ESS cases consisting of either unilateral or bilateral maxillary antrostomy, partial or total ethmoidectomy, sphenoidotomy, or frontal sinusotomy, with septoplasty and inferior turbinate reductions as needed. Postoperative therapeutic regimens included continued daily nasal saline rinses and appropriate therapeutics, as needed, on an individual basis. Study participants were observed through the standard of care for up to 18 months and asked to complete postoperative follow-up at 6 month intervals, either during normal clinical appointments or using follow-up packets mailed via the United States Postal Service.

Exclusion Criteria

Due to physiologic differences in underlying disease processes, treatment considerations, and potential confounding of olfactory results, patients with either ciliary dyskinesia or oral corticosteroid dependent comorbidities were excluded from final analyses. Additionally, study participants who did not complete preoperative olfactory testing or failed to provide study-related follow-up evaluations within 18-months were excluded from final analyses.

Clinical Measures of Disease Severity

Measures of preoperative disease severity, collected during routine clinical assessments, were used simultaneously for study purposes. High resolution computed tomography (CT) imaging was utilized to evaluate sinonasal inflammation, with staging by each enrolling physician in accordance with the Lund-Mackay system (total score range: 0–24) which quantifies the severity of image opacification in the maxillary, ethmoidal, sphenoidal, ostiomeatal complex, and frontal sinus regions.21 Postoperative CT scores were not collected due to risk of elevated radiation exposure and divergence from the standard of care.

Bilateral paranasal sinuses were also preoperatively evaluated using rigid, fiberoptic endoscopes (SCB Xenon 175, Karl Storz, Tuttlingen, Germany) and quantified by each enrolling physician using the Lund-Kennedy scoring system (total score range: 0–20) which quantifies pathologic states within the paranasal sinuses including the presence and severity of nasal polyposis, discharge, edema, scarring, and crusting.22 Higher scores on either staging system reflect worse disease severity.

Brief Smell Identification Test

Bilateral olfactory identification was evaluated both before endoscopic sinus surgery and during each postoperative evaluation using the BSIT instrument (Sensonics, Inc., Haddon Heights, NJ). The BSIT is a validated, cross-cultural, 12-item, non-invasive instrument designed to quantify olfactory identification utilizing microencapsulated odorant strips. Respondents are asked to activate and correctly identify the odorant from one correct response and three distractors. The BSIT has a ‘scratch-and-sniff” design activated using #2 pencil strikes. A higher number of correct scores (total score range: 0–12) reflects superior olfactory identification. Total scores can be interpreted as either ‘normal olfaction’ (BSIT ≥ 9), or as some severity of olfactory dysfunction (BSIT ≤ 8), although alternative thresholds have been proposed for patients with CRS.12, 13

Patient Reported Outcome Measures (PROMs)

Additional assessments of patient-reported symptom severity and olfactory function include the use of the 22-item SinoNasal Outcome Test (SNOT-22; ©2006, Washington University, St. Louis, MO).23 The SNOT-22 is a validated survey instrument developed to operationalize sinonasal symptom severity for a host of sinonasal conditions. Item scores involve Likert scale responses (item score range: 0=”No problem” to 5=”Problem as bad as it can be”) which are summarized (total score range: 0 –110) whereas higher scores reflect worse overall symptom severity. The SNOT-22 consists of a single, specific item (#21) to capture symptom severity associated with “Sense of smell and/or taste”. This survey item score was utilized as an olfactory-specific, anchor-based response for this investigation.

Data Management and Analysis

Participant data was de-identified and safeguarded using unique study identification number assignment before secure transfer to OHSU from each enrollment location. Data was manually entered into a password protected, relational database (Access, Microsoft Corp., Redmond, WA) compliant with the Health Insurance Portability and Accountability Act of 1996.24 Statistical analyses were conducted using SPSS v.24.0 statistical software (IBM Corp., Armonk, NY). Final cohort data was evaluated descriptively while approximate distribution normality was verified for all ordinal or continuous measures.

Distribution-based, effect size methodology was used to determine the MCID for BSIT total scores, as this method defines threshold values for ‘small,’ ‘medium,’ and ‘large’ effect sizes, without the need for multiple baseline assessments.15 To confirm the results of this approach in relation to a patient-reported outcome measure, an anchor-based analysis with linear regression modeling was completed. This anchor-based approach compared the determined MCID to postoperative changes in olfactory specific SNOT-22 item response scores.

Average within-subject differences between preoperative and postoperative PROMs were assessed using Wilcoxon signed rank testing. Simple, bivariate linear regression was used to evaluate associations between BSIT and SNOT-22 anchor-based response scores, without adjustment for surgical extent. Unadjusted effect estimates (β) and standard errors (SE) are reported where appropriate. One-way t-testing was also completed across patient subgroups to evaluate whether observed mean differences were significantly different from MCID value determinations. All statistical analyses used a conventional 0.050 alpha level to identify significance.

RESULTS

Final Cohort and Baseline Evaluations

A total of 312 CRS patients electing ESS were identified for study inclusion and enrolled between March, 2011 and June, 2015. A total of 8 participants were excluded due to comorbid ciliary dyskinesia, while 14 participants were excluded due to comorbid corticosteroid dependent conditions [asthma (n=8); sinusitis (n=4); other (n=2)], allowing for 290 participants for final analyses. Preoperative olfactory dysfunction was present in 92 (~32%) study participants, with an average postoperative follow-up of 15.3 [SD±5.1] months. Additional preoperative characteristics of the final study cohort are described in Table 1, while the prevalence and extent of ESS procedures are described for the final cohort in Table 2.

Table 1.

Demographic factors and patient characteristics in final cohort (n=290)

Characteristics: Mean [± SD] Range N (%)
Follow-up (months) 15.3 [±5.1] [3 – 39]
Age at enrollment (years) 51.1 [±15.3] [18 – 86]
White/Caucasian 242 (83%)
Hispanic/Latino 13 (5%)
Males 142 (49%)
Asthma 96 (33%)
Nasal polyposis (CRSwNP) 100 (35%)
Deviated septum 112 (39%)
Turbinate hypertrophy 43 (15%)
Allergic rhinitis 119 (41%)
ASA sensitivity 16 (6%)
Depression 46 (16%)
Smoking / tobacco use 16 (6%)
Previous sinus surgery 152 (52%)
Diabetes mellitus (Type I/II) 23 (8%)
Computed tomography total score 11.8 [±5.9] [0 – 24]
Endoscopy total score 5.9 [±3.7] [0 – 18]
BSIT total score 8.9 [±3.0] [1 – 12]
Normal olfaction (BSIT ≥ 9) 198 (68%)
SNOT-22 total score 53.2 [±19.5] [4 – 106]
SNOT-22 item #21 score 2.7 [±1.8] [0 – 5]
Open in a new tab

BSIT, brief smell identification test; SD, standard deviation; N, sample size; RAST, radioallergosorbent testing; SNOT-22, 22-item SinoNasal Outcome Test; ASA, acetylsalicylic acid. CRS, chronic rhinosinusitis.

Table 2.

Frequency of surgical procedures (n=290)

Procedure: Right
N (%)		 Bilateral
N (%)		 Left
N (%)
Maxillary antrostomy 253 (87%) ----- 262 (90%)
Partial ethmoidectomy 40 (14%) ----- 42 (15%)
Total ethmoidectomy 213 (73%) ----- 217 (75%)
Sphenoidotomy 193 (67%) ----- 193 (67%)
Middle turbinate resection 41 (14%) ----- 36 (12%)
Inferior turbinate reduction 49 (17%) ----- 51 (18%)
Frontal sinusotomy (Draf IIa) 134 (46%) ----- 134 (46%)
Frontal sinusotomy (Draf IIb) 21 (7%) ----- 21 (7%)
Frontal sinusotomy (Draf III) ----- 8 (3%) -----
Septoplasty ----- 103 (36%) -----
Imaging guidance ----- 202 (70%) -----
Open in a new tab

Distribution-based determination of MCID

Average BSIT total scores for the entire cohort were utilized to calculate MCID values for the BSIT instrument using a distribution-based, effect size approach, as shown in Figure 1. Given that the possible BSIT scores are discrete values and that the MCID is the minimum difference within a pre-determined range, this approach defines the BSIT MCID as a change in score of 1.0 unit; categorized as the difference between a small and medium effect size.

Figure 1.

Figure 1

Open in a new tab

Distribution-based, effect size algorithm and determination for the BSIT MCID. Standard deviation of BSIT scores at baseline = 3.03. BSIT, Brief Smell Identification Test. MCID, minimal clinically important difference.

Anchor-based Confirmation of MCID Value

To demonstrate an anchor-based association between the defined MCID value and patient-reported changes in olfactory function, response scores of the survey item (#21) of the SNOT-22 were used to assess “Sense of smell and/or taste”. Average, within-subject item scores significantly improved (p<0.001) postoperatively from 2.7 [SD±1.8] to 1.6 [SD±1.6]. For participants with baseline normal olfactory function (BSIT ≥ 9) within-subject mean scores significantly improved (p<0.001) from 2.2 [SD±1.7] to 1.3 [SD±1.4]. Participants with baseline olfactory dysfunction (BSIT ≤ 8) also reported significant (p<0.001) within-subject mean item scores (3.9 [SD±1.5] to 2.4 [SD±1.8]), a larger magnitude of item score improvement (p=0.062) on average. Simple regression modeling further evaluated the linear association between postoperative changes in BSIT scores and postoperative changes in SNOT-22 anchor scores. Each single MCID unit improvement in unadjusted BSIT scores was found to be associated with a postoperative reduction in patient-reported sense of smell and/or taste (β= −0.44 [SE±0.09]; t= −4.86; p<0.001). The MCID value for BSIT scores was found to significantly associate with changes in patient-reported taste/smell symptom severity.

Subgroup Improvements in Average Olfactory Function

Among the overall cohort, a significant postoperative change (∆) in mean total BSIT scores was not observed following ESS (mean Δ=0.13, Table 3); with the observed mean differences significantly less than the MCID criterion of 1.0 unit (p<0.001). Clinically apparent change in mean BSIT scores was found among CRS patients with baseline olfactory dysfunction (mean Δ=2.28); with a change in BSIT score that was significantly greater than the MCID criterion of 1.0 units. This finding was consistent in patients both with and without comorbid nasal polyposis (mean Δ=2.52 and mean Δ=1.95, respectively). While statistically significant worsening in average BSIT scores were demonstrated among patients without baseline olfactory dysfunction (mean Δ= −0.86); this mean change was not significantly greater when compared against the MCID criterion.

Table 3.

Mean Change in BSIT Scores from Baseline Patient Category and Null Hypothesis Specification

H0: ΔM = 0 H0: ΔM =
|MCID=1.0|
Baseline Category: Mean Δ 95% CI p-value p-value
Overall  (n=290) 0.13 (−0.19, 0.46) p=0.622 p<0.001
With Polyposis  (n=100) 0.82 (0.15, 1.49) p=0.017 p=0.596
Without Polyposis  (n=190) −0.23 (−0.56, 0.11) p=0.181 p<0.001
With Olfactory Dysfunction  (n=92) 2.28 (1.68, 2.88) p<0.001 p<0.001
Without Olfactory Dysfunction  (n=198) −0.86 (−1.15, −0.58) p<0.001 p=0.353
With Polyposis and Olfactory Dysfunction  (n=54) 2.52 (1.59, 3.45) p<0.001 p=0.002
With Polyposis without Olfactory Dysfunction  (n=46) −1.17 (−1.76, −0.59) p<0.001 p=0.555
Without Polyposis with Olfactory Dysfunction  (n=38) 1.95 (1.30, 2.60) p<0.001 p=0.005
Without Polyps and Olfactory Dysfunction  (n=152) −0.77 (−1.10, −0.44) p<0.001 p=0.174
Open in a new tab

Mean Δ; Last available BSIT score – preoperative BSIT score; Ho, null hypothesis; MCID, minimal clinically important difference

DISCUSSION

This study utilizes distribution-based modeling to calculate the MCID for measurement of olfactory dysfunction in CRS patients undergoing ESS, with demonstration of a BSIT score of 1.0 representing a threshold value for clinically apparent change. Anchor-based modeling was then utilized to demonstrate a significant association between the calculated MCID value and patient-reported olfactory outcomes. The BSIT instrument was utilized for this study due to a relatively low time-burden which is uniquely suited for use in a robust clinical practice.

The calculation of the MCID for a given health state or treatment modality is an inexact science, without consensus regarding the most appropriate methodology for a given population or diagnostic instrument. Two broad strategies for MCID modeling include distribution-based and anchor-based approaches.15, 25 Distribution-based approaches utilize statistical characteristics of the sample (e.g., the standard deviation of diagnostic scores pre-treatment or the mean change in scores over time) to define an MCID. Options for distribution-based modeling include calculation of the standard error of measurement,26 reliable change index,27 effect size,28 paired t-statistic,29 growth curve analysis,30 standard response mean31 and responsiveness statistic.32 While distribution-based approaches are readily available and can be completed without the use of arbitrary external referents, (e.g., patient reported outcome measures) there are several limitations, including dependence on the characteristics of a given sample and the potential to define a MCID value that does not reach statistical significance.15, 25

Anchor-based approaches use external referents when defining MCID values, with modeling options such as comparison to known populations,27 comparison to disease/non-disease criteria,33, 34 preference ratings,35 global ratings of change,36 prognosis of future events37 and changes in disease related outcome.38 Limitations to anchor-based modeling include potential unavailability of external referents, recall bias, and the dependence on arbitrary cut-point thresholds to define MCID values.15, 25 This technique was utilized by Hopkins et al. to calculate the MCID value of 9 for the SNOT-22 instrument.23

In our approach, we utilized a combined strategy with distribution-based effect sizes to determine a MCID value for the BSIT instrument, followed by anchor-based modeling with utilization of olfactory-specific SNOT-22 item scores to evaluate statistical significance. In this analysis, the calculated MCID value has a higher specificity than the results of effect size modeling, with ½ standard deviation of mean baseline BSIT scores (Table 1) equivalent to the calculated threshold for medium effects (Figure 1). Similar combined approaches have been previously demonstrated for various determinations of MCID values.27, 38, 39

This study demonstrates that CRS patients with abnormal baseline olfactory function (BSIT scores ≤ 8), surpass a MCID threshold for improvement in olfactory function following ESS. Clinical improvements were demonstrated among patients with and without nasal polyposis, with baseline olfactory dysfunction representing the most important variable related to clinical improvements in olfaction following ESS. These findings are consistent with prior study, which utilized Sniffin’ Sticks and the SIT-40 to evaluate olfactory outcomes following ESS.9, 40, 41

The defined MCID is limited to the outcomes assessment of olfactory function among a heterogeneous population of patients with CRS following ESS. In considering possible methods for determining the MCID for a given health outcome, it is useful to consider the distinction between an MCID as an indicator of treatment effectiveness for an arbitrary treatment modality versus a specific intervention. In principle, one could define an MCID for BSIT scores that would hold irrespective of which treatment was applied (e.g., medical management including nasal sprays, or elective sinus surgery). Therefore, while this investigation defines a MCID value that may be used to counsel a heterogeneous group of patients before ESS, it should not be used to evaluate outcomes among different treatments for CRS without further study demonstrating external validity.

CONCLUSIONS

The minimal clinically meaningful change in BSIT scores may be defined as an absolute difference of at least 1.0 unit following ESS. Clinically-relevant improvements may be restricted to CRS patients with baseline olfactory dysfunction, without regard to patient age or nasal polyposis.

Acknowledgments

Financial Disclosures: Timothy Smith, Jeremiah A. Alt, Todd Bodner, and Jess Mace are currently supported by a grant for this investigation from the National Institute on Deafness and Other Communication Disorders (NIDCD), one of the National Institutes of Health, Bethesda, MD., USA (R01 DC005805; PI/PD: TL Smith). Public clinical trial registration (www.clinicaltrials.gov) ID# NCT02720653. This funding organization did not contribute to the design or conduct of this study; preparation, review, approval or decision to submit this manuscript for publication.

Footnotes

Potential Conflicts of Interest: None

There are no financial disclosures for Joshua M. Levy.

References

  1. Kohli P, Naik AN, Harruff EE, Nguyen SA, Schlosser RJ, Soler ZM. The prevalence of olfactory dysfunction in chronic rhinosinusitis. Laryngoscope. 2017;127(2):309–320. doi: 10.1002/lary.26316. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. 2.Litvack JR, Fong KJ, Mace J, James KE, Smith TL. Predictors of olfactory dysfunction in patients with chronic rhinosinusitis. Laryngoscope. 2008;118(12):2225–2230. doi: 10.1097/MLG.0b013e318184e216. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. 3.Rosenfeld RM, Piccirillo JF, Chandrasekhar SS, et al. Clinical practice guideline (update): adult sinusitis. Otolaryngol Head Neck Surg. 2015;152(2 Suppl):S1–S39. doi: 10.1177/0194599815572097. [DOI] [PubMed] [Google Scholar]
  4. 4.Katotomichelakis M, Simopoulos E, Tripsianis G, et al. Improvement of olfactory function for quality of life recovery. Laryngoscope. 2013;123(11):E10–E16. doi: 10.1002/lary.24113. [DOI] [PubMed] [Google Scholar]
  5. 5.DeConde AS, Mace JC, Alt JA, Schlosser RJ, Smith TL, Soler ZM. Comparative effectiveness of medical and surgical therapy on olfaction in chronic rhinosinusitis: a prospective, multi-institutional study. Int Forum Allergy Rhinol . 2014;4(9):725–733. doi: 10.1002/alr.21350. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. 6.DeConde AS, Mace JC, Alt JA, Rudmik L, Soler ZM, Smith TL. Longitudinal improvement and stability of the SNOT-22 survey in the evaluation of surgical management for chronic rhinosinusitis. Int Forum Allergy Rhinol . 2015;5(3):233–239. doi: 10.1002/alr.21458. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. 7.Alt JA, Mace JC, Buniel MCF, Soler ZM, Smith TL. Predictors of olfactory dysfunction in rhinosinusitis using the brief smell identification test. Laryngoscope. 2014;124(7):E259–E266. doi: 10.1002/lary.24587. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. 8.Litvack JR, Mace J, Smith TL. Does olfactory function improve after endoscopic sinus surgery? Otolaryngol Head Neck Surg. 2009;140(3):312–319. doi: 10.1016/j.otohns.2008.12.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. 9.Levy JM, Mace JC, Sansoni ER, Soler ZM, Smith TL. Longitudinal improvement and stability of olfactory function in the evaluation of surgical management for chronic rhinosinusitis. Int Forum Allergy Rhinol. 2016;6(11):1188–1195. doi: 10.1002/alr.21800. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. 10.Doty RL. Office procedures for quantitative assessment of olfactory function. Am J Rhinol. 2007;21(4):460–473. doi: 10.2500/ajr.2007.21.3043. [DOI] [PubMed] [Google Scholar]
  11. 11.Hummel T, Sekinger B, Wolf SR, Pauli E, Kobal G. 'Sniffin‘ Sticks’: Olfactory performance assessed by the combined testing of odor identification, odor discrimination and olfactory threshold. Chem Senses. 1997;22(1):39–52. doi: 10.1093/chemse/22.1.39. [DOI] [PubMed] [Google Scholar]
  12. 12.Doty RL, Marcus A, Lee WW. Development of the 12-item Cross-Cultural Smell Identification Test (CC-SIT) Laryngoscope. 1996;106(3 Pt 1):353–356. doi: 10.1097/00005537-199603000-00021. [DOI] [PubMed] [Google Scholar]
  13. 13.El Rassi E, Mace JC, Steele TO, et al. Sensitivity analysis and diagnostic accuracy of the Brief Smell Identification Test in patients with chronic rhinosinusitis. Int Forum Allergy Rhinol. 2016;6(3):287–292. doi: 10.1002/alr.21670. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. 14.Jaeschke R, Singer J, Guyatt GH. Measurement of health status. Ascertaining the minimal clinically important difference. Control Clin Trials. 1989;10(4):407–415. doi: 10.1016/0197-2456(89)90005-6. [DOI] [PubMed] [Google Scholar]
  15. 15.Wright A, Hannon J, Hegedus EJ, Kavchak AE. Clinimetrics corner: a closer look at the minimal clinically important difference (MCID) J Man Manip Ther . 2012;20(3):160–166. doi: 10.1179/2042618612Y.0000000001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. 16.Rosenfeld RM, Andes D, Bhattacharyya N, et al. Clinical practice guideline: adult sinusitis. Otolaryngol Head Neck Surg. 2007;137(3 Suppl):S1–S31. doi: 10.1016/j.otohns.2007.06.726. [DOI] [PubMed] [Google Scholar]
  17. 17.Levy JM, Mace JC, Smith TL, Soler ZM. Influence of interpersonal traits on patient outcomes in the treatment of chronic rhinosinusitis. Int Forum Allergy Rhinol. 2016 Nov 11; doi: 10.1002/alr.21886. Epub ahead of print. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. 18.Alt JA, Smith TL, Schlosser RJ, Mace JC, Soler ZM. Sleep and quality of life improvements after endoscopic sinus surgery in patients with chronic rhinosinusitis. Int Forum Allergy Rhinol. 2014;4(9):693–701. doi: 10.1002/alr.21364. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. 19.Levy JM, Mace JC, DeConde AS, Steele TO, Smith TL. Improvements in psychological dysfunction after endoscopic sinus surgery for patients with chronic rhinosinusitis. Int Forum Allergy Rhinol. 2016;6(9):906–913. doi: 10.1002/alr.21776. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. 20.Steele TO, Rudmik L, Mace JC, DeConde AS, Alt JA, Smith TL. Patient-centered decision making: the role of the baseline SNOT-22 in predicting outcomes for medical management of chronic rhinosinusitis. Int Forum Allergy Rhinol. 2016;6(6):590–596. doi: 10.1002/alr.21721. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. 21.Lund VJ, Mackay IS. Staging in rhinosinusitus. Rhinology. 1993;31(4):183–184. [PubMed] [Google Scholar]
  22. 22.Lund VJ, Kennedy DW. Staging for rhinosinusitis. Otolaryngol Head Neck Surg. 1997;117(3 Pt 2):S35–S40. doi: 10.1016/S0194-59989770005-6. [DOI] [PubMed] [Google Scholar]
  23. 23.Hopkins C, Gillett S, Slack R, Lund VJ, Browne JP. Psychometric validity of the 22-item Sinonasal Outcome Test. Clin Otolaryngol. 2009;34(5):447–454. doi: 10.1111/j.1749-4486.2009.01995.x. [DOI] [PubMed] [Google Scholar]
  24. 24.United States Department of Health and Human Services, Office for Civil Rights Headquarters. [Accessed January 5, 2017];Health Information Privacy. Available at: https://www.hhs.gov/hipaa/
  25. 25.Crosby RD, Kolotkin RL, Williams GR. Defining clinically meaningful change in health-related quality of life. J Clin Epidemiol . 2003;56(5):395–407. doi: 10.1016/s0895-4356(03)00044-1. [DOI] [PubMed] [Google Scholar]
  26. 26.Wyrwich KW, Tierney WM, Wolinsky FD. Further evidence supporting an SEM-based criterion for identifying meaningful intra-individual changes in health-related quality of life. J Clin Epidemiol. 1999;52(9):861–873. doi: 10.1016/s0895-4356(99)00071-2. [DOI] [PubMed] [Google Scholar]
  27. 27.Jacobson NS, Truax P. Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. J Consult Clin Psychol. 1991;59(1):12–19. doi: 10.1037//0022-006x.59.1.12. [DOI] [PubMed] [Google Scholar]
  28. 28.Kazis LE, Anderson JJ, Meenan RF. Effect sizes for interpreting changes in health status. Med Care. 1989;27(3 Suppl):S178–S189. doi: 10.1097/00005650-198903001-00015. [DOI] [PubMed] [Google Scholar]
  29. 29.Husted JA, Cook RJ, Farewell VT, Gladman DD. Methods for assessing responsiveness: a critical review and recommendations. J Clin Epidemiol. 2000;53(5):459–468. doi: 10.1016/s0895-4356(99)00206-1. [DOI] [PubMed] [Google Scholar]
  30. 30.Speer DC, Greenbaum PE. Five methods for computing significant individual client change and improvement rates: support for an individual growth curve approach. J Consult Clin Psychol. 1995;63(6):1044–1048. doi: 10.1037//0022-006x.63.6.1044. [DOI] [PubMed] [Google Scholar]
  31. 31.Stucki G, Liang MH, Fossel AH, Katz JN. Relative responsiveness of condition-specific and generic health status measures in degenerative lumbar spinal stenosis. J Clin Epidemiol. 1995;48(11):1369–1378. doi: 10.1016/0895-4356(95)00054-2. [DOI] [PubMed] [Google Scholar]
  32. 32.Guyatt GH, Bombardier C, Tugwell PX. Measuring disease-specific quality of life in clinical trials. CMAJ. 1986;134(8):889–895. [PMC free article] [PubMed] [Google Scholar]
  33. 33.Deyo RA, Inui TS, Leininger J, Overman S. Physical and Psychosocial Function in Rheumatoid Arthritis: Clinical Use of a Self-administered Health Status Instrument. Arch Intern Med. 1982;142(5):879–882. [PubMed] [Google Scholar]
  34. 34.Testa MA, Lenderking WR. Interpreting Pharmacoeconomic and Quality-of-Life Clinical Trial Data for Use in Therapeutics. PharmacoEconomics. 1992;2(2):107–117. doi: 10.2165/00019053-199202020-00004. [DOI] [PubMed] [Google Scholar]
  35. 35.Llewellyn-Thomas HA, Williams JI, Levy L, Naylor CD. Using a trade-off technique to assess patients' treatment preferences for benign prostatic hyperplasia. Med Decis Making. 1996;16(3):262–282. doi: 10.1177/0272989X9601600311. [DOI] [PubMed] [Google Scholar]
  36. 36.Stucki G, Liang MH, Fossel AH, Katz JN. Relative responsiveness of condition-specific and generic health status measures in degenerative lumbar spinal stenosis. J Clin Epidemiol. 1995;48(11):1369–1378. doi: 10.1016/0895-4356(95)00054-2. [DOI] [PubMed] [Google Scholar]
  37. 37.Mossey JM, Shapiro E. Self-rated health: a predictor of mortality among the elderly. Am J Public Health. 1982;72(8):800–808. doi: 10.2105/ajph.72.8.800. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. 38.Kolotkin RL, Crosby RD, Williams GR. Integrating Anchor-Based and Distribution-Based Methods to Determine Clinically Meaningful Change in Obesity-Specific Quality of Life. Qual Life Res. 2002;11:670. [Google Scholar]
  39. 39.Cella D, Eton DT, Fairclough DL, et al. What is a clinically meaningful change on the Functional Assessment of Cancer Therapy-Lung (FACT-L) Questionnaire? Results from Eastern Cooperative Oncology Group (ECOG) Study 5592. J Clin Epidemiol. 2002;55(3):285–295. doi: 10.1016/s0895-4356(01)00477-2. [DOI] [PubMed] [Google Scholar]
  40. 40.Pade J, Hummel T. Olfactory function following nasal surgery. Laryngoscope. 2008;118(7):1260–1264. doi: 10.1097/MLG.0b013e318170b5cb. [DOI] [PubMed] [Google Scholar]
  41. 41.Hsu CY, Wang YP, Shen PH, Weitzel EK, Lai JT, Wormald PJ. Objective olfactory outcomes after revision endoscopic sinus surgery. Am J Rhinol Allergy. 2013;27(4):e96–e100. doi: 10.2500/ajra.2013.27.3939. [DOI] [PubMed] [Google Scholar]

RESOURCES