Abstract
OBJECTIVE
The objective of this investigation was to evaluate endoscopic sinus surgery (ESS) outcomes for chronic rhinosinusitis (CRS) between medical centers to determine if differences in quality-of-life (QOL) outcomes were detectable. In addition, we sought to identify significant, independent cofactors toward the development of an ESS-specific risk-adjustment model so that ESS outcomes may be appropriately compared between institutions and health care providers.
STUDY DESIGN
Prospective, multi-center, observational cohort.
METHODS
Study participants electing ESS for CRS were enrolled and randomly selected in equal numbers from three academic clinical practices in North America between April, 2011 and May, 2015. The magnitude of average 6-month postoperative improvement in patient-related outcome measures (PROMs) was compared between enrollment sites using multivariate linear regression modeling.
RESULTS
A total of 228 participants met inclusion criteria and were included for final analyses (n=76 per site). The prevalence of septal deviation/septoplasty and oral corticosteroid dependent conditions was significantly different between enrollment sites (p≤0.004). Each enrollment site generated significant within-subject improvement across all PROMs after ESS (p<0.001), however, average unadjusted magnitudes of improvement were significantly different between sites for the primary outcome measure. After controlling for baseline PROMs, septal deviation, steroid dependent conditions, and medication use variables, enrollment site was no longer associated with significant outcome differences (p=0.535).
CONCLUSION
Comparison of surgeon outcomes of ESS is feasible and must take into account a number of baseline patient characteristics. Further studies will be critical toward developing an ESS-specific risk-adjustment model and enabling a robust comparison of surgeon outcomes.
MeSH Key Words: Outcome Assessment (Health Care), Patient Outcome Assessment, Surgeons, Quality Improvement, Sinusitis, Quality of Life
INTRODUCTION
As value-based healthcare and quality improvement initiatives become more entrenched in modern medical culture, numerous systems of collecting and describing patient-reported outcome measures (PROMs) have emerged across medical practices and specialties.1–3 While PROMs are a well-known tool to evaluate both patient safety and satisfaction, they can also be used to measure processes of care, health service delivery, and institutional performance. Previous findings using PROMs have, in fact, shown that outcome measures among surgeons and institutions can be highly variable for procedures such as: abdominal aortic aneurysm repair, carotid endarterectomy, cancer surgery, pancreatic resection, and laparoscopic Nissen fundoplication.4–8 For endoscopic sinus surgery (ESS) for chronic rhinosinusitis (CRS), a change or improvement in disease-specific, quality-of-life (QOL) has been adopted as a primary PROM of interest for this patient population.9–11 In support of this guideline, single and multi-institutional studies using a variety of patient-based outcomes for ESS have been widely published in the last decade.12–18 A comparison of ESS outcomes for CRS between institutions and/or surgeons is, however, currently lacking and will be a necessary component of value-based healthcare for the practice of Rhinology.
The objective of this investigation was to evaluate ESS outcomes for CRS between several academic medical centers to determine if differences in QOL outcomes could be detected. In addition, we sought to identify factors that might impact outcomes that would need to be considered in the development of an ESS-specific risk-adjustment model so that outcomes may be appropriately compared between institutions and health care providers.
MATERIALS and METHODS
Study Population and Inclusion Criteria
Study participants consisted of adult patients (≥18 years of age) referred to tertiary surgeons with a confirmed diagnosis of medically refractory CRS. Diagnostic criteria for CRS consisted of guidelines endorsed by the American Academy of Otolaryngology.19 Study participants elected to pursue ESS following unsuccessful medical regimens including, but not limited to, at least 1 course (>14 days) of culture-directed or broad-spectrum antibiotics, at least one trial of either topical corticosteroids (> 21 days) or a 5 day course of oral corticosteroid therapy, and daily saline irrigations (>21 days). These treatment criteria were develop based on a consensus of expert opinion of the co-authors prior to enrollment.
Participants provided written, informed consent in English and agreed to complete preoperative and postoperative study evaluations per the study protocol. An academic, Institutional Review Board (IRB) provided study approval with continuing annual reviews with each enrollment center. Study participants were prospectively enrolled, observed, and evaluated through the standard of care surrounding ESS for CRS for 6 months postoperatively. Study participation was voluntary and did not alter the standard of care.
Surgical Intervention
Surgical approach was directed by the enrolling clinician and guided by symptomatic, radiographic, and endoscopic indications of disease severity. Endoscopic sinus surgery procedures consisted of maxillary antrostomy, partial or total ethmoidectomy, sphenoidotomy, or frontal sinusotomy procedures, with septoplasty and inferior turbinate reductions as adjunctive procedures as determined necessary by the treating surgeon. All surgical cases were followed with appropriate evidence-based postoperative therapeutic regimens including daily nasal saline rinsing, topical corticosteroid therapy, and other appropriate medical therapies as determined necessary by the treating surgeon.20
Clinical Measures of Disease Severity
Radiographic and endoscopic diagnostic evaluations of disease severity were collected during preoperative clinical assessments and utilized for study objectives. High resolution computed tomography (CT) was utilized to evaluate sinonasal disease severity using 1.0mm contiguous images in both sagittal and coronal planes. Images were staged by each on-site enrolling surgeon in accordance with the Lund-Mackay scoring system.21
The paranasal sinuses were evaluated bilaterally using rigid, fiberoptic 0–30° endoscopes by each on-site enrolling physician. Endoscopic exams were staged in accordance with the Lund-Kennedy scoring system.22 Endoscopy scores were also collected during 6-month clinical follow-up.
Disease-Specific, Patient-Reported Outcome Measures
The primary PROM of interest to this investigation was the 22-item SinoNasal Outcome Test (SNOT-22), a validated instrument developed to quantify symptom severity associated with sinonasal conditions (©2006, Washington University, St. Louis, MO).23,24 Individual item scores are measured using patient selected Likert scale responses where higher scores indicate worse symptom severity as follows: 0= ‘No problem’; 1=‘Very mild problem’; 2=‘Mild or slight problem’; 3=‘Moderate problem’; 4=‘Severe problem’; 5=‘Problem as bad as it can be’. Higher total scores on the SNOT-22 suggest worse patient functioning or symptom severity (total score range: 0–110). The 22-items of the SNOT-22 survey have previously been categorized and summarized into 5 distinct domains including: rhinologic symptoms (score range: 0–30), extra-nasal rhinologic symptoms (score range: 0–15), ear / facial symptoms (score range: 0–25), psychological dysfunction (score range: 0–35), and sleep dysfunction (score range: 0–25) as previously described.25 The minimal clinically important difference (MCID) for SNOT-22 total scores has been previously described as a within-subjects improvement of at least 8.9 points.23
The secondary outcome of interest was the Rhinosinusitis Disability Index (RSDI), a 30-item survey instrument developed to quantify symptom severity associated with CRS in a complimentary fashion. The RSDI consists of 3 domains which evaluate the impact of CRS on a respondent’s physical (range: 0–44), functional (range: 0–36), and emotional (range: 0–40) domains. Individual item scores are measured using patient selected Likert scale responses where higher scores indicate worse symptom severity as follows: 0= ‘Never’; 1=‘Almost never’; 2=‘Sometimes’; 3=‘Almost always’; and 4=‘Always’. Higher total scores on the RSDI suggest worse patient functioning or symptom severity (total score range: 0–120).26 The MCID for RSDI total scores have previously been defined as a within-subjects improvement of at least ½ standard deviation (SD) of the preoperative mean score.14,27 Each on-site enrolling surgeon was blinded to survey responses for the entire study duration.
Medication Use Measures
Participants were asked to provide additional information regarding perioperative medication use for treatment of CRS specifically. At both preoperative and 6-month postoperative evaluations, patients recalled the number of days (out of the previous 90 days) they utilized medications for the treatment of their sinus disease including: Topical nasal steroid sprays (e.g., fluticasone, mometasone, etc); Topical nasal steroid drops / irrigations (e.g., prednisolone drops, budesonide respules, etc); Decongestants; Oral / systemic antibiotics; Oral / systemic corticosteroids (e.g. prednisone, methylprednisolone); Antihistamines; Leukotriene modifiers (e.g. montelukast, zafirlukast, etc); Saline irrigation rinses.
Study Exclusion Criteria
Study participants were excluded from final analyses if they did not complete all preoperative enrollment procedures and evaluations or had not yet completed the 6-month postoperative follow-up time period.
Data Management and Statistical Analysis
Protected health information was removed and study data was safeguarded using unique study identification (ID) number assignments for each participant. Study data was securely transferred to Oregon Health & Science University from each enrollment site for manual entry into a password protected relational database (Access, Microsoft Corp, Redmond, WA). All statistical analyses were completed using commercially available software (SPSS v.22, IBM Corp., Armonk, NY). In order to avoid enrollment bias by treating surgeons, blinding of all study data to provider/institution was performed and maintained throughout the entire project, including all co-authors. All unique study ID numbers were removed and each patient entry received alternative random number assignment. In order to blind all investigators and statisticians, each institution was reassigned a unique site ID designation by personnel not associated with this investigation. The site with the lowest number of participants with appropriate follow up yielded a sample of n=76. Therefore, n=76 subjects were then randomly chosen from each of the other sites for final analysis.
Data was evaluated descriptively while normality or skewness was verified for all ordinal or continuous measures. Patient characteristics, comorbid status, and surgical procedures were all compared for omnibus differences across enrollment sites using either one-way analysis of variance (ANOVA; F-test), Kruskall-Wallis, or Chi-square (χ2) testing, with two-sided bivariate multiple comparison testing where appropriate. Further ANOVA testing was used to compare differences in mean preoperative and postoperative QOL scores between enrollment sites. Two-sided matched pairs t-testing was utilized to identify significant within-subjects improvement in QOL across each enrollment site. Test statistics were provided for all normally distributed data where appropriate.
Simple stepwise linear regression was used to identify significant independent risk factors associated with 6-month postoperative mean improvement (postoperative scores minus preoperative scores) for SNOT-22 and RSDI total scores when appropriate. Preliminary models included Enrollment Site as the main exposure variable of interest and all other independent factors screened for univariate significance (p<0.250). A total of 25 variables were additionally screened for univariate, independent significance including preoperative patient characteristics, preoperative QOL status, and 6-month postoperative medication use cofactors. Final models used a manual forward selection (p<0.100) and backwards elimination (p<0.050) process and multi-collinearity was evaluated using variance inflation factors (VIFs). Significant covariates were introduced into final models to assess potential confounding of the effect estimate for enrollment site difference. Any covariate resulting in an absolute change of ±10% in the effect estimate value for the site variable was considered a confounder.28,29 Unadjusted and adjusted regression coefficients (β), standard errors (SE), 95% confidence intervals, and estimates of type-I error (p-values) are reported. The percentage of final model variance explained by each model was calculated using coefficients of multiple determination (R2).
RESULTS
Final Cohort Characteristics
Final analysis was conducted on a cohort of 228 study participants (76 patients from each site) with CRS who met inclusion criteria, completed all preoperative study requirements, and underwent ESS between April, 2011 and May, 2015. Final cohort characteristics and preoperative clinical measures of disease severity are delineated across enrollment sites in Table 1. Important baseline factors were similar among the sites including: age, gender, prior sinus surgery, polyposis, aspirin sensitivity, allergy, depression, and tobacco abuse. The prevalence of both septal deviation and oral corticosteroid dependent conditions was significantly different between enrollment sites (both p<0.004). Clinical measures of disease severity (CT and Endoscopy scores) were also found to be significantly different between enrollment sites (p≤0.006; Table 1).
Table 1.
Comparison of preoperative patient characteristic between enrollment sites (n=228)
| Enrollment Site #1 (n=76) | Enrollment Site #2 (n=76) | Enrollment Site #3 (n=76) | ||
|---|---|---|---|---|
| Patient Characteristics: | Mean [SD] / N(%) | Mean [SD] / N(%) | Mean [SD] / N(%) | p-value |
| Age (years) | 53.7 [16.9] | 48.6 [14.9] | 52.8 [15.6] | 0.103 |
| Males | 32 (42%) | 29 (38%) | 34 (45%) | 0.710 |
| Previous Sinus Surgery | 41 (54%) | 40 (53%) | 36 (47%) | 0.692 |
| Septal deviation | 12 (16%) | 33 (43%) | 41 (54%) | <0.001 |
| Nasal polyposis | 23 (30%) | 25 (33%) | 30 (40%) | 0.468 |
| Asthma | 24 (32%) | 34 (45%) | 22 (29%) | 0.092 |
| ASA sensitivity | 7 (9%) | 6 (8%) | 2 (3%) | 0.223 |
| Allergy (history) | 20 (26%) | 17 (22%) | 16 (21%) | 0.726 |
| Allergy (testing) | 40 (53%) | 31 (41%) | 29 (38%) | 0.160 |
| Depression (history) | 6 (8%) | 15 (20%) | 13 (17%) | 0.099 |
| Current smoker | 2 (3%) | 3 (4%) | 3 (4%) | 0.878 |
| Current alcohol consumption | 38 (50%) | 31 (41%) | 42 (55%) | 0.219 |
| Ciliary dysfunction / CF | 4 (5%) | 2 (3%) | 3 (4%) | 0.707 |
| Corticosteroid dependency | 13 (17%) | 8 (11%) | 1 (1%) | 0.004 |
| Diabetes mellitus (Type I/II) | 6 (8%) | 3 (4%) | 9 (12%) | 0.196 |
| CT total scores | 10.9 [5.5] | 10.2 [5.8] | 13.3 [6.6] | 0.006 |
| Endoscopy total scores | 5.4 [3.6] | 5.0 [4.0] | 7.1 [3.5] | 0.002 |
N, sample size; SD, standard deviation; ASA, acetylsalicyclic acid (aspirin); CF, cystic fibrosis; CT, computed tomography; p-values are reflective of omnibus test results and reflect significant differences between at least two independent enrollment sites.
Baseline Quality of Life, Medication Use, and Surgical Procedures Performed Between Sites
Average preoperative scores for each disease-specific QOL outcome measure were compared between enrollment sites (Table 2). Average, preoperative QOL status was comparable between all sites with the exception of worse sleep dysfunction domain scores as measured by the SNOT-22 survey (p=0.009). Mean preoperative days of medication usage for sinus disease were additionally compared between enrollment sites (Table 3). Study participants used similar durations of preoperative medications across enrollment sites with the exception of variations in exposure to decongestants (p=0.004) and saline irrigation rinses (p=0.021).
Table 2.
Comparison of mean preoperative disease-specific QOL scores between enrollment sites
| Preoperative disease-specific QOL scores: | Enrollment Site #1 (n=76) | Enrollment Site #2 (n=76) | Enrollment Site #3 (n=76) | ||
|---|---|---|---|---|---|
| Mean [SD] | Mean [SD] | Mean [SD] | F-test | p-value | |
| SNOT-22 total scores | 49.4 [22.1] | 50.5 [17.1] | 55.2 [20.3] | 1.83 | 0.162 |
| SNOT-22 Rhinologic domain | 15.9 [6.3] | 15.9 [5.7] | 16.4 [6.6] | 0.21 | 0.815 |
| SNOT-22 ExtraNasal domain | 8.9 [3.6] | 8.6 [3.4] | 7.9 [3.8] | 1.44 | 0.240 |
| SNOT-22 Ear/Facial domain | 8.5 [5.3] | 9.0 [4.7] | 10.1 [5.6] | 1.75 | 0.176 |
| SNOT-22 Psychological domain | 14.1 [9.8] | 14.8 [8.4] | 16.7 [7.8] | 1.84 | 0.161 |
| SNOT-22 Sleep dysfunction domain | 12.1 [7.5] | 12.3 [6.2] | 15.2 [6.7] | 4.81 | 0.009 |
| RSDI total scores | 44.6 [26.5] | 43.1 [22.8] | 46.9 [24.4] | 0.46 | 0.631 |
| RSDI Physical domain | 18.4 [9.7] | 16.9 [7.8] | 19.1 [9.0] | 1.20 | 0.303 |
| RSDI Functional domain | 14.6 [9.1] | 14.3 [8.8] | 15.0 [8.9] | 0.11 | 0.899 |
| RSDI Emotional domain | 11.7 [9.6] | 11.8 [8.7] | 12.8 [8.5] | 0.34 | 0.712 |
N, sample size; SD, standard deviation; QOL, quality of life; SNOT-22, 22-item SinoNasal Outcome Test; RSDI, Rhinosinusitis Disability Index; p-values are reflective of omnibus test results and reflect significant differences between at least two independent enrollment sites.
Table 3.
Comparison of mean preoperative medication use days (of previous 90) between enrollment sites
| Preoperative medication use (days of previous 90): | Enrollment Site #1 (n=76) | Enrollment Site #2 (n=76) | Enrollment Site #3 (n=76) | |
|---|---|---|---|---|
| Mean [SD] | Mean [SD] | Mean [SD] | p-value | |
| Topical nasal steroid sprays | 36.8 [37.5] | 40.3 [39.2] | 46.3 [37.8] | 0.224 |
| Topical nasal steroid drops/irrigations | 7.8 [22.3] | 12.4 [28.0] | 8.3 [25.6] | 0.233 |
| Decongestants | 16.2 [29.3] | 18.4 [31.3] | 29.6 [36.1] | 0.004 |
| Oral / systemic antibiotics | 22.5 [25.3] | 13.9 [19.1] | 16.3 [22.5] | 0.083 |
| Oral / systemic corticosteroids | 14.1 [24.1] | 13.4 [23.9] | 10.9 [17.8] | 0.648 |
| Antihistamines | 24.9 [36.6] | 27.4 [38.3] | 31.1 [37.0] | 0.317 |
| Leukotriene modifiers | 14.7 [31.4] | 16.7 [33.4] | 12.2 [39.4] | 0.838 |
| Saline irrigation rinses | 34.0 [36.6] | 50.0 [37.0] | 43.3 [36.8] | 0.021 |
N, sample size; SD, standard deviation; p-values are reflective of omnibus test results and reflect significant differences between at least two independent enrollment sites.
The frequencies of surgical procedures between enrollment sites are described in Table 4 and are similar with the exception of septoplasty which was performed significantly more commonly at enrollment sites #2 and #3 relative to enrollment site #1.
Table 4.
Comparison of surgical procedures between enrollment sites
| Procedures: | Enrollment Site #1 (n=76) | Enrollment Site #2 (n=76) | Enrollment Site #3 (n=76) | ||
|---|---|---|---|---|---|
| N (%) | N (%) | N (%) | χ2 (df=2) | p-value | |
| Maxillary antrostomy | 70 (92%) | 73 (96%) | 74 (97%) | 2.48 | 0.289 |
| Partial ethmoidectomy | 16 (21%) | 8 (11%) | 19 (25%) | 5.56 | 0.062 |
| Total ethmoidectomy | 54 (71%) | 63 (83%) | 59 (78%) | 3.04 | 0.219 |
| Sphenoidotomy | 52 (68%) | 49 (65%) | 55 (72%) | 1.10 | 0.578 |
| Middle turbinate resection | 12 (16%) | 17 (22%) | 11 (15%) | 1.88 | 0.391 |
| Inferior turbinate reduction | 13 (17%) | 20 (26%) | 20 (26%) | 2.41 | 0.300 |
| Septoplasty | 14 (18%) | 36 (47%) | 33 (43%) | 16.18 | <0.001 |
| Frontal sinusotomy | 53 (70%) | 50 (66%) | 45 (59%) | 1.89 | 0.389 |
N, sample size; χ2, Chi-square test statistic; df, degrees of freedom
Postoperative Improvements in Quality of Life Between Sites
Unadjusted, mean improvement in each QOL measure was evaluated between all three enrollment sites to compare magnitudes of 6-month postoperative improvement (Table 5). All enrollment sites reported significant postoperative improvement 6 months after ESS across QOL measures (p<0.001). The magnitude of mean improvement was found to be significantly different between enrollment sites for some QOL outcomes including the primary outcome SNOT 22 (−20.5 vs. −26.8 vs. −29.1; p=0.027).
Table 5.
Unadjusted comparisons of mean postoperative improvement in disease-specific QOL scores between enrollment sites (n=228)
| Absolute mean improvement in disease-specific QOL scores: | Enrollment Site #1 (n=76) | Enrollment Site #2 (n=76) | Enrollment Site #3 (n=76) | ||
|---|---|---|---|---|---|
| Mean [SD] | Mean [SD] | Mean [SD] | F-test | p-value | |
| SNOT-22 total scores | −20.5 [22.7]* | −26.8 [17.4]* | −29.1 [20.4]* | 3.66 | 0.027 |
| SNOT-22 Rhinologic domain | −7.0 [8.1]* | −8.4 [5.6]* | −9.2 [7.3]* | 1.76 | 0.174 |
| SNOT-22 ExtraNasal domain | −3.9 [4.5]* | −4.1 [3.5]* | −4.1 [3.7]* | 0.07 | 0.929 |
| SNOT-22 Ear/Facial domain | −3.6 [5.1]* | −5.1 [4.3]* | −6.1 [4.5]* | 5.48 | 0.005 |
| SNOT-22 Psychological domain | −5.2 [8.6]* | −8.0 [7.9]* | −8.3 [8.0]* | 3.21 | 0.042 |
| SNOT-22 Sleep dysfunction domain | −4.6 [6.4]* | −6.4 [6.3]* | −6.9 [6.9]* | 2.60 | 0.076 |
| RSDI total scores | −18.8 [20.7]* | −21.5 [20.2]* | −26.4 [22.7]* | 2.52 | 0.082 |
| RSDI Physical domain | −7.7 [8.6]* | −8.3 [8.1]* | −11.9 [9.5]* | 5.35 | 0.005 |
| RSDI Functional domain | −6.6 [8.2]* | −7.4 [7.6]* | −8.3 [8.2]* | 0.86 | 0.426 |
| RSDI Emotional domain | −4.5 [6.8]* | −5.8 [7.2]* | −6.1 [7.1]* | 1.14 | 0.323 |
N, sample size; SD, standard deviation; QOL, quality of life; SNOT-22, 22-item SinoNasal Outcome Test; RSDI, Rhinosinusitis Disability Index;
indicates significant within-subjects improvement over time using matched pairs t-testing (p<0.001).
Comparison of Postoperative Medication Use Between Sites
Average days of postoperative medication use (out of the previous 90) for treatment of sinus disease were also compared between enrollment sites in a similar fashion (Table 6). Significant differences were found in topical steroid use (sprays, drops/irrigations) postoperatively suggesting enrollment site #1 either prescribed these less frequently, or patients enrolled at that site were less compliant with prescribed therapy.
Table 6.
Unadjusted comparisons of mean 6-month postoperative medication use days (out of the previous 90) between enrollment sites
| Postoperative medication use (days of previous 90): | Enrollment Site #1 (n=76) | Enrollment Site #2 (n=76) | Enrollment Site #3 (n=76) | |
|---|---|---|---|---|
| Mean [SD] | Mean [SD] | Mean [SD] | p-value | |
| Topical nasal steroid sprays | 23.6 [36.7]** | 36.8 [40.0] | 51.1 [41.5] | <0.001 |
| Topical nasal steroid drops/irrigations | 16.6 [33.9] | 32.8 [39.6]* | 23.1 [36.3]** | 0.009 |
| Decongestants | 9.2 [23.1] | 12.2 [28.2]** | 11.3 [26.7]* | 0.665 |
| Oral / systemic antibiotics | 4.7 [10.4]* | 6.1 [12.0]** | 6.3 [17.6]* | 0.497 |
| Oral / systemic corticosteroids | 8.6 [25.2]** | 6.3 [19.6]** | 6.0 [18.9]* | 0.935 |
| Antihistamines | 14.7 [31.2]** | 22.4 [38.0] | 23.9 [37.9] | 0.136 |
| Leukotriene modifiers | 13.4 [30.1] | 16.5 [33.9] | 12.5 [31.1] | 0.418 |
| Saline irrigation rinses | 58.1 [37.9]* | 65.8 [33.9]** | 54.3 [38.3]** | 0.103 |
N, Sample size; SD, standard deviation;
indicates significant within-subjects improvement over time using Wilcoxon Signed Rank testing (p<0.001).
indicates significant within-subjects improvement over time using Wilcoxon Signed Rank testing (p<0.050). Negative mean values reflect overall fewer average days of medication use reported postoperatively.
Linear Regression Modeling
The final unadjusted and adjusted stepwise linear regression model for cofactors independently associated with improvement in total SNOT-22 scores are described in Table 7. Objective disease severity as measured by CT and endoscopy varied across sites but these factors did not impact outcomes and did not require adjustment. The effect estimates on SNOT-22 change scores for baseline QOL, septoplasty, postoperative medication use, and oral corticosteroid dependent condition are described in the order of increasing magnitude. After adjusting for all independent cofactors, enrollment site was no longer a significant variable associated with 6-month postoperative improvement in SNOT-22 scores (p=0.535). Final independent model cofactors were able to explain ~41% of overall model variance while no evidence of multicollinearity between variables was found (VIFs < 2.0).
Table 7.
Unadjusted and adjusted linear regression modeling outcomes for mean postoperative improvement in SNOT-22 total scores
| SNOT-22 Total Score Improvement Model: | |||||
|---|---|---|---|---|---|
| Unadjusted Univariate Model: | β | SE | 95% CI | p-value | R2 |
| Enrollment Site / Location | −4.3 | 1.6 | (−7.5, −1.1) | 0.009 | 0.029 |
| Final Adjusted Model: | |||||
| Septal deviation / septoplasty | −6.9 | 2.4 | (−11.7, −2.2) | 0.004 | |
| Enrollment Site / Location | −0.9 | 1.4 | (−3.7, 1.9) | 0.535 | |
| Preoperative SNOT-22 total scores | −0.6 | 0.1 | (−0.7, −0.5) | <0.001 | |
| Antihistamine use days | 0.1 | 0.03 | (0.01, 0.1) | 0.021 | |
| Oral / systemic antibiotic use days | 0.2 | 0.1 | (0.1, 0.4) | 0.004 | |
| Corticosteroid dependent conditions | 8.8 | 4.0 | (0.9, 16.7) | 0.029 | 0.411 |
SNOT-22, 22-Item SinoNasal Outcome Test; SE, standard error; CI, confidence interval; R2, coefficient of multiple determination; Final models screened all patient characteristic variables, baseline measures of disease severity, and 6-month postoperative medication use (average reported days) variables. Negative effect estimates (β) are associated with greater postoperative improvement in SNOT-22 total scores.
DISCUSSION
These data show that clinically meaningful improvements in QOL, following ESS for CRS, occurred in all three centers we evaluated. However, without adjustment for multiple cofactors, there was a statistically significant difference in improvement between the three enrollment centers, with a ~20 point improvement in SNOT-22 total scores on the low end and ~30 point improvement on the higher end on average. While the baseline characteristics of patients treated at each of the centers were relatively similar, important predictors of improvement were identified including baseline SNOT-22 total scores and the prevalence of oral steroid dependent comorbidity. Due to the fact that the effect estimate for the main exposure variable was confounded by other cofactor differences, inclusion of baseline/preoperative QOL scores was warranted and improved overall model efficiency.30 In addition, important and significant differences in unadjusted, average SNOT-22 total score improvements were found between enrollment sites for patients with baseline septal deviation and the subsequent performance of concurrent septoplasty.
In final regression models, mean differences in outcomes between enrollment sites became non-significant after accounting for differences in baseline QOL, oral steroid dependent diseases, and performance of concurrent septoplasty. In addition to comparing postoperative differences in mean SNOT-22 improvement scores, the unadjusted prevalence of participants achieving at least one MCID in disease-specific QOL scores was different between enrollment sites. A significantly lower prevalence (p=0.013) of improvement was reported for enrollment site #1 for SNOT-22 total scores compared to enrollment sites #2 and #3 (68% vs. 87% and 83%, respectively). These data demonstrate proof of concept that differences in QOL outcomes of ESS for CRS can be detected between centers, using a variety of analytical approaches, and important factors must be considered when interpreting such a comparison including clinical and treatment differences between the sites.
Baseline characteristics that have been previously implicated in potentially reducing improvement in PROMs following ESS include prior ESS, ASA intolerance, depression, psychological distress/anxiety, and tobacco abuse.14,31–36 All of these factors were evaluated as part of this investigation but were found to occur at similar prevalence between enrollment sites. In future studies that include non-tertiary sites where some of these factors will likely occur with differing frequencies, these potential predictive factors will need to be carefully considered in any comparison of outcomes and adjusted for if necessary.
We did not anticipate a difference in the rate of septal deviation or the performance of concurrent septoplasty between the sites. Even more surprising was the apparent impact of septoplasty on outcomes (Table 8). Some studies have suggested that concurrent septoplasty does not associate with treatment outcomes of ESS.37 It is clear to all who perform this surgery that the decision to perform concurrent septoplasty is based on several factors including extent of deviation, potential impact on access to the sinuses for ESS and postoperative debridement, postoperative drug delivery, and baseline patient-specific symptoms including nasal airway obstruction, among others. It is unclear whether performance of septoplasty was based on fundamental differences in surgical philosophy or differences in the prevalence of deviation across centers, but none of the other surgical procedures differed among the centers including procedures considered more advanced. It is also interesting that differences in total SNOT-22 outcomes were driven by differences in the Ear/Facial and Psychological domains of the instrument. While it would seem logical that septoplasty would improve nasal obstruction and therefore the Rhinologic domain, the impact of septoplasty may be more all-encompassing if it improves drug delivery post operatively. The reasons why septal deviation impacts outcomes and indications for performing septoplasty are still unclear and this topic deserves further study.
Table 8.
Unadjusted mean SNOT-22 total score improvement across enrollment site and septoplasty groups
| Enrollment Site: | Septoplasty | No Septoplasty | p-value |
|---|---|---|---|
| #1 (n=76) | −29.5 [17.7] | −18.8 [23.2] | 0.150 |
| #2 (n=76) | −31.8 [17.7] | −23.0 [16.3] | 0.056 |
| #3 (n=76) | −31.5 [18.6] | −26.3 [22.3] | 0.170 |
| ALL (n=228) | −31.3 [18.0] | −21.9 [21.2] | 0.001 |
N, sample size; SNOT-22, 22-Item SinoNasal Outcome Test;
The magnitude of association of oral steroid dependent diseases with reduced SNOT-22 total score improvement deserves further discussion. It is apparent to sinus surgeons that oral steroid dependent sinusitis or asthma would be indicative of a particularly severe form of inflammation that might impact disease-specific outcomes and 16/22 steroid dependent patients, collectively, fell into this category. Unfortunately, the classification of steroid dependence and the requirement for patients to be on chronic steroids for any condition can be arbitrary depending on varying clinician practices. Chronic steroid use will also potentially impact baseline QOL, endoscopy, and CT scores and could potentially modify reported effect estimates. Interestingly, 6 of 22 steroid dependent patients suffered autoimmune diseases. These data suggest that patients with autoimmune diseases that are oral steroid dependent experience less improvement with sinus surgery. Future studies with strict criteria for inclusion of steroid dependent subjects are another area for future research.
Limitations of our study include the inclusion of sites with only fellowship trained, tertiary surgeons which limits the variability in baseline patient factors and the generalizability of these findings. However, this comparison is novel, quite challenging to perform, and a necessary step in today’s healthcare environment. Another limitation is the recall-based medication use measures, which are subject to recall bias. Despite this, critical pre-operative medication use for the treatment of CRS, such as antibiotic and steroid therapy, appeared similar at all three sites. On the other hand, postoperative use of topical steroids (eg. sprays, drops, irrigations) were utilized by patients significantly less at enrollment site #1 suggesting that either surgeons prescribed these less frequently or patient compliance with these medications was reduced at that site. Analysis demonstrated that less postoperative topical steroid was significantly associated with less improvement in SNOT-22 but was clinically irrelevant (i.e. trivial effect size).
Throughout this investigation, the identification of enrollment sites was blinded for all authors and contributors as a condition of entering the study. This was accomplished in order to avoid patient selection bias that might occur if surgeons knew the data would become public. Our objective was not to report outcomes for specific institutions or surgeons, rather to demonstrate that such a comparison is possible and to begin an analysis of confounding factors for which we must account in such comparisons. The future of clinical research consortiums will likely be impacted by the reporting of these data and we have treated these data with considerable sensitivity.
CONCLUSION
Comparison of surgeon outcomes of ESS is feasible and must take into account a number of different baseline characteristics including baseline QOL, septal deviation, and steroid dependent illnesses. Further studies using a variety of sites will be critical to enhancing baseline variability of important patient factors (e.g., history of prior surgery, aspirin intolerance, depression/psychosocial distress) toward developing an ESS-specific risk-adjustment model and enabling a robust comparison of surgeon outcomes.
Footnotes
Level of Evidence:2c
Conflicts of Interest: None to report.
Relevant Financial Disclosures: Timothy L. Smith, Jess C. Mace, Peter H. Hwang, Jeremiah A. Alt, and Zachary M. Soler are supported by a grant for this investigation from the National Institute on Deafness and Other Communication Disorders (NIDCD), one of the National Institutes of Health, Bethesda, MD., USA (R01 DC005805; PI/PD: TL Smith). Clinical trial public registration (www.clinicaltrials.gov) ID# NCT01332136. This funding organization did not contribute to the design or conduct of this study; collection, management, analysis, or interpretation of the data; preparation, review, approval or decision to submit this manuscript for publication. Zachary M. Soler is also supported by another grant from the NIDCD (R03 DC013651; PI/PD: ZM Soler) which is not affiliated with this investigation.
References
- 1.Bryan S, Davis J, Broesch J, et al. Choosing your partner for the PROM: a review of evidence on patient-reported outcome measures for use in primary and community care. Healthc Policy. 2014;10(2):38–51. [PMC free article] [PubMed] [Google Scholar]
- 2.Mapi Research Trust. [Accessed January 18th, 2016];PROQOLID [Searchable database] Available at: http://www.proqolid.org/
- 3.Krumholz HM, Keenan PS, Brush JE, Jr, et al. Standards for measures used for public reporting of efficiency in health care: a scientific statement for the American Heart Association Interdisciplinary Council on Quality of Care and Outcomes Research and the American College of Cardiology Foundation. Circulation. 2008;118(18):1885–1893. doi: 10.1161/CIRCULATIONAHA.108.190500. [DOI] [PubMed] [Google Scholar]
- 4.McPhee JT, Robinson WP, 3rd, Eslami MH, Arous EJ, Messina LM, Schanzer A. Surgeon case volume, not institutional case volume, is the primary determinant of in-hospital mortality after elective open abdominal aortic aneurysm repair. J Vasc Surg. 2011;53(3):591–599. doi: 10.1016/j.jvs.2010.09.063. [DOI] [PubMed] [Google Scholar]
- 5.Feasby TE, Quan H, Ghali WA. Hospital and surgeon determinants of carotid endarterectomy outcomes. Arch Neurol. 2002;59(12):1877–1881. doi: 10.1001/archneur.59.12.1877. [DOI] [PubMed] [Google Scholar]
- 6.Bilimoria KY, Bentrem DJ, Feinglass JM, et al. Directing surgical quality improvement initiatives: comparison of perioperative mortality and long-term survival for cancer surgery. J Clin Oncol. 2008;26(28):4626–4633. doi: 10.1200/JCO.2007.15.6356. [DOI] [PubMed] [Google Scholar]
- 7.Riall TS, Nealon WH, Goodwin JS, Townsend CM, Jr, Freeman JL. Outcomes following pancreatic resection: variability among high-volume providers. Surgery. 2008;144(2):133–140. doi: 10.1016/j.surg.2008.03.041. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Varban OA, McCoy TP, Westcott C. A comparison of pre-operative comorbidities and post- operative outcomes among patients undergoing laparoscopic nissen fundoplication at high- and low-volume centers. J Gastrointest Surg. 2011;15(7):1121–1127. doi: 10.1007/s11605-011-1492-z. [DOI] [PubMed] [Google Scholar]
- 9.Rudmik L, Hopkins C, Peters A, Smith TL, Schlosser RJ, Soler ZM. Patient-reported outcome measures for adult chronic rhinosinusitis: A systematic review and quality assessment. J Allergy Clin Immunol. 2015;136(6):1532–1540. doi: 10.1016/j.jaci.2015.10.012. [DOI] [PubMed] [Google Scholar]
- 10.Benninger MS, Hopkins C, Tantilipikorn P. Measuring outcomes in rhinosinusitis. Am J Rhinol Allergy. 2014;28(3):249–254. doi: 10.2500/ajra.2014.28.4041. [DOI] [PubMed] [Google Scholar]
- 11.Vila PM, Schneider JS, Piccirillo JF, Lieu JE. Understanding quality measures in Otolaryngology-Head and Neck Surgery. 2016;142(1):86–90. doi: 10.1001/jamaoto.2015.2687. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Hopkins C, Rudmik L, Lund VJ. The predictive value of the preoperative Sinonasal Outcome Test-22 score in patients undergoing endoscopic sinus surgery for chronic rhinosinusitis. Laryngoscope. 2015;125(8):1779–1784. doi: 10.1002/lary.25318. [DOI] [PubMed] [Google Scholar]
- 13.Smith TL, Kern R, Palmer JN, et al. Medical therapy vs surgery for chronic rhinosinusitis: a prospective, multi-institutional study with 1-year follow-up. Int Forum Allergy Rhinol. 2013;3(1):4–9. doi: 10.1002/alr.21065. [DOI] [PubMed] [Google Scholar]
- 14.Smith TL, Litvack JR, Hwang PH, et al. Determinants of outcomes of sinus surgery: a multi- institutional prospective cohort study. Otolaryngol Head Neck Surg. 2010;142(1):55–63. doi: 10.1016/j.otohns.2009.10.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Meier JC, Remenschneider AK, D’Amico L, Holbrook EH, Gray ST, Metson R. A new model for collection of clinical outcomes data in patients with chronic rhinosinusitis. Int Forum Allergy Rhinol. 2014;4(5):411–415. doi: 10.1002/alr.21282. [DOI] [PubMed] [Google Scholar]
- 16.Remenschneider AK, D’Amico L, Gray ST, Holbrook EH, Gliklich RE, Metson R. The EQ-5D: a new tool for studying clinical outcomes in chronic rhinosinusitis. doi: 10.1002/lary.24715. [DOI] [PubMed] [Google Scholar]
- 17.DeConde AS, Mace JC, Alt JA, Schlosser RJ, Smith TL, Soler ZM. Comparative effectiveness of medical and surgical therapy on olfaction in chronic rhinosinusitis: a prospective, multi- institutional study. Int Forum Allergy Rhinol. 2014;4(9):725–733. doi: 10.1002/alr.21350. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Alt JA, Smith TL, Schlosser RJ, Mace JC, Soler ZM. Sleep and quality of life improvements after endoscopic sinus surgery in patients with chronic rhinosinusitis. Int Forum Allergy Rhinol. 2014;4(9):693–701. doi: 10.1002/alr.21364. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Rosenfeld RM, Piccirillo JF, Chandrasekhar SS, et al. Clinical practice guideline (update): adult sinusitis. Otolaryngol Head Neck Surg. 2015;152(2 Suppl):S1–S39. doi: 10.1177/0194599815572097. [DOI] [PubMed] [Google Scholar]
- 20.Rudmik L, Soler ZM, Orlandi RR, et al. Early postoperative care following endoscopic sinus surgery: an evidence-based review with recommendations. Int Forum Allergy Rhinol. 2011;1(6):417–430. doi: 10.1002/alr.20072. [DOI] [PubMed] [Google Scholar]
- 21.Lund VJ, Mackay IS. Staging in rhinosinusitis. Rhinology. 1993;31(4):183–184. [PubMed] [Google Scholar]
- 22.Lund VJ, Kennedy DW. Staging for rhinosinusitis. Otolaryngol Head Neck Surg. 1997;117(3 Pt 2):S35–40. doi: 10.1016/S0194-59989770005-6. [DOI] [PubMed] [Google Scholar]
- 23.Hopkins C, Gillett S, Slack R, Lund VJ, Browne JP. Psychometric validity of the 22-item Sinonasal Outcome Test. Clin Otolaryngol. 2009;34:447–454. doi: 10.1111/j.1749-4486.2009.01995.x. [DOI] [PubMed] [Google Scholar]
- 24.Piccirillo JF, Merritt MG, Jr, Richards ML. Psychometric and clinimetric validity of the 20-item Sinonasal Outcome Test (SNOT-20) Otolaryngol Head Neck Surg. 2002;126:41–47. doi: 10.1067/mhn.2002.121022. [DOI] [PubMed] [Google Scholar]
- 25.DeConde AS, Bodner TE, Mace JC, Smith TL. Response shift in quality of life after endoscopic sinus surgery for chronic rhinosinusitis. JAMA Otolaryngol Head Neck Surg. 2014;140(8):712– 719. doi: 10.1001/jamaoto.2014.1045. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Benninger MS, Senior BA. The development of the Rhinosinusitis Disability Index. Arch Otolaryngol Head Neck Surg. 1997;123:1175–1179. doi: 10.1001/archotol.1997.01900110025004. [DOI] [PubMed] [Google Scholar]
- 27.Norman GR, Sloan JA, Wyrwich KW. Interpretation of changes in health-related quality of life: the remarkable universality of half a standard deviation. Med Care. 2003;41:582–592. doi: 10.1097/01.MLR.0000062554.74615.4C. [DOI] [PubMed] [Google Scholar]
- 28.Sullivan LM. In: Essentials of Biostatistics in Public Health. Riegelman R, editor. Boston, MA: Jones and Bartlett Publishers; 2008. p. 190. [Google Scholar]
- 29.Maldonado G, Greenland S. Simulation study of confounder-selection strategies. Am J Epidemiol. 1993;138(11):923–936. doi: 10.1093/oxfordjournals.aje.a116813. [DOI] [PubMed] [Google Scholar]
- 30.Glymour MM, Weuve J, Berkman LF. When is baseline adjustment useful in analyses of change? An example with education and cognitive change. Am J Epidemiol. 2005;162:267–278. doi: 10.1093/aje/kwi187. [DOI] [PubMed] [Google Scholar]
- 31.Robinson JL, Griest S, James KE, Smith TL. Impact of aspirin intolerance on outcomes of sinus surgery. Laryngoscope. 2007;117(5):825–830. doi: 10.1097/MLG.0b013e3180333121. [DOI] [PubMed] [Google Scholar]
- 32.Smith TL, Mendolia-Loffredo S, Loehrl TA, Sparapani R, Laud PW, Nattinger AB. Predictive factors and outcomes in endoscopic sinus surgery for chronic rhinosinusitis. Laryngoscope. 2005;115(12):2199–2205. doi: 10.1097/01.mlg.0000182825.82910.80. [DOI] [PubMed] [Google Scholar]
- 33.Mace J, Michael YL, Carlson NE, Litvack JR, Smith TL. Effects of depression on quality of life improvement after endoscopic sinus surgery. Laryngoscope. 2008;118(3):528–534. doi: 10.1097/MLG.0b013e31815d74bb. [DOI] [PubMed] [Google Scholar]
- 34.Davis GE, Yueh B, Walker E, Katon W, Koepsell TD, Weymuller EA. Psychiatric distress amplifies symptoms after surgery for chronic rhinosinusitis. Otolaryngol Head Neck Surg. 2005;132:189–196. doi: 10.1016/j.otohns.2004.09.135. [DOI] [PubMed] [Google Scholar]
- 35.Steele TO, Mace JC, Smith TL. Does comorbid anxiety predict quality of life outcomes in patients with chronic rhinosinusitis following endoscopic sinus surgery? Int Forum Allergy Rhinol. 2015;5(9):829–838. doi: 10.1002/alr.21543. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Rudmik L, Mace JC, Smith TL. Smoking and endoscopic sinus surgery: does smoking volume contribute to clinical outcome. Int Forum Allergy Rhinol. 2011;1(3):145–152. doi: 10.1002/alr.20045. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Rudmik L, Mace J, Ferguson BJ, Smith TL. Concurrent septoplasty during endoscopic sinus surgery for chronic rhinosinusitis: does it confound outcomes assessment? Laryngoscope. 2011;121(12):2679–2683. doi: 10.1002/lary.22361. [DOI] [PMC free article] [PubMed] [Google Scholar]