The 2017 13^th Annual David W Kennedy, M.D. Lecture: The evolution of outcomes in sinus surgery for chronic rhinosinusitis: past, present, and future

Abstract

This is an edited transcript from the 2017 13th Annual David W Kennedy, M.D. Lecture, presented to the American Rhinologic Society during the 63^rd annual meeting.

LECTURE

I would like to thank the American Rhinologic Society for the great honor and privilege of giving this lecture. I hope my words are worthy of your time and, Dr. Kennedy, worthy of your named lectureship.

I have no industry conflicts of interest. My primary conflict is that I, like many of you, am a sinus surgeon and I view these data through that lens. We should all keep that in mind.

My story begins before the turn of the century when pioneers of endoscopic sinus surgery presented early retrospective case series demonstrating success rates of sinus surgery on the order of 90–95%.¹ Next came the 1990s and what I like to call the quality-of-life (QOL) outcomes revolution. This is the decade that so inspired my interest in outcomes research. We saw the development of validated QOL instruments, prospective data collection, and calculations of cohort mean data. The top graph of Figure 1 demonstrates cohort mean data showing improvement in QOL over time, after surgical intervention. The lower graph in Figure 1 is a “spaghetti plot” of the individual patient data that make up that mean data. We see that most patients generally follow the mean but as you can see, some patients actually reported worse QOL after sinus surgery. We have to remember that our reported means are not representative of the entire population and that outliers do exist. We should probably focus on and learn from those outliers.

Figure 1.

The upper graph shows a “clean” quality-of-life improvement using mean scores. The lower graph is a spaghetti plot demonstrating individual patient variability in outcomes for the same cohort.

Also during the outcomes revolution of the 1990s, we asked the question: Are we interested in the ultimate QOL achieved or with the change in QOL achieved with an intervention?² This concept was nicely demonstrated in patients with chronic rhinosinusitis (CRS) and comorbid depression.^3,4 Patients without depression ultimately achieved a much higher QOL following sinus surgery than patients with depression. However, the magnitude of change or improvement in QOL after surgery was the same in both groups. That is to say that patients with depression experienced just as much improvement in QOL following sinus surgery as patients without depression. They just do so from a different baseline; patients with depression start with a lower QOL. This was a critical distinction and it became clear that it was change in QOL that was the most important outcome to us.

And again, during the QOL outcomes revolution of the 1990s, we faced an emerging dilemma; the lack of association between patient-based measures of disease like QOL, and clinical measures of disease like CT severity in CRS. It turns out that there is not much association between what a patient experiences and what we observe in our clinical testing.⁵ Numerous studies at the time found very low correlation between CT score severity and patient reported symptoms and validated QOL measures.^5,6 I recall one occasion after I gave a lecture about this topic, I was very junior at the time, and a very prominent rhinologist approached me and asked: “Tim, so which one is right, the QOL reported by the patient or the CT scan severity findings? One must be a correct measurement of the disease and the other an incorrect measurement?” Well, first of all, I couldn’t believe he knew my name. After I recovered from that, I explained to him that clinical tests like CT and sinonasal endoscopy, and patient-based QOL instruments are often measuring different aspects of the underlying disease. Sometimes these measures are related and sometimes they are not but they are very seldom equivalent. However, we know that both types of measures/outcomes are important in evaluating treatment effectiveness. For instance, when a patient presents to our office after endoscopic sinus surgery, we fundamentally want to know two things.^7,8 First, how do they feel, what is their quality of life? Second, we want to know what the ethmoid mucosa looks like on sinonasal endoscopy.⁸ We know from our experience that these two measures of disease commonly do not correlate but both of them drive treatment decision making. Both outcomes are actionable pieces of information. We should also understand that this lack of correlation between patient-based measures and clinical tests is not unique to the field of rhinology. In fact, it is ubiquitous in medicine.

After the turn of the century, we saw movement from single institutional studies to multi-institutional cohorts reporting observational outcomes on larger patient populations.⁹ We began to struggle with the concept of statistically significant improvement as compared to clinically significant improvement. In our early outcomes studies, we found statistically significant improvement in many subgroups of CRS, that is, patients who suffer aspirin intolerance, patients who are smokers, or suffer comorbid allergy or asthma, or had prior sinus surgery, all demonstrated statistically significant improvement with p values less than 0.05.¹⁰ But was this improvement meaningful to the patient and how do we define that? And so we asked what is a minimal clinically important difference, an “MCID”, and how do we define that? Once defined, we found that compared to the 90–95% success rates reported in the retrospective literature, that actually 70–75% of patients experienced clinically significant improvement in disease-specific QOL after endoscopic sinus surgery. And once we’ve defined our primary outcome, we can utilize that with a host of patient factors to begin breaking down this syndrome we call CRS into many rarer diseases, let’s call them clusters.^11–13 We can begin to endotype these disease processes and treat the individual patient at the molecular level- and we would enter the area of personalized medicine. But that’s an entirely different lecture for another day, likely to be given by one of you sitting in the audience today. I will look forward to that.

Meanwhile, while we were doing all of this outcomes work, a Cochrane review of endoscopic sinus surgery was released in 2006 that concluded that endoscopic sinus surgery does not confer additional benefit to that obtained by medical treatment. This was a very powerful statement. These conclusions were made based on three studies, none of which compared surgery to medical therapy in patients who had failed an initial trial of appropriate medical management. These conclusions were not generalizable to the patients we actually treat. And this ushered in the era of comparison groups. We recognized that we needed better data that was generalizable to the patients that we actually treat. And so we asked: what really happens to patients who have failed appropriate medical management? We know that some of these patients elect sinus surgery and others elect to continue medical management. And while there are real selection biases at play, we decided to observe these patients prospectively. In a multi-institutional study, we enrolled patients who failed what was considered appropriate medical management.^14–16 To our surprise, half of these patients chose to continue medical therapy while the other half elected endoscopic sinus surgery. Also to our great surprise, both of these groups had similar CT severity at baseline. We were convinced that the patients with more severe disease on CT scan would elect sinus surgery and those with less disease would elect to continue with medical therapy. But it was not so. As it turns out, patients don’t care what their CT looks like; what drove their decision making was their quality of life.^17–21 As we followed these patients over the course of one year, we found that one third of the initial medical cohort chose to “cross over” to endoscopic sinus surgery during that year. And what drove that decision? It turns out it was quality-of-life. In Figure 2, the dashed line represents patients who elected endoscopic sinus surgery from the outset. They began with the lowest level of QOL and ultimately achieved the highest level of QOL and greatest improvement in QOL of any group. The solid line represents patients who initially elected ongoing medical therapy. They started with a substantially higher level of QOL compared to patients who elected surgery, but they did not ultimately achieve the QOL or the improvement in QOL achieved by surgical patients. The dotted line represents the cross over group. These patients experienced an initial deterioration in their QOL evidently driving their decision to crossover and pursue endoscopic sinus surgery. And interestingly, the crossover group did not achieve the degree of improvement in QOL as did patients who originally elected sinus surgery. This finding has spurred a new realm of research regarding the timing of endoscopic sinus surgery and its impact on outcomes.

Figure 2.

Quality-of-life as a function of time in patients who elect sinus surgery, continued medical management, and cross over from continued medical management to sinus surgery.

Despite all of this work, a major third party payer has recently drafted a policy labeling endoscopic sinus surgery “investigational and experimental.” We all know what this means. Thankfully, we have some good data to refute this.

There is a Zen teaching that says something like: the main thing is to keep the main thing, the main thing. And in the realm of CRS, quality of life appears to be the main thing. But it is not be only thing and I would like to talk to you about some other important outcomes and measures in the area of econometrics. Econometrics are critical to health policy decision making. Let’s start with health state utility. This is a value between 0 and 1.0 that we can all place on our current state of health. A score of 0 represents death and a score of 1.0 represents perfect health. In the United States, the mean health utility is 0.81 and the mean health utility of patients with CRS is substantially lower at 0.65.²² A health utility score of approximately 0.65 is also experienced by patients with Parkinson’s disease, moderate asthma, and end-stage renal disease with hemodialysis. It is amazing to see how patients perceive their disease, their suffering, and their quality-of-life. It is equally amazing to see how clinicians perceive disease processes. When I present these data to people outside of this room, they often shake their heads in disbelief. I’m afraid those of us in this room are privy to the understanding of the suffering caused by chronic rhinosinusitis and we need to advocate. Who are we to say how much one can suffer based on the diagnosis we have given them? But I feel like clinicians do that all too often.

Perhaps even more interesting is that we can measure a change in a person’s health utility based on an intervention.²² We know that an MCID in health utility scores is a value of 0.03. We know that primarily endoscopic sinus surgery improves health utility by three times that value, or 0.09 and revision endoscopic sinus surgery improves health utility twice the MCID or 0.06. These improvements in a health utility are similar to those that occur with total joint replacement and coronary angioplasty.

Perhaps most interesting of all is the fact that we can return a patient’s health utility to the US norm utilizing endoscopic sinus surgery followed by continued medical management.²³ This is very powerful information indeed. These are the types of treatments that we should invest in, that is treatments that return health utility to normal values.

We can then convert health utility to quality adjusted life years (QALYs) and use this to inform a cost effectiveness study comparing surgery to the option of ongoing medical therapy.^24–26 It is important to select the appropriate study time horizon because we know that the typical patient entering our clinical trials is 46 years of age. We also know that we do not have a cure for this chronic inflammatory disease and that patients are likely to suffer this disease and require treatment for years and decades to come. Some are likely to experience complications of treatment including not only complications of surgery but also complications of ongoing medical management, for example long-term steroid exposure.²⁷ When we perform such an analysis we find that endoscopic sinus surgery costs less to produce more QALYs than the option of medical management alone. A sensitivity analysis concludes with 95% certainty that endoscopic sinus surgery is the more cost effective option.

And so now we enter the era of quality management where our outcomes become increasingly important. I would first ask you: are you a “high quality” provider? Payers are categorizing us and patient incentives are being applied. For example, one payer defined a “high quality provider” as a clinician who performed one or fewer nasal endoscopy procedures on a patient with chronic rhinosinusitis per year. A “low quality” provider performed more than one nasal endoscopy per patient per year. Patient incentives were applied so that the co-pay to see a “high quality provider” was less than the co-pay to see a “low quality provider.” And we ask ourselves, how can they do this? Well, if we do not rigorously define quality ourselves, others will define it for us, perhaps not so rigorously. We need to engage in quality management.

Some of the initial quality management investigations have examined geographic variation in the rates of surgical procedures.²⁸ We found that endoscopic sinus surgery is performed at a rate of about 1/1000 in the United States, similar to the rate of spine surgery. However, the rates of sinus surgery differ by geographic location. Why is it that you are 3.5× more likely to receive sinus surgery in South Dakota as compared to Vermont, a rate of 1.8/1000 vs 0.51/1000? When health policy makers see this information without a clear explanation, they conclude that perhaps we need more regulation.

The next level of scrutiny in the realm of quality is to examine individual provider data and compare that to some national benchmark. It is imperative to quality improvement. But how do we do this for sinus surgery? We have not yet defined the outcome and broadly collected the data, though we are getting better at this. We have not developed the disease severity index so that we can appropriately compare surgeons in different practice settings. We have not established the time frame for outcome measurement. All of these things are imperative to quality improvement. And so I ask, shall we do it, or shall we have it done for us?

First, to the issue of the appropriate time frame for outcome measurement: how long is long enough? We found that when we followed patients with two different QOL instruments after endoscopic sinus surgery, their QOL improved and stabilized at 6 months postoperatively and remained stable for approximately 2 years after sinus surgery.²⁹ Therefore, we concluded that 6 months was an appropriate and adequate time frame for outcome measurement.

Next, we performed a blinded evaluation of 3 sites in our clinical research consortium looking at ~75 patients per site.³⁰ Our primary outcome was the 22-item SinoNasal Outcome Test (SNOT-22) at 6 months follow-up. Do you think there is a difference in outcomes between surgeons at different tertiary care, academic sites? I did not think there would be, but there was. The “worst” performing surgeon had a mean improvement of ~20 points on the SNOT- 22. The “best” performing surgeon had a mean improvement of ~30 points on the SNOT- 22. I will remind you that the MCID for a SNOT- 22 total score is ~9 points. Therefore, the difference in outcomes between these surgeons on initial evaluation was clinically significant.

We developed risk adjustment models and found that two factors played important roles in differentiating between the surgeons. The higher performing surgeons performed septoplasty with endoscopic sinus surgery more commonly. The lowest performing surgeon had a higher proportion of patients with systemic steroid-dependent conditions. And when we accounted for these two variables, the differences between the surgeons resolved. That is, our risk-adjustment model leveled the playing field and helped make appropriate comparisons. Also this year, we performed an evaluation comparing surgeons in Canada where we examined 5 year revision surgery rates for each surgeon, a different outcome. Interestingly, there were outliers–high performers and low performers– and the same two variables, systemic steroid use and performance of septoplasty informed the risk assessment model. In addition, and not surprisingly, polyposis also impacted revision surgery rates substantially. We need to consider all of these factors as we develop our risk adjustment models.³¹

It appears that comparison of surgeon outcomes is feasible and that the risk adjustment might include: baseline QOL, polyp status, systemic steroid requirements, and the performance of septoplasty as an adjunct to the sinus surgery. We now need to study community sites to further develop our risk adjustment model.

And so in summary, I can say that we have come a long way. Disease-specific QOL has established itself as the primary outcome measure. If you are not collecting this in your clinics, I would advise you to do so and to become more familiar with your own outcomes utilizing these instruments.³² We must remember that QOL, not CT disease severity, drives patient decision making in chronic rhinosinusitis. We, as surgeons, perhaps continue to value the CT severity too highly-we give it too much credit to inform our treatment decision making and to predict our outcomes. I recognize there is a careful balance to be achieved here. I am not advocating that we routinely operate on patients with no evidence of inflammatory disease on the CT scan. I am trying to convey the fact that the data suggest that the information currently supplied to us in the form of sinus CT appears to be over emphasized by clinicians.³³ We should keep this possibility in mind.

We have learned that rather than 90–95% success rates, more realistically, 70–75% of patients will have clinically significant improvement with endoscopic sinus surgery. Patients who have failed appropriate medical management and who elect endoscopic sinus surgery have larger quality of life gains, and fewer future antibiotic/oral steroid exposures than those who elect to continue medical management without surgery.³⁴ We have yet to fully understand whether the timing of sinus surgery impacts these outcomes.

We understand that health utility gains with endoscopic sinus surgery are significant in the greater procedural realm and on par with procedures like total joint replacement and coronary angioplasty. Endoscopic sinus surgery followed by continued medical management appears capable of returning health utility to normal values. When we consider an appropriate time horizon of 30 years, it appears that endoscopic sinus surgery is a cost effective option in patients who have failed appropriate medical management. Clinicians often think of costs and benefits in the near-term. Since we are treating one of the most common chronic diseases to affect humans, we need to get better at thinking long-term with regard to costs and outcomes.

We now enter an area of quality management and value-based care where our outcomes will increasingly determine reimbursement. We desperately need to become more involved in quality measurement and quality management.

Finally, I would like to thank the many co-investigators who have substantially contributed to this work over many years. Perhaps the greatest honor of my career is to watch those who have worked alongside me take this field to a level that I never dreamed of. I’m in no position to offer advice to most of you; the fact is, I’ve sought advice from many of you. But to my junior colleagues in the audience today, I would say if you want to contribute something to this field, figure out what you are interested in, find like-minded people, and collaborate with them. That’s a great first step and puts you at a huge advantage. I’m sure some of you will transcend that advice and become something called a multiplier- that is what I aspire to be. Multipliers are leaders who look beyond their own genius and focus their energy on extracting and extending the genius of others.³⁵ This is what David Kennedy has been for our field over the past 30 years—thank you, David.

Thank you again so much for your time and for this great honor.