Measuring Outcomes in Nasal Surgery

Many desire outcome measurements in surgery, but few can agree on the measuring tools, and fewer yet desire to be measured. This conflict underlies the quandary of measuring outcomes in any surgical disease process, perhaps more so in the case of surgical procedures that address the form and function of the nose. Nasal procedures that address functional and/or aesthetic concerns—septoplasty, rhinoplasty, nasal valve surgery, turbinoplasty, and septorhinoplasty—are oftentimes so intermingled in their purposes and proposed clinical outcomes that the success of the intervention can be difficult to quantify. Yet the health care and academic environments often demand clear and distinct measurements for comparisons, reimbursements, research purposes, and certifications.

The health care environment for the nasal surgeon today is filled with lengthy preauthorizations, denial of services, confusing coding schemes, and at times conflicting satisfactory outcomes. At the root of the issue is that gold standard outcome measures for nasal surgery remain elusive and imperfect (and it always will be so, to some degree). The purpose of this commentary is not to provide a comprehensive review of the evidence to support or disprove the effectiveness of nasal surgery; rather it is (1) to use nasal surgery as a case illustration of the interrelationships between primary research studies, clinical practice guideline development, and performance measures and (2) to explore how we can strave to make our imperfect measures less imperfect.

Primary research studies—basic science, translational, and clinical—serve as the foundation and provide the substance for clinical practice patterns, guideline development, and performance measures. In a recent systematic review of the literature,¹ the quality of evidence in the existing literature for functional rhinoplasty and/or nasal valve repair was graded at an aggregate level of C (predominantly case series or case reports) by Oxford Centre for Evidence-Based Medicine criteria. The inherent limitations of a surgically managed disease process may pre-clude achieving a level A grade owing to ethical and logistic issues in performing randomized controlled trials. However, improvements in study design are possible with the evolution of standardized outcome measures. Comparison cohort study designs could raise the body of evidence to a grade B.

PATIENT-REPORTED OUTCOME MEASURES

Realities

One of the more difficult and controversial aspects of creating a study is deciding on appropriate and meaningful outcome measures. For patient-reported measures, there appears to be a progression in the literature from ad hoc, simple patient satisfaction questionnaires to the use of quality of life (QOL) measures. As detailed previously,² many relevant and psychometrically validated QOL instruments are already in use, ranging from the disease specific to the general—eg, Nasal Obstruction Septoplasty Effectiveness,³ Rhinoplasty Outcomes Evaluation,⁴ Glasgow Benefit Inventory,⁵ and Derriford Appearance Scales 59⁶ and 24.⁷ The presumption behind the use of all of these instruments is that the impact of the clinical problem of interest on patient QOL is important. Therefore any successful treatment should improve patient QOL, and thus QOL measurement is crucial to any clinical outcomes study. There are some well-performed, unmatched, prospective, observational studies in the literature using QOL measures that demonstrate compelling evidence that our surgical interventions are effective.^3,8,9 Yet we can do more to make these patient-reported measures more powerful and meaningful.

Possibilities

What is most often lacking in QOL studies is a comparison group—historical or contemporaneous. Creation of a meaningful comparison group would raise the study design to a cohort study level. One example would be to compare QOL scores or changes in scores with those found in other disease processes for which studies have used the same QOL scale to measure relative impact on patient-perceived improvement. As our body of literature grows with studies that use QOL measures, we effectively create a portfolio of historical cohorts. Another example would be to compare the QOL findings of a nonsurgical intervention with those of a surgical one (eg, the use of a nasal steroid spray vs nasal valve surgery). The nonsurgical group could be observed for some period to measure effectiveness with QOL scales, and if the medical treatment was found to fail in any case, that patient for whom it failed could cross over to the surgical group.

The next step in the evolution of QOL studies is to make these scores more meaningful and predictive—anchor QOL scores or changes in QOL scores to objective tests or other measures (eg, missed days of work, over-the-counter medication use, and financial impact) and use baseline QOL scores to predict patient outcomes. These types of studies require large numbers of patients and therefore lend themselves best to multi-investigator study design.

OBJECTIVE OUTCOME MEASURES

Realities

The use of objective measures as a surrogate measure for success is not only appealing to clinicians but also necessary as the foundation for potential randomized controlled trials or other interventional studies. However, objective tests such as rhinomanometry and acoustic rhinometry are not universally available or accepted, and their limitations in usefulness and reproducibility have made them less appealing to clinicians. The clinical meaningfulness of some of these objective surrogate measures for outcomes has remained controversial.

In addition, physical examination findings are subjective and have been shown to be vulnerable to examination bias.¹⁰ While observer assessments have been found to be quite sensitive for identifying anatomic deformities, they have low specificity in relation to subjective measures: surgeons tend to see deformities that may be asymptomatic to the patient. The creation or acceptance of a gold standard objective test that is widely available and accepted would create outcome measures that could potentially be used for future comparison cohort studies.

Finally, correlation of objective measures with patient-reported symptoms would help establish the clinical meaningfulness of both of these outcome measures. Presently, existing objective tests that measure nasal airway patency have not proven to be consistently correlative to patient reported symptoms.

Possibilities

One exciting area of research, and perhaps the next frontier in objective measures, is the use of computer-assisted measures ranging from facial anthropometrics to nasal airflow modeling using computational fluid dynamic principles. The ability to quantify, standardize, alter, and analyze digitized facial features through the use of facial anthropometrics would offer the prospect of providing more accurate and reproducible measures.¹¹ Some of this technology is available and is currently being used, but broader use in clinical and academic circles is yet to come. Some of the barriers to this technology relate to costs, access, universal acceptance, and inherent limitations of the available commercial software.

Another promising area of objective measure development relates to the use of bioengineering tools to investigate airflow and air conditioning in the nasal cavity. Using computer-aided design software, anatomically accurate 3-dimensional computational models can be generated from patient-specific digital data captured by computed tomographic scans. Computational fluid dynamic techniques allow for the merger of anatomy with physiology by creating a virtual model of the nasal cavity with computed measures of airflow, heat transfer, and air humidification. Furthermore, the computed nasal geometry can be virtually modified to reflect new patterns of airflow and heat and water vapor transport based on predicted results of proposed surgical techniques, ie, virtual surgery.¹²

Regardless of the creation of better objective measures, correlation with patient-reported QOL, satisfaction, and symptoms are critical for the measure to have relevance and be accepted. Even if the technology is too expensive, cumbersome, or inaccessible at first, creating a gold standard will be important as newer measures arise that can then be tested against the standard.

DEVELOPMENT OF GUIDELINES AND CONSENSUS STATEMENTS

The robustness and strength of the primary studies can be summarized with systematic reviews and meta-analyses (if applicable). Summations of these studies potentially allow for interpretations and recommendations that could form the basis for clinical practice guidelines and clinical consensus statements. In an ideal setting, a clinical practice guideline should be created by a multidisciplinary panel of experts who represent the multidisciplinary nature of the clinical entity and who follow a rigid and standardized protocol to carefully evaluate literature rich in high-level evidence. However, obviously, this scenario is not always possible, and compromises often need to be made. Whatever limitations and drawbacks are known to be present in a set of guidelines or a consensus statement must be made transparent to all.

Realities

Nasal surgeons and their patients are quite aware of the pressures intrinsic to a third-party payer system in which guidelines or policies are cited to deny surgical procedures or other interventions. One example of a common third-party requirement is a medical trial of nasal steroids prior to authorization of septoplasty or septorhinoplasty. In some cases nasal steroids might have good effect. However, it is obvious to both the surgeon and the patient in many cases that use of an expensive nasal steroid spray will only result in the medication running out of the affected nostril as it hits the macerated mucosa of the def1ected septum. The inevitable consequences will be poor outcome, poor patient satisfaction, increased cost, and unnecessary patient discomfort. So, how did such a policy become so universal, and on what body of evidence was it based? The problem is that no literature, to my knowledge, demonstrates that such a treatment plan is unfavorable. In fact, there should be studies in the literature that demonstrate that nasal steroid sprays can he more effective after surgically correcting the deviated septum because the medication can then be deposited where it was designed to be effective. However, to my knowledge, no such study exists in the literature. We need primary studies that address issue-targeted clinical questions that can shape our day-to-day practices before any recommendations can be made at the guideline level.

In addition, our medical societies need to be more active in shaping health care policy and guiding treatment. Third-party payers would welcome input from physicians, and if we do not embrace collaboration, the alternative is that nonphysicians’ policies and rules will continue to be made without the input of those who have the most skill and knowledge to truly help our patients. In this context, the American Academy of Otolaryngology–Head Neck Surgery along with sister societies, including the American Academy of Facial Plastic and Reconstructive Surgery, have formed a panel to develop a clinical consensus statement for nasal valve and functional septorhinoplasty surgery. It will be the first consensus statement developed by the academy, and it is yet a work in progress. A consensus statement method was chosen instead of the larger, more comprehensive clinical practice guideline method owing to the less-than-compelling strength of the primary literature. I serve as the chair of the consensus statement panel, which is composed of respected colleagues in facial plastic surgery, rhinology, and sleep medicine. We hope to have a meaningful document published in early 2010.

Possibilities

The development of a consensus statement is promising, but overreliance on such documents can be problematic. There are intrinsic shortcomings to these documents, including biases and oversimplifications. Also, these statements can become quickly outdated in a rapidly changing health care environment. ^l3 Nevertheless, we must strive to continually improve our current situation for the benefit of our patients. Infrastructure at the society level will need to continually evolve to embrace the creation, development, and updating of all such guidelines or consensus statements. The creation of these documents will serve as the foundation for discussions surrounding coding changes and perhaps performance measures. Nasal surgeons can use these documents in their practices for guidance in patient management as well as to justify management strategies to third-party payers. However, it is still too early to tell if such documents will be useful; the full impact of these documents is yet to be determined. The success of this first nasal valve clinical consensus statement will be monitored, and reaction to it will serve as a guide for future guidelines or consensus statements.

PERFORMANCE MEASURES

Realities

Performance measures tend to make physicians uneasy, and rightfully so. Guidelines and consensus statements can be used to critique the quality of a physician’s care. It is important that such documents (1) be well designed and well written; (2) have only limited biases; and (3) be based on a thorough and systematic review of the primary literature. In some geographic locations and for specific disease processes, pay-for-performance models have been and are still being used, at times based on the best available primary studies or guidelines.

The reality is that nasal surgeons to some degree have the built-in performance measure of market pressure driven solely by patient satisfaction. Our outcomes, whether they are functional airway or external deformity correction, are often readily apparent to our patients, the general public, and our peers. However, quantification of our successfully outcomes is necessary on multiple levels. In addition, certifying boards with maintenance of certification requirements will demand quality and outcome measurements for individual practitioners to maintain their board certifications. The exact details and specifics of the requirements depend on the specialty board and are still evolving. However, at the core of the recuirements will be documentation of quality patient care and successful outcomes. One example of such documentation for an individual practitioner could be the collection and analysis of patient QOL data before and after nasal surgery, demonstrating improvement. This documentation could then be submitted to the certifying boards as part of the overall maintenance of certification process.

Possibilities

Public reporting of outcomes is a reality for some specialties and diseases such as cardiac artery bypass graft surgery. One could argue the validity of some of these measures and reporting structures, but the reality is that such information is becoming more available and accessible. Could nasal surgery outcomes become more publicly available? What measures would be used? As electronic medical records (EMRs) become more commonplace, will there ever be a nationalized EMR system? Could deviations from guidelines or outcomes of nasal surgery be tracked by EMRs? Some of these practices might seem farfetched and exaggerated, but the possibilities exist, and so we must shape outcome measures and reporting structures that have relevance and meaning.

CONCLUSIONS

In conclusion, 1 am hopeful that this commentary serves as a motivation for nasal surgeons to continue to take a lead role in shaping patient care–related decisions in today’s health care environment while keeping an eye on the future landscape. Active and thoughtful contributions at the level of primary research studies, guideline development, and performance measures are needed by all of us who are passionate about nasal surgery and the patients who benefit from our care. We are fortunate to have a deep pool of talented colleagues—researchers and clinicians—across many specialties of medicine. It is time for us to pool our resources to shape the future possibilities into realities that most benefit our patients.