Validity of the flap risk score in predicting nasoseptal flap use after endoscopic transsphenoidal pituitary mass resection

Abstract

Introduction

There remains a paucity of literature examining the decision algorithm for use of nasoseptal flap (NSF) after endoscopic endonasal approaches (EEA) to pituitary adenoma resection. In 2018, we published the first ever flap risk score (FRS) to predict the use of NSF. We present here a validity study examining the FRS as applied to our center.

Methods

A retrospective review was completed of consecutive patients undergoing EEA from January 2015 to March 2021. The sensitivity, specificity, and predictive value of the FRS were calculated. A multivariate logistic model was used to determine the relative weight imaging characteristics in predicting need for NSF. The relative weighting of the FRS was then re-optimized.

Results

A total of 376 patients underwent EEA for pituitary adenoma resection, with 113 (30.1%) requiring NSF. The FRS had a sensitivity and specificity of 43.4% and 94.7%, respectively. Sphenoid sinus extension increased the odds of needing a NSF equivalent to 19 mm of tumor height, as opposed to 6 mm in the original 2018 cohort. The re-optimized model had sensitivity and specificity of 79.6% and 76.4%, respectively.

Conclusion

We present a validity study examining the utility of FRS in predicting the use of NSF after EEA for pituitary adenoma resection. Our results show that while FRS is still predictive of the need for NSF after EEA, it is not as predictive now as it was for its original cohort. Therefore, a more comprehensive model is necessary to more accurately stratify patients’ preoperative risk for NSF.

1. Introduction

Pituitary adenomas are composed of a variety of subtypes categorized primarily by tumor size and presence or absence of hormone production.¹ Pituitary tumors account for approximately 15% of all non-metastatic central nervous system (CNS) tumors.^2,3 Although previously resected microscopically, the endoscopic endonasal approach (EEA) has become standard in many practices due to associated decreased length of stay and lower complication rates.^1,4 However, the EEA comes with its own challenges, including but not limited to potential for carotid artery injury, cerebrospinal fluid (CSF) leak, and need for complex skull base reconstruction via endoscopic approach.^{4, 5, 6} Although various techniques have been described to reconstruct the skull base after EEA, there remains a lack of published data to assist the clinician in predicting those patients that will require this reconstruction preoperatively.^{7, 8, 9}

At this institution, patients who are offered surgical resection of pituitary adenomas are screened by the attending neurosurgeon (K.O.R.) for their likelihood of needing a nasoseptal flap (NSF) for skull base reconstruction, either due to a defect that will be too large or complex to repair or for high volume CSF leak. In 2018, we published the first ever flap risk score (FRS) with tumor height and extension into sphenoid sinus as the primary predictors for need for NSF. Specifically, the FRS was defined as maximal tumor height in millimeters (mm), plus 6 if the tumor extended into the sphenoid sinus. FRS >35 was considered predictive of need for NSF.²

For the skull base repair, the majority of patients undergo abdominal fat graft harvest with multi-layered closure via synthetic dural analogs. Patients in whom NSF reconstruction is certain or likely are referred to otolaryngology preoperatively. If NSF reconstruction is certain, the otolaryngologist performs the sellar exposure, and harvests the NSF at the beginning of the case prior to tumor resection. In all other instances, the neurosurgeon performs the exposure and otolaryngology is called intraoperatively if the decision is made during the case that NSF is necessary. In these instances, the NSF is harvested and positioned after tumor resection is complete. As surgeon experience and patient complexity have grown we sought to re-evaluate our model for a modern patient cohort. Now, 3 years later, we examine this our previously published FRS using a larger cohort of patients undergoing EEA for pituitary mass resection in order to determine the validity of this tool.²

2. Methods

2.1. Patient workflow and NSF decision-making

This study was approved by the University of Alabama Birmingham (UAB) institutional review board. A retrospective review was completed of consecutive patients undergoing EEA for pituitary adenomas from January 2015 to March 2021. The FRS included patients from 2009 until January 2015 but was not published until 2018. The January 2015 timeframe marked the start of the implementation of the FRS in practice at our institution despite its official publication in 2018. All patients underwent EEA by the senior neurosurgeon (K.O.R.). Patients are first evaluated in a multidisciplinary pituitary adenoma clinic. Indications for surgery and study inclusion included those patients with pituitary adenomas who demonstrated radiographic tumor growth, large tumor size with associated visual compromise, apoplexy, or a hyperfunctioning tumor. Patients were excluded if they did not have pituitary adenomas, such as Rathke's cleft cysts, meningiomas or craniopharyngiomas. The preoperative MRI was reviewed by the senior neurosurgeon, and NSF was planned if there is a suspected >2 cm (cm) skull base defect, significant bony erosion of the sella (often evidenced by tumor extension into the sphenoid sinus), or a large tumor with suprasellar extent heralding potential high-volume CSF leak. In the event a NSF is not needed, the neurosurgery team performs the skull base repair. In most cases, a free abdominal fat graft is harvest and placed within sellar defect. A dural onlay graft (Biodesign®, Durepair®, or Surgisis®) covers the fat graft, and finally a sellar buttress (Omnipore®) completes the repair. If a NSF is planned pre-operatively, an otolaryngologist harvests the NSF at the beginning of the case and provides exposure to the sella. If an NSF is not planned pre-operatively, the neurosurgery team performs the exposure and attempts skull base repair with an abdominal fat graft. Intraoperative criteria for need of rescue NSF include the failure to adequately secure the free graft material with sellar buttress under a bony edge on at least 3 sides, a large bony defect (generally >3 cm) or high volume CSF leak. If one or more of these criteria are met, intraoperative consultation with otolaryngology is requested and a rescue NSF is harvested.

2.2. Statistical analysis

Tumor height, extension into the sphenoid sinus, and need for NSF skull base reconstruction were obtained via electronic medical record (EMR) review. The sensitivity, specificity, and predictive values of the FRS in this cohort were calculated. A receiver operating characteristic (ROC) curve was constructed and area under the curve (AUC) was calculated.

Next, multivariate logistic regression was utilized to determine the relative weight of tumor height and sphenoid sinus invasion in predicting need for NSF in this more modern cohort. Again, an ROC curve was constructed with this model and AUC was calculated. A predictive cutoff was determined via Youden Index optimization and the sensitivity, specificity, and predictive values of the re-optimized model were calculated. Statistical analysis performed using Stata SE 16.1 (StataCorp, College Station, TX).¹⁰

3. Results

A total of 376 patients underwent EEA for pituitary adenoma resection between January 2015 and March 2021, of whom 113 (30.1%) required NSF. All surgeries utilized a transsphenoidal transsellar approach to the tumor and were performed by the senior author (K.O.R.). Of the total population, 102 (27.4%) patients had a functional adenoma. Based upon preoperative imaging, 120 (31.9%) patients had obvious tumor extension into the sphenoid sinus and mean tumor height was 23.3 mm (±10.9 mm). The mean tumor extension above the anterior clinoid process was 8.0 mm (±6.9 mm). Intraoperatively, 111 (29.5%) of patients had an apparent CSF leak. The FRS had a sensitivity and specificity of 43.4% and 94.7%, respectively, in this cohort, with positive and negative predictive values of 76.6% and 79.8%, respectively. ROC curve for the original FRS applied to the validation cohort is illustrated in Fig. 1. The AUC for this curve was 0.8102.

Fig. 1.

ROC curve for the original FRS applied to the validation cohort.

Multivariate logistic regression showed that in the validation cohort, sphenoid sinus extension increased the odds of needing a NSF by an amount equivalent to 19 mm of tumor height, as opposed to 6 mm in the original 2018 FRS cohort. However, this regression only explained 27% of variability in need for NSF, as opposed to 43% in the original cohort. Using this new weighting system, a re-optimized FRS (rFRS) was calculated for each patient. By Youden Index optimization, the re-optimized cutoff for predicting need for NSF was rFRS >33. In this cohort, the rFRS had a sensitivity of 79.6% and a specificity of 76.4%, respectively. The positive and negative predictive values were 59.2% and 89.7%, respectively. An ROC curve was also constructed for this model, and is presented in Fig. 2. The AUC for this model was 0.8334.

Fig. 2.

ROC curve for the re-optimized FRS applied to the validation cohort.

4. Discussion

With the steadily increasing number of EEA in the United States and the variety of skull base reconstruction options available, clinicians are in need of a simple, easy to use tool to preoperatively assess the need for skull base reconstruction in a patient with pituitary mass.^1,8,11 The need for NSF is most apparent with large tumors with resultant sellar expansion or erosion into the sphenoid sinus.^2,6 With that in mind, we published the first ever FRS in 2018. In the original model, tumor height and erosion into the sphenoid sinus were predictive of need for NSF.²

Here, we present a validity study applying our FRS to a larger cohort of our patients over a longer clinical time period. When applied to the current cohort, the original FRS had lower sensitivity, but higher specificity. The ability of the FRS to discriminate between patients who did and did not require NSF in this cohort was still adequate, though not perfect, based on the AUC of the ROC curve. The rFRS was calculated using the adjusted weights of tumor height and sphenoid sinus extension. This rFRS had improved sensitivity (79.6%) but lower specificity (76.4%) compared to the FRS. The predictive accuracy of the rFRS was slightly better than the original FRS. In either model the specificity remains >75% with adequate predictive accuracies (AUC >0.800). This suggests the FRS is capable of identifying which patients are most in need of NSF and can aid in the preoperative planning both with patient counseling and the early involvement of otolaryngology. By applying the FRS in practice, surgeons can hopefully minimize the need for intraoperative rescue NSFs which may prolong surgery and provide unanticipated morbidity and discomfort for patients postoperatively whom were not expecting NSF.

Interestingly, multivariate logistic regression of the validation cohort demonstrated that sphenoid sinus tumor extension increased the odds of needing a NSF equivalent to 19 mm of additional tumor height. This is in comparison to the 6 mm value given to sphenoid sinus extension in the 2018 paper.² We postulate that as the experience of the senior author has grown, larger tumor heights have become more manageable in terms of avoiding a CSF leak, which would have previously required NSF reconstruction. However, sphenoid sinus extension and resultant skull base destruction remains a challenging problem in this patient population that almost always requires a NSF.

The logistic regression model that was used to construct the FRS explained 43% of the variability in need for NSF in that cohort. At that time, the authors commented that the less than 50% variability explained by their model suggests “there are factors beyond preoperative imaging that are involved in the decision to use NSF.”² In this validation cohort, only 27% of the variability in need for FRS was explained by the logistic regression. This indicates that even with re-optimization, these two variables are not sufficiently predictive of need for NSF, and suggests that a more comprehensive model is needed to more accurately predict need for NSF preoperatively. Presently, there is still much unexplained variability of our model even when applied to our own institution. The limitations of our previous model are now apparent and highlight the need to further identify the causes of uncertainty. The current study emphasizes the importance of further studying practice patterns of skull base surgeons across institutions prior to re-attempting a new model. In order to determine the various practices and decision-making processes, a survey of anterior skull base surgeons is necessary and is currently underway. The need for a more predictive score is useful such that appropriate preoperative consultation and planning with otolaryngology can be performed. NSF are not without added morbidity and discomfort, at least in the immediate postoperative setting, of which patients should be informed.¹² There has been a growing trend to reduce implementation of NSF except when necessary due to concerns over nasal quality of life. However, to date, no major study has found a long-term quality of life reduction in patients with NSF.^13,14 Additionally, capable and willing otolaryngology colleagues must be available to assist in the necessary postoperative management of NSF patients to reduce morbidity.

There are several limitations our current study. Namely, this is a single center, single surgeon experience. There is a standard workflow at this institution for how the pre-operative or intra-operative decision to use a NSF to reconstruct the skull base is made. This workflow is not uniform across all centers. Additionally, EMR review may not capture more nuanced decision making regarding need for NSF. There may also be unique, intraoperative variables that may play a bigger role in need for NSF than we previously hypothesized. Furthermore, the validity of any single surgeon series is limited by variability in operator experience and technical skill. A survey to understand nationwide practice patterns and a multi-institutional external validity study of a revised model would likely serve to devise a more predictive and generalizable predictive score.

5. Conclusion

As EEA surgeries become more commonplace, it is necessary to predict need for complex skull base repair preoperatively. Although the our previously published FRS is still predictive of the need for NSF after EEA for pituitary adenoma resection, a more comprehensive model is necessary to better stratify patients’ preoperative need for NSF. We plan to better understand the various factors that predict need for NSF via a national survey and subsequently this national data and our growing patient experience to develop a new and improved FRS.

Funding

No financial support has been given.

CRediT authorship contribution statement

Arsalaan Salehani: Conceptualization, Data curation, Writing – original draft, Writing – review & editing. Matthew Parr: Conceptualization, Formal analysis, Methodology, Writing – original draft, Writing – review & editing. Travis J. Atchley: Data curation, Investigation, Methodology, Writing – original draft, Writing – review & editing. Sasha Howell: Data curation, Investigation, Writing – review & editing. Dagoberto Estevez-Ordonez: Formal analysis, Methodology, Validation, Writing – review & editing. Nicholas M.B. Laskay: Data curation, Investigation, Methodology, Writing – review & editing. Kristen Riley: Conceptualization, Project administration, Resources, Supervision, Writing – review & editing.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Abbreviations List

AUC

area under the curve

CNS

central nervous system

CSF

cerebrospinal fluid

EEA

endoscopic endonasal approach(es)

EMR

electronic medical record

FRS

flap risk score

mm

millimeters

NSF

nasoseptal flap

rFRS

re-optimized flap risk score

ROC

receiver operating characteristic

UAB

University of Alabama at Birmingham