Nomograms to predict naming decline after temporal lobe surgery in adults with epilepsy

Abstract

Objective

To develop and externally validate models to predict the probability of postoperative naming decline in adults following temporal lobe epilepsy surgery using easily accessible preoperative clinical predictors.

Methods

In this retrospective, prediction model development study, multivariable models were developed in a cohort of 719 patients who underwent temporal lobe epilepsy surgery at Cleveland Clinic and externally validated in a cohort of 138 patients who underwent temporal lobe surgery at one of 3 epilepsy surgery centers in the United States (Columbia University Medical Center, Emory University School of Medicine, University of Washington School of Medicine).

Results

The development cohort was 54% female with an average age at surgery of 36 years (SD 12). Twenty-six percent of this cohort experienced clinically relevant postoperative naming decline. The model included 5 variables: side of surgery, age at epilepsy onset, age at surgery, sex, and education. When applied to the external validation cohort, the model performed very well, with excellent calibration and a c statistic (reflecting discriminatory ability) of 0.81. A second model predicting moderate to severe postoperative naming decline included 3 variables: side of surgery, age at epilepsy onset, and preoperative naming score. This model generated a c statistic of 0.84 in the external validation cohort and showed good calibration.

Conclusion

Externally validated nomograms are provided in 2 easy-to-use formats (paper version and online calculator) clinicians can use to estimate the probability of naming decline in patients considering epilepsy surgery for treatment of pharmacoresistant temporal lobe epilepsy.

Surgery is an effective treatment option for temporal lobe epilepsy (TLE), but is often associated with postoperative language declines. Dysnomia occurs in up to 60% of adults after dominant temporal lobe resection (TLR)^1–4 and can be observed following a number of surgical approaches.^2,5 Deficits often present as word-finding difficulties and can negatively affect social discourse and reduce quality of life.⁶ A number of risk factors for naming decline have been identified, including dominant-sided surgery,^1,7,8 older age at seizure onset,^1,8–13 better preoperative naming performance,^1,4,14 absence of hippocampal sclerosis,^12,15,16 and older age at surgery.^1,7 Given the elective nature of epilepsy surgery and high prevalence of naming decline following TLR, it is imperative to develop risk models to predict naming outcome and improve patient counseling. However, unlike episodic memory,^17,18 few studies have used multivariable models to predict naming outcome,^1,7,13,15 and there is a need for externally validated models to predict naming outcome for individual patients.

Nomograms are statistical tools that permit simultaneous consideration of numerous, often contradictory, clinical risk factors to make a comprehensive risk assessment for a particular clinical outcome in a particular patient given unique demographic and disease characteristics. These models are widely used for clinical decision-making in other areas of medicine, but have only recently been applied to epilepsy surgery outcomes.¹⁹

The objective of this study was to develop and externally validate models to predict the probability of postoperative naming decline in adults following TLR for pharmacoresistant epilepsy.

Methods

Standard protocol approvals, registrations, and patient consents

This was a retrospective, prediction model development and validation study. All data for the study were obtained from existing institutional review board (IRB)–approved data registries at Cleveland Clinic, Columbia University, Emory University School of Medicine, and University of Washington School of Medicine. The need for informed consent was waived because of use of deidentified data.

Participants

This prediction model development study included data from an IRB-approved neuropsychology registry for older adolescents and adults who underwent epilepsy surgery at Cleveland Clinic between 1990 and 2016. Individuals were included if they met the following criteria: (1) age 16 or older, (2) underwent any type of TLR, (3) completed preoperative and postoperative neuropsychological evaluations that included the Boston Naming Test (BNT),²⁰ (4) no history of prior neurosurgery, and (5) complete data for all predictors of interest.

For model validation, we obtained data from 3 independent epilepsy surgery centers in different regions of the country: Columbia University Medical Center (New York), University of Washington (Seattle), and Emory University School of Medicine (Atlanta). The same inclusion/exclusion criteria were applied to the validation cohort, although patients only needed complete data for predictors included in the final prediction models. Variables used for the validation cohort were coded using the same criteria employed for the development cohort.

Outcome

Naming outcome was assessed using the 60-item BNT,²⁰ a well-validated and widely used measure of visual confrontation naming. Patients are asked to name 60 line drawings (e.g., bed). If they cannot generate the name, a stimulus cue (e.g., a piece of furniture) or a phonemic cue (e.g., it begins with the sound bĕ-) is provided. Items correctly named spontaneously or with a stimulus cue are awarded 1 point for a total of 60 points possible. Patients completed this measure prior to and approximately 6–12 months following TLR as part of clinical neuropsychological evaluations. Change scores were calculated and classified into one of 2 categories (decline or no decline) using established, epilepsy-specific reliable change indices (≥5 raw points; 80% confidence interval [CI]) for clinically meaningful change.²¹ While our primary goal was to develop a model clinicians could use to predict any clinically meaningful decline, we thought it desirable to develop a separate model clinicians could use to predict more severe naming declines possibly indicative of a broader aphasic syndrome. For this second model, outcome was classified using a postoperative change of ≥11 raw points. While somewhat arbitrary, this cut point was selected as it represents more than double the reliable change threshold and is consistent with the cut point used to define moderate to severe naming decline in prior research.¹

Predictors

Ten candidate predictors were considered for model development: sex, education, age at surgery, age at epilepsy onset, duration of epilepsy, side of surgery (dominant or nondominant), presence or absence of hippocampal sclerosis on preoperative MRI, lesional or nonlesional preoperative MRI, hippocampal resection (resected or spared), and baseline BNT raw score. These variables were selected for their known association with naming outcome following TLR and the ease with which they can be obtained in the course of routine presurgical assessment in most epilepsy centers. Several a priori–defined interactions were also assessed: BNT score by side of surgery, duration, hippocampal sclerosis, and nonlesional MRI; and hippocampal sclerosis by hippocampal resection.

Side of temporal lobe surgery (dominant or nondominant) was coded using side of surgery (left or right) and results of available language lateralization procedures (fMRI or Wada testing). In right-handed individuals without a lateralization procedure, left-sided resections were coded as dominant and right-sided resections nondominant. Left-handed individuals without a language lateralization procedure were excluded, as were patients with symmetric bilateral language representation. To ensure methods used for characterizing language lateralization did not overly influence the final models, model-building methods were repeated using only patients with confirmed left language dominance via fMRI or Wada testing.

Statistical analyses

Baseline descriptive statistics were calculated for development and validation cohorts and compared using independent-samples t tests (with Satterthwaite correction for inequality of variances, if warranted), Wilcoxon rank-sum tests, and Pearson χ² tests. Model development methods were identical for the model predicting decline of ≥5 points on the BNT and the model predicting ≥11 points. Best subsets selection was used to determine which predictors to include in final logistic regression models. This technique uses a branch-and-bound algorithm, generating best models for each possible number of predictor variables.²² The global χ² statistic of each candidate model was compared and the final model with the best, most parsimonious fit was selected. Continuous predictors were assessed for linearity. If warranted, variable transformation was performed or models were fit using piecewise linear terms or restricted cubic splines.

The models built from the development cohort were applied to the validation cohort and performance was assessed. The discriminatory ability of each model was evaluated using the concordance (c) statistic. The c statistic, equivalent to the area under the receiver operating characteristic (ROC) curve, ranges from 0.5 to 1.0, with a value of 0.5 indicating that the model performs no better than chance, and a value of 1.0 indicating that the model correctly classifies 100% of patients. External calibration was evaluated by plotting predicted outcomes against observed outcomes and deriving a confidence band, known as the calibration belt.²³ Nomograms were created based on final logistic regression models, providing a visual representation for each model. Online risk calculators were also developed using established methods.²⁴

While seizure outcome was not included as a predictor in our models given it would not be known when predictions are being made, we conducted Pearson χ² tests to examine the relationship between naming and seizure outcomes.

Statistical analyses were conducted using SAS (SAS Institute, Cary, NC) Studio v.3.5, R v.3.3.0 rms, and givitiR packages.^25,26

Data availability statement

The datasets analyzed in the current study are not publicly available because of restricted access, but further information about the datasets is available from the corresponding author on reasonable request.

Results

Model development

A total of 719 patients were included in the development cohort. The validation cohort consisted of 138 patients. Demographic and disease characteristics of each cohort are summarized in table 1. Cohorts were generally similar, although the validation cohort had a lower percentage of Caucasian patients (p < 0.001) and slightly more education on average (p = 0.009).

Table 1.

Characteristics and comparison of development and validation cohorts

Predicting decline of ≥5 points (clinically relevant decline)

Twenty-six percent (n = 187; 88% dominant) of development cohort patients and 30% (n = 41; 93% dominant) of validation cohort patients experienced clinically relevant postoperative naming decline. Descriptive statistics and unadjusted associations between each predictor and naming decline are summarized in table 2. Five variables were included in the final model: side of resection, age at epilepsy onset, age at surgery, sex, and education. Model discrimination and calibration were excellent, with c statistics of 0.83 in the development cohort and 0.81 in the validation cohort. ROC curves are presented in figure 1A and the calibration belt for the model applied to validation cohort is presented in figure 2A. The nomogram is presented in figure 3 and available as an online risk calculator at riskcalc.org:3838/CognitiveAfterEpilepsySurgery/.

Table 2.

Descriptive statistics and unadjusted association between each predictor and postoperative naming decline in development sample

Figure 1. Receiver operating characteristic (ROC) curves.

(A) ROC for predicting clinically meaningful naming decline (≥5 points) and (B) ROC for predicting moderate to severe naming decline (≥11 points). The blue line is the ROC curve for the model in the development cohort and the red line is the ROC curve for the model applied to the validation cohort.

Figure 2. Model calibration.

(A) Calibration of model predicting clinically meaningful naming decline (≥5 points) in the validation cohort after temporal lobe surgery. The model was applied to the validation cohort and predictions generated from the model were plotted against actual patient outcomes. The 80% (light gray) and 90% (dark gray) confidence bands neither approach nor cross the ideal 45° line, indicating that model predictions are reliable estimates of actual risk. Confidence bands are relatively narrow and even throughout, indicating that uncertainty associated with predictions is consistent and minor. (B) Calibration of model predicting moderate to severe naming decline (≥11 points). The 80% (light gray) and 90% (dark gray) confidence bands do not cross the ideal line, but noticeably wider confidence bands at the higher end of possible probabilities indicate greater uncertainty than at the lower end of risk.

Figure 3. Prognostic nomogram to predict clinically meaningful naming decline (≥5 points).

To use the nomogram, locate the patient's position on the scale associated with each predictor. The top axis displays prognostic points. Connect the position on each variable axis with the Points (top) axis to determine the number of points corresponding to the appropriate variable position. Total the points for all variables, then find the appropriate position on the Total points axis and connect it with the associated position on the Risk of decline (bottom) axis to determine the patient's individual risk. For example, a patient who was 15 at epilepsy onset (22 points), had a dominant-sided resection (85 points), was 60 at the time of surgery (21 points), and was female (12 points) with 10 years education (−12 points) would have a total of 128 points and a corresponding approximate 58% risk of clinically meaningful postoperative naming decline.

Model coefficients and odds ratios (ORs) are presented in table 3. When controlling for covariates, those with dominant-sided resections had 14 times the odds of decline (OR 14.07, 95% CI 8.48–23.34, p < 0.001) and every year older a patient was at epilepsy onset yielded a 5% increased odds of decline (OR 1.05, 95% CI 1.03–1.07, p < 0.001). Age at surgery, sex, and years of education were not significantly associated with decline but increased the discriminative ability of the model.

Table 3.

Naming decline models: Model coefficients, odds ratios (ORs), and p values^a

Predicting decline of ≥11 points (moderate to severe decline)

Eleven percent (n = 75; 92% dominant) of the development cohort and 14% (n = 19; 95% dominant) of the validation cohort experienced moderate to severe declines in naming performance following surgery. Descriptive statistics and unadjusted associations are summarized in table 2. Three variables were included in the model predicting moderate to severe decline: side of surgery, age at epilepsy onset, and preoperative BNT score. Model performance was good, with a c statistic of 0.83 in the development cohort and 0.84 in the validation cohort. ROC curves are presented in figure 1B, and the calibration belt for the model applied to the validation cohort is presented in figure 2B. The nomogram is presented in figure 4 and as an online risk calculator at riskcalc.org:3838/CognitiveAfterEpilepsySurgery/.

Figure 4. Prognostic nomogram to predict moderate to severe naming decline (≥11 points).

To use the nomogram, locate the patient's position on the scale associated with each predictor. The top axis displays prognostic points. Connect the position on each variable axis with the Points (top) axis to determine the number of points corresponding to the appropriate variable position. Total the points for all variables, then find the appropriate position on the Total points axis and connect it with the associated position on the Risk of decline (bottom) axis to determine the patient's individual risk. For example, a patient who was 15 at epilepsy onset (22 points) and had a dominant temporal resection (95 points) and a baseline Boston Naming Test (BNT) score of 50 (43 points) would have a total of 160 points and a corresponding approximate 17% risk of moderate to severe postoperative naming decline.

Model coefficients and ORs are presented in table 3. When controlling for covariates, those with dominant-sided resections had 16 times the odds of decline (OR 15.98, 95% CI 6.68–38.26, p < 0.001) and every year older a patient was at epilepsy onset yielded a 5% increased odds of decline (OR 1.05, 95% CI 1.03–1.07, p < 0.001). Baseline BNT score served to increase the discriminative ability of the model.

Predicting decline in subgroups based on confidence in language lateralization

When analyses were re-run using only patients with confirmed left-hemisphere language dominance (n = 382 from development cohort), included variables, coefficients, and discriminatory ability remained largely unchanged for both models, with c statistics of 0.83 and 0.87, respectively. Similar findings were observed when analyses were re-run using only right-handed patients, without a language lateralization procedure (n = 300), with c statistics of 0.86 and 0.85. The models for moderate/severe decline in these analyses had fewer events per variable available given the smaller sample sizes; thus the c statistics for these models may be artificially inflated to some extent.

Relationship to seizure outcome

Seizure freedom was not associated with naming decline.

Discussion

This study developed and externally validated multivariable models to simultaneously account for multiple clinical predictors and estimate the probability of cognitive decline following epilepsy surgery for an individual patient given their unique clinical characteristics. The nomograms reported here provide clinicians with practical, externally validated tools for quick and effortless individualized prediction of postoperative naming outcome after TLR using readily available clinical information.

Consistent with existing literature, side of surgery^1,7,8 and age at epilepsy onset^1,9–12 were the most important variables in predicting postoperative naming decline. Given the hemispheric specialization of the human brain, it is not surprising that naming declines are observed almost exclusively following left-sided (dominant) resections. In fact, the few studies that have examined naming outcome after right-sided (nondominant) temporal resections suggest the base rate of naming decline is quite low (i.e., ≤6%) and may actually reflect unidentified atypical language representation.^1,15,27,28 The relationship between age at epilepsy onset and naming outcome after TLR has also been rather consistent across studies.^1,8–13 Research suggests that patients with early epilepsy onset are more likely to have intrahemispheric language reorganization,^9,29 which may explain the lower risk for naming decline in patients with left-hemisphere language dominance and younger epilepsy onset.

A number of other factors, variably associated with naming outcome in prior studies (i.e., age, sex, education, baseline naming performance),^1,4,7,14,15 increased the discriminative ability of our models. While inclusion of other variables and more detailed variable characterization (e.g., type and extent of resection, imaging results, language mapping, fMRI language asymmetry) may have improved model performance, we intentionally limited our set of predictors to demographic and clinical variables readily available in most surgical centers and coded them in a way that would ensure consistency across sites. This will permit more widespread use of these nomograms with information that is easily ascertainable in the course of clinical care, which we view as a strength of these models.

The model predicting any clinically meaningful decline had excellent discrimination and calibration when applied to the external validation cohort. When presented with 2 patients, one from each outcome group (i.e., decline and no decline), the model would correctly classify patients 81% of the time. The calibration belt indicated that predictions generated by the model are very good representations of actual risk. This model's excellent performance suggests that it is suitable for routine clinical use to predict naming decline after TLR.

While our primary goal was to develop a model to predict any clinically meaningful decline in naming, we also developed a second model to predict more severe naming declines that may be indicative of a broader aphasic syndrome in some patients. Our prior research has indicated that up to 17% of adults who undergo left TLR demonstrate substantial naming declines¹ and that patients who experience objective naming declines endorse subjective functional deficits, particularly when declines are severe (i.e., 20+ points).³⁰ Note that the number of patients who demonstrated moderate to severe postoperative naming declines in this study was relatively small, statistically speaking (n = 75 in development cohort and n = 19 in validation cohort). Individual variability exerts greater influence in small samples and likely affected our model. While discrimination of the model predicting moderate to severe naming decline is excellent (c statistic of 0.83) in the validation sample, confidence bands for prediction are noticeably wider than those of the other model, particularly as the predicted probability exceeds 20% (figure 2B). Because of this greater uncertainty in prediction, this model would benefit from further validation in larger samples to better address suitability and refine model fit.

These models were developed on a large, well-characterized cohort of patients who underwent TLR in the Midwest and were validated in a cohort of patients from 3 epilepsy centers in disparate regions of the United States (East, West, South). Thus, we expect good generalizability to other large US epilepsy centers that use the BNT to assess preoperative naming ability. Future validation studies will be needed to determine if these models have utility for surgical patients in other regions of the world.

There are several study limitations that deserve mention. First, the methods used to characterize language lateralization are relatively simplistic and may not fully represent actual language representation for some patients. While these methods broaden the sample of patients to whom these equations can be applied (e.g., those without a language lateralization procedure), they do not fully account for the variations in language representation commonly observed in patients with epilepsy^31,32 and should not be used in patients with atypical language lateralization. Further, this study only included patients who had undergone open TLR. Results may not generalize to less invasive surgical techniques (e.g., laser ablation), particularly given recent evidence to suggest less language morbidity following such procedures.^33,34 We also included a wide range of surgery types ranging from lesionectomy to standard anterior temporal lobectomy. While past research has failed to find significant differences in naming outcome as a function of resection type,^5,12 additional research in this regard may be useful to explore ways to enhance model performance, particularly as invasive monitoring techniques become more sophisticated and tailored resections/ablations more commonplace. Our models are limited to predicting visual naming outcome as assessed with the 60-item BNT.²⁰ There are other language measures as well as additional neurocognitive and behavioral outcomes to consider in patients undergoing temporal lobectomy. Ideally, we envision a toolbox of nomograms, such as those reported here, that can be used to simultaneously predict multiple outcomes (e.g., seizure, cognitive, mood, quality of life).

Finally, it is important to consider patient goals in the surgical decision-making process. Each individual has his or her own unique set of values that will undoubtedly affect surgical decision-making. For some patients, the chance of seizure freedom may far outweigh any risk of postoperative naming decline. For others, even a small decline in their ability to communicate may result in outcome dissatisfaction despite seizure freedom. Future research is needed to examine patient perspectives and experiences as well as the functional consequences of postoperative cognitive declines and possible relationships with other cognitive changes. A recognition of the many complexities involved in the surgical decision-making process and outcome satisfaction is necessary to develop useful clinical tools to predict the many outcomes that may be important to patients and their families.

This study provides easy-to-use nomograms to predict naming outcome on the BNT in adults considering TLR for treatment of epilepsy. We hope clinicians will find these tools useful in consolidating multiple, often contradictory, risk factors to improve patient counseling regarding individual risk for postoperative naming decline. We view these models as an important first step toward the development of a set of comprehensive, well-validated tools for predicting neurobehavioral outcomes following epilepsy surgery. The use of such tools will improve preoperative decision-making and patient counseling as well as increase clinician confidence in their preoperative risk assessment.

Glossary

BNT

Boston Naming Test

CI

confidence interval

IRB

institutional review board

OR

odds ratio

ROC

receiver operating characteristic

TLE

temporal lobe epilepsy

TLR

temporal lobe resection

Author contributions

Dr. Busch: designed and conceptualized study, collected and interpreted data, drafted/revised the manuscript for intellectual content. O. Hogue: analyzed and interpreted data, compiled tables, designed figures, drafted/revised the manuscript for intellectual content. Dr. Kattan: designed and conceptualized study, interpreted data, revised manuscript for intellectual content. Dr. Hamberger: collected and interpreted data, revised manuscript for intellectual content. Dr. Drane: collected and interpreted data, revised manuscript for intellectual content. Dr. Hermann: interpreted data, revised manuscript for intellectual content. Dr. Kim: collected and interpreted data, revised manuscript for intellectual content. L. Ferguson: collected data, revised manuscript for intellectual content. Dr. Bingaman: interpreted data, revised manuscript for intellectual content. Dr. Gonzalez-Martinez: interpreted data, revised manuscript for intellectual content. Dr. Najm: interpreted data, revised manuscript for intellectual content. Dr. Jehi: designed and conceptualized study, interpreted data, revised manuscript for intellectual content.

Study funding

Primary support for this study was provided by the Cleveland Clinic Epilepsy Center. Contributions to this project from Daniel Drane and Michelle Kim were supported by grants (K02NSO70960, R01NSO88748) received from the NIH/National Institute of Neurological Disorders and Stroke and from Medtronic, Inc. (A12257978FN:1056035).

Disclosure

R. Busch, O. Hogue, M. Kattan, and M. Hamberger report no disclosures relevant to the manuscript. D. Drane's lab receives funds from Medtronic, Inc. to serve as the core center for managing neuroimaging and cognitive analysis of data used in their multisite trial of stereotactic laser amygdalohippocampectomy. B. Hermann, M. Kim, L. Ferguson, W. Bingaman, J. Gonzalez-Martinez, I. Najm, and L. Jehi report no disclosures relevant to the manuscript. Go to Neurology.org/N for full disclosures.

Publication history

Received by Neurology May 22, 2018. Accepted in final form August 17, 2018.