Predicting postoperative delirium severity in older adults: The role of surgical risk and executive function

Abstract

Objectives:

Delirium is an important postoperative complication, yet predictive risk factors for postoperative delirium severity remain elusive. We hypothesized that the NSQIP risk calculation for serious complications (NSQIP-SC) or risk of death (NSQIP-D), and cognitive tests of executive function (Trail Making Test A and B [TMTA, TMTB]), would be predictive of postoperative delirium severity. Further, we demonstrate how advanced statistical techniques can be used to identify candidate predictors.

Methods/Design:

Data from an ongoing perioperative prospective cohort study of 100 adults (≥65yo) undergoing non-cardiac surgery were analyzed. In addition to NSQIP-SC, NSQIP-D, TMTA and TMTB; participant age, sex, American Society of Anesthesiologists (ASA) score, tobacco use, surgery type, depression, Framingham risk score, and preoperative blood pressure were collected. The Delirium Rating Scale-R-98 (DRS) measured delirium severity, the Confusion Assessment Method (CAM) identified delirium. LASSO and Best Subsets linear regression were employed to identify predictive risk factors.

Results:

Ninety-seven participants with a mean age of 71.68±4.55, 55% male (31/97 CAM+, 32%) and a mean Peak DRS of 21.5±6.40 were analyzed. LASSO and Best Subsets regression identified NSQIP-SC and TMTB to predict postoperative delirium severity (p<0.001, Adj. R2: 0.30). NSQIP-SC and TMTB were also selected as predictors for postoperative delirium incidence (AUROC 0.81, 95%CI 0.72–0.90).

Conclusions:

In this cohort, we identified NSQIP Risk score for Serious Complications and a measure of executive function, TMT-B, to predict postoperative delirium severity using advanced modeling techniques. Future studies should investigate the utility of these variables in a formal delirium severity prediction model.

Introduction

Delirium, a type of acute brain failure, is a common surgical complication experienced by approximately 50% of patients, incurring an estimated annual U.S. cost of $152 billion.^1–3 It is a crucial public health concern as it is significantly associated with increased mortality⁴ and morbidity in terms of cognitive decline^5,6 and the loss of independence.⁷ A recent study identified an association between delirium severity and subsequent 3-year cognitive decline; the rate of cognitive decline nearly tripled in those that experienced the most severe delirium.⁶ Another study demonstrated that delirium severity was associated with increased risk of in-hospital mortality and discharge to an institutional facility instead of returning home.⁸ Taken together, these studies demonstrate the detrimental impact of high delirium severity on patient outcomes. While there are well-established predictors for delirium incidence, few studies to-date have outlined candidate predictors for delirium severity in older adults, and to our knowledge, a prediction model for postoperative delirium severity is not available.^9,10 Further, prior delirium prediction modeling employed traditional statistical techniques in samples with limited events leading to model overfitting.^9,11–13 One way to overcome this limitation is to use advanced statistical methodologies such as Least Absolute Shrinkage and Selection Operator (LASSO) and Best Subsets Regression.^14–18 These statistical methods show efficiencies in modeling when compared to traditional regression procedures, reduce bias introduced by standard univariate selection, and limit model overfitting.¹⁸ Statistical shrinkage methods such as LASSO are recommended by the Transparent Reporting of a multivariable prediction Model for Individual Prognosis or Diagnosis (TRIPOD) guidelines, which is an evidenced based guide for prediction model development and validation.^19,20

The ability to identify moderate to high-risk individuals prior to their surgery is critical to delirium prevention. To identify potential candidate predictors, we considered the pathogenesis of delirium and sought to identify both predisposing (age, depression, and medical comorbidities) and precipitating (surgery) variables. The Cognitive Disintegration Model²¹ posits that an individual with increasing risk, or vulnerability, to delirium will require less of a precipitating stimulus to cross over the “Delirium Threshold” and become delirious. This is illustrated in the cognitive trajectory in Figure 1. In contrast, an individual with fewer predisposing risk factors will require a large stimulus to precipitate delirium. Therefore, it is crucial to consider the future precipitating event in delirium severity prediction models when possible. The online surgical risk calculator built and copyrighted by the American College of Surgeons, National Surgical Quality Improvement Program (ACS NSQIP), may be ideal for delirium severity prediction as it includes several predisposing risk factors and the estimated magnitude of the precipitating event, the surgery.^22–24 Built using data from 1,414,006 patients including 1,557 distinct surgical procedures, the NSQIP risk score has been widely validated and applied to predict outcomes in various surgical populations, but has not been applied in delirium severity risk assessment.^25–29 We hypothesized that NSQIP risk of serious complications (NSQIP-SC) would be a stronger predictor of delirium over NSQIP risk of death (NSQIP-D) as the causal relationship between serious complications and delirium is likely stronger than the association between delirium and the risk of death.³⁰ As delirium is a cognitive disorder, we further hypothesized that cognitive data (that is not included in the NSQIP calculation) could enhance the prediction of the surgical risk scores. Our recent systematic review identified that current delirium prediction models do not evaluate specific cognitive domains, such as executive function.⁹ Executive function encompasses diverse higher order cognitive functions such as attention and problem-solving.³¹ Significant associations between preoperative executive function and postoperative delirium incidence have been reported and is plausible given the extensive impairment of executive functions in delirium.^32–34

Figure 1.

illustrates the Cognitive Trajectory. The relationship between cognitive abilities (predisposing, y axis) and the precipitating event, i.e. surgery over time (x-axis) is shown, with each individual trajectory displayed with a horizontal line. The dashed line, situated above the x-axis of “time”, represents the “Delirium Threshold.” (A) Trajectory #1 (gray line, numbered 1) displays an individual with maximum cognitive abilities. They have a surgery, but do not cross the “Delirium Threshold.” Trajectory #2 (blue line, numbered 2) contrasts #1 by showing an individual with decreased cognitive abilities. This individual undergoes the same surgery and crosses over the “Delirium Threshold” to experience delirium. (B) Trajectory #3 (black-dashed line, numbered 3) returns to an individual with maximum cognitive abilities. A sufficiently large precipitating event will push this individual across the “Delirium Threshold”, inducing delirium. Trajectory #1 (gray line) is transposed onto this graph to show the difference in magnitude and impact of the precipitating event. (C) When developing a prediction model for delirium, it may be important to consider not only the predisposing risk factors, but also the influence of the precipitating event. A surgical risk score such as NSQIP combines both predisposing risk and the future-precipitating event into one score, which may be optimal for postoperative delirium severity prediction.

The purpose of this study is twofold. First, to examine the potential of the NSQIP risk scores and a measure of executive function to predict postoperative delirium severity among other potential candidate predictors. Second, to demonstrate the ability of advanced statistical modeling procedures, outlined by the TRIPOD guidelines, to identify candidate predictors in a sample with limited events. Our exploratory aim repeated these advanced modeling techniques in postoperative delirium incidence to identify candidate predictors. These findings will benefit individuals who are working towards identifying patients at moderate to high risk for postoperative delirium severity including researchers and clinicians. Results of this study, including the advanced modeling procedures, should be used to inform future studies focused on developing prediction models for postoperative delirium severity.

Methods

Source of Data and Participants

This analysis is a sub-study drawn from an ongoing prospective perioperative cohort study that is approved by the University of Wisconsin-Madison, Health Sciences Institutional Review Board (#2015–0374) and registered with ClinicalTrials.govgov (ref: NCT03124303, NCT01980511).

Between August 2015 and May 2018, 1,054 potential participants were screened from vascular, urology, general and spine surgical clinics with the consent of their surgeons (HL, ST, LT, PR, JK, MD, BH, RDS). As shown in Figure 2, 100 subjects were recruited and 97 were included in the final analysis.

Figure 2.

displays the inclusion and exclusion criteria and a flowchart detailing study screening, recruitment, consent, and attrition numbers.

Outcome Measurement

The primary outcome was delirium severity. The exploratory outcome was delirium incidence. Data was collected by the trained research team composed of the principal investigator (physician), graduate students (nurse, neuroscience), medical and undergraduate nursing students.

Delirium Severity and Incidence: Pre- and postoperatively, participants were formally assessed for delirium severity and incidence using the widely validated Confusion Assessment Method-long form (CAM),³⁵ 3D-CAM,³⁶ and Delirium Rating Scale-R-98 (DRS),³⁷ twice daily, between the hours of 0500–1000 and 1600–2200 on postoperative days one through four. The DRS-R-98 (DRS) is a 16-item assessment tool that measures delirium symptoms and severity. The maximum score is 44-points, an increasing score indicates worse delirium.³⁷ The CAM and 3D-CAM were administered concurrently to provide both a comprehensive view of delirium symptoms while providing a structured interview format. If the participant was CAM positive at postoperative day-four PM assessment, the participant was followed until delirium resolved (delirium duration). If participants were ventilated in the intensive care unit (ICU), the CAM-ICU³⁸ was administered. The Delirium Rating Scale-R-98 was applied in both settings due to its robust psychometric properties and administered by trained research staff.³⁷

Predictors

Preoperatively, participants underwent an interview and completed assessments of executive function, functional status, depression, and delirium using the Confusion Assessment Method (CAM).³⁵ Executive function was assessed through two well- validated and widely used measures, Trail Making Test A (TMTA) and Trail Making Test B (TMTB).³⁹ These tests are quick to administer and require participants to connect a series of circles in ascending order. Scoring is based on time to completion; a longer completion time indicates worse executive function. Functional ability was assessed using the Instrumental Activities of Daily Living (iADL).⁴⁰ Depression was assessed with the Geriatric Depression Scale-15 (GDS).⁴¹ Demographics, vitals, comorbidities, outpatient medications and American Society of Anesthesiologists (ASA) classification score were collected. Vascular surgery was selected a priori to be included as a covariate to examine whether surgical type was sufficient to predict delirium or if the surgical risk score provided important information for the prediction model. Preoperative comorbidities as predisposing risk factors were assessed through composite measures including the ASA score, the Framingham Cardiovascular Disease Ten-year Risk Calculator (Framingham CVD), and the NSQIP surgical risk calculator.^22,42–44 Composite measures provide the ability to combine several risk factors into a single score, which is advantageous to statistical modeling and clinical application. The ACS NSQIP online surgical risk calculator⁴⁵ (http://riskcalculator.facs.org) was used to obtain the risk scores for serious complications (NSQIP-SC) and death (NSQIP-D). This calculator employs twenty patient preoperative risk factors and pairs these with the Current Procedural Terminology (CPT) code, providing a risk score specific to each procedure. Risk is calculated from both predisposing factors and precipitating factors (Figure 1). The input variables are age, sex, functional status (independent, partially dependent, dependent), emergency case, ASA classification, steroid use for chronic condition, ascites within 30-day prior to surgery, systemic sepsis within 48-hours prior to surgery, ventilator dependency, disseminated cancer, diabetes, hypertension with medications, congestive heart failure (within 30-days prior to surgery), dyspnea, current smoker (within 1-year), history of severe COPD, dialysis, acute renal failure, height and weight as well as surgical procedure. There are 1,557 distinct CPT codes, ranging from minor surgeries such as a cholecystectomy to major surgeries such as thoracoabdominal aortic aneurysm repair.

Sample Size

Sample size was based on linear regression and determined using the rule of 8–10 outcome events (delirium) per variable.⁴⁶ The decision to analyze was made after 100 participants were recruited with a delirium incidence rate of 32%.

Missing Data

Missing data was identified in the following variables (#missing): TMTB (1), GDS15 (1), TMTA (6), and Tobacco Pack Years (10). Little’s test of missing completely at random (MCAR) was not significant indicating that the missing data were missing completely at random and likely, did not influence the analysis. Multiple imputation for these missing values were completed using single value regression analysis. As recommended by Jackobsen et al. (2017), the analysis was completed using data generated from the multiple imputations and a sensitivity analysis was completed using list-wise deletion.^47–49

Statistical Analysis

Patient characteristics were described using means ± standard deviations for continuous variables and frequency counts with percentages for categorical variables. Dependent on the distribution of the data, continuous variables were compared using Student’s t-test or Mann-Whitney U-test. Categorical variables were compared using x². The primary outcome variable, delirium severity, was measured using the Peak DRS Total Score (DRS) for linear regression. Significance was notated with a p-value ≤0.05. NCSS v12.0, Stata/IC v15.0 and R v1.1453 were used for statistical analysis. HL and RB conducted the statistical analysis.

The DRS score was transformed using the Box-Cox Method⁵⁰ with the optimal Lambda value due to the positive skew, please refer to Figure 3-A and B for raw and transformed plots. The independent variables NSQIP-SC, NSQIP-D, TMTA, TMTB, tobacco pack years, and GDS15 demonstrated a positive skew. These were not transformed as the distribution of the independent variables does not violate modeling assumptions.^51,52 The errors of the regression model were examined and those were normally distributed. The assumptions of the regression model were examined and met. First, to identify candidate predictors for delirium severity and demonstrate the development of a prediction model, we employed LASSO and Best Subsets regression. We did not employ univariate statistics to select candidate predictors as this may lead to poor performing predictors and overfitting.⁵³ To counter the effects of small sample sizes and reduce bias within data, we employed a statistical shrinkage regression technique, using Least Absolute Shrinkage and Selection Operator (LASSO).⁵⁴ This technique reduces the noise within the data, allowing true signals to be detected and avoids common problems such as model overfitting. Further, this technique has shown to out-perform traditional statistical modeling procedures such as forward stepwise regression, leading to an efficient and precise modeling procedure.¹⁸ Candidate variables demonstrating the smallest Mallow’s Cp value,⁵⁵ indicating precise predictors, were then applied in Best Subsets regression. Best Subsets regression is an automated regression approach that evaluates all possible combinations of candidate predictors.⁵⁶ The output provides a set models with model fit statistics. Model selection was based on assessment of model fit using Akaike information criteria (AIC), Bayesian information criteria (BIC), and adjusted R².⁵⁷ Second, to evaluate the predictive ability of NSQIP-SC over NSQIP-D (as composites of the predisposing and precipitating factors), linear regression models were completed, nested, and compared to ASA classification and Framingham risk (as measures of predisposing only).

Figure 3.

illustrates the postoperative delirium symptom severity prediction model. Box A is a histogram showing the data distribution of the Peak Delirium Rating Scale Score (DRS). This value was transformed using the Box-Cox Method with an optimal lambda value of 0.35 achieving a near Gaussian distribution and is shown on the histogram in Box B. Boxes C-E display the predicted burden of delirium symptoms based on the NSQIP-SC and TMTB prediction model (Box C) and univariate analysis of NSQIP-SC (Box D) and TMTB (Box E). The statistics from each regression model are shown in the upper left hand corner of each box. The univariate NSQIP-SC regression model was analyzed with 97 participants. Due to one missing assessment of TMTB, Box C and E are analyzed with 96 participants.

The regression modeling procedures outlined in the paragraph above for linear regression were repeated for the logistic regression model to select candidate predictors for delirium incidence. The area under the receiver operating characteristic curve (AUROC) with 95% CI was calculated. Calibration was assessed through goodness-of-fit tests calculated by the Hosmer-Lemeshow statistic. Sensitivity, specificity, positive predictive and negative predictive values were calculated and reported.

Results

Thirty-one participants (32%) experienced postoperative delirium with a mean peak DRS severity total score of 21.48 (±SD 6.40). Forty-two percent of delirium cases were hypoactive. The median delirium duration was one day (24 hours). Participant characteristics are summarized in Table 1. Delirious patients had higher preoperative NSQIP risk scores of serious complications (NSQIP-SC) and death (NSQIP-D), worse executive function tests, were more likely to have had a vascular surgery, and higher ASA status (univariate, p<0.05). Significant pairwise correlations were demonstrated between the DRS and NSQIP-SC, NSQIP-D, TMTA and TMTB (univariate, p<0.05).

Table :1.

Description of sample and significant differences between no delirium and delirium

Variable	Mean (SD) n/N (%) N=97	No DeliriumN=66	Delirium N=31
Age	71.68 (4.55)	71.71 (4.79)	71.61 (4.09)
Sex	55 (male, 55%)	39 (59%)	16 (52%)
Years of Education	3 (<12yrs, 3%)	1 (1%)	2 (6%)
	29 (12yrs, 30%)	19 (29%)	10 (32%)
	65 (>12yrs, 67%)	46 (70%)	19 (61%)
NSQIP-SC	17.56 (11.66)	13.93 (9.34)	25.27 (12.49)***
NSQIP-D	2.52 (3.87)	1.85 (3.45)	3.95 (4.38)**
Framingham CVD	35.86 (19.6)	35.99 (20.74)	26.52 (16.98)
ASA	2.64 (3.87)	2.56 (0.61)	2.84 (0.52)*
Preoperative SBP	133 (17)	133 (17)	135 (17)
Preoperative DBP	74 (10)	74 (11)	74 (10)
Preoperative PP	59 (16)	59 (16)	61 (17)
Preoperative MAP	94 (11)	94 (12)	94 (10)
Tobacco Pack Years	19 (24)	17 (22)	25 (29)
Current tobacco user	19 (yes, 16%)	10 (15%)	6 (19%)
Past tobacco user	65 (yes, 67%)	43 (65%)	22 (71%)
Type of Surgery-Vascular	38 (vascular, 39%)	22 (33%)	16 (51%)*
Other	12 (general, 12%)	44 (66%)	15 (48%)
	36 (Spine, 37%)
	11 (Urology, 11%)
GDS15	2.47 (2.49)	2.35 (2.48)	2.74 (2.53)
Preoperative TMTA	42.10 (16.67)	39.53 (13.785)	47.58 (20.67)
Preoperative TMTB	98.73 (52.4)	89.92 (46.45)	117.48 (60.02)*
Peak DRS Total Score	11.5 (8.3)	6.82 (3.74)	21.48 (6.40)***
Delirium duration (median)			1 day
Delirium subtypes (%)			Hypoactive: 42
			Mixed: 32
			Hyperactive: 19
			RASS 0: 6

Significance levels:

* = p<0.05

**p<0.001

***p<0.0001

Abbreviations: ASA=American Society of Anesthesiologists classification score, DBP=Diastolic Blood Pressure, DRS=Delirium Rating Scale-98-R, Framingham CVD=Framingham Cardiovascular Risk Score, GDS15=Geriatric Depression Scale-15, MAP=Mean Arterial Pressure, NSQIP_SC=National Surgical Quality Improvement Program Risk for Serious Complications, NSQIP-D=National Surgical Quality

Improvement Program Risk for Death, PP=Pulse Pressure, SBP=Systolic Blood Pressure, SD=Standard Deviation, TMTA=Trail Making Test A, TMTB=Trail Making Test B

Predictor Selection, Model Development, and Performance

LASSO identified (NSQIP-SC, vascular surgery, Framingham CVD, Preoperative pulse pressure and mean arterial pressure, Tobacco pack years, TMTA and TMTB) as predictors for postoperative delirium severity and are displayed in Supplemental Figure S-2. These variables were applied to Best Subsets regression. The model demonstrating optimal fit statistics was a two-factor linear regression model contained preoperative NSQIP-SC and TMTB. This two-factor model reports an adjusted R² of 0.30 (p<0.001), thus explaining 30% of the variability in observed delirium symptoms. For every 1-percent increase in the NSQIP-SC score, the peak DRS total score will increase by 0.29 points. Alternatively, a 10% increase in the NSQIP-SC score will increase the Peak DRS total score by 2.9 points. Further model details are displayed in supporting information, Table S-1 and Figure 3-C-E. Age, sex, NSQIP-D, ASA, tobacco pack years, vascular surgery, Framingham CVD, GDS15, TMTA, and Pre-BP were not identified as significant predictors in Best Subsets modeling procedures. As a sensitivity analysis, these statistical procedures were completed following list-wise deletion due to missing values (n=79). This analysis identified the same two-factor model containing NSQIP-SC and TMTB.

LASSO identified NSQIP-SC and TMT-B as predictors for postoperative delirium severity. To further examine how NSIQP-SC compared to the other surgical/vascular composite risk scores of NSQIP-D, Framingham CVD, and ASA status, we used standard linear regression models. Preoperative NSQIP-SC was confirmed as a predictor of postoperative peak DRS using simple linear regression models for NSQIP-SC (p<0.0001, AdjR²: 0.184). NSQIP-D was also significantly associated with DRS (p=0.04, AdjR²: 0.03). However, when nested, the NSQIP-SC model demonstrated higher adjusted R², and lower AIC (1.525) and BIC (−290.718) metrics, providing support for that predictor over NSQIP-D. Similar to logistic regression for postoperative delirium incidence, ASA (p=0.04, AdjR²: 0.03) and Framingham CVD score (p=0.69, AdjR²: 0.01) did not demonstrate a strong predictive relationship with DRS. Further model comparison statistics are illustrated in Table 2.

Table 2–

Linear and Logistic Regression Model Statistics for NSQIP-SC, NSQIP-D, ASA and Framingham 10-year Cardiovascular Risk. These models were nested to compare performance.

Linear Regression with Transformed DRS
F-Stat p-value	Variable	Coef.	Std. Error (95%CI)	p-value	Std. Betas	Adj R- Squared	AIC BIC
22.68	NSQIP-SC	0.02	0.004 (0.01–0.03)	<0.001	0.44	0.18	1.525
							−16.19
<0.0001
4.25	NSQIP-D	0.03	0.01 (0.001–0.06)	<0.05	0.21	0.03	1.695
<0.05							0.33
4.18	ASA	0.20	0.096 (0.01–0.39)	<0.05	0.21	0.03	1.696
<0.05							0.40
0.16	Framing.	0.001	0.003 (−0.004–0.007)	P=0.69	0.04	0.002	1.737
P=0.69							4.14
Logistic Regression with Delirium Incidence
LR chi2 p-value	Variable	Odds Ratio	Std. Error (95%CI)	p-value	AUROC (95%CI)	McKelvey& Zavoina’s Adj R2	AIC BIC’
−50.67	NSQIP-SC	1.09	0.03 (1.05–1.15)	<0.001	0.76 (0.66–0.87)	0.25	1.086
<0.0001							−15.63
−57.80	NSQIP-D	1.15	0.07 (1.02–1.29)	<0.05	0.73 (0.62–0.84)	0.08	1.233
<0.05							−1.38
−58.44	ASA	2.24	0.86 (1.06–4.76)	<0.05	0.63 (0.53–0.73)	0.07	1.246
<0.05							−0.09
−60.74	Framing.	1.00	0.11 (0.98–1.02)	P=0.82	0.53 (0.40–0.65)	−0.03	1.294
P=0.82							4.52

Abbreviations: Adj. =adjusted, AIC= Akaike information criteria, ASA=American Society of Anesthesiologists, AUROC=Area Under the Receiver Curve Operator, BIC=Bayesian Information Criteria, CI=Confidence Interval, DRS=Delirium Rating Scale-R-98, Framing. =Framingham Cardiovascular Risk Score, LR=Likelihood ratio, NSQIP-SC=National Surgical Quality Improvement Program-Risk of Serious Complications, NSQIP-D=National Surgical Quality Improvement Program-Risk of Death, Std. =Standard.

This study was designed in 2013–2014, prior to the publications of the CAM-S severity tool.⁵⁸ To examine the reproducibility of these results in different severity tools, we crosswalked the Peak DRS Total Scores to both CAM-S long form and short form scores using the Network for Investigating Delirium: Unifying Scientists (NIDUS) BASIL harmonization tool.^59,60 These results are provided in the supplemental section, Table S-3, Figure S-4, and Table S-5.

Exploratory outcome: Candidate predictors and Model Development for Delirium Incidence

A two-factor logistic regression model containing preoperative NSQIP-SC and TMTB was identified by LASSO and Best Subsets regression to predict postoperative delirium incidence. Hosmer-Lemeshow goodness-of-fit test was not significant (p=0.37), indicating accurate model calibration. The model demonstrated moderate predictive ability (AUROC 0.81, 95% CI: 0.72–0.90), a 5-percent increase in the NSQIP-SC score increased the probability of delirium incidence by 10% (Figure 4). Age, sex, NSQIP-D, ASA, tobacco pack years, vascular surgery, Framingham CVD, GDS15, TMTA, and Pre-BP were not identified as significant predictors. Table 2 displays the nested model comparison between NSQIP-SC, NSQIP-D, ASA, and Framingham CVD. Logistic regression model classification metrics are shown in supporting information, in Table S-1.

Figure 4.

illustrates the predictive ability of the NSQIP-SC and TMTB model for postoperative delirium incidence. (A) Displays the Area Under the Receiver Operator Curve statistic (AUROC). (B) Demonstrates the predicted probability of postoperative delirium incidence based on the % NSQIP-SC score. This is holding TMTB constant at zero.

Discussion

Our analysis of a prospective perioperative cohort using the advanced statistical methods of LASSO and Best Subsets regression identified NSQIP-SC to be a predictor of postoperative delirium severity and incidence. The preoperative NSQIP-SC score is a composite variable, combining data on both predisposing risk factors for delirium and surgical severity, (i.e. the precipitating event for delirium) and is well positioned to contribute information on the risk of delirium severity. This analysis demonstrates that as the risk for surgical complications increases, so does the risk for severe delirium. Further, NSQIP-SC out-performed age, ASA classification, Framingham Cardiovascular Disease Risk, and type of surgery alone when modeling procedures were applied. This suggests that NSQIP-SC may provide a more accurate estimation of delirium risk compared to age, comorbidities, and acute illness in surgical patients. The executive function measure, TMT-B, was also identified as a predictor for both postoperative delirium severity and incidence. Previous research has identified executive function to be significantly associated with delirium incidence.^32–34 However, as identified by our recent systematic review of delirium prediction models in older adults, an executive function measure has not been applied previously.⁹ Given the breakdown in executive function during delirium and prior data on the predisposition to delirium by impaired cognition, incorporating a cognitive variable appears biologically important; our data show that it is statistically important too. This study expands current knowledge by examining the utility of NSQIP risk scores and executive function in predicting delirium severity and incidence. Further, this study demonstrates the use of advanced statistical modeling procedures to select predictors, and to develop a prediction model in a sample with low event numbers.

The NSQIP-D score was significantly associated with both postoperative delirium severity and incidence, however, it was not selected as a predictor in our variable selection, rather NSQIP-SC was selected. Delirium often results from a complicated perioperative course, particularly following a major surgery, hence a relationship with NSQIP-SC is plausible. Furthermore, a recent systematic review questioned the strength of the association between postoperative delirium and mortality, hence a priori we hypothesized that NSQIP-SC would perform better than NSQIP-D for predicting postoperative delirium severity and incidence.³⁰ This was supported by several statistical measures in our dataset. The recommended modeling procedures, LASSO and Best Subsets regression,¹⁹ did not identify NSQIP-D, vascular burden, comorbidities, smoking history, and depression as important predictors although these variables have been identified as significant risk factors for postoperative delirium incidence in prior research. This may be due to a number of factors. First, their prevalence in this population of study may not be sufficient for prediction. In order for a risk factor to also be an accurate and useful predictor, it must be sufficiently prevalent in the at-risk population.⁶¹ This prevalence is key as when predicting risk of an individual surgical patient, each factor selected in any model should exert meaningful influence on the prediction. Secondly, late-life depression, vascular burden, and tobacco use often co-occur leading to overlapping data capture. Variables that capture similar information fail to contribute important information during modeling procedures.⁹

The strengths of this study include its prospective perioperative design, statistical methods chosen, and rigorous delirium assessments including outcomes based on severity and incidence of delirium. This study represents a novel application of a two statistical methods to select candidate predictors and develop a prediction model in a sample size with a low number of outcome events. Further, NSQIP-SC is a readily available online tool that has potential for broad application in delirium-focused clinical care. The identification of at-risk individuals prior to surgery would provide an opportunity to develop a targeted plan of care centered on delirium prevention.^62,63 The NSQIP-SC score combines several potential preoperative risk factors for delirium (age, functional status, current tobacco use, vascular burden) with the precipitating event, the planned surgery, and provides a single risk score that is easy to interpret, i.e., a 10% increase in NSQIP-SC results in a 2.9 point increase in delirium severity. Given that the patient becomes delirious postoperatively, quantifying the potential impact of this precipitating event is clearly a key feature of a delirium prediction model.

This study has several limitations to consider. While these models were built using statistical methods optimized for modeling a limited number of events, the small sample size may still have had an effect on the results. However, in this study we were able to model 97 events for delirium severity. Nonetheless, the recommendations by the TRIPOD Statement¹⁹, a guide for prediction modeling and transparent reporting, were followed to use statistical shrinkage procedures to minimize model overfitting. The NSQIP online risk tool is copyrighted and the American College of Surgeons does not currently allow this tool to be incorporated into electronic health records. Nevertheless, it is available online, providing free access to clinicians. The population is largely homogenous in terms of years of education and ethnicity. Patients with a dementia diagnosis were excluded from participation. This exclusion limits the generalizability of our findings to patients with dementia. In larger and more diverse populations, additional factors may enhance model performance.

Conclusion

In summary, this analysis of a prospective perioperative cohort study identified NSQIP-SC and a measure of executive function, TMTB, to be predictors of postoperative delirium severity. Two advanced statistical procedures, LASSO and Best Subsets, were used to select candidate predictors and develop a prediction model is a small sample size with limited outcome events. Future studies should focus on the broad external validation of these models following the statistical methodology demonstrated in this study and are outlined by the TRIPOD guidelines.

Key points:

• This study addresses an important knowledge gap by examining how surgical risk and executive function predict postoperative delirium severity. To our knowledge, no studies to-date have investigated these associations.

• This study used advanced statistical methods to identify candidate predictors of postoperative delirium severity; the National Surgical Quality Improvement Program (NSQIP-SC)-Risk for Serious Complications and an executive function measure, TMT-B.

• A ten percent increase in the NSQIP-SC score resulted in a 2.9 point increase in the peak Delirium Rating Scale Score. This is a straight-forward interpretation that could be readily implemented into clinical practice.

• Both variables, NSQIP-SC and TMTB, were also identified as robust predictors of postoperative delirium incidence.