Cardiovascular Risk Scores to Predict Perioperative Stroke in Non-Cardiac Surgery

Abstract

Background and Purpose:

Perioperative stroke is associated with significant morbidity and mortality. Conventional cardiovascular risk scores have not been compared to predict acute stroke after non-cardiac surgery.

Methods:

Patients undergoing non-cardiac surgery between 2009-2010 were identified from the United States National Surgical Quality Improvement Program (n=540,717). Patients were prospectively followed for 30 days postoperatively for the primary outcome of stroke. Established cardiovascular and perioperative risk scores (CHADS₂, CHA₂DS₂VASC, RCRI, Mashour et. al. risk score, MICA, and NSQIP ACS Surgical Risk Calculator [ACS-SRC]) were assessed to predict perioperative stroke.

Results:

Stroke occurred in the perioperative period of 1,474 non-cardiac surgeries (0.27%). Patients with perioperative stroke were older, more frequently male, had lower body mass index, and were more likely to have undergone vascular surgery or neurosurgery than patients without stroke (p < 0.001 for each comparison). All risk prediction models were associated with increased risk of perioperative stroke (c-statistic (AUC) range 0.743-0.836). The MICA (AUC 0.833) and ACS-SRC (AUC 0.836) risk scores had the most favorable test characteristics and a greater ability to discriminate perioperative stroke when compared to RCRI, CHADS_2, CHA₂DS₂VASC, and Mashour risk scores (p for comparison <0.001, Delong’s). Risk scores did not provide consistent discriminative ability across surgery types and were least predictive in vascular surgery (AUC range 0.588-0.672).

Conclusion:

The MICA and ACS-SRC surgical risk scores provide excellent risk discrimination for perioperative stroke in most patients undergoing non-cardiac surgery. Stroke prediction was less optimal in patients undergoing vascular surgery.

Introduction

Despite improvements in medical optimization and surgical technique, risk of perioperative stroke has increased in the past decade.¹ Guidelines recommend evaluating stroke risk during preoperative cardiovascular risk assessment.² A number of cardiovascular risk stratification tools are used to predict perioperative complications, including the Revised Cardiac Risk Index (RCRI)³, the Myocardial Infarction or Cardiac Arrest (MICA) calculator,³ and the American College of Surgeons Surgical Risk Calculator (ACS-SRC). These risk stratification tools do not include stroke as an endpoint, or do not evaluate the risk of stroke independent of other events in the composite outcome.⁴ CHADS₂ and CHA₂DS₂VASC risk scores have been shown to predict postoperative stroke in patients undergoing cardiac procedures even in the absence of atrial fibrillation⁵, but their utility has not been evaluated in non-cardiac surgery. Mashour et al. derived a perioperative stroke risk model from patients at low-risk for perioperative stroke undergoing non-cardiac surgery and identified 10 independent predictors of stroke.⁶ However, the utility of risk scores for perioperative stroke has not been established in a broadly representative population of patients undergoing non-cardiac surgery. The purpose of the present study is to compare the effectiveness of existing cardiovascular risk stratification scores in predicting perioperative stroke after non-cardiac surgery.

Methods

Data were obtained from the American College of Surgeons National Quality Improvement Project (NSQIP). The NSQIP is a large national multicenter registry that collects data on over 150 perioperative variables from patients undergoing surgery at over 250 participating centers in 2009-2010. At each site, certified nurse reviewers prospectively enroll a systematic sample of the first 40 patients on the operating log over an 8-day cycle. Data are obtained from patient medical records, physician office records, and follow-up telephone interviews. Patients are followed after hospital discharge up to 30 days postoperatively. The accuracy and reproducibility of NSQIP data have been previously validated. As data were obtained from a large, de-identified surgical database, this study was exempt from IRB review. Data is available for research purposes at all participating NSQIP sites.

We performed a retrospective cohort study of participants age ≥ 18 undergoing non-cardiac surgery between 2009-2010. Due to changes in NSQIP data collection methods, history of ischemic heart disease and stroke were only collected between 2009-2010. Cases with incomplete data (n=129,380) or outlier body mass index data (n=10,852) were also excluded, as described below. Patients undergoing cardiac surgery and interventional radiology procedures as the index surgery were excluded (n=9,177). Eligible surgeries were categorized by the specialty of the operating surgeon into the following subgroups: general surgery, gynecology, neurosurgery, orthopedics, otolaryngology, plastic surgery, thoracic, urology and vascular surgery.

Clinical Characteristics:

Demographic data, including age, sex, and race/ethnicity, were obtained from the dataset. Body mass index (BMI) was calculated from height and weight data. BMIs calculated to be greater than three times the interquartile range above the third quartile or less than the first quartile, which comprised 1.6% of cases, were excluded as outliers that were presumed to be due to inaccurate data entry. Preoperative characteristics, including demographics, American Society of Anesthesiologist (ASA) Physical Status classifications, and surgery type were tabulated for patients with and without post-operative stroke.

Outcome:

Stroke was defined as per the NSQIP, with development of an embolic, thrombotic, or hemorrhagic vascular accident or stroke with motor, sensory, or cognitive dysfunction (e.g., hemiplegia, hemiparesis, aphasia, sensory deficit, impaired memory) that persists for 24 or more hours.⁷ Secondary outcomes of interest included length of stay, mortality, as well as pre- and post-stroke mortality, myocardial infarction, cardiac arrest, pneumonia, unplanned reintubation, sepsis, septic shock, pulmonary embolism, and deep vein thrombosis. To minimize missing data, certified NSQIP reviewers tracked both in and out of hospital outcomes 30 days post-operatively via follow up phone calls. Hospitals reporting a follow up rate of less than 80% were excluded. Missing data on outcome timing was imputed by median imputation as discussed in the supplementary methods. We compared secondary outcomes in the post-stroke population to patients without stroke using chi-square tests.

Risk Scores:

Components of six cardiovascular and perioperative risk scores (ACS-SRC, CHADS₂, CHA₂DS₂VASC, Mashour, MICA, and RCRI) were identified in the NSQIP database. Risk scores were calculated for all eligible surgical cases. Definitions of the individual components of each risk score are shown in Supplemental Table I.

Statistical Analysis:

Odds of perioperative stroke were calculated by risk score. C-statistics (AUC) and 95% confidence intervals (CI) were calculated using sensitivity and specificity for each risk score to identify perioperative stroke. Two-way comparisons were made between receiver operating characteristic (ROC) curves using Delong’s test. Risk scores between 0.5-0.7 were considered poorly discriminative, between 0.7-0.8 were considered moderately discriminative, and >0.8 were considered highly discriminative.^8,9

Sensitivity Analysis

A sensitivity analysis was performed for the outcome of early stroke after surgery, defined as strokes occurring within 3 days post-operatively. In this analysis, patients with strokes that occurred ≥3 days after surgery were combined with patients without any documented stroke during 30-day follow up. ROC curves for stroke at 3 and 30-days were compared using Delong’s test. A sensitivity analysis was performed after excluding outcomes that occurred on the same day as stroke except for death which we know must have occurred post-stroke.

Results

Perioperative stroke occurred in 1,474 of 540,717 cases identified in 2009-2010, with 272 cases per 100,000 surgeries. Patients with stroke were older, more frequently male, and had lower BMI (p < 0.001 for each comparison, Table 1). Stroke was most frequent following vascular surgery and neurosurgery, with 993 and 627 per 100,000 cases, respectively (Figure 1).

Table 1.

Demographic Information of Patients with and without Perioperative Stroke

	No Stroke No. (%)	Stroke No. (%)	P-value
Male sex	233,148 (43.2)	716 (48.6)	<0.001
Age, mean ±SD	57.7 (±16.6)	69.7 (±12.9)	<0.001
Race
White	419,623 (77.8)	1,148 (77.9)	0.95
Black	57,538 (10.7)	172 (11.7)	0.23
Asian	11,399 (2.1)	28 (1.9)	0.55
American Indian	1,681 (0.3)	8 (0.5)	0.29
Native Hawaiian or Pacific Islander	3,069 (0.6)	12 (0.8)	0.23
Unknown/Not Reported	45,933 (8.5)	106 (7.2)	0.05
Ethnicity
Hispanic	37,489 (7.0)	55 (3.7)	0.02
BMI, mean ±SD	29.8 (±8.1)	28.1 (±7.0)	<0.001
ASA Class
1	35,640 (6.6)	3 (0.2)	<0.001
2	234,249 (43.4)	150 (10.2)	<0.001
3	229,598 (42.6)	841 (57.1)	<0.001
4	38,419 (7.1)	446 (30.3)	<0.001
5	1,337 (0.2)	34 (2.3)	<0.001
Surgery Type
General Surgery	364,195 (67.5)	606 (41.1)	<0.001
Gynecology	16,983 (3.1)	7 (0.5)	<0.001
Neurosurgery	12,361 (2.3)	78 (5.3)	<0.001
Orthopedics	42,888 (8.0)	67 (4.5)	<0.001
Otolaryngology	8,101 (1.5)	6 (0.4)	<0.001
Plastics	5,637 (1.0)	5 (0.3)	<0.001
Thoracic	5,047 (0.9)	12 (0.8)	0.60
Urology	18,262 (3.4)	33 (2.2)	0.003
Vascular	65,769 (12.2)	660 (44.8)	<0.001

BMI: Body Mass Index

ASA: American Society Anesthesiologists

Figure 1. Frequency of Perioperative Stroke by Non-Cardiac Surgical Subtype.

Stroke, defined as development of an embolic, thrombotic, or hemorrhagic vascular accident or stroke with motor, sensory, or cognitive dysfunction (e.g., hemiplegia, hemiparesis, aphasia, sensory deficit, impaired memory) that persists for 24 or more hours, per 100,000 surgery.

Patients who suffered stroke had a significantly higher 30-day mortality than patients without perioperative stroke (22.3% vs 1.6%, p<0.0001). Patients with perioperative stroke also had increased length of stay (10 days vs 2 days, p<0.0001), higher rate of myocardial infarction (2.85% vs 0.5%, p<0.0001), cardiac arrest, pneumonia, unplanned reintubation, sepsis, septic shock, pulmonary embolism, and deep vein thrombosis (Table 2).

Table 2.

30-day Perioperative Outcomes in Patients with and without Perioperative Stroke

	No Stroke No. 539,243	Stroke No. 1,474	P-value
Death	8,518 (1.58)	328 (22.25)	< 0.0001
MI	2,686 (0.50)	42 (2.85)	< 0.0001
Cardiac Arrest	2,231 (0.41)	32 (2.17)	< 0.0001
Pneumonia	8,436 (1.56)	131 (8.92)	< 0.0001
Reintubation	7,566 (1.40)	158 (10.73)	< 0.0001
Sepsis	10,187 (1.89)	103 (6.99)	< 0.0001
Septic Shock	5,201 (0.96)	67 (4.55)	< 0.0001
Pulmonary Embolism	1,967 (0.36)	16 (1.09)	< 0.0001
Deep Vein Thrombosis	4,015 (0.74)	65 (4.41)	< 0.0001
Length of Stay	2.00 (1.00 - 5.00)	10.00 (5.00 - 19.00)	< 0.0001

For patients without stroke, secondary outcomes occurred day 0-30 perioperatively. For patients with stroke, secondary after (at least 1 calendar day post-stroke), or on the same day that the stroke occurred. P values reflect chi-squared comparisons between secondary outcomes in the non-stroke versus post-stroke populations.

Patients with perioperative stroke had significantly higher median risk scores (Table 3). Odds of perioperative stroke increased with incremental increases in risk scores (Table 3). In all non-cardiac surgeries, MICA (AUC 0.833, 95% CI 0.825-0.842) and ACS-SRC (AUC 0.836, 95% CI 0.827-0.845) risk scores were highly discriminative for perioperative stroke, with significantly greater c-statistics for stroke than CHADS_2, CHA₂DS₂VASC, Mashour and RCRI models (p<0.001, Figure 2, Supplemental Table II).

Table 3:

Comparing Risk Scores: Range per Risk Score, Median Risk Score in Stroke and Non-Stroke Patients, Odds of Stroke Associated with Incremental Increase in Risk Score

Risk Score	Risk Score Range	Risk Score without Stroke Median (IQR)	Risk Score with Stroke Median (IQR)	P-Value	Odds Ratio (95%CI) Stroke
ACS-SRC	0-23	4 (2 - 6)	7 (5 - 9)	< 0.0001	1.34 (1.32-1.35)
CHADS	0-6	1 (0 - 2)	2 (1 - 3)	< 0.0001	1.96 (1.90-2.04)
CHADSVASC	0-9	2 (1 - 3)	3 (2 - 5)	< 0.0001	1.69 (1.64-1.73)
MASHOUR	0-9	2 (1 - 3)	3 (3 - 4)	< 0.0001	2.11 (2.03-2.20
MICA	0-9	2 (1 - 3)	4 (3 - 5)	< 0.0001	1.95 (1.89-2.00)
RCRI	0-6	0 (0 - 0)	1 (0 - 2)	< 0.0001	2.45 (2.35-2.55)

IQR: Interquartile Range

CI: Confidence Interval

Figure 2. Risk Model Receiver Operating Characteristic Curves to Predict Perioperative Stroke.

C-statistics, calculated using the sensitivity and specificity of each risk score for the outcome of perioperative stroke, are displayed within the figure. Risk scores between 0.5-0.7 were considered poorly discriminative, between 0.7-0.8 were considered moderately discriminative, and >0.8 were considered highly discriminative.^8,9

ROC: Receiver Operating Characteristics

RCRI: Revised Cardiac Risk Index

MICA: Myocardial Infarction or Cardiac Arrest

MASHOUR: Mashour et al. risk score for perioperative stroke

ACS-SRC: American College of Surgeons Surgical Risk Calculator

When stratified by surgery, the RCRI model had consistently poor discrimination for perioperative stroke (AUC <0.7 for general, gynecologic, neurologic, orthopedic, otolaryngology, thoracic, urologic and vascular surgery types). ASC-SRC and MICA had consistently high discriminative accuracy for perioperative stroke (AUC > 0.8 for general, gynecologic, otolaryngology, plastic surgery types, AUC > 0.7 for neurologic, orthopedic, thoracic, and urologic subtypes). There were no significant differences in c-statistics between the CHADS_2, CHA₂DS₂VASC and RCRI risk scores across surgery types. All risk scores had poor discriminative accuracy for stroke in patients undergoing vascular surgery (AUC <0.7), but the ACS-SRC had significantly greater accuracy than the other risk scores in this surgical subgroup (Figure 3).

Figure 3. Prediction of Perioperative Stroke by Risk Model and Surgery Type. C-statistics with 95%confidence intervals are shown.

C-statistics and 95% confidence intervals are displayed by surgery type, defined by specialty of the operating surgeon. A c-statistic of greater than 0.7 (vertical line) indicates a model with acceptable accuracy in discriminating patients at risk for stroke⁹.

RCRI: Revised Cardiac Risk Index

MICA: Myocardial Infarction or Cardiac Arrest

MASHOUR: Mashour et al. risk score for perioperative stroke

ACS-SRC: American College of Surgeons Surgical Risk Calculator

Sensitivity Analysis

Thirty one percent of all strokes occurred within 3 days of the procedure. Post-operative day 4 was the median for stroke occurrences (95% CI 4-5). All risk scores had similar test characteristics for early stroke compared with stroke prediction at 30-days (Supplemental Table III).

Discussion

Perioperative stroke is a rare but potentially devastating complication of non-cardiac surgery. In a large multicenter US registry, one-fifth of patients with perioperative stroke after non-cardiac surgery died within 30 days. Although existing risk scores demonstrated adequate discrimination for perioperative stroke in the overall surgical population, accuracy varied by risk score and subtype of non-cardiac surgery. The RCRI, a well-established score that incorporates 6 factors to determine risk for perioperative major cardiac complications, had the poorest discriminative ability. CHADS₂ and CHA₂DS₂VASC scoring systems, which may help predict perioperative stroke in cardiac procedures,⁵ did not improve accuracy when compared to RCRI. Instead, two newer risk scores, the MICA and ACS-SRC, had greater discriminative ability for perioperative stroke after non-cardiac surgery. The Mashour risk score, which was developed specifically for the outcome of stroke, was inferior to the MICA and ACS-SRC and similar diagnostic performance RCRI, CHADS_2, and CHA₂DS₂VASC risk scores.

The frequency of post-operative stroke was three-fold higher after vascular procedures, which represented 12% of all non-cardiac surgeries, compared with non-vascular surgeries. Stroke after vascular surgery accounted for 45% of all strokes. Unfortunately, all risk scores had poor accuracy in vascular surgery (AUC <0.7). Patients undergoing vascular surgery had a greater burden of comorbid disease, including hypertension, prior cardiovascular disease, severe chronic obstructive pulmonary disease (COPD), kidney disease, and dependent functional status (Supplemental Table IV), contributing to a heterogeneous risk profile. Given the high burden of cardiovascular disease, many of these patients may have received antithrombotic medications prior to surgery, putting them at increased risk of hemorrhagic stroke. Perioperative beta blocker use is also associated with an increased risk of stroke.¹⁰ Finally, patients who underwent vascular surgery were more likely to have stroke on the first post-operative day than patients undergoing non-vascular procedures (39% vs. 20%). Surgical factors, including vascular manipulation and intraoperative hemodynamics, may contribute to early strokes in this surgical subgroup.

Despite the rising burden of cardiovascular risk factors among patients undergoing non-cardiac surgery and increases in perioperative stroke reported over the past decade, few risk models relevant to non-cardiac surgery have been developed to predict perioperative stroke. The Mashour risk score, developed for perioperative stroke, includes acute renal failure, age > 62, ASA class, current smoking, dialysis, history of transient ischemic attack or cerebral vascular accident, hypertension, recent myocardial infarction, history of severe COPD, and BMI of 35– 40 kg/m².⁶ This risk score was derived using a subset of low risk patients, excluding those undergoing major vascular and neurologic procedures. In the present analysis, which included higher-risk patients, the Mashour risk score performance was modest (C-statistic 0.773). In the more generalizable population used in this analysis, ACS-SRC and MICA risk scores were more accurate in identifying patients at risk for stroke. The MICA risk score includes pre-operative creatinine, surgery type, age, ASA class and functional status (Supplemental Table I). The ACS-SRC includes all factors included in the MICA score, as well as emergency case, steroid use, ascites, sepsis within 48 hours prior to surgery, ventilator dependence, disseminated cancer, diabetes, hypertension, congestive heart failure, symptoms of dyspnea at rest or with exertion, current smoking, history of COPD, dialysis, acute renal failure and BMI.

Ideal risk stratification tools should provide accurate estimates of all cardiovascular events in order to determine optimal perioperative interventions. There may be opportunities for improved surgical risk prediction by incorporating additional clinical factors, such as dose and timing of initiation of perioperative beta-blockers.

Limitations:

We acknowledge there are several limitations to this manuscript. First, this is a retrospective analysis of a large national registry with robust data collection in the first 30 post-operative days. The NSQIP database was designed for quality improvement rather than survey purposes. Although 30-day follow up is systemically performed by trained nurses, data on perioperative stroke are subject to sampling, reporting and recall bias. Although no binary outcomes were reported as missing, the timing of each outcome could not be determined in <1% of cases. In these cases, there was an invalid date that precluded the determination of the sequence of events. Missing data did not impact the overall binary outcomes reported in the manuscript. Granularity of timing was limited to day, therefore timing secondary outcomes that occurred on the same day as stroke could not be determined, and were categorized as post-stroke. This leaves potential for overestimation of association of post-operative outcome with stroke. However, after excluding outcomes that occurred on the same day as stroke, outcomes of all secondary events remained significantly higher in the post-stroke population (Supplemental Table V). Second, the NSQIP did not consistently collect information on perioperative medical management, such as perioperative statin or beta blocker use. These factors may be unmeasured confounders. Third, the Mashour, MICA, ACS-SRC were developed using the NSQIP dataset, giving these scores an advantage as they were originally validated in this population. Fourth, we could not distinguish primary ischemic strokes from hemorrhagic strokes as this information was not ascertained at the time of data collection. Risk factors for ischemic versus hemorrhagic stroke may differ,¹¹ limiting the predictive ability of perioperative risk models based on standard clinical data.

Conclusion:

In conclusion, MICA and ACS-SRC surgical risk scores provided excellent risk assessment for perioperative stroke among patients undergoing non-cardiac surgery, precluding the need to use separate scores to determine risk of stroke versus overall cardiovascular risk. The simple, 5-component MICA score was as accurate as the complex, 23-component ACS-SRC. All risk scores provided suboptimal accuracy in identifying patients at risk for stroke after vascular surgery. Future studies are needed to establish accurate risk models for stroke after vascular procedures.