Key Points
Question
Which patients with asymptomatic carotid stenosis are unlikely to benefit from carotid endarterectomy?
Findings
Among 2325 veterans in this cohort study that used Veterans Administration and Medicare data, a risk prediction tool (Carotid Mortality Index) based on 23 candidate variables and a simpler model based on the number of 4 key comorbidities (any cancer, chronic obstructive pulmonary disease, congestive heart failure, and chronic kidney disease [the 4C model]) were developed to identify patients at higher risk of 5-year mortality. The models were internally validated.
Meaning
Study results suggest that both the Carotid Mortality Index and the 4C model may be used to inform clinicians whether a patient will live long enough to benefit from carotid endarterectomy.
This cohort study uses Veterans Administration and Medicare data to develop a risk prediction tool to improve patient selection for carotid endarterectomy among patients with asymptomatic carotid stenosis.
Abstract
Importance
Randomized clinical trials have demonstrated that patients with asymptomatic carotid stenosis are eligible for carotid endarterectomy (CEA) if the 30-day surgical complication rate is less than 3% and the patient’s life expectancy is at least 5 years.
Objective
To develop a risk prediction tool to improve patient selection for CEA among patients with asymptomatic carotid stenosis.
Design, Setting, and Participants
In this cohort study, veterans 65 years and older who received both carotid imaging and CEA in the Veterans Administration between January 1, 2005, and December 31, 2009 (n = 2325) were followed up for 5 years. Data were analyzed from January 2005 to December 2015. A risk prediction tool (the Carotid Mortality Index [CMI]) based on 23 candidate variables identified in the literature was developed using Veterans Administration and Medicare data. A simpler model based on the number of 4 key comorbidities that were prevalent and strongly associated with 5-year mortality was also developed (any cancer in the past 5 years, chronic obstructive pulmonary disease, congestive heart failure, and chronic kidney disease [the 4C model]). Model performance was assessed using measures of discrimination (eg, area under the curve [AUC]) and calibration. Internal validation was performed by correcting for optimism using 500 bootstrapped samples.
Main Outcome and Measure
Five-year mortality.
Results
Among 2325 veterans, the mean (SD) age was 73.74 (5.92) years. The cohort was predominantly male (98.8%) and of white race/ethnicity (94.4%). Overall, 29.5% (n = 687) of patients died within 5 years of CEA. On the basis of a backward selection algorithm, 9 patient characteristics were selected (age, chronic kidney disease, diabetes, chronic obstructive pulmonary disease, any cancer diagnosis in the past 5 years, congestive heart failure, atrial fibrillation, remote stroke or transient ischemic attack, and body mass index) for the final logistic model, which yielded an optimism-corrected AUC of 0.687 for the CMI. The 4C model had slightly worse discrimination (AUC, 0.657) compared with the CMI model; however, the calibration curve was similar to the full model in most of the range of predicted probabilities.
Conclusions and Relevance
According to results of this study, use of the CMI or the simpler 4C model may improve patient selection for CEA among patients with asymptomatic carotid stenosis.
Introduction
Carotid endarterectomy (CEA) is one of the most common types of major surgery in the United States and is 1 of only 2 primary prevention surgical procedures available. Randomized clinical trials1,2 published in 1995 and 2004 showed that, among carefully selected patients with asymptomatic carotid stenosis and experienced surgeons, the procedure reduced the risk of stroke and death compared with medical therapy alone.
Revascularization with CEA is a trade-off between higher perioperative short-term risks in exchange for a lower long-term risk of stroke. Based on the aforementioned trials,1,2 guidelines concluded that candidate patients need to have a 30-day surgical complication rate less than 3% and a life expectancy of at least 5 years for the longer-term benefits of revascularization to outweigh the higher short-term risks of the procedure.3 Therefore, patient selection is a key factor in determining who benefits from intervention.4,5,6 In a randomized clinical trial, patients are excluded based on comorbid conditions; however, such exclusions may not be observed in routine practice.7,8,9 If in routine clinical practice patient selection does not conform to the original exclusion criteria outlined in the trials, patients may not live long enough to benefit from intervention. The development of risk prediction tools that can assist surgeons in identifying patients who are most likely to live long enough to benefit from CEA can improve patient outcomes.
Multiple studies10,11,12,13,14 have developed risk prediction tools focused on reducing 30-day complication rates. Few studies15,16,17 have focused on developing prediction models to identify patients who will live long enough to benefit from revascularization. In this cohort study, we use national Veterans Administration (VA) and Medicare data to examine factors predictive of 5-year mortality among older veterans with asymptomatic carotid stenosis who underwent CEA.
Methods
Cohort
We included all veterans who received both carotid imaging and CEA within 1 year of carotid imaging in the VA between January 1, 2005, and December 31, 2009. The dates of analysis were January 2005 to December 2015. We first identified 13 383 index carotid imaging tests among those 65 years and older performed in the VA between 2005 and 2009 using a previously developed natural language processing algorithm.18 This algorithm identified all veterans who had received carotid ultrasonography, magnetic resonance angiography, and computed tomographic angiography in the VA and who had at least 50% or moderate stenosis. We then manually reviewed the carotid imaging tests and excluded patients with less than 70% stenosis and patients with a stroke or transient ischemic attack in the prior 6 months. We then identified all veterans who had received CEA in the VA within 1 year of the carotid image (n = 2532). We excluded 181 veterans who had a stroke or transient ischemic attack between the imaging and CEA and excluded 26 patients who received a coronary artery bypass graft on the same day as carotid revascularization. The final cohort included 2325 veterans. These patients were followed up for 5 years from the date of CEA (2005-2015). This study was approved by the University of California, San Francisco Institutional Review Board.
Outcome
Our primary outcome was mortality due to any cause within 5 years of CEA. We also collected data on complications within 30 days of the procedure, including stroke, acute myocardial infarction, and death. We used the high-specificity algorithm by Tirschwell and Longstreth19 to identify strokes within 30 days. The VA Suicide Data Repository and the VA Vital Status Files were used to identify 30-day mortality.20 We identified 30-day acute myocardial infarction rates using the International Classification of Diseases, Ninth Revision (ICD-9) code 435.x on hospital discharges for admissions within 30 days of CEA.
Candidate Predictor Variables
Candidate predictor variables were selected based on review of published models in the literature.10,14,15,16,21 We identified previously published predictors of 1-year and 5-year survival.15,17,21 Because this literature was limited, we also included variables from a previously validated model of 30-day surgical complications that focused on frailty and function as predictors of 30-day surgical complications because these variables may be relevant to outcomes among older adults.14 We identified 23 candidate variables in the literature that were previously found to be predictive of either 30-day complication rates, 1-year mortality, or 5-year mortality as summarized in Table 1. Demographic variables, including age, sex, and race/ethnicity, were derived from VA and Medicare data. Data on comorbid conditions were constructed with ICD-9 codes using both VA and Medicare data. We required at least 2 outpatient visits or 1 inpatient visit with a diagnosis code to classify a patient as having a comorbid condition. Using a previously developed algorithm (eTable 1 in the Supplement), we identified all veterans who had a cancer diagnosis in the 5 years preceding CEA.22 Nonmetastatic prostate cancer was not included in this variable given the generally favorable prognosis and high prevalence of this condition among older men. The dementia variable was constructed using a combination of ICD-9 codes and data on use of medications (eg, donepezil hydrochloride) in the year before CEA. Data on medications were extracted from the VA Pharmacy Files. Data on body mass index were extracted from the Corporate Data Warehouse.20 Logical Observation Identifier Names and Codes (LOINC) were used to identify laboratory test data. We extracted data on kidney function, hemoglobin level, platelet count, and international normalized ratio from the Corporate Data Warehouse. We considered patients to have evidence of functional impairment if they were hemiplegic, had a pressure ulcer in the past year, were admitted to a nursing home, received home-based primary care, had a home health aide, or were dispensed a wheelchair, walker, or cane. Data on hemiplegia were extracted using ICD-9 codes, and data on pressure ulcers were extracted from the VA health factors files. Data on nursing home use and home health aide use were extracted from VA encounter files and VA Fee Basis files and Medicare data. Data on mobility devices were extracted from the VA prosthetics file and Medicare durable medical equipment files. Data on ejection fraction and pulmonary function were extracted from the medical record by trained abstractors (M.G., A.S.A., and A.J.Z.).
Table 1. Characteristics of 2325 Veterans Who Received Carotid Endarterectomy in the Veterans Administration Between 2005 and 2010.
| Variable | Value |
|---|---|
| Age, mean (SD), y | 73.74 (5.92) |
| Age group, y, No. (%) | |
| ≥80 | 398 (17.1) |
| 75-79 | 531 (22.8) |
| 70-74 | 631 (27.1) |
| 65-69 | 765 (32.9) |
| Male, No. (%) | 2298 (98.8) |
| Race/ethnicity, No. (%) | |
| White | 2195 (94.4) |
| Black | 98 (4.2) |
| Other | 28 (1.2) |
| Unknown | 4 (0.2) |
| Comorbid conditions, No. (%) | |
| Hypertension | 2170 (93.3) |
| Coronary artery disease | 1243 (53.5) |
| CKD | 1150 (49.5) |
| Diabetes | 932 (40.1) |
| COPD | 635 (27.3) |
| Any cancer diagnosis in the past 5 y preceding CEA, No. (%)a | 603 (25.9) |
| CHF | 409 (17.6) |
| Valvular disease | 311 (13.4) |
| Atrial fibrillation | 279 (12.0) |
| Dementia | 160 (6.9) |
| Remote stroke or TIA in the prior 6 mo | 150 (6.5) |
| Dialysis in the past 3 mo | 11 (0.5) |
| Tobacco use, No. (%) | |
| Current tobacco smoker | 778 (33.5) |
| Former tobacco smoker | 1324 (56.9) |
| Nonsmoker | 223 (9.6) |
| Contralateral stenosis levels, No. (%) | |
| Occluded | 159 (6.8) |
| 70%-99% | 330 (14.2) |
| <70% | 1836 (79.0) |
| BMI category, No. (%) | |
| Underweight | 25 (1.1) |
| Normal or healthy weight | 578 (24.9) |
| Overweight | 1030 (44.3) |
| Obese | 683 (29.4) |
| Unknown | 9 (0.4) |
| Any evidence of functional impairment, No. (%) | 242 (10.4) |
| Hemiplegia or another paralytic syndrome | 59 (2.5) |
| Pressure ulcer in the past year | 28 (1.2) |
| Use of nursing home | 17 (0.7) |
| Use of home-based primary care | 35 (1.5) |
| Use of home health aide | 113 (4.9) |
| Use of wheelchair | 17 (0.7) |
| Use of walker or cane | 45 (1.9) |
| Percent weight loss, No. (%) | |
| >15 | 28 (1.2) |
| 10-15 | 51 (2.2) |
| 5-9 | 173 (7.4) |
| <5 | 2056 (88.4) |
| Unknown | 17 (0.7) |
| Laboratory values, No. (%) | |
| Hemoglobin level <10 g/dL | 37 (1.6) |
| Platelet count <125 × 103/µL | 68 (2.9) |
| INR >1.5 | 117 (5.0) |
| Medications, No. (%) | |
| Taking statin | 1840 (79.1) |
| Taking antiplatelet medication | 1996 (85.8) |
| 4C model comorbid conditions, No. (%) | |
| 0 | 607 (26.1) |
| 1 | 909 (39.1) |
| 2 | 570 (24.5) |
| 3 or 4 | 239 (10.3) |
| 30-d Complication rate (combined) | 114 (4.9) |
| Stroke | 53 (2.3) |
| Acute myocardial infarction | 44 (1.9) |
| Death | 17 (0.7) |
| 5-y Mortality | 687 (29.5) |
Abbreviations: BMI, body mass index; CEA, carotid endarterectomy; CHF, congestive heart failure; CKD, chronic kidney disease; 4C model, any cancer, chronic obstructive pulmonary disease, congestive heart failure, and chronic kidney disease; COPD, chronic obstructive pulmonary disease; INR, international normalized ratio; TIA, transient ischemic attack.
SI conversion factors: To convert hemoglobin level to grams per liter, multiply by 10.0; to convert platelet count to ×109/L, multiply by 1.0.
Does not include diagnoses of prostate cancer.
Statistical Analysis
We summarized categorical variables by frequency and percentage; continuous variables were summarized with means (SDs). We also present 5-year mortality curves, stratified by the count of the 4C model comorbidities, estimated as the complement of the Kaplan-Meier survival curves. We conducted a simple univariate analysis to examine the association of each candidate predictor variable with the odds of 5-year mortality. We also report the unadjusted estimates of the association between the count of the following 4 comorbidities with 5-year mortality: (1) any cancer in the past 5 years, (2) chronic obstructive pulmonary disease (COPD), (3) congestive heart failure (CHF), and (4) chronic kidney disease (CKD) with estimated glomerular filtration rate (eGFR) less than 60 mL/min/1.73 m2. Odds ratios (ORs) and 95% CIs derived from simple logistic regression of 5-year mortality with each of the predictors are listed in Table 2. For comorbid variables with data on level of severity (eg, CKD [eGFR], COPD [forced expiratory volume in the first second of expiration], and diabetes [insulin or no insulin]), we compared the discrimination (area under the curve [AUC]) and fit (Akaike information criterion and Nagelkerke R2) of the model that incorporated their dichotomously coded (yes or no) versions with those of the model that incorporated the noncollapsed version of the comorbid variable, and the best-performing version was retained in further analyses (Table 2). To reduce the number of predictors to a set small enough to be clinically expedient, we performed model selection using backward variable elimination (with a threshold of 2-sided P < .005).23 We also tested different versions of comorbid conditions (eg, cancer in the past 1 year vs cancer in the past 5 years) by allowing them to compete in the backward elimination process. All pairwise interactions between the 9 variables in the resulting model (age, chronic kidney disease, diabetes, chronic obstructive pulmonary disease, any cancer diagnosis in the past 5 years, congestive heart failure, atrial fibrillation, remote stroke or transient ischemic attack, and body mass index) were tested and retained if 2-sided P < .001. We calculated the AUC and Nagelkerke R2 for the resultant model in the full cohort. We then used a bootstrap approach to obtain optimism-corrected measures of model performance. To do so, we (1) applied all model-building steps described previously in each bootstrap sample, (2) assessed the performance of this model in the bootstrap sample and in the original data set, and (3) took the difference of the 2 estimates obtained in (2) to get an estimate of optimism. Finally, the mean of 500 bootstrapped estimates of optimism was subtracted from the initial (full cohort model) estimate of the AUC and Nagelkerke R2 to obtain the bootstrap optimism-corrected estimates of performance.
Table 2. Univariate Predictors of 5-Year Mortality.
| Variable | No. Who Died/Total No. in the Subgroup (%) | Univariate OR (95% CI) |
|---|---|---|
| Age group, y | ||
| 70-74 | 180/631 (28.5) | 1.30 (1.02-1.65) |
| 75-79 | 157/531 (29.6) | 1.36 (1.06-1.75) |
| ≥80 | 170/398 (42.7) | 2.42 (1.87-3.10) |
| 65-69 | 180/765 (23.5) | 1 [Reference] |
| Race/ethnicitya | ||
| Black | 31/98 (31.6) | 1.10 (0.71-1.70) |
| Other | 4/28 (14.3) | 0.40 (0.14-1.14) |
| White | 651/2195 (29.7) | 1 [Reference] |
| Comorbid Conditions | ||
| CHF with EF >35% | 111/237 (46.8) | 2.61 (1.98-3.40) |
| CHF with EF ≤35% | 92/172 (53.5) | 3.40 (2.50-4.70) |
| No CHF | 484/1916 (25.3) | 1 [Reference] |
| CKD with eGFR 30-60 mL/min/1.73 m2 | 348/1052 (33.1) | 1.55 (1.29-1.87) |
| CKD with eGFR <30 mL/min/1.73 m2 | 55/98 (56.1) | 4.01 (2.60-6.10) |
| No CKD | 284/1175 (24.2) | 1 [Reference] |
| Dialysis in the past 3 mo | 9/11 (81.8) | 10.86 (2.30-50.00) |
| No dialysis in the past 3 mo | 678/2314 (29.3) | 1 [Reference] |
| Mild COPD | 125/386 (32.4) | 1.38 (1.08-1.75) |
| Moderate or severe COPD | 126/249 (50.6) | 2.95 (2.20-3.90) |
| No COPD | 436/1690 (25.8) | 1 [Reference] |
| Any cancer diagnosis in the past 5 y | 227/603 (37.6) | 1.66 (1.36-2.00) |
| No cancer diagnosis in the past 5 y | 460/1722 (26.7) | 1 [Reference] |
| Diabetes taking insulin | 94/234 (40.2) | 1.81 (1.36-2.40) |
| Diabetes not taking insulin | 216/698 (30.9) | 1.21 (0.99-1.47) |
| No diabetes | 377/1393 (27.1) | 1 [Reference] |
| Dementia | 68/160 (42.5) | 1.85 (1.33-2.60) |
| No dementia | 619/2165 (28.6) | 1 [Reference] |
| Remote stroke or TIA in the prior 6 mo | 63/150 (42.0) | 1.80 (1.28-2.50) |
| No remote stroke or TIA | 624/2175 (28.7) | 1 [Reference] |
| Coronary artery disease | 417/1243 (33.5) | 1.52 (1.27-1.82) |
| No coronary artery disease | 270/1082 (25.0) | 1 [Reference] |
| Atrial fibrillation | 129/279 (46.2) | 2.29 (1.78-3.00) |
| No atrial fibrillation | 558/2046 (27.3) | 1 [Reference] |
| Valvular disease | 125/311 (40.2) | 1.74 (1.36-2.20) |
| No valvular disease | 562/2014 (27.9) | 1 [Reference] |
| Vascular Risk Factors | ||
| Current tobacco smoker | 255/778 (32.8) | 1.53 (1.08-2.10) |
| Former tobacco smoker | 378/1324 (28.5) | 1.25 (0.90-1.74) |
| Nonsmoker | 54/223 (24.2) | 1 [Reference] |
| Hypertension | 646/2170 (29.8) | 1.18 (0.82-1.70) |
| No hypertension | 41/155 (26.5) | 1 [Reference] |
| Contralateral stenosis levels | ||
| Occluded | 52/159 (32.7) | 1.22 (0.86-1.73) |
| 70%-99% | 112/330 (33.9) | 1.29 (1.01-1.66) |
| <70% | 523/1836 (28.5) | 1 [Reference] |
| BMI category | ||
| Underweight | 18/25 (72.0) | 4.64 (1.91-11.30) |
| Overweight | 277/1030 (26.9) | 0.66 (0.53-0.83) |
| Obese | 186/683 (27.2) | 0.68 (0.53-0.86) |
| Unknown | 0/9 | 0 |
| Normal or healthy weight | 206/578 (35.6) | 1 [Reference] |
| Any evidence of functional impairment | ||
| Yes | 98/242 (40.5) | 1.73 (1.31-2.30) |
| No | 589/2083 (28.3) | 1 [Reference] |
| Laboratory Values | ||
| Hemoglobin level <10 g/dL | 22/37 (59.5) | 3.58 (1.85-6.90) |
| Hemoglobin level ≥10 g/dL | 665/2288 (29.1) | 1 [Reference] |
| Platelet count <125 × 103/µL | 30/68 (44.1) | 1.92 (1.18-3.10) |
| Platelet count ≥125 × 103/µL | 657/2257 (29.1) | 1 [Reference] |
| INR ≥1.5 | 49/117 (41.9) | 1.77 (1.21-2.60) |
| INR <1.5 | 638/2208 (28.9) | 1 [Reference] |
| Medications | ||
| Taking statin | 519/1840 (28.2) | 0.74 (0.60-0.92) |
| Not taking statin | 168/485 (34.6) | 1 [Reference] |
| Taking antiplatelet medication | 594/1996 (29.8) | 1.08 (0.83-1.39) |
| Not taking antiplatelet medication | 93/329 (28.3) | 1 [Reference] |
| 4C model comorbid conditions | ||
| 1 | 224/909 (24.6) | 1.41 (1.10-1.82) |
| 2 | 213/570 (37.4) | 2.58 (1.98-3.40) |
| 3 or 4 | 136/239 (56.9) | 5.71 (4.10-7.90) |
| 0 | 114/607 (18.8) | 1 [Reference] |
Abbreviations: BMI, body mass index; CHF, congestive heart failure; CKD, chronic kidney disease; 4C model, any cancer, chronic obstructive pulmonary disease, congestive heart failure, and chronic kidney disease; COPD, chronic obstructive pulmonary disease; EF, ejection fraction; eGFR, estimated glomerular filtration rate; INR, international normalized ratio; OR, odds ratio; TIA, transient ischemic attack.
SI conversion factors: To convert hemoglobin level to grams per liter, multiply by 10.0; to convert platelet count to ×109/L, multiply by 1.0.
Four individuals had unknown race/ethnicity.
Backward variable elimination is known to produce estimated regression coefficients that are biased and CIs that do not have a coverage probability of 95%. To improve estimation of regression coefficients and 95% CIs, we used the zero-corrected bootstrap (number of bootstrapped samples = 3000) model selection procedure. In a given bootstrap sample, predictor variables that are not selected for inclusion in the final regression model have their regression coefficient set to zero. Regression coefficients are averaged across the bootstrap samples, and nonparametric percentile bootstrap 95% CIs are then constructed for each regression coefficient.24
Finally, a multivariable logistic model consisting only of age group and a count of the number of the 4 comorbidities (any cancer, COPD, CHF, and CKD) was created (the 4C model). The AUC and Nagelkerke R2 were calculated, and internal validation was performed by correcting for optimism using 500 bootstrapped samples.25
To test whether the count of comorbidities was as good as allowing the individual comorbidity coefficient to vary, likelihood ratio tests comparing the 4C model (with the 4C model augmented with each comorbid condition separately) were performed. This analysis demonstrated that the simplifying assumption of equal weight for each comorbidity was reasonable for COPD, CHF, and CKD but not for cancer. However, the Nagelkerke R2 measure of model fit was similar for both the parsimonious and 4C models.
To examine and compare calibration of the 2 risk models (Carotid Mortality Index [CMI] and 4C), the observed occurrence of mortality within 5 years was regressed against the model-predicted probabilities using loess smoothing.26 Optimism-corrected calibration curves (number of bootstrap samples = 500) and the 45-degree reference line represent ideal model fit and were overlaid on the same plot.
A 5-year mortality CMI was created by multiplying the zero-corrected bootstrap regression coefficients by 10 and rounding, and then summing each patient score accordingly. The resulting score was approximately normally distributed, with a mean (SD) of 8.6 (8.5). The distribution of risk scores was divided into quintiles, and 5-year mortality (and 95% CI) for these 5 groups was estimated. We also present predicted probabilities and 95% CIs from the 4C model. All analyses were performed with statistical software (SAS Enterprise Guide, version 7.1; SAS Institute Inc).
Results
Cohort Characteristics
In total, 2325 veterans were included in the cohort. The mean (SD) age of the cohort was 73.74 (5.92) years. The cohort was predominantly male (98.8%) and of white race/ethnicity (94.4%). The most prevalent comorbidities were coronary artery disease (53.5% [n = 1243]), CKD (49.5% [n = 1150]), diabetes (40.1% [n = 932]), COPD (27.3% [n = 635]), any cancer (25.9% [n = 603]), and CHF (17.6% [n = 409]). Approximately 10% (n = 242) of veterans who received CEA had functional impairment (Table 1).
30-Day Complication Rate and 5-Year Mortality
The 30-day complication rate (composite of stroke, acute myocardial infarction, and death) was 4.9% (n = 114), with a combined stroke and death rate of 3.0% (n = 70) (Table 1). Overall, 29.5% (n = 687) of patients died within 5 years of CEA. Veterans 80 years and older had an observed mortality rate of 42.7% (170 of 398) (Table 2 and eFigure 1 in the Supplement). Patients with 0, 1, 2, and 3 or 4 comorbidities had observed mortality rates of 18.8% (114 of 607), 24.6% (224 of 909), 37.4% (213 of 570), and 56.9% (136 of 239), respectively (Table 2 and eFigure 2 in the Supplement).
Univariate Analyses
Among the 23 candidate variables evaluated, 20 were nominally associated with 5-year survival. Factors most strongly associated with mortality within 5 years included the following: age 80 years and older (OR, 2.42; 95% CI, 1.87-3.10), CHF with ejection fraction of 35% or less (OR, 3.40; 95% CI, 2.50-4.70), CKD with eGFR of less than 30 mL/min/1.73 m2 (OR, 4.01; 95% CI, 2.60-6.10), dialysis in the past 3 months (OR, 10.86; 95% CI, 2.30-50.00), moderate or severe COPD (OR, 2.95; 95% CI, 2.20-3.90), diabetes taking insulin (OR, 1.81; 95% CI, 1.36-2.40), dementia (OR, 1.85; 95% CI, 1.33-2.60), atrial fibrillation (OR, 2.29; 95% CI, 1.78-3.00), underweight (OR, 4.64; 95% CI, 1.91-11.30), and hemoglobin level of less than 10 g/dL (OR, 3.58; 95% CI, 1.85-6.90) (Table 2) (to convert hemoglobin level to grams per liter, multiply by 10.0).
Multivariable Models
Carotid Mortality Index
A risk prediction tool (CMI) based on 23 candidate variables identified in the literature was developed using VA and Medicare data. On the basis of our backward selection algorithm, we selected 9 patient characteristics (age, CKD, diabetes, COPD, any cancer diagnosis in the past 5 years, CHF, atrial fibrillation, remote stroke or transient ischemic attack, and body mass index) for the final logistic model for 5-year mortality. The final model had an AUC of 0.706. These 9 variables represent 97.5% of the potential AUC if all 23 candidate variables had been included in the model. Internal validation by bootstrapping analysis demonstrated an optimism-corrected AUC of 0.687 for the CMI (Table 3). A nomogram is provided in eTable 2 in the Supplement. An online calculator is also available (https://is.gd/CEA_Risk_Calculator).
Table 3. Comparison of 2 Multivariable Prediction Risk Models.
| Variable | Adjusted OR (95% CI) | |
|---|---|---|
| Zero-Corrected CMI | 4C Model | |
| Age group, y | ||
| 70-74 | 1.29 (0.98-1.68) | 1.23 (0.96-1.58) |
| 75-79 | 1.33 (0.99-1.79) | 1.19 (0.92-1.55) |
| ≥80 | 2.26 (1.69-3.32) | 2.05 (1.57-2.70) |
| 65-69 | 1 [Reference] | 1 [Reference] |
| CHF | 2.37 (1.80-3.12) | NA |
| No CHF | 1 [Reference] | NA |
| CKD with eGFR 30-60 mL/min/1.73 m2 | 1.30 (1.05-1.60) | NA |
| CKD with eGFR <30 mL/min/1.73 m2 | 2.76 (1.90-4.30) | NA |
| No CKD | 1 [Reference] | NA |
| Mild COPD | 1.19 (0.89-1.56) | NA |
| Moderate or severe COPD | 2.33 (1.73-3.20) | NA |
| No COPD | 1 [Reference] | NA |
| Any cancer diagnosis in the past 5 y | 1.58 (1.38-1.93) | NA |
| No cancer diagnosis in the past 5 y | 1 [Reference] | NA |
| Diabetes, taking insulin | 1.96 (1.50-2.65) | NA |
| Diabetes, not taking insulin | 1.41 (1.13-1.74) | NA |
| No diabetes | 1 [Reference] | NA |
| Remote stroke or TIA in the prior 6 mo | 2.01 (1.70-2.69) | NA |
| No remote stroke or TIA | 1 [Reference] | NA |
| Atrial fibrillation | 1.73 (1.52-2.16) | NA |
| No atrial fibrillation | 1 [Reference] | NA |
| BMI category | ||
| Underweight | 5.78 (2.33-17.3) | NA |
| Overweight | 0.68 (0.53-0.87) | NA |
| Obese | 0.65 (0.48-0.88) | NA |
| Unknown | 0.00 (0.00-0.00) | NA |
| Normal or healthy weight | 1 [Reference] | NA |
| 4C model comorbid conditions | ||
| 1 | NA | 1.31 (1.02-1.70) |
| 2 | NA | 2.33 (1.78-3.10) |
| 3 or 4 | NA | 5.22 (3.70-7.30) |
| 0 | NA | 1 [Reference] |
| Measures of model fit and discrimination | ||
| AUC | 0.706 | 0.661 |
| Optimism-corrected AUC | 0.687 | 0.657 |
| Nagelkerke R2 | 0.16 | 0.10 |
| Optimism-corrected Nagelkerke R2 | 0.13 | 0.09 |
Abbreviations: AUC, area under the curve; BMI, body mass index; CHF, congestive heart failure; CKD, chronic kidney disease; CMI, Carotid Mortality Index; 4C model, any cancer, chronic obstructive pulmonary disease, congestive heart failure, and chronic kidney disease; COPD, chronic obstructive pulmonary disease; eGFR, estimated glomerular filtration rate; NA, not applicable; OR, odds ratio; TIA, transient ischemic attack.
4C Model
The interaction between age and the number of comorbidities was not statistically significant. The count of the 4C model comorbidities was strongly associated with 5-year mortality, with the number of comorbidities demonstrating increasing odds of 5-year mortality with increasing comorbidity as follows: 1 (OR, 1.31; 95% CI, 1.02-1.70), 2 (OR, 2.33; 95% CI, 1.78-3.10), and 3 or 4 (OR, 5.22; 95% CI, 3.70-7.30). The model AUC was 0.661, and the optimism-corrected AUC was 0.657 (Table 3). The 4C model had slightly worse discrimination compared with the CMI model; however, the calibration curve was similar to the CMI model in most of the range of predicted probabilities.
Comparison of Model Performance
The 4C model had slightly worse fit (Table 3) than the CMI. However, calibration was as good as for the CMI for most of the range of predicted probabilities (Figure 1), with both models underestimating risk at higher levels (ie, calibration curves lay above the perfect-fit line).
Figure 1. Calibration of the CMI vs the 4C Model.
CMI indicates Carotid Mortality Index; 4C model, any cancer, chronic obstructive pulmonary disease, congestive heart failure, and chronic kidney disease.
Predicted Probabilities of 5-Year Mortality Based on the CMI and the 4C Model
Figure 2 shows the estimated probability of 5-year mortality based on quintiles of the CMI. eFigure 3 in the Supplement shows the estimated probability of 5-year mortality based on age and the number of comorbidities using the 4C model.
Figure 2. Estimated Probability of 5-Year Mortality Based on the CMI Binned Into 5 Groups.
Estimated mortality by quintile of the Carotid Mortality Index (CMI). The number of patients is 2325, and the number of events is 687. Error bars indicate 95% CIs.
Discussion
In this cohort study that used VA and Medicare data, 29.5% (687 of 2325) of veterans 65 years and older who received CEA died within 5 years. Outcomes among older veterans are similar to those of patients with Medicare coverage and patients revascularized in other health systems.27,28,29,30 Our data suggest that health systems can improve patient selection to ensure that patients will achieve a net benefit from carotid revascularization. We developed both simple and more complex risk prediction tools that can be used to improve patient selection for CEA in the VA.
Our study replicates and builds on the findings of 2 other published 5-year risk prediction models in the literature. One was developed in a sample of 4114 patients with asymptomatic carotid stenosis cared for by the Vascular Study Group of New England.16 This model found that age, diabetes, CHF, COPD, renal function, dialysis dependency, severity of contralateral stenosis, and statin use were associated with 5-year mortality. The other, an analysis of both 1-year and 5-year outcomes using the CEA Vascular Quality Initiative (VQI) registry, found that age, diabetes, coronary artery disease, COPD, renal function, dialysis dependency, and absence of statin use were associated with mortality.21,31 Contralateral stenosis was not evaluated in the VQI-derived model. In addition, patients whose data were included in the VQI registry who had evidence of a major medical contraindication (CHF class III/IV, left ventricular ejection fraction <30%, unstable angina, recent acute myocardial infarction, or COPD with forced expiratory volume in the first second of expiration <30%) were not eligible for CEA. Therefore, it appears that the severity of disease in the VQI population was different from that of our cohort, although the mean age was similar. We confirmed that all variables identified in the VQI model21,31 and by the Vascular Study Group of New England16 were important predictors of 5-year survival. However, our final model included several factors that were not evaluated in these articles. For example, the model based on the VQI registry did not include CHF as a variable, and neither study included cancer. Both of these variables contributed to model performance and were included in our final model. Our model did not include dialysis as a factor because only 11 patients in our sample were on dialysis. Our data also confirm prior work32 suggesting that patients on dialysis are not appropriate candidates for CEA, with 5-year mortality herein among those with dialysis in the past 3 months of 81.8% (9 of 11) (Table 2). The present study builds on the prior literature and includes all relevant variables identified in this earlier work.
This emerging body of research16,17 suggests that these risk prediction models should be used to improve patient selection for CEA. The CMI was developed based on a population of veterans with carotid stenosis who received CEA in the VA and is especially informative to practice in the VA. General mortality models (eg, the Lee Index33) have been developed using data from the US adult population and can also be incorporated into assessments regarding the benefits of CEA among patients with asymptomatic disease. The decision to operate could be individualized with use of multiple different mortality calculators15,16,17,33 that have been developed in recent years. However, at this juncture, it is unclear what 5-year mortality risk is acceptable from a public health perspective to ensure benefit from revascularization for an individual patient. Given that multiple risk scores developed in different populations have found comparable results, risk scores estimating 5-year mortality should be incorporated into updated guidelines on the management of patients with asymptomatic carotid stenosis. Expert panels should weigh in on what constitutes an acceptable risk of 5-year mortality before consideration of surgery and provide guidance to clinicians on how many patients should be exposed to a potentially unnecessary surgical procedure to prevent 1 stroke. Five-year survival among participants who received medical therapy in asymptomatic carotid stenosis trials could offer a basis for guidelines and provide the outer bounds of an acceptable 5-year mortality range to target using risk prediction tools.
To facilitate implementation clinically, we also developed a simpler model that can be easily remembered in clinical practice (any cancer in the past 5 years, COPD, CHF, and CKD [the 4C model]). This model, based on the number of 4 key comorbidities that were prevalent and strongly associated with 5-year mortality, demonstrated that patients with multiple 4C model comorbidities are less likely to benefit from revascularization. While the model is limited by equally weighting any cancer diagnosis, COPD, CHF, and CKD, it is easy to remember and provides a quick assessment of potential 5-year survival.
Limitations
Our study has several limitations that deserve comment. First, we did not include patient self-reported assessments of function because these data are unavailable in the VA electronic medical record. We used measures that were suggestive of poor function in the medical record, such as a recent pressure ulcer, recent admission to a nursing home, and use of mobility devices. This may be the reason why functional status was not retained in our final model, although other investigators have found it to be an important predictor of survival.33 Second, similar to other models,33 we found that cancer predicts poor survival. Given our sample size, we could not determine which cancers were the most important to consider and combined all types of cancer. The results of our study and other studies suggest that cancer diagnosis is important. A patient’s cancer type and overall prognosis should be weighed carefully in individual decision making. Third, we used backward selection to select predictor variables for inclusion in the final model; however, the candidate predictor variables with which we began were not selected based on convenience or availability but were selected based on prior research demonstrating that these factors were important in other 5-year mortality models. Fourth, the models in this work underwent only internal validation. However, we used computer-intensive methods to adjust our estimates of model performance for optimism. Future work should focus on external validation in a more recent period in the VA. Fifth, models derived in older male veterans may not be generalizable to younger veterans or the general adult population; however, CEA is more commonly performed among older men. Therefore, while the models are limited in generalizability, they are still instructive.
Conclusions
Both a risk prediction tool based on 9 variables (the CMI) and a simpler model based on the number of 4 key comorbidities (the 4C model) developed in this study can improve patient selection for CEA among patients with asymptomatic carotid stenosis in the VA. Five-year risk stratification scores should be incorporated into guidelines and routine practice to identify asymptomatic patients who are most likely to benefit from carotid revascularization.
eTable 1. Cancers Included in the “Cancer Variable” in the CMI Model
eTable 2. Nomogram to Estimate 5-Year Mortality
eFigure 1. Five-Year Mortality Curves by Age Group
eFigure 2. Five-Year Mortality by Number of Comorbidities (No C, 1C, 2C, and 3 or More Cs), Among All Ages
eFigure 3. Estimated Probability of 5-Year Mortality Based on Age and Number of Comorbidities Using the 4C Model
References
- 1.Executive Committee for the Asymptomatic Carotid Atherosclerosis Study. Endarterectomy for asymptomatic carotid artery stenosis. JAMA. 1995;273(18):1421-1428. doi: 10.1001/jama.1995.03520420037035 [DOI] [PubMed] [Google Scholar]
- 2.Halliday A, Mansfield A, Marro J, et al. ; MRC Asymptomatic Carotid Surgery Trial (ACST) Collaborative Group . Prevention of disabling and fatal strokes by successful carotid endarterectomy in patients without recent neurological symptoms: randomised controlled trial. Lancet. 2004;363(9420):1491-1502. doi: 10.1016/S0140-6736(04)16146-1 [DOI] [PubMed] [Google Scholar]
- 3.Chaturvedi S, Bruno A, Feasby T, et al. ; Therapeutics and Technology Assessment Subcommittee of the American Academy of Neurology . Carotid endarterectomy: an evidence-based review: report of the Therapeutics and Technology Assessment Subcommittee of the American Academy of Neurology. Neurology. 2005;65(6):794-801. doi: 10.1212/01.wnl.0000176036.07558.82 [DOI] [PubMed] [Google Scholar]
- 4.Chaturvedi S. Has evidence changed practice? appropriateness of carotid endarterectomy after the clinical trials. Neurology. 2007;69(2):224. doi: 10.1212/01.wnl.0000271911.54805.13 [DOI] [PubMed] [Google Scholar]
- 5.Halm EA, Tuhrim S, Wang JJ, Rojas M, Hannan EL, Chassin MR. Has evidence changed practice? appropriateness of carotid endarterectomy after the clinical trials. Neurology. 2007;68(3):187-194. doi: 10.1212/01.wnl.0000251197.98197.e9 [DOI] [PubMed] [Google Scholar]
- 6.Halm EA, Tuhrim S, Wang JJ, et al. . Racial and ethnic disparities in outcomes and appropriateness of carotid endarterectomy: impact of patient and provider factors. Stroke. 2009;40(7):2493-2501. doi: 10.1161/STROKEAHA.108.544866 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Murthy VH, Krumholz HM, Gross CP. Participation in cancer clinical trials: race-, sex-, and age-based disparities. JAMA. 2004;291(22):2720-2726. doi: 10.1001/jama.291.22.2720 [DOI] [PubMed] [Google Scholar]
- 8.Krumholz HM, Gross CP, Peterson ED, et al. . Is there evidence of implicit exclusion criteria for elderly subjects in randomized trials? evidence from the GUSTO-1 study. Am Heart J. 2003;146(5):839-847. doi: 10.1016/S0002-8703(03)00408-3 [DOI] [PubMed] [Google Scholar]
- 9.Masoudi FA, Havranek EP, Wolfe P, et al. . Most hospitalized older persons do not meet the enrollment criteria for clinical trials in heart failure. Am Heart J. 2003;146(2):250-257. doi: 10.1016/S0002-8703(03)00189-3 [DOI] [PubMed] [Google Scholar]
- 10.Calvillo-King L, Xuan L, Zhang S, Tuhrim S, Halm EA. Predicting risk of perioperative death and stroke after carotid endarterectomy in asymptomatic patients: derivation and validation of a clinical risk score. Stroke. 2010;41(12):2786-2794. doi: 10.1161/STROKEAHA.110.599019 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Press MJ, Chassin MR, Wang J, Tuhrim S, Halm EA. Predicting medical and surgical complications of carotid endarterectomy: comparing the risk indexes. Arch Intern Med. 2006;166(8):914-920. doi: 10.1001/archinte.166.8.914 [DOI] [PubMed] [Google Scholar]
- 12.Chaudhry SA, Afzal MR, Kassab A, Hussain SI, Qureshi AI. A new risk index for predicting outcomes among patients undergoing carotid endarterectomy in large administrative data sets. J Stroke Cerebrovasc Dis. 2016;25(8):1978-1983. doi: 10.1016/j.jstrokecerebrovasdis.2016.01.023 [DOI] [PubMed] [Google Scholar]
- 13.Gupta PK, Ramanan B, Mactaggart JN, et al. . Risk index for predicting perioperative stroke, myocardial infarction, or death risk in asymptomatic patients undergoing carotid endarterectomy. J Vasc Surg. 2013;57(2):318-326. doi: 10.1016/j.jvs.2012.08.116 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Melin AA, Schmid KK, Lynch TG, et al. . Preoperative frailty Risk Analysis Index to stratify patients undergoing carotid endarterectomy. J Vasc Surg. 2015;61(3):683-689. doi: 10.1016/j.jvs.2014.10.009 [DOI] [PubMed] [Google Scholar]
- 15.Conrad MF, Kang J, Mukhopadhyay S, Patel VI, LaMuraglia GM, Cambria RP. A risk prediction model for determining appropriateness of CEA in patients with asymptomatic carotid artery stenosis. Ann Surg. 2013;258(4):534-538. [DOI] [PubMed] [Google Scholar]
- 16.Wallaert JB, Cronenwett JL, Bertges DJ, et al. ; Vascular Study Group of New England . Optimal selection of asymptomatic patients for carotid endarterectomy based on predicted 5-year survival. J Vasc Surg. 2013;58(1):112-118. doi: 10.1016/j.jvs.2012.12.056 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Carmo M, Barbetta I, Bissacco D, et al. . Development and validation of a score to predict life expectancy after carotid endarterectomy in asymptomatic patients. J Vasc Surg. 2018;67(1):175-182. doi: 10.1016/j.jvs.2017.05.107 [DOI] [PubMed] [Google Scholar]
- 18.Mowery DL, Chapman BE, Conway M, et al. . Extracting a stroke phenotype risk factor from Veteran Health Administration clinical reports: an information content analysis. J Biomed Semantics. 2016;7:26. doi: 10.1186/s13326-016-0065-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Tirschwell DL, Longstreth WT Jr. Validating administrative data in stroke research. Stroke. 2002;33(10):2465-2470. doi: 10.1161/01.STR.0000032240.28636.BD [DOI] [PubMed] [Google Scholar]
- 20.U.S. Department of Veterans Affairs. The researcher’s guide to VA data: VA Information Research Center (VIReC). http://www.virec.research.va.gov/. Accessed September 19, 2011.
- 21.DeMartino RR, Brooke BS, Neal D, et al. . Development of a validated model to predict 30-day stroke and 1-year survival after carotid endarterectomy for asymptomatic stenosis using the Vascular Quality Initiative. J Vasc Surg. 2017;66(2):433-444.e2. doi: 10.1016/j.jvs.2017.03.427 [DOI] [PubMed] [Google Scholar]
- 22.Rassekh SR, Lorenzi M, Lee L, Devji S, McBride M, Goddard K. Reclassification of ICD-9 codes into meaningful categories for oncology survivorship research. J Cancer Epidemiol. 2010;2010:569517. doi: 10.1155/2010/569517 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Austin PC. Bootstrap model selection had similar performance for selecting authentic and noise variables compared to backward variable elimination: a simulation study. J Clin Epidemiol. 2008;61(10):1009-1017.e1. doi: 10.1016/j.jclinepi.2007.11.014 [DOI] [PubMed] [Google Scholar]
- 24.Austin PC. Using the bootstrap to improve estimation and confidence intervals for regression coefficients selected using backwards variable elimination. Stat Med. 2008;27(17):3286-3300. doi: 10.1002/sim.3104 [DOI] [PubMed] [Google Scholar]
- 25.Harrell FE Jr, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996;15(4):361-387. doi: [DOI] [PubMed] [Google Scholar]
- 26.Austin PC, Steyerberg EW. Graphical assessment of internal and external calibration of logistic regression models by using loess smoothers. Stat Med. 2014;33(3):517-535. doi: 10.1002/sim.5941 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Go C, Avgerinos ED, Chaer RA, et al. . Long-term clinical outcomes and cardiovascular events after carotid endarterectomy. Ann Vasc Surg. 2015;29(6):1265-1271. doi: 10.1016/j.avsg.2015.03.031 [DOI] [PubMed] [Google Scholar]
- 28.Jalbert JJ, Nguyen LL, Gerhard-Herman MD, et al. . Outcomes after carotid artery stenting in Medicare beneficiaries, 2005 to 2009. JAMA Neurol. 2015;72(3):276-286. doi: 10.1001/jamaneurol.2014.3638 [DOI] [PubMed] [Google Scholar]
- 29.Hertzer NR, Arison R. Cumulative stroke and survival ten years after carotid endarterectomy. J Vasc Surg. 1985;2(5):661-668. doi: 10.1016/0741-5214(85)90035-7 [DOI] [PubMed] [Google Scholar]
- 30.Kragsterman B, Björck M, Lindbäck J, Bergqvist D, Pärsson H; Swedish Vascular Registry (Swedvasc) . Long-term survival after carotid endarterectomy for asymptomatic stenosis. Stroke. 2006;37(12):2886-2891. doi: 10.1161/01.STR.0000248967.44015.88 [DOI] [PubMed] [Google Scholar]
- 31.Eslami MH, Rybin D, Doros G, Farber A. An externally validated robust risk predictive model of adverse outcomes after carotid endarterectomy. J Vasc Surg. 2016;63(2):345-354. doi: 10.1016/j.jvs.2015.09.003 [DOI] [PubMed] [Google Scholar]
- 32.Yuo TH, Sidaoui J, Marone LK, Makaroun MS, Chaer RA. Revascularization of asymptomatic carotid stenosis is not appropriate in patients on dialysis. J Vasc Surg. 2015;61(3):670-674. doi: 10.1016/j.jvs.2014.10.002 [DOI] [PubMed] [Google Scholar]
- 33.Lee SJ, Lindquist K, Segal MR, Covinsky KE. Development and validation of a prognostic index for 4-year mortality in older adults. JAMA. 2006;295(7):801-808. doi: 10.1001/jama.295.7.801 [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
eTable 1. Cancers Included in the “Cancer Variable” in the CMI Model
eTable 2. Nomogram to Estimate 5-Year Mortality
eFigure 1. Five-Year Mortality Curves by Age Group
eFigure 2. Five-Year Mortality by Number of Comorbidities (No C, 1C, 2C, and 3 or More Cs), Among All Ages
eFigure 3. Estimated Probability of 5-Year Mortality Based on Age and Number of Comorbidities Using the 4C Model