A Validated Risk Model to Predict 90-Day VTE Events in Postsurgical Patients

Abstract

Background:

VTE is the proximate cause of 100,000 deaths in the United States each year. Perioperative VTE risk among surgical patients varies by 20-fold, which highlights the importance of risk stratification to identify high-risk patients, in whom chemoprophylaxis can decrease VTE risk, and low-risk patients, for whom the risk-benefit relationship of prophylaxis may be unfavorable.

Methods:

We used data from a statewide surgical quality collaborative for surgical procedures performed between 2010 and 2012. Regression-based techniques identified predictors of 90-day VTE while adjusting for procedural complexity and comorbid conditions. A weighted risk index was created and was validated subsequently in a separate, independent dataset.

Results:

Data were available for 10,344 patients, who were allocated randomly to a derivation or validation cohort. The 90-day VTE rate was 1.4%; 66.2% of the derivation cohort and 65.5% of the validation cohort received chemoprophylaxis. Seven risk factors were incorporated into a weighted risk index: personal history of VTE, current cancer, sepsis/septic shock/systemic inflammatory response syndrome, age ≥ 60 years, BMI ≥ 40 kg/m², male sex, and family history of VTE. Prediction for 90-day VTE was similar in the derivation and validation cohorts (areas under the receiver operator curve, 0.72 and 0.70, respectively). An 18-fold variation in 90-day VTE rate was identified.

Conclusions:

A weighted risk index quantifies 90-day VTE risk among surgical patients and identifies an 18-fold variation in VTE risk among the overall surgical population.

Annually, 12 million hospitalized patients are at risk of VTE, which includes DVT and pulmonary embolus (PE).^1,2 Nearly 600,000 adults are given a diagnosis of VTE each year, and PE is responsible for > 100,000 deaths annually in the United States.^3‐5 This staggering number of PE-associated deaths is higher than the annual mortality from breast cancer and motor vehicle accidents combined.^6,7

Patients with VTE have unacceptably high short-term mortality and long-term morbidity, which makes prevention of VTE paramount. Prevention begins with an understanding of a patient’s risk. Studies have shown that a 10- to 20-fold difference in VTE risk exists among the overall surgical population.^8‐11 The American College of Chest Physicians (ACCP) guidelines on VTE prevention have long been the standard for conceptualizing VTE risk.¹² The most recent ACCP guidelines, Antithrombotic Therapy and Prevention of Thrombosis, 9th ed: ACCP Evidence-Based Clinical Practice Guidelines, acknowledge the importance of individualized risk stratification.¹² However, the guidelines note that the available models lack rigorous statistical development, extensive validation or both in large, independent patient populations. The labor-intensive nature of the existing models, which require yes/no answers to between 18 and 40 questions, makes accurate and reliable completion by physicians unlikely.^12‐15 Among the existing risk models that have been developed rigorously from large databases,^8,15 none was able to incorporate VTE-centric risk factors such as a personal or family history of VTE, central venous catheter (CVC), or thrombophilia. The existing risk models were also developed to predict 30-day VTE risk, although studies have shown that the postoperative risk of VTE may extend to 90 days after surgery.¹⁶

In this study, we used prospective data from a statewide clinical registry that includes VTE-specific risk factors to create a risk-prediction tool for 90-day VTE in surgical patients. We also validated the predictive capacity of this risk model in a separate, independent dataset of surgical patients.

Materials and Methods

This study was part of a larger, statewide, observational, quality-improvement initiative. No care interventions were mandated. Thus, signed patient consent was waived. Analyses of deidentified Michigan Surgical Quality Collaborative (MSQC) data were approved by the University of Michigan institutional review board (HUM0025951).

The MSQC is a partnership among 52 Michigan hospitals, Blue Cross Blue Shield of Michigan, and the Blue Care Network. The MSQC methodology has been described previously.^17,18

This project was limited to inpatient, nonemergent surgical cases (Table 1). The VTE data collection tool was affixed to the virtual workstation at 10 MSQC sites and was launched automatically when inclusion criteria were satisfied. Exclusion criteria included age < 18 years and admission for palliative care. Patients with recently diagnosed VTE for which they were actively receiving anticoagulation treatment were also excluded.

Table 1.

—Derivation Cohort Categorized by Primary Procedure Type

CPT Range of Primary Procedure	Type of Operation by Organ System or Area of Body	Total Patients (N = 6,768), No. (%)	Patients with DVT/PE (n = 95), No. (% Incidence Within the CPT Group)
10000-19999	Integument	455 (6.7)	2 (0.4)
20000-29999	Musculoskeletal	301 (4.5)	4 (1.3)
30000-33999	Respiratory and cardiovascular	110 (1.6)	2 (1.8)
34000-37799	Arteries and veins	1,391 (20.6)	9 (0.6)
38000-39999	Hemic and lymphatic system, mediastinum, and diaphragm	70 (1.0)	2 (2.9)
40000-43499 and 69500-69650	Head and neck, esophagus	70 (1.0)	1 (1.4)
43500-43999	Foregut (stomach, including gastric bypass procedure)	775 (11.5)	11 (1.4)
44000-46999	Hindgut (small bowel, large bowel, rectum, and anus)	1,988 (29.4)	40 (2.0)
47000-48999	Liver, biliary system, and pancreas	733 (10.8)	11 (1.5)
49000-49490	Miscellaneous peritoneal procedures	123 (1.8)	3 (2.4)
49491-49999	Herniorrhaphy	467 (6.9)	6 (1.3)
50000-53999	Urinary system	16 (0.2)	0
54000-59999	Genital system (male or female)	99 (1.5)	2 (2.0)
60000-60999	Endocrine	111 (1.6)	2 (1.8)
61000-64999	Nervous system structures	59 (0.9)	0

CPT = Current Procedural Terminology; PE = pulmonary embolus.

Data acquisition took place between March 2010 and October 2012. At the time of the project, the MSQC used the American College of Surgeons’ National Surgical Quality Improvement Program platform to standardize data collection, using rigorous definitions for risk factors and outcomes. Thus, data from all MSQC sites underwent regular external audits to ensure data fidelity.¹⁷ Data acquisition was performed by formally trained case reviewers. Excellent interrater reliability, with > 98.5% variable agreement during audits, has been shown previously.¹⁷ Reviewers were required to make patient contact at 90 days via phone call or letter or to conduct a thorough review of medical records to identify postoperative complications diagnosed or managed at other institutions. Patients who lacked a 90-day follow-up were not included in the final database provided to the authors. Consequently, demographic data were not available for patients who lacked a 90-day follow-up.

Independent variables (Table 2) were defined rigorously. Definitions are summarized in e-Appendix 1 ^{(375.4KB, pdf)} . The primary outcome was 90-day, non-CVC-associated VTE, including patients with either DVT or PE. Upper-extremity DVT included clots in the jugular, subclavian, axillary, or brachial veins. Lower-extremity DVT included clots in the vena cava, femoral, tibial, or popliteal veins. Visceral DVT (eg, portal or mesenteric vein) or cerebral sinus thrombosis were not included in the primary outcome. PE included clots in the pulmonary vasculature. All VTE events were diagnosed using an objective imaging study. To avoid confounding in the final regression model, patients with CVC-associated DVT were considered to have only DVT if DVT was present at another anatomic site without a CVC. Similarly, PEs in the setting of isolated CVC-associated DVT were not considered PE events.

Table 2.

—Bivariate Statistics Comparing Rate of VTE in Patients Who Did or Did Not Have Individual Risk Factors

Risk Factor	No DVT/PE (n = 6,673), No. (%)	Yes DVT/PE (n = 95), No. (%)	P Value	OR (95% CI)
Age ≥ 60 y	3,652 (54.7)	65 (68.4)	.009	1.79 (1.16-2.77)
BMI ≥ 40 kg/m2	1,640 (24.6)	26 (27.4)	.53	1.16 (0.73-1.82)
Male sex	2,823 (42.3)	50 (52.6)	.044	1.51 (1.01-2.27)
Ethnicity
White	5,094 (76.3)	73 (76.8)	Reference	Reference
Black	814 (12.2)	12 (12.6)	.93	1.03 (0.56-1.90)
Asian	715 (10.7)	9 (9.5)	.72	0.88 (0.44-1.76)
Other	50 (0.8)	1 (1.1)	.74	1.40 (0.19-10.24)
Smoking
0 pack-y	4,233 (63.4)	62 (65.3)	Reference	Reference
< 25 pack-y	939 (14.1)	13 (13.7)	.85	0.95 (0.52-1.73)
≥ 25 pack-y	1,501 (22.5)	20 (21.1)	.72	0.91 (0.55-1.51)
Diabetes requiring medication	1,464 (21.9)	23 (24.2)	.60	1.14 (0.71-1.82)
COPD	605 (9.1)	11 (11.6)	.40	1.31 (0.70-2.48)
Pneumonia	52 (0.8)	3 (3.2)	.01	4.15 (1.27-13.54)
Congestive heart failure	79 (1.2)	4 (4.2)	.008	3.67 (1.31-10.23)
History of stroke or TIA	795 (11.9)	10 (10.5)	.68	0.87 (0.45-1.68)
Peripheral vascular disease	641 (9.6)	6 (6.3)	.28	0.64 (0.28-1.46)
Sepsis, septic shock, or SIRS	481 (7.2)	14 (14.7)	.005	2.22 (1.25-3.95)
Coronary disease	1,150 (17.2)	21 (22.1)	.21	1.36 (0.84-2.22)
Prior operation within 30 d	181 (2.7)	4 (4.2)	.37	1.58 (0.57-4.34)
Quadriplegia or paraplegia	58 (0.9)	2 (2.1)	.20	2.45 (0.59-10.19)
Varicose veins	381 (5.7)	3 (3.2)	.29	0.54 (0.17-1.71)
Hypercoagulable disorder (any)	53 (0.8)	1 (1.1)	.78	1.33 (0.18-9.71)
Personal history of DVT or PE	514 (7.7)	16 (16.8)	.001	2.43 (1.41-4.18)
Family history of DVT or PE	202 (3.0)	6 (6.3)	.065	2.16 (0.93-4.99)
Inflammatory bowel disease	215 (3.2)	1 (1.1)	.23	0.32 (0.04-2.30)
Leg immobilization	17 (0.3)	0	.62	…
Hip, pelvis, or proximal femur fracture	30 (0.5)	0	.51	…
Central venous catheter	508 (7.6)	17 (17.9)	< .001	2.64 (1.55-4.50)
Multisystem trauma	29 (0.4)	0	.52	…
Current cancer	920)13.8)	29 (30.5)	< .001	2.75 (1.77-4.28)
Females onlya
Pregnant or postpartum	22 (0.6)	0	.61	…
Oral contraceptives or HRT	229 (6.0)	1 (2.2)	.29	0.36 (0.05-2.62)

HRT = hormone replacement therapy; SIRS = systemic inflammatory response syndrome; TIA = transient ischemic attack. See Table 1 legend for expansion of other abbreviations.

^aNo DVT/PE: n = 3,847; Yes DVT/PE: n = 45.

Statistical Analysis

Data analysis was performed using the Stata11 statistical package (StataCorp LP), and descriptive statistics on DVT, PE, and VTE incidence were generated. Bivariate statistics examined the relationship of individual variables to the VTE outcome. For improved clinical relevance, continuous variables (age, BMI, and pack-years of smoking) were transformed empirically into categorical variables.

Risk Model Generation:

Patients were allocated randomly to a derivation (66.6%) or validation (33.3%) cohort. The derivation cohort was used for model creation. Analyses are depicted in flowchart form in Figure 1.

Figure 1.

Flowchart for risk model creation. CPT = Current Procedural Terminology.

Collinearity diagnostics were performed between all independent variables. No variable demonstrated a variance inflation factor > 1.25. The mean variance inflation factor was 1.10. All variables were entered into a forward stepwise regression with entry criteria P < .15. Variables meeting the entry criteria were included in the final model.

We created a linear risk adjustment score using the log odds of the risk predicted by logistic regression modeling. This score represented the predicted aggregate risk of VTE based on patient-level data for variables not included in the final model. Procedural complexity was quantified using Raval et al’s¹⁹ “bucket” hierarchical framework, which has been shown to improve procedural risk estimates when compared with relative value units, broad Current Procedural Terminology (CPT) range groups, or both. We created a linear risk adjustment score based on the “bucket” into which the index CPT code was placed. Both risk adjustment and procedural risk scores were entered into the final model as continuous variables.

Logistic regression was performed, with 90-day VTE as the dependent variable. Independent variables included those meeting the criteria of the forward stepwise regression, the risk adjustment score, and the procedural risk score. The bootstrap resampling technique (1,000 repetitions) generated bias-corrected CIs around risk factors in our final model.²⁰ A P value of < .10 was predefined as the cutoff for inclusion in the final risk model.

Weighted risk scores produce clinically relevant improvement in VTE risk stratification.^8,14,15 We created a weighted risk score by dividing the β coefficient for each predictor with P < .10 by the smallest β coefficient of the predictors. This value was rounded to the nearest integer.²¹ The total risk score was treated as a continuous variable. Model fit was examined using the area under the receiver operator curve (AUROC).

Testing the Model in an Independent Dataset:

The discriminating capacity of the total risk score was determined using the AUROC for the derivation and validation cohorts. Stratified analysis that examined 90-day VTE rate by risk score in both cohorts was performed. We then calculated 95% CIs around proportions using the Wilson interval without continuity correction.

Results

Data were available for 10,344 patients. Among the 6,768 patients in the derivation cohort, the 90-day rate of upper-extremity DVT was 0.59%, CVC-associated DVT was 0.30%, and lower-extremity DVT was 0.77%. The overall rate of non-CVC-associated DVT was 1.32%, and 90-day PE occurred in 0.62%. The overall rate of non-CVC-associated VTE was 1.40%.

Bivariate statistics showed multiple associations between independent variables and 90-day VTE (Table 2). Eight variables met the entry criteria of the initial forward stepwise logistic regression model (Fig 1, Table 3). These variables, in addition to the risk adjustment and procedural risk scores, were placed into the final model. The Hosmer and Lemeshow test for the final model showed a χ² value of 13.21, degrees of freedom of 8, and P = .1049.

Table 3.

—Adjusted OR and Bias-Corrected CIs for Independent Variables Identified by the Stepwise Logistic Regression Model (90-d VTE Was the Dependent Variable)

Risk Factor	Regression Coefficient	Adjusted OR	95% CI Based on Original Sample	Bias-Corrected 95% CIa	P Valuea
Current cancer	1.07	2.93	1.79-4.78	1.68-5.10	< .001
Personal history of DVT or PE	0.76	2.14	1.22-3.75	1.21-3.80	.009
Sepsis, septic shock, or SIRS	0.60	1.83	0.97-3.43	0.91-3.67	.091
Age ≥ 60 y	0.23	1.26	1.00-1.58	0.99-1.60	.062
BMI ≥ 40 kg/m2	0.22	1.24	0.98-1.58	0.97-1.60	.087
Family history of DVT or PE	0.79	2.21	0.94-5.21	0.85-5.72	.103
Male sex	0.40	1.50	0.97-2.31	0.95-2.37	.085
Central venous catheter	0.47	1.60	0.90-2.85	0.87-2.93	.130
Risk adjustment scoreb	0.69	1.99	1.28-3.10	1.31-3.03	.001
Procedural risk scorec	−0.56	0.57	0.03-13.46	0.025-12.75	.723

See Table 1 and 2 legends for expansion of abbreviations.

^aCalculated using bias-corrected SE estimates obtained using the bootstrap resampling technique (1,000 repetitions).

^bRisk adjustment score is the predicted risk of VTE based on patient-level data for variables not included after stepwise regression, including oral contraceptives or HRT, COPD, pneumonia, congestive heart failure, coronary disease, prior operation within 30 d, cigarette use, diabetes requiring medication, stroke, peripheral vascular disease, quadriplegia/paraplegia, varicose veins, hypercoagulable state, and inflammatory bowel disease.

^cProcedural risk score is the predicted risk of VTE based on procedural complexity.¹⁹

In the final bootstrapped model, the bias-corrected P value for family history of VTE (P = .103) did not meet our de novo cutoff of P < .10. However, a family history of VTE has face validity for inclusion. In addition, the model with family history of VTE provided a clinically relevant improvement in risk stratification. Finally, the model that included family history of VTE explained a larger proportion of the variability in VTE outcome (AUROC, 0.69 vs AUROC, 0.68). Thus, the final weighted risk model that included age ≥ 60 years, BMI ≥ 40 kg/m², male sex, sepsis/septic shock/systemic inflammatory response syndrome (SIRS), personal history of VTE, family history of VTE, and current cancer (Fig 2) was applied to the validation cohort.

Figure 2.

Weighted risk model for VTE risk stratification. Hx = history; SIRS = systemic inflammatory response syndrome.

In total, 3,576 patients were allocated randomly to the validation cohort. The 90-day VTE rate between the derivation (95 of 6,768 patients [1.40%]) and validation (50 of 3,576 patients [1.40%]) cohorts were not significantly different (P = .98). The observed 90-day VTE rates were similar between cohorts at each risk level (Fig 3). The validation cohort AUROC (0.70) was similar to the derivation cohort AUROC (0.69).

Figure 3.

Rate of VTE stratified by risk score in the derivation and validation cohorts.

When stratified by risk score, there were no clear trends in the rates of overall chemoprophylaxis or chemoprophylaxis within 24 or 72 h (Table 4). The proportion of patients who received no chemoprophylaxis was not significantly different between the derivation and validation cohorts (33.8% vs 34.5%, P = .44). Among patients who received prophylaxis, there was no significant difference between the derivation and validation cohorts in the proportion of patients who received prophylaxis within 24 h (60.1% vs 58.7%, P = .26) or 72 h (97.5% vs 97.2%, P = .49).

Table 4.

—Proportion of Patients Who Received Chemoprophylaxis Stratified by Risk Score and Timing of Chemoprophylaxis (N = 10,344)

Risk Score	Prophylaxis Within 24 h	Prophylaxis Within 72 h	Prophylaxis During Hospitalization
0 (n = 1,179)	487 (41.3)	706 (59.9)	715 (60.6)
1-2 (n = 3,865)	1,707 (44.2)	2,620 (67.8)	2,694 (69.7)
3-5 (n = 3,531)	1,218 (34.5)	2,124 (60.2)	2,183 (61.8)
6-8 (n = 1,520)	576 (37.9)	1,036 (68.2)	1,065 (70.1)
> 8 (n = 249)	80 (32.1)	163 (65.5)	169 (67.9)

Data are presented as No. (%).

Discussion

This analysis of data from a statewide surgical quality collaborative demonstrates that an 18-fold variation in VTE risk exists among the overall surgical population and that a weighted risk scoring system can be used to quantify 90-day VTE risk after surgery. The capacity of our risk model to risk discriminate has several implications. The model allows the identification of a specific high-risk group of patients who would benefit from additional research to examine the appropriate timing, intensity, and duration of prophylaxis. Previous work has shown that chemoprophylaxis significantly reduces VTE risk in high-risk patients, and that this population may have an additional benefit from extended-duration prophylaxis.^22‐24 Our risk model also identifies a very-low-risk group of patients (90-day VTE risk of one in 400 patients); the risk and cost of chemoprophylaxis in this group requires further examination.

In 2008, then-Surgeon General Steven K. Galson published his “Call to Action” on VTE prevention.⁵ This treatise identified VTE as a major public health problem and specifically advocated for the development of evidence-based practices for VTE risk stratification and prevention. Postoperative DVT and PE rates are considered to be quality indicators by major policymakers and payers.^25‐27 A thorough understanding of patient-level risk is the most important first step toward evidence-based VTE prevention.

The concept that VTE risk can be quantified in surgical patients is well known. The Caprini score was created based on expert opinion > 20 years ago but has subsequently been validated retrospectively in multiple populations.^9‐11,28 However, the model requires yes/no answers to 40 risk factor questions, which may limit physician compliance with completion.¹³ The Rogers score was published in 2007 after being developed using rigorous statistical methods based on data from the Veterans Administration Patient Safety Study. The study was limited to the variables available in the Patient Safety Study, which does not include VTE-centric risk factors.¹⁵

The 2012 ACCP consensus guidelines on VTE prevention recommend use of an individualized risk-stratification tool to understand VTE risk. The guidelines also note that the existing tools are cumbersome.¹² “Reduced” models with fewer risk factors have been shown to predict risk similarly to full models.²⁹ The addition of more risk factors to existing risk models may impede, not improve, risk stratification.³⁰ The AUROC for the presented model (0.70) is comparable to the AUROC for the 2007 Rogers score (0.76).¹⁵ The presented risk model requires only seven risk factors, which is two- to sixfold fewer factors than existing risk models.^14,15 This fact is particularly relevant because more-complex risk models may not be completed accurately by providers.¹³

Limitations

Patients who lacked a 90-day follow-up were excluded from this dataset. Because demographic data were not available for these patients, we cannot be certain whether systematic differences were present between patients with and without a 90-day follow-up.

The goal of this study was to create a risk-prediction tool that could risk stratify patients prior to surgery. Thus, to create this model, we used only risk factors known or able to be estimated in the preoperative setting. We did not incorporate VTE prophylaxis or postoperative complications into this model. Models that predict VTE risk based on pre-, intra-, and postoperative events do exist. However, by definition, such models can predict a patient’s VTE risk only at the end of their hospitalization, when the opportunity for acute prophylaxis has passed.

Several factors may have contributed to the 30% variability in VTE outcome not explained by the presented model. This was an observational study, and providers administered mechanical and pharmacologic VTE prophylaxis on a case-by-case basis. Prophylaxis type, timing, and duration were not standardized. In addition, several potentially important risk factors (eg, hypercoagulable disorder and quadriplegia/paraplegia) occurred infrequently in our dataset and may have been incorporated into the final model were a larger data pool available. We recommend that the calculated risk score be used as an initial guide to estimate perioperative VTE risk. However, physicians must use their own judgment to take into account potentially important and plausible risk factors that we were unable to incorporate rigorously into the presented model.

The perceived risk of hemorrhage is a recognized barrier to physicians providing chemoprophylaxis after surgery. We did not analyze postoperative bleeding requiring transfusion or operative drainage in this study because the chemoprophylaxis regimen was not standardized. Meta-analyses have shown that < 1% of patients require an additional operative procedure because of chemoprophylaxis-associated bleeding.³¹ However, the available meta-analysis data examined the surgical population as a whole. Bleeding risk was not stratified by VTE risk, which is a marker of the aggressiveness of prophylaxis regimen. We recommend that further work present data on the risk-stratified response to prophylaxis side by side with the risk-stratified data on bleeding complications. This approach will allow physicians to make informed decisions about the risks and benefits of prophylaxis for individual patients.

Clinical examination underestimates total VTE burden.^22,23,32,33 Screening studies in asymptomatic high-risk surgical patients have shown VTE rates > 15%.^34‐36 This study did not screen routinely for asymptomatic VTE. Thus, our reported VTE rates likely underestimate the true VTE burden. The clinical significance of asymptomatic VTE remains unclear. Of note, ACCP guidelines do not recommend screening for asymptomatic patients, including those who are high risk.¹²

Conclusions

The simple, seven-factor VTE risk model presented here allows the identification of an 18-fold variation in VTE risk among postoperative patients. This risk model allows physicians to quantify VTE risk in the preoperative setting, which will facilitate a data-driven discussion with patients about the estimated perioperative VTE risk and will improve the process of informed surgical consent. This research represents the first of three steps in prediction modeling research, which include development and internal validation, external validation, and demonstration that use of the prediction model has an effect at the patient level. The latter two steps represent important directions for further research.

Supplementary Material

Online Supplement

Abbreviations

ACCP

American College of Chest Physicians

AUROC

area under the receiver operator curve

CPT

Current Procedural Terminology

CVC

central venous catheter

MSQC

Michigan Surgical Quality Collaborative

PE

pulmonary embolus

SIRS

systemic inflammatory response syndrome