Derivation and External Validation of a Risk Assessment Model of Venous Thromboembolism in Hospitalized Chinese Patients

Abstract

Aim

To develop and validate a risk assessment model (RAM) of venous thromboembolism (VTE) in hospitalized Chinese patients.

Methods

We reviewed data from 300 patients with VTE and 300 non-VTE patients at Beijing Shijitan Hospital. The risk factors related to VTE were analyzed, and the VTE RAM (Shijitan (SJT) version) was developed according to the weight of each risk factor. A total of 407 patients with VTE and 533 non-VTE patients were enrolled for external validation. The sensitivity, specificity, Youden index, receiver operating curve (ROC), and area under the ROC curve (AUC) were used to evaluate the performance of VTE RAM (SJT version) compared with Caprini RAM and Padua RAM.

Results

The VTE RAM (SJT version) contained six risk factors (age >60 years, lower limb edema, chronic obstructive pulmonary disease (COPD), central venous catheterization (CVC), VTE history, and D dimer). In the external validation group, for medical patients, the AUC value of SJT RAM (0.82 ± 0.03) is significantly higher than Caprini RAM (0.76 ± 0.04; P < 0.05), SJT RAM has a higher sensitivity, specificity, and Youden index than Caprini RAM (P < 0.05), which means that the SJT RAM has a much better predictive value than Caprini RAM. While SJT RAM and Padua RAM have the similar predictive value for medical patients (P > 0.05). For surgical patients, the AUC value of SJT RAM (0.72 ± 0.04) is significantly higher than the value of Padua RAM (0.66 ± 0.04; P < 0.05), SJT RAM has a higher sensitivity, specificity, and Youden index than Padua RAM (P < 0.05), which shows that the VTE RAM has better predictive value than Padua RAM. While SJT RAM and Caprini RAM have the similar predictive value for surgical patients (P > 0.05).

Conclusion

The SJT RAM derived from general hospitalized Chinese patients will be time-saving for physicians and has a better predictive ability for patients at risk of VTE.

Introduction

Venous thromboembolism (VTE) includes deep vein thrombosis (DVT) and pulmonary embolism (PE) and is the most likely preventable fatal disease. Identifying high-risk patients can not only reduce the mortality and morbidity caused by VTE but also reduce the medical tangles caused by sudden death in hospitals. Brandyn D. Lau mentioned that VTE was an adverse event that could be prevented during hospitalization. Risk assessment is the first step in VTE prevention management.¹ In China, Zhenguo Zhai et al. collected 105,723 VTE patients from 2007 to 2016 and found that the hospitalization rate related to VTE increased from 3.2 to 17.5 per 100,000 population.² He also analyzed the data from 2000–2001 to 2010−2011 and showed that the incidence rate of DVT increased from 17.1 per 100,000 to 30.0 per 100,000, PE increased from 3.9 per 100,000 to 8.7 per 100,000, and 51.8% of hospitalized patients were at risk of VTE.³ The Chinese expert consensus on the prevention of VTE recommended the Caprini RAM for surgical patients and Padua RAM for medical patients.⁴ It is well known that Caprini RAM and Padua RAM were derived from western populations, and these two RAM are very complicated, Caprini RAM contains more than 40 risk factors, and Padua RAM has almost 11 risk factors. But there is no unified data for VTE RAM in hospitalized Chinese patients. The aim of this study was to develop and externally validate VTE RAM based on hospitalized Chinese patients.

Methods

Ethical Approval

This study was approved by the Medical ethics committee of Beijing Shijitan Hospital Affiliated with Capital Medical University (No. 2020-2).

Subjects

Development of VTE RAM: We reviewed data from 300 VTE patients and 300 non-VTE patients admitted to Beijing Shijitan (SJT) Hospital from January 2019 to December 2020. External validation of VTE RAM: Data from 407 VTE patients and 533 non-VTE patients admitted to Beijing Chaoyang Hospital from October 2021 to March 2022 were collected.

VTE group: The inclusion criteria were as follows: age ≥ 18 years and ≥ 2 days of hospitalization. DVT was confirmed by color Doppler ultrasound and/or venography of the upper and lower limbs, with (or not) PE diagnosed by CT pulmonary angiography (CTPA) or pulmonary ventilation perfusion (V/Q) scanning. The exclusion criteria were as follows: superficial venous thrombosis, anticoagulant (including oral and subcutaneous injection) or thrombolytic drug use at admission or after admission, and incomplete clinical data.

Non-VTE group: We selected patients who were admitted to the same departments during the periods mentioned above. The inclusion criteria were as follows: age ≥ 18 years and ≥ 2 days of hospitalization. Non-DVT patients were diagnosed by color Doppler ultrasound and/or venography of the upper and lower limbs, with non-PE patients confirmed by CTPA or V/Q. The exclusion criteria were as follows: superficial venous thrombosis, anticoagulant (including oral and subcutaneous injection) or thrombolytic drug use at admission or after admission, and incomplete clinical data.

Data collect: We collected some clinical characteristics and the value of D-dimer. The clinical characteristics were as following: age, sex, bed rest, lower limb edema (include unilateral or bilateral), acute infection, chronic obstructive pulmonary disease, respiratory failure, heart failure, coronary heart disease, nephrotic syndrome, septicemia, cancer, thrombocytosis, central vein catheterization, and VTE history.

Statistical Analyses

1. Model development: The risk factors related to VTE were analyzed by logistic regression. The relative risk ratio and 95% confidence interval (CI) were calculated. Each regression coefficient was divided by the minimum coefficient as the score for each item. All of the weighted items were summed as an individual patient's total score. Different risk stratifications were defined according to our pilot study and reference.⁵ The Hosmer–Lemeshow goodness-of-fit test was used to compare the observed and predicted VTE events.

2. Model validation: The sensitivity, specificity, and Youden index were analyzed. Based on sensitivity and specificity, an ROC curve was drawn, and the AUC value was defined as the AUC.

Measurement data were expressed as`x ± s. If the distribution of data was normal, a parametric model was used to calculate the P value and confidence interval. For the non-normal distribution of data, the nonparametric rank sum test was used to calculate the P value and CI. For discrete features, Fisher's exact test was used to calculate the p value. The count data were expressed as percentages (%), and the χ² test was used for comparisons between groups. R statistical software (version 3.4.1) was used for all statistical analyses. P < 0.05 denotes statistical significance.

Results

VTE RAM (SJT Version) Development

Data from 300 VTE patients and 300 non-VTE patients were collected retrospectively from January 2019 to December 2020, including those from gastrointestinal surgery 67 (11.17%), orthopedics 31 (5.17%), thoracic surgery 53 (8.83%), urologic surgery 8 (1.33%), obstetrics and gynecology 11 (1.83%), cerebral surgery 4 (0.67%), neurology 72 (12%), intensive care unit 68 (11.33%), geriatrics 10 (1.67%), respiratory 154 (25.67%), cardiovascular 32 (5.33%), nephrologic 30 (5%), hematologic 21 (3.5%), and traditional Chinese medicine 6 (1%), rheumatologic and immunologic 2 (0.33%), endocrine 7 (1.17%), oncologic 2 (0.33%), and emergency 22 (3.67%). The characteristics of the patients showed a higher prevalence rate of VTE patients who were experiencing age >60 years, bed rest, lower limb edema, acute infection, COPD, respiratory failure, coronary heart disease, septicemia, central venous catheterization, VTE history and elevated D-dimer. (Table 1)

Table 1.

Characteristics of VTE Patients in the Development Group.

Variables	N	Prevalence of VTE (%)	χ2	P
Age (years)			12.27	<0.001
≤ 60	117	35.04
>60	483	53.62
Sex			2.41	0.120
Female	286	53.50
Male	314	46.82
Bed rest			7.17	0.007
Yes	160	59.38
No	440	46.59
Lower limb edema			7.46	0.006
Yes	91	63.74
No	509	47.54
Acute infection			4.96	0.026
Yes	180	57.22
No	420	46.90
COPD			6.49	0.011
Yes	62	66.13
No	538	48.14
Respiratory failure			12.39	<0.001
Yes	78	69.23
No	522	47.13
Heart failure			2.10	0.147
Yes	79	58.23
No	521	48.75
Coronary heart disease			5.03	0.025
Yes	203	56.65
No	397	40.60
Nephrotic syndrome			1.50	0.220
Yes	25	36.00
No	575	50.61
Septicemia			6.90	0.009
Yes	65	66.15
No	535	48.04
Cancer			3.50	0.061
Yes	180	56.11
No	420	47.38
Thrombocytosis			1.18	0.277
Yes	22	36.36
No	578	50.52
CVC			23.93	<0.001
Yes	53	83.02
No	547	46.80
VTE history			8.21	0.004
Yes	33	75.76
No	567	48.50
D-dimer			110.59	<0.001
<ULN	162	25.31
≥ 1XULN and ≤ 2XULN	134	32.09
>2XULN	304	71.05

COPD, chronic obstructive pulmonary disease; CVC, central vein catheterization; ULN, upper limit of normal.

Logistic regression analysis showed that age, lower limb edema, COPD, central venous catheterization, VTE history, and elevated D-dimer were significantly correlated with the occurrence of VTE. (P < 0.05) (Table 2)

Table 2.

Logistic Regression Analysis of VTE Risk Factors.

Characteristic	β	Std. Error	Z value	P value
Age >60 years	0.420	0.250	2.036	0.042
Bed rest	0.194	0.266	0.728	0.466
Lower limb edema	0.298	0.326	1.979	0.048
Acute infection	0.001	0.283	0.004	0.997
COPD	0.488	0.369	2.042	0.041
Respiratory failure	0.335	0.456	0.733	0.463
Coronary heart disease	0.138	0.250	0.554	0.580
Septicemia	0.035	0.461	0.076	0.939
CVC	1.732	0.477	3.623	0.000
VTE history	1.231	0.478	2.592	0.01
D-dimer	0.960	0.122	7.461	0.000

COPD, chronic obstructive pulmonary disease; CVC, central vein catheterization

The developed VTE RAM (SJT version) was composed of six items, including age>60 years, lower limb edema, COPD, VTE history, CVC, and D-dimer. Each β coefficient was divided by the smallest absolute value of the coefficient as the score for each item (Table 3).

Table 3.

Risk Assessment Model of VTE (SJT Version).

Risk factor	Score
Age >60 years	1
Lower limb edema	1
COPD	2
VTE history	4
CVC	6
D-dimer ≥ 1XULN and ≤ 2XULN	3
D-dimer>2XULN	6

COPD, chronic obstructive pulmonary disease; CVC, central vein catheterization; ULN, upper limit of normal

According to the results of our pilot study and the literature,⁶ we stratified VTE RAM (SJT) version) according to the cumulative score as follows: 0 low risk; 1 moderate risk; 2–4 high risk; and ≥ 5 highest risk. Based on AUC values of SJT, Caprini and Padua RAMs, the AUC value of SJT RAM (0.75 ± 0.03) is significantly higher than the value of Padua RAM (0.53 ± 0.03; P < 0.05), SJT RAM (0.75 ± 0.03) vs Caprini RAM (0.73 ± 0.03; P > 0.05), which showed that the developed model had better predictive ability than Padua RAM for the all retrospective hospitalized patients. SJT RAM and Caprini RAM have a similar predictive ability for all retrospective hospitalized patients (Figure 1).

Figure 1.

ROC curve of the VTE RAM (SJT version) in all retrospective patients.

The overall fitting degree of the model was evaluated by the fitting degree, and the agreement degree between the predicted event and the observed event was evaluated by the Hosmer–Lemeshow (H–L) goodness-of-fit test. In the H–L test, if p > 0.05, it is considered that the fitting degree of the model is good, indicating that the agreement between the predicted VTE event and the observed VTE event is high. The H–L test result of our developed VTE RAM (p = 0.9987 > 0.05) indicates excellent agreement (Figure 2).

Figure 2.

Hosmer–Lemeshow goodness-of-fit test of VTE RAM (SJT version).

VTE RAM (SJT Version) External Validation

We analyzed a total of 940 patients for external validation. Based on AUC values of SJT, Caprini, and Padua RAMs, the AUC value of SJT RAM (0.75 ± 0.03) vs Caprini RAM (0.73 ± 0.03); (P > 0.05), and SJT RAM (0.75 ± 0.03) vs Padua RAM (0.72 ± 0.03); (P > 0.05), which indicates that the SJT RAM has similar predictive value for all medical patients and surgical patients as Caprini RAM and Padua RAM. (Figure 3)

Figure 3.

ROC curve of the VTE RAM (SJT version) in all external validation patients.

We analyzed 408 medical patients (204 VTE patients and 204 non-VTE patients) for external validation. the AUC value of SJT RAM (0.82 ± 0.03) is significantly higher than the value of Caprini RAM (0.76 ± 0.04; P < 0.05), and SJT RAM (0.82 ± 0.03) vs Padua RAM (0.78 ± 0.03; P > 0.05), which means that the SJT RAM has a much better predictive value than Caprini RAM for medical patients. While SJT RAM and Padua RAM have the similar predictive value for medical patients (Figure 4).

Figure 4.

ROC curve of the VTE RAM (SJT version) in medical patients.

We analyzed 532 surgical patients (203 VTE patients and 329 non-VTE patients) for external validation. The AUC value of SJT RAM (0.72 ± 0.04) is significantly higher than the value of Padua RAM (0.66 ± 0.04; P < 0.05), SJT RAM (0.72 ± 0.04) vs Caprini RAM (0.72 ± 0.04), which shows that the SJT RAM has better predictive value than Padua RAM for surgical patients. While SJT RAM and Caprini RAM have the similar predictive value for surgical patients (Figure 5).

Figure 5.

ROC curve of the VTE RAM (SJT version) in surgical patients.

We found SJT RAM, Caprini RAM, and Padua RAM have a similar sensitivity and specificity and Youden index for all patients (P > 0.05). For internal medicine patients, SJT RAM has a higher sensitivity, specificity, and Youden index than Caprini RAM (P < 0.05), while SJT RAM and Padua RAM have a similar sensitivity, specificity, and Youden index (P > 0.05). For surgery patients, SJT RAM has a higher sensitivity, specificity, and Youden index than Padua RAM (P < 0.05), while SJT RAM and Caprini RAM have a similar sensitivity, specificity, and Youden index (P > 0.05) (Table 4).

Table 4.

Comparison of Different RAMs in External Groups.

	RAM	Sensitivity	Specificity	Youden Index
All Patients	Caprini	0.648	0.687	0.335
	Padua	0.701	0.644	0.345
	SJT	0.729	0.690	0.419
	P Value	>0.05	>0.05	>0.05
Internal medicine patients	Caprini	0.764	0.568	0.38
	Padua	0.812	0.694	0.459
	SJT	0.840	0.680	0.520
	P Value	<0.05a	<0.05a	<0.05a
Surgery patients	Caprini	0.729	0.599	0.328
	Padua	0.631	0.402	0.232
	SJT	0.778	0.599	0.377
	P Value	<0.05b	<0.05b	<0.05b

RAM, risk assessment model; SJT, Shijitan

a, SJT vs Caprini; b, SJT vs Padua.

Discussion

In the United States, each VTE patient needs to spend nearly $18,000 – $23,000 every year. VTE events have brought serious economic burdens to society.⁷ It is an important task for VTE prevention management to identify high-risk patients and carry out corresponding prophylaxis. In the United Kingdom, the National Institute for Health and Care Excellence (NICE) analyzed 22 studies and found that nearly 40% of low-risk medical and surgical patients may receive thromboprophylaxis inappropriately.⁸ The Padua RAM and Caprini RAM were constructed by expert consensus, and the International Medical Prevention Registry on Venous Thromboembolism (IMPROVE) score was derived from logistic regression studies.^9–11 The American College of Chest Physicians (ACCP) recommended Caprini RAM for non-orthopedic surgical patients and Padua RAM for internal medical patients. At that same time, ACCP guidelines emphasize that internal medical patients should use institution-specific VTE risk assessment models.¹² Our VTE RAM was constructed and validated based on the real-world data of two comprehensive teaching hospitals, not from any RCT research.

This study showed that age, lower limb edema, COPD, CVC, VTE history, and D-dimer elevation were significantly correlated with the occurrence of VTE (p < 0.05). It is well known that an increase in age is accompanied by the aging of organs, a decrease in activity, and chronic comorbid diseases. In a review of VTE epidemiology, Richard H. also mentioned that the incidence of VTE at the age of >60 years increased sharply.¹³ Almost all VTE risk assessment models include age, which is easy to obtain and does not require complex calculation formulas. Blondon et al.¹⁴ finished a prospective study to compare whether the current VTE risk assessment models were better than the age model that had age as the only predictor. The results demonstrated that the age model was not significantly different from the current Caprini RAM, Padua RAM, and IMPROVE score (P = 0.266). Lower limb edema is a common clinical symptom and sign of VTE, and it was proven to be moderate-level evidence of VTE (or 1.88; 95% CI, 1.23−2.90).¹⁵ COPD has become the fifth leading cause of death worldwide.¹⁶ Harenberg et al.¹⁷ found that even COPD in the stable period will be one of the main causes of VTE. A Spanish retrospective study compared the incidence rate of PE and mortality during hospitalization between COPD and non-COPD patients. They found that the incidence rate and mortality of PE patients with COPD were significantly higher than those without COPD.¹⁸ In the process of developing the model (SJT version), CVC was associated with the occurrence of VTE. Bo et al.¹⁹ proved that the incidence of CVC-related VTE was 2%−11%, and the number was higher in cancer patients. Catheter-related VTE (CRT) accounts for approximately 10% of DVT cases. Many factors lead to CRT, such as aging, catheter retention time, catheter/vein >0.45, type and material. The catheter activated the coagulation pathway triggered by tissue factors and promoted blood coagulation. The American Society of Hematology guideline recommends that CRT patients should receive anticoagulation until the catheter is removed.²⁰ In the prospective study of Joffe et al.,²¹ only 3.6%–4.3% of CRTs were symptomatic. A peripherally inserted central catheter (PICC) has a higher probability of CRT due to a damaged local vascular wall and slow blood flow.^{22, 23} CVC was proven to be an independent risk factor for VTE, and it has nothing to do with the location of CVC.²⁴

We incorporated the VTE history into the VTE RAM (SJT version), which is another important risk factor for VTE.²⁵ The recurrence rate of VTE within 5 years in patients with a previous history of VTE was almost 10%−30%.²⁶ A retrospective cohort study involving 9472 trauma patients revealed that there were more patients with a VTE history than those without a VTE history (2.4% vs 0.1%; p < 0.001).²⁷ In 2019, a case-control study reported that the probability of VTE recurrence in surgical patients with a VTE history was 6-fold higher than that in those without a VTE history.²⁸ What makes our VTE RAM different from Caprini RAM and Padua RAM was that we included D-dimer. There is no doubt that D-dimer is closely related to VTE. Patients at high VTE risk with increasing D-dimer could benefit from extended anticoagulation.²⁹ Some domestic studies have tried to combine the Caprini RAM and D-dimer, and the combined RAM has a good predictive ability for ICU high-risk VTE patients.³⁰ In the IMPROVEDD study, two points were added to the IMPROVE score if D-dimer was more than 2-fold upper limit of normal, and the IMPROVEDD score was found to improve VTE risk stratification and the predictive ability of VTE through 77 days in acute medical inpatients.³¹ Eichinger et al.³² also incorporated D-dimer into the IMPROVE score and discovered that it can screen for recurrence of VTE after stopping anticoagulation. In our study, considering the different lab methods and normal value range of D-dimer in every hospital, we did not add a fixed value of D-dimer, and only the level of elevation was considered, which was in agreement with the above IMPROVEDD study.

In clinical practice, a simpler VTE risk assessment model will result in better compliance for health care providers. The VTE RAM (SJT version) included six risk factors (age, lower limb edema, COPD, CVC, VTE history, and D-dimer) that were easily available at admission. In external validation, SJT RAM and Caprini RAM have similar predictive value for surgical patients. And SJT RAM and Padua RAM also have similar predictive value for medical patients. Our previous research also confirmed that Caprini RAM for surgical patients and Padua RAM for medical patients among Chinese hospitalized patients.³³ At present, there are few VTE risk assessment models for general hospitalized patients, especially in China. Many teams constructed VTE risk assessment models for specific diseases based on the data of their institutions. Wang and colleagues³⁴ collected data from 81,505 surgical inpatients in Southwest Hospital of China from January 1, 2019, to June 18, 2021, and built the SW model. This model internally verified that the predictive ability of VTE in surgical patients was better than that of the Caprini RAM. Lok et al.³⁵ developed a VTE risk assessment model including five risk factors (including overweight, advanced material age, multiple priority, obesity, and primary postpartum hemorrhage) by analyzing 859 Chinese obstetrics and gynecology patients. This model avoided inappropriate anticoagulation. Yao et al.³⁶ established VTE risk assessment models including age ≥ 69 years, preoperative plasma D-dimer ≥ 0.49 mg/L, stage IV cancer and transfusion. The AUC of this model was 0.769, while that of the Caprini RAM was 0.656. Tian et al.³⁷ of Chaoyang Hospital built a VTE risk assessment model for patients undergoing thoracic surgery. Qu et al.³⁸ simplified the Caprini RAM in 2015 and built the gynecological Caprini (G-Caprini) RAM for patients undergoing gynecological surgery. The G-caprini RAM has six risk factors: varicose veins, bed rest ≥48 h, length of operation ≥3 h, latitude, hypertension, and age ≥50 years. It effectively identified low-risk VTE patients. Some constructed risk assessment models for cancer patients, Marlise Alexander constructed VTE RAM, which is applicable to non-small cell carcinoma (NSCLC) patients, including chemotherapy, D-dimer, and fibrinogen.³⁹ Li et al.⁴⁰ built and validated a VTE risk assessment model for lung cancer, including cancer type, age, sex, bed rest, CVC, and anticancer treatment. Muñoz Martín et al.⁴¹ built a clinical-genetic risk model for predicting VTE events in patients with cancer. This risk score can better identify cancer patients with VTE risk than the Khorana score. In addition, there were some VTE risk assessment models by novel methods. Yang y constructed a VTE risk assessment model for tumor patients through machine learning (ML) technologies, which also has better predictive ability than Padua RAM (AUC 0.973 ± 0.006 vs 0.791 ± 0.022) (p < 0.001).⁴² Ahmad built a VTE recurrence risk assessment model by detecting single-nucleus polymers (SNPs). This eight-gene mutation model was revealed to have good predictive ability for the recurrence of VTE in men, especially those without obvious risk factors.⁴³

The similarities between SJT RAM and Caprini RAM are that they include age, lower limb edema, COPD, VTE history, and CVC. Age and VTE history are the common items of SJT RAM and Padua RAM, which are easily available at admission. The major difference between SJT RAM and Caprini RAM and Padua RAM is that our model includes D-dimer, while the other two RAMs do not. Another difference is the number of RAM, since Caprini RAM has nearly forty risk factors and Padua RAM has eleven risk factors. Nevertheless, VTE RAM (SJT version) has only six risk factors, so physicians and nurses will save much time in assessing VTE risk for hospitalized patients. Therefore, the working burden of medical staff will be reduced. In addition, our VTE RAM (SJT version) was proven to be available for all hospitalized patients rather than specific diseases, as illustrated above. Therefore, our model is applicable to a wider range of inpatients.

The limitation of our study is that the SJT RAM does not include surgical items such as anesthesia time and surgical operation types, which may be the reason why the predictive ability of surgical patients is lower than that of internal medical patients. However, the risk factors we included are easily obtained information at admission. Fewer risk factors are included, less time will be spent, and the compliance of medical staff will also be higher. According to a previous study, most patients with VTE were diagnosed within 3 months after admission, and 67% of them were confirmed within 1 month after admission.⁴⁴ Hospitalized patients need to be dynamically assessed during admission, transfer, and discharge. One study found that VTE risk stratification was different on Day 1 and Day 3 after admission (4.7 ± 1.7 vs 4.2 ± 1.8; p = .008).⁴⁵ Therefore, it is particularly important to carry out dynamic evaluations of inpatients. In the next step, we will study the weights of anesthesia time and surgical operation types and add these specific surgical items to this VTE RAM (SJT version), which is applicable to surgical patients due to surgery, when the patients’ risk status will change during hospitalization.

As VTE prevention management has received increasing attention in China, more evidence about VTE risk assessment models for hospitalized Chinese patients is needed. The SJT RAM we built based on institution inpatients was proven effective through external validation. It can help screen patients with VTE risk, and improve awareness of VTE risk prevention. In the future, we plan to conduct a clinical study on VTE prevention according to the risk stratification of SJT and the bleeding risk of patients. If the patient has VTE risk and high bleeding risk, the patient should receive mechanical prevention, then dynamically assess the patient's VTE risk and bleeding risk. If the patient still has the VTE risk, but the risk of bleeding is low, the patient should receive drug prophylaxis. VTE-related serious complications and corresponding medical expenses will be reduced.