Abstract
Background
Antithrombotic guidance statements for hospitalized patients with coronavirus disease 2019 (COVID‐19) suggest a universal thromboprophylactic strategy with potential to escalate doses in high‐risk patients. To date, no clear approach exists to discriminate patients at high risk for venous thromboembolism (VTE).
Objectives
The objective of this study is to externally validate the IMPROVE‐DD risk assessment model (RAM) for VTE in a large cohort of hospitalized patients with COVID‐19 within a multihospital health system.
Methods
This retrospective cohort study evaluated the IMPROVE‐DD RAM on adult inpatients with COVID‐19 hospitalized between March 1, 2020, and April 27, 2020. Diagnosis of VTE was defined by new acute deep venous thrombosis or pulmonary embolism by Radiology Department imaging or point‐of‐care ultrasound. The receiver operating characteristic (ROC) curve was plotted and area under the curve (AUC) calculated. Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated using standard methods.
Results
A total of 9407 patients were included, with a VTE prevalence of 2.9%. The VTE rate was 0.4% for IMPROVE‐DD score 0‐1 (low risk), 1.3% for score 2‐3 (moderate risk), and 5.3% for score ≥ 4 (high risk). Approximately 45% of the total population scored high VTE risk, while 21% scored low VTE risk. IMPROVE‐DD discrimination of low versus medium/high risk showed sensitivity of 0.971, specificity of 0.218, PPV of 0.036, and NPV of 0.996. ROC AUC was 0.702.
Conclusions
The IMPROVE‐DD VTE RAM demonstrated very good discrimination to identify hospitalized patients with COVID‐19 as low, moderate, and high VTE risk in this large external validation study with potential to individualize thromboprophylactic strategies.
Keywords: COVID‐19, hospitalized patients, risk assessment, thromboembolism, thrombosis
Essentials.
No clear approach exists to discriminate high‐risk hospitalized patients with coronavirus disease 2019 (COVID‐19) for venous thromboembolism (VTE).
Our study included a large hospitalized COVID‐19 population within a multihospital health system.
The IMPROVE‐DD VTE risk score classified approximately 45% of the population into high VTE risk.
The IMPROVE‐DD score has the potential to individualize strategies to prevent VTE in this population.
1. INTRODUCTION
The coronavirus disease 2019 (COVID‐19) pandemic has been associated with elevated rates of thrombotic events. The majority of events represent venous thromboembolism (VTE) and include classic macrovessel disease such as deep vein thrombosis (DVT) or pulmonary embolism (PE), as well as microvessel disease and in situ fatal thrombosis. 1 , 2 Although initial reports suggested VTE rates of 46% or higher in hospitalized patients – especially those with critical illness – subsequent larger US studies have shown much lower VTE rates of 1.7% to 3.6%. 3 , 4 , 5 Antithrombotic guidance statements on hospitalized patients with COVID‐19 suggest a universal thromboprophylactic strategy with potential to escalate doses in high‐risk groups, though identifying and discriminating these groups remains a challenge. 6
The International Medical Prevention Registry on Venous Thromboembolism and D‐Dimer (IMPROVE‐DD) risk assessment model (RAM) expands upon a well‐validated RAM – the IMPROVE VTE RAM ‐ and incorporates a novel biomarker, an elevated d‐dimer (Dd). 7 , 8 Elevated Dd appears to be highly predictive of increased thrombotic risk and poor outcomes in patients hospitalized with COVID‐19. 9 Our aim was to externally validate the IMPROVE‐DD RAM for VTE in a large cohort of hospitalized patients with COVID‐19.
2. METHODS
This retrospective cohort study included adult patients aged ≥ 18 years old with COVID‐19 diagnosed by polymerase chain reaction and hospitalized in 1 of 13 acute care hospitals across an integrated health care network in the New York metropolitan region between March 1, 2020, and April 27, 2020. Patients were excluded if they were on the obstetrics service; death or discharge had not been reached by April 30, 2020; length of stay was <8 hours; key variables were missing; or a VTE event had occurred within the first 8 hours after presentation. The study was performed with institutional review board approval and waiver of informed consent. Data were obtained from the enterprise inpatient electronic health record (Sunrise Clinical Manager, Allscripts, Chicago, IL). All data and outcomes were tracked until April 30, 2020.
VTE was defined as new acute DVT or PE events diagnosed by imaging performed by the Department of Radiology or by point‐of‐care lower extremity ultrasound and verified by manual review of the radiology reports by two attending radiologists in a routine care setting where there was no policy for screening. We collected IMPROVE‐DD variables, patient demographics, comorbidities, and treatment. We defined history of VTE and cancer (including cancer in remission as well as current cancer undergoing anticancer treatment) from International Classification of Diseases, Tenth Revision (ICD‐10) codes. Immobility was scored for all hospitalized patients. A patient was considered to have been admitted to the intensive care unit (ICU) if care included vasopressors, ventilation, or admission to a named ICU. Mechanical ventilation was designated a surrogate for lower limb paralysis. Maximum Dd was the highest Dd throughout hospitalization for patients without VTE, or the highest Dd before VTE for patients with VTE, using a cutoff of ≥ 2 times the upper limit of local laboratory normal. Thrombophilia status was unknown. We identified major comorbidities by ICD‐10 coding: coronary artery disease, chronic kidney disease or end‐stage renal disease, peripheral arterial disease or peripheral vascular disease, and cerebrovascular disease. Thromboprophylaxis was determined to be none, treatment dose, or prophylaxis dose based on the highest dose before the first diagnosed VTE or highest anticoagulant dose before discharge (deceased or alive) for patients without VTE.
The receiver operating characteristic (ROC) curve was plotted and area under the curve (AUC) calculated. Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated using standard methods. The observed prevalence of VTE was used in the calculation of PPV and NPV. We also conducted a sensitivity analysis that included for none, prophylactic, and treatment dose anticoagulation in the logistic regression model. We then plotted the ROC curve for the model using IMPROVE‐DD VTE RAM alone and for the model using IMPROVE‐DD VTE RAM adjusted for anticoagulant dose. The areas under these correlated ROC curves were then compared using the nonparametric approach. 10 All analyses were performed with SAS version 9.4 (SAS institute, Cary, NC, USA).
3. RESULTS
A total of 11 265 patients were considered. After 1858 exclusions, 9407 patients met study criteria. Overall prevalence of VTE was 2.9%. Patient characteristics and distribution of model VTE risk factors are listed in Table 1. The VTE rate by IMPROVE ‐DD score was 0.4% for a score of 0‐1 (low risk), 1.3% for a score of 2‐3 (moderate VTE risk), and 5.3% for a score of ≥ 4 (high VTE risk) (Table 2). Approximately 45% of the total population was identified as high VTE risk, while approximately 21% was identified as low risk. Across the three groups of low, moderate, and high VTE risk as identified by the model, no anticoagulants were given in 15.9%, 11.9%, and 6.8% of patients, respectively; prophylactic‐dose anticoagulants were given in 81.3%, 77.7%, and 61.2% of patients, respectively; and treatment‐dose anticoagulants given in 2.9%, 10.4%, and 32.0% of patients, respectively. The AUC of the ROC was 0.702. A sensitivity analysis that adjusted for none, prophylactic, and treatment dose anticoagulation found that the AUC of the ROC for the model changed to 0.715 (P < .042), a difference that is not clinically meaningful. Discriminating low versus medium/high VTE risk showed a sensitivity of 0.971, specificity of 0.218, PPV of 0.036, and NPV of 0.996.
TABLE 1.
Patient characteristics and VTE risk factors
| Patient characteristics | N (%) |
|---|---|
| All | 9407 (100) |
| IMPROVE‐DD variables (points) | |
| Prior episode of VTE (3) | 247 (2.6) |
| Thrombophilia (2) a | NA |
| Paralysis of the lower extremity during hospitalization (2) b | 1557 (16.6) |
| Cancer (2) | 727 (7.7) |
| Max d‐dimer > 2× ULN (2) c | 4364 (46.4) |
| Immobilization for at least 7 days (1) | 9407 (100) |
| ICU stay (1) | 2203 (23.4) |
| Age > 60 (1) | 5785 (61.5%) |
| Patient characteristics | |
| BMI | |
| Unknown | 2029 (21.6) |
| ≤35 | 6154 (65.4 |
| >35 | 1224 (13.0) |
| Male | 5580 (59.3) |
| Age | |
| 18‐59 | 3407 (36.2) |
| 60‐75 | 3365 (35.8) |
| 75+ | 2635 (28.0) |
| Comorbidity | |
| Coronary artery disease | 1200 (12.8) |
| Heart failure | 773 (8.2) |
| Peripheral artery/vascular disease | 379 (4.0) |
| Cerebrovascular disease | 551 (5.9) |
| ESRD or CKD | 783 (8.3) |
| Treatment | |
| Hospital anticoagulation | |
| None | 979 (10.4) |
| Prophylaxis dose | 6675 (71.0) |
| Treatment dose | 1753 (18.6) |
| Home or hospital antiplatelet | |
| None | 4531 (48.2) |
| Present | 2804 (29.8%) |
| NA | 2072 (22.0%) |
Abbreviations: BMI, body mass index; CKD, chronic kidney disease; ESRD, end‐stage renal disease; ICU, intensive care unit; ULN, upper limit of normal; VTE, venous thromboembolism.
Unknown for all patients.
On vent used as surrogate.
d‐dimer not done on 2566 subjects.
TABLE 2.
Observed VTE events of IMPROVE‐DD VTE RAM in Hospitalized Patients COVID‐19 Based on Score Thresholds
| Patient characteristics | VTE, n (%) | No VTE, n (%) |
|---|---|---|
| IMPROVE‐DD | ||
| 0‐1, low risk | 8 (0.4) | 1988 (99.6) |
| 2‐3, moderate risk | 40 (1.3) | 3093 (98.7) |
| 4‐12, high risk | 226 (5.3) | 4052 (94.7) |
| Total | 274 (2.9) | 9133 (97.1) |
Abbreviations: COVID‐19, coronavirus disease 2019; RAM, risk assessment model; VTE, venous thromboembolism.
4. DISCUSSION
Predicting VTE risk in hospitalized patients with COVID‐19 remains an important and difficult clinical issue, as the majority of thromboembolic events in this population are venous in origin. 1 The IMPROVE‐DD VTE RAM, a modification of the well‐validated IMPROVE VTE RAM in hospitalized medically ill patients, showed very good model discrimination and excellent NPV in predicting VTE risk in a large cohort of hospitalized patients with COVID‐19. When compared to expected and observed VTE rates from previous derivation and external validation efforts of the original IMPROVE VTE RAM, the model also showed excellent calibration across the three cutoffs of low, moderate, and high VTE risk. 8 , 11 To our knowledge, this is the first dedicated external validation study using a VTE RAM in hospitalized patients with COVID‐19.
International guidelines are moving toward an individualized, risk‐adapted approach to thromboprophylaxis in hospitalized medically ill patients, 6 which would include the subset of hospitalized patients with COVID‐19. Pre–COVID‐era studies in medically ill patients, including those with pneumonia and sepsis, have suggested a much lower percentage (10% to ~ 25%) of high‐VTE‐risk patients compared to our study’s findings. 12 Although nearly 80% of hospitalized patients with COVID‐19 in our study were at moderate or high VTE risk, a sizable proportion (~21%) were at a low VTE risk of ~ 0.4%; such patients may be subject to potential harms in a universal anticoagulant thromboprophylaxis policy. 6 Conversely, the 45% of hospitalized patients with COVID‐19 deemed at high VTE risk by our study (score of ≥ 4) may be considered for escalated dose thromboprophylaxis or therapeutic anticoagulation (as is being studied in current randomized trials) or considered for extended postdischarge thromboprohylaxis as suggested by current guidelines – including the most recent Scientific and Standardization Guidance Statement from the International Society on Thrombosis and Haemostasis. 6 , 13
The model’s AUC of 70%, sensitivity of 97%, and specificity of 22% are in line with previous validation studies of routinely used medical VTE RAMs (eg, Khorana and IMPROVE VTE scores). 11 , 14 Previous efforts using VTE risk scores in hospitalized patients with COVID‐19 were underpowered, used surrogate outcomes, or included ad hoc risk factors that were not included within the original derivation study. 15 , 16 In our study, nearly half of patients had elevated Dd, approximately 25% had an ICU stay, and >60% were above age 60. This distribution of risk factors follows previous studies predicting VTE and poor outcomes in hospitalized patients with COVID‐19, especially with elevated Dd. 9 Study limitations included the inability to capture thrombophilia as a risk factor (although this would apply to a very small percentage of patients and not expected to alter model characteristics appreciably), as well as the reliance on surrogates to capture immobility (hospital stay) and lower limb paralysis (mechanical ventilation). Another study limitation is that maximal (and not initial) d‐dimers were used, which may affect the utility of the RAM when used during hospital admission.
In conclusion, in this large external validation study the IMPROVE‐DD VTE RAM demonstrated very good discrimination to identify hospitalized patients with COVID‐19 as low, moderate, or high VTE risk. Approximately half the patients were at high VTE risk, while 21% were low VTE risk. These results have potential to individualize VTE thromboprophylactic strategies in this complex hospitalized medically ill population.
AUTHOR CONTRIBUTIONS
ACS contributed to the study design. NK and ML contributed to data acquisition. All authors contributed to data analysis and interpretation.
RELATIONSHIP DISCLOSURE
SLC received funding support from the Association of University Radiologists GE Radiology Research Academy Fellowship (GERRAF) and Siemens Healthineers and was a consultant for Infervision 2019. ACS is a consultant for Boehringer Ingelheim, Janssen, Bayer, and Portola. All other authors declare no conflicts of interest.
ACKNOWLEDGMENTS
The authors thank the COVID‐19 Consortium Group at Northwell Health/Feinstein Institutes for Medical Research as well as the Broxmeyer Fellowship in Clinical Thrombosis.
Spyropoulos AC, Cohen SL, Gianos E, et al; On Behalf of the COVID‐19 Consortium Group . Validation of the IMPROVE‐DD risk assessment model for venous thromboembolism among hospitalized patients with COVID‐19. Res Pract Thromb Haemost. 2021;5:296–300. 10.1002/rth2.12486
Handling Editor: Dr Cihan Ay.
REFERENCES
- 1. Spyropoulos AC, Weitz JI. Hospitalized COVID‐19 patients and venous thromboembolism: A perfect storm. Circulation. 2020;142:129–32. [DOI] [PubMed] [Google Scholar]
- 2. Wichmann D, Sperhake J‐P, Lütgehetmann M, Steurer S, Edler C, Heinemann A, et al. Autopsy findings and venous thromboembolism in patients with COVID‐19: A prospective cohort study. Ann Intern Med. 2020;173:268–77. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Hanif A, Khan S, Mantri N, Hanif S, Saleh M, Alla Y, et al. Thrombotic complications and anticoagulation in COVID‐19 pneumonia: a New York City hospital experience. Ann Hematol. 2020;99:2323–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Bilaloglu S, Aphinyanaphongs Y, Jones S, Iturrate E, Hochman J, Berger JS. Thrombosis in hospitalized patients with COVID‐19 in a New York City health system. JAMA. 2020;324:799–801. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Tang N, Bai H, Chen X, Gong J, Li D, Sun Z. Anticoagulant treatment is associated with decreased mortality in severe coronavirus disease 2019 patients with coagulopathy. J Thromb Haemost. 2020;18:1094–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Spyropoulos AC, Levy JH, Ageno W, Connors JM, Hunt BJ, Iba T, et al. Subcommittee on Perioperative, Critical Care Thrombosis, Haemostasis of the Scientific, Standardization Committee of the International Society on Thrombosis, Haemostasis+. Scientific and Standardization Committee Communication: clinical guidance on the diagnosis, prevention and treatment of venous thromboembolism in hospitalized patients with COVID‐19. J Thromb Haemost. 2020;18:1859–65. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Gibson CM, Spyropoulos AC, Cohen AT, Hull RD, Goldhaber SZ, Yusen RD, et al. The IMPROVEDD VTE Risk Score: Incorporation of D‐Dimer into the IMPROVE Score to Improve Venous Thromboembolism Risk Stratification. TH Open. 2017;1:e56–65. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Spyropoulos AC, Anderson FA, FitzGerald G, Decousus H, Pini M, Chong BH, et al. Predictive and associative models to identify hospitalized medical patients at risk for VTE. Chest. 2011;140:706–14. [DOI] [PubMed] [Google Scholar]
- 9. Tang N, Li D, Wang X, Sun Z. Abnormal coagulation parameters are associated with poor prognosis in patients with novel coronavirus pneumonia. J Thromb Haemost. 2020;18:844–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. DeLong ER, DeLong DM, Clarke‐Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44:837–45. [PubMed] [Google Scholar]
- 11. Mahan CE, Liu Y, Turpie AG, Vu JT, Heddle N, Cook RJ, et al. External validation of a risk assessment model for venous thromboembolism in the hospitalised acutely‐ill medical patient (VTE‐VALOURR). Thromb Haemost. 2014;112:692–9. [DOI] [PubMed] [Google Scholar]
- 12. Spyropoulos AC, Ageno W, Cohen AT, Gibson CM, Goldhaber SZ, Raskob G. Prevention of venous thromboembolism in hospitalized medically ill patients: A U.S. perspective. Thromb Haemost. 2020;120:924–36. [DOI] [PubMed] [Google Scholar]
- 13. Cohoon KP, Sanctis YD, Haskell L, McBane RD, Spiro TE. Rivaroxaban for thromboprophylaxis among patients recently hospitalized for acute infectious diseases: a subgroup analysis of the MAGELLAN study. J Thromb Haemost. 2018;16:1278–87. [DOI] [PubMed] [Google Scholar]
- 14. Khorana AA, Kuderer NM, Culakova E, Lyman GH, Francis CW. Development and validation of a predictive model for chemotherapy‐associated thrombosis. Blood. 2008;111:4902–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Tsaplin S, Schastlivtsev I, Lobastov K, Zhuravlev S, Barinov V, Caprini J. The validation of the original and modified Caprini score in COVID‐19 patients. medRxiv Cold Spring Harbor Laboratory Press; 2020. [Google Scholar]
- 16. Zeng D‐X, Xu J‐L, Mao Q‐X, Liu R, Zhang W‐Y, Qian H‐Y, et al. Association of Padua prediction score with in‐hospital prognosis in COVID‐19 patients. QJM Int J Med. 2020;113(11):789–93. [DOI] [PMC free article] [PubMed] [Google Scholar]