Risk Stratification Model: Lower-Extremity Ultrasonography for Hospitalized Patients with Suspected Deep Vein Thrombosis

Abstract

Background

The Wells score for deep venous thrombosis (DVT) has a high failure rate and low efficiency among inpatients.

Objective

To create and validate an inpatient-specific risk stratification model to help assess pre-test probability of DVT in hospitalized patients.

Design

Prospective cohort study of hospitalized patients undergoing lower-extremity ultrasonography studies (LEUS) for suspected DVT. Demographics, physical findings, medical history, medications, hospitalization, and laboratory and imaging results were collected. Samples were divided into model derivation (patients undergoing LEUS 11/1/2012–12/31/2013) and validation cohorts (LEUS 1/1/2014–5/31/2015). A DVT prediction rule was derived using the recursive partitioning algorithm (decision tree-type approach) and was then validated.

Participants

Adult inpatients undergoing LEUS for suspected DVT from November 2012 to May 2015, excluding those with DVT in the prior 3 months, at a 793-bed, urban academic quaternary-care hospital with ~50,000 admissions annually.

Main Measures

The primary outcome was the presence of proximal DVT, and the secondary outcome was the presence of any DVT (proximal or distal). Model sensitivity and specificity for predicting DVT were calculated.

Key Results

Recursive partitioning yielded four variables (previous DVT, active cancer, hospitalization ≥ 6 days, age ≥ 46 years) that optimized the prediction of proximal DVT and yield in the derivation cohort. From this decision tree, we stratified a scoring system using the validation cohort, categorizing patients into low- and high-risk groups. The incidence rates of proximal DVT were 2.9% and 12.0%, and of any DVT were 5.2% and 21.0%, for the low- and high-risk groups, respectively. The AUC for the discriminatory accuracy of the Center for Evidence-Based Imaging (CEBI) score for risk of proximal DVT identified on LEUS was 0.73. Model sensitivity was 98.1% for proximal and 98.1% for any DVT.

Conclusions

In hospitalized adults, specific factors can help clinicians predict risk of DVT, identifying those with low pre-test probability, in whom ultrasonography can be safely avoided.

INTRODUCTION

Deep venous thrombosis (DVT) is the third most common cardiovascular disease, affecting approximately 600,000 people in the United State annually ^{1, 2}. The diagnosis of lower-extremity DVT poses challenges to clinicians, as its associated symptoms are often nonspecific, and physical examination findings have both low sensitivity and specificity ¹. To guide clinicians in the evaluation of patients with suspected DVT, Wells et al. derived a scoring system to stratify patients into three risk categories—low, moderate, and high—based on ten clinical predictors. The Wells criteria for DVT have been extensively validated in the outpatient setting, but have demonstrated less optimal performance among hospitalized patients, showing sensitivity as low as 83–88% ^{3, 4}.

The relatively high false-negative rate of the Wells score among hospitalized patients may be due to differences in demographics, risk factors, and comorbidities between inpatient and outpatient cohorts. Hospitalized patients are more likely to have risk factors for DVT, including heart failure, cancer, immobilization, and recent surgery ^{3, 5}. Further, inpatients are more susceptible to developing DVT, heightening the need for a risk stratification scoring system specific to the hospital setting ⁴.

The objective of this study was to develop and validate a novel inpatient-specific risk stratification model for DVT.

METHODS

Study Setting

This institutional review board-approved prospective, observational cohort study was conducted at a 793-bed urban, academic quaternary-care hospital with approximately 50,000 admissions annually. The requirement for informed consent was waived due to the observational nature of the study.

Study Population

We included all adult hospitalized patients admitted between November 1, 2012, and May 31, 2015, with suspected DVT and who underwent a lower-extremity venous duplex ultrasound study (LEUS). Patients with DVT diagnosed within the prior 3 months were excluded to avoid capturing studies that were done for follow-up. For patients with multiple LEUS during the study period, only the first was included (Fig. 1).

Figure 1.

Cohort identification flow chart. To avoid capturing follow-up studies, patients diagnosed with deep venous thrombosis (DVT) within prior three months were excluded and only the first study for patients undergoing additional studies for multiple LEUS studies was included.

Data Sources and Collection

Using a computerized physician order entry (CPOE) system, ordering providers entered the ten Wells criteria predictors at the time of LEUS order entry: active cancer (treatment ongoing or within previous 6 months, or palliative), immobilization, surgery within the previous 12 weeks, tenderness of the leg, leg swelling, calf swelling (at least 3 cm larger than contralateral side), edema, collateral veins present, previously documented DVT, and alternative diagnosis at least as likely as DVT (Fig. 2). In hopes of improving the performance beyond Wells criteria, other potential DVT predictors were generated by an expert multispecialty panel consisting of two radiologists (CBB, RK) and two cardiologists (SZG, GP), and were abstracted from the electronic health record. These included length of hospital stay at the time of LEUS, patient age, body mass index, gender, history of heart disease, history of inflammatory bowel disease, current pregnancy, use of hormones, and use of anticoagulants.

Figure 2.

Computer physician order entry system during LEUS order data capture for components of the Wells score displayed at the time of lower-extremity venous duplex ultrasound study (LEUS) order entry.

We used a natural language processing (NLP) algorithm developed using the General Architecture for Text Engineering (GATE) framework to identify the presence of DVT on LEUS reports. The NLP algorithm was previously validated and found to have 100% sensitivity and specificity in identifying the presence of DVT ³. The location of DVT was subsequently manually classified as proximal or isolated distal by an internal medicine attending physician (IKI). The thrombus anatomic location was classified as proximal if the popliteal or more proximal veins were involved and distal if confined to the calf veins. Since proximal DVT is associated with increased adverse clinical outcomes, cases where proximal and distal DVT co-occurred were classified as proximal.

Outcomes and Measures

Our primary outcome was the presence of proximal DVT. Our secondary outcome was the presence of any DVT, proximal or distal.

Sample Size

We divided our study population into two cohorts: a derivation cohort (patients undergoing LEUS 11/1/2012–12/31/2013) and a validation cohort (patients undergoing LEUS 1/1/2014–5/31/2015). Using the derivation cohort, we created a prediction rule for DVT by a recursive partitioning algorithm with a decision tree approach. Silveira et al. previously calculated a sample size of 800 patients to provide 80% power to detect a difference in incidence of proximal DVT of 0.5% to 1% ³.

Recursive Partitioning Algorithm

We used recursive partitioning to find the best combinations of predictor variables that were sensitive for detecting DVT. Recursive partitioning is a statistical method for multivariable analysis that uses decision tree to classify members of a population by splitting it into sub-populations based on independent variables, and has been used in the development of other clinical decision rules^6–8. Quantitative variables were converted to binary by JMP Pro 10 (SAS Institute Inc., Cary, NC, USA), which minimized entropy. Threshold criteria of 9 % efficiency and 4 % failure rate were set a priori. We defined efficiency as the proportion of patients in the low-risk group and failure rate as the mean predicted probability of DVT in patients classified as low risk ⁹. To maximize clinical utility, we selected the tree with the fewest nodes (predictors), identifying the factors most predictive of risk of DVT.

Score Creation

Points were assigned for significant predictors based on logistic regression odds ratios from the derivation cohort (Table 2). The points were summed to determine a final score to stratify patients into low- and high-risk groups. Applying the scoring system to the validation cohort, we computed the sensitivity, negative predictive value (NPV), specificity, and likelihood ratio as measures of the model’s performance. Chi-square tests were performed using JMP Pro 10 (SAS Institute Inc., Cary, NC) to detect differences in the frequency of DVT between the low- and high-risk groups. A two-tailed p-value of <0.05 was considered statistically significant.

Table 2.

Center for Evidence-Based Imaging (CEBI) Score Point System

CEBI score criteria	Points
Previous documented deep venous thrombosis (DVT)	+6
Active cancer (treatment ongoing or within previous 6 months or palliative)	+1
Long hospital stay (≥6 days)	+1
Age ≥ 46 years	+1

Score = 0: low risk

Score = 1–7: high risk

All authors had full access to all of the data (including statistical reports and tables) in the study and can take responsibility for the integrity of the data and the accuracy of the data analysis.

RESULTS

Characteristics of 2960 inpatients with suspected DVT divided into the derivation (n = 1135) and validation (n = 1825) cohorts are shown in Table 1. The majority of patients were recently bedridden or had major surgery, were on venous thromboembolism pharmacologic prophylaxis, and were 46 years of age or older. Between the two cohorts, there were significant differences in age (derivation mean 60 years old, validation 64 years old; p = .0001), use of pharmacologic DVT prophylaxis (derivation 57.4%, validation 73.4%; p = 0.041), immobilization (derivation 70.2%, validation 66.3%; p = 0.017), possibility of an alternative diagnosis (derivation 22.5%, validation 27.5%; p = 0.003), localized tenderness along the distribution of DVT (derivation 17.9%, validation 22.1%; p = 0.009), and recently bedridden or major surgery (derivation 70.2%, validation 66.3%; p = 0.017).

Table 1.

Characteristics of 2960 Inpatients with Suspected Deep Vein Thrombosis (DVT), Divided into Derivation and Validation Cohorts

Characteristic	Derivation Cohort (n = 1135) N (%)	Validation Cohort (n = 1825) N (%)	p-Value*
Age (mean [years] ± standard deviation)	60 ± 17	64 ± 16	0.0001*
Age as a binary variable (≥ 46 years)	920 (81.1)	1568 (85.9)	0.0004*
Female	598 (52.3)	990 (54.2)	0.24
Recently bedridden or major surgery	802 (70.2)	1211 (66.3)	0.017*
Active cancer	418 (36.6)	698 (38.2)	0.44
Paralysis, paresis, or recent plaster immobilization of the lower extremity	205 (17.9)	403 (22.1)	0.009*
Entire leg swollen	479 (41.9)	748 (41.0)	0.64
Localized tenderness along the distribution of the deep venous system	439 (38.4)	558 (30.6)	0.0001*
Pitting edema	331 (29.0)	513 (28.1)	0.64
Calf swelling	321 (28.1)	475 (26.0)	0.20
Alternative diagnosis at least as likely as deep vein thrombosis	257 (22.5)	501 (27.5)	0.003*
Previously documented DVT	167 (14.6)	285 (15.6)	0.43
Proximal DVT	137 (12.07)	206 (11.29)	0.52
Any DVT	241 (21.2)	362 (19.8)	0.36
Collateral superficial veins	52 (4.5)	70 (3.8)	0.52
Use of pharmacologic DVT prophylaxis	651 (57.4)	1340 (73.4)	0.041*
Number of days in hospital	5.4	4.8	0.28
Long stay as a binary variable (≥ 6 days)	337 (29.69)	482 (26.4)	0.052

*Statistically significant result

Incidence of Deep Venous Thrombosis

LEUS identified proximal DVT in 343 of the 2960 patients and proximal or distal DVT in 603 patients. The overall incidence of proximal DVT was 11.6%, with 12.1% in the derivation cohort and 11.3% in the validation cohort. The rate of any DVT was 20.4%, with 21.2% in the derivation cohort and 20.0% in the validation cohort.

Decision Tree

Significant predictors were identified from the recursive partitioning model based on our pre-determined criteria. The variables selected were previous DVT, active cancer, long hospital stay (≥ 6 days), and a binary age variable (≥ 46 years).

Center for Evidence-Based Imaging (CEBI) Score

Based on odds ratios obtained from logistic regression, we assigned scores to categorize patients into low- and high-risk groups: history of DVT (6 points from an odds ratio of 6.12), active cancer (1 point from an odds ratio of 1.33), hospital stay of ≥ 6 days before imaging (1 point from an odds ratio of 0.96), and age ≥ 46 years (1 point from an odds ratio of 1.02). Patients were considered low-risk (score = 0) or high-risk (score > 1; Table 2).

In the derivation cohort, the frequency of proximal DVT in the low- and high-risk categories was 1.9% (2/106) and 13.12% (135/1029), respectively (p < 0.0001). The frequency of any DVT was 3.8% (4/106) in the low-risk group and 23.0% (237/1029) in the high-risk group (p < 0.0001).

The validation cohort had similar results. Proximal DVT was found in 2.9% (4/136) of patients in the low-risk group and 12.0 (202/1689) in the high-risk group (p < 0.0001). For any DVT, the frequency was 5.2% (7/136) in the low-risk and 21.0% (355/1689) in the high-risk group (p < 0.0001).

The AUC for the discriminatory accuracy of the Wells score for risk of proximal DVT was 0.73 and for any DVT was 0.68 (Fig. 3). Overall, the NVP of the score for classifying proximal DVT was 97.1%, sensitivity was 98.1%, the likelihood ratio was 0.22, and efficiency was 7.5% (Table 3). Efficiency was defined as the proportion of patients in the low-risk group ⁹.

Figure 3.

Area under the receiver operating characteristic (ROC) curve. AUC proximal DVT = 0.73; AUC any DVT = 0.68.

Table 3.

Performance of the CEBI Score in the Validation Cohort

	Low Risk		High Risk
Proximal DVT on LEUS	Positive	Negative	Positive	Negative
No. of patients	4	132	202	1487
Test performance
Sensitivity, % (95% CI)	98.1 (95.1–99.2)
Specificity, % (95% CI)	8.2 (6.9–9.6)
Negative predictive value, %	97.1 (92.7–98.9)
Positive likelihood ratio	1.07 (1.04–1.09)
Negative likelihood ratio	0.24 (0.09–0.64)
Any DVT on LEUS	Positive	Negative	Positive	Negative
No. of patients	7	129	355	1334
Test performance
Sensitivity, % (95% CI)	98.1 (96.1–99.1)
Specificity, % (95% CI)	8.8 (7.5–10.4)
Negative predictive value, %	94.9 (89.8–97.5)
Positive likelihood ratio	1.08 (1.05–1.10)
Negative likelihood ratio	0.22 (0.10–0.47)

DISCUSSION

In this prospective observational study, we created a novel inpatient-specific risk stratification model for lower-extremity DVT which achieved sensitivity of 98.1% for identifying proximal DVT among hospitalized patients and an NPV of 97.1%. The test characteristics for our model were significantly higher than the original Wells criteria for DVT, which showed sensitivity of 83–88% in the inpatient population ⁴. However, the increased sensitivity came at the cost of a 4% reduction in efficiency ³. Our results show that a setting-specific risk stratification model has greater predictive accuracy, potentially due to the differences between characteristics of outpatient and hospitalized patients with suspected DVT. Hospitalized patients are typically older, twice as likely to have active cancer, five times as likely to have had recent major surgery, and 2.5 times as likely to have had recent immobilization ⁴. Furthermore, inpatients are more likely to develop venous thromboembolism ¹⁰.

The four risk factors that we identified have been associated with DVT in previous studies. Piotrowski et al. noted that age and length of stay helped to differentiate cases of deep venous thrombosis from those with no DVT among high-risk trauma patients ¹¹. Similarly, Geerts et al. found that older age was one of five independent risk factors for DVT (odds ratio 1.05 per year of age) ¹². The strongest predictor in our score, a personal history of DVT, has been reported to carry a threefold risk of DVT, and has also been reported by others as one of the stronger predictors for acquiring the disease ¹³.

Due to the simplicity of the identified variables, implementation of our model in a computerized clinical decision support system would be relatively easy. The four identified attributes are already documented in the EHR, and could be ‘harvested’, minimizing additional work for physicians in providing them again in the computerized order entry process.

While the AUC values for the discriminatory accuracy of the CEBI score for risk of proximal DVT were only fair, the purpose of our model was to identify a subset of patients at low risk (i.e., < 3% probability of having a positive test) in whom an LEUS may be of low utility and could be safely bypassed. While we feel that a < 3% chance is a reasonable, acceptable risk, we also believe that the ultimate decision regarding imaging should be made by patients and their physicians. Some risk-averse patients may not accept the 3% risk, while many cost-conscious patients will. The CEBI score is a tool that can facilitate such shared decision-making.

Our study does have limitations. First, it was performed at a single academic medical center. Thus, the generalizability of our findings in other settings is unclear, particularly as negative and positive predictive values are dependent on disease prevalence, and the disease prevalence of DVT may vary from that at our quaternary-care hospital. Second, it is possible that our ordering providers did not thoroughly and accurately enter data into the structured CPOE form. However, we previously selected a sample for chart review and found greater than 90% concordance between data captured and that in the electronic health records ³. As our cohort included only those patients with suspected DVT who underwent LEUS, it is possible that some patients who did not undergo imaging or underwent imaging with other cross-sectional modalities were excluded, resulting in differential test ordering bias. Also, we may not have included all potentially relevant variables in our model. Finally, we did not explore the potential benefits of adding D-dimer testing to the diagnostic pathway, as use of the test is rare among hospitalized patients.

Contributors

All authors contributed to the acquisition, analysis, and interpretation of the data, as well as the drafting and final approval of this manuscript. Emily Alper and Dr. Ip had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Prior Presentations

This abstract was presented as a poster session at the Radiological Society of North America Conference in Chicago in November 2016.

Funding

No funding was received for this study.

Compliance with Ethical Standards

Conflict of Interest

Dr. Piazza has received research grants from Bristol Myers Squibb, Daiichi Sankyo, the Thrombosis Research Institute, and Janssen, and consulting fees from Merck, eXIthera, and Zafgen. Dr. Khorasani is a consultant to ‬Medicalis Corporation‬. U.S. Patent 6,029,138 is held by Brigham and Women’s Hospital on clinical decision support-related software, licensed to Medicalis Corporation in 2000. As a result of licensing, Brigham and Women’s Hospital and its parent organization, Partners HealthCare, Inc., have equity and royalty interests in Medicalis. The authors have no other relationships or activities that could appear to have influenced the submitted work.

The data sets during and/or analyzed during the current study are available from the corresponding author on reasonable request.