Computer Surveillance of Patients at High Risk for and with Venous Thromboembolism

Abstract

Venous thromboembolism (VTE), including deep vein thrombosis (DVT) and pulmonary embolism (PE), may be the number one preventable cause of death associated with hospitalization. Numerous evidence-based guidelines for effective VTE prophylaxis therapy exist. However, underuse is common due to the difficulty in integrating VTE risk assessment into routine patient care. Previous studies utilizing computer decision support to identify high-risk patients report improved use of prophylaxis therapy and reduced VTE. However, those studies did not report the sensitivity, specificity or positive predictive value of their methods to identify patients at high risk. We report an evaluation of a computerized tool to identify patients at high risk for VTE that found a sensitivity of 98% and positive predictive value of 99%. Another computer program used to detect VTE had a sensitivity of 92%, specificity of 99% and a positive predictive value of 97% to identify DVT and a sensitivity of 100%, specificity of 98% and positive predictive value of 89% to identify PE. These tools were found to provide a dependable method to identify patients at high risk for and with VTE.

Introduction

Venous thromboembolism (VTE), including deep vein thrombosis (DVT) and pulmonary embolism (PE), is a frequent complication associated with hospitalization and affects the morbidity, mortality, length of stay and costs of millions of patients each year throughout the world^1–3. While risk of VTE among surgical patients is well defined^1,5, the risk for VTE in medicine patients is often overlooked and many VTE, including fatal PE, occur in this population after hospital discharge^6–9. Despite the development of numerous evidence-based guidelines for effective VTE prophylaxis, underuse and inadequate use is common in U.S. and European hospitals^2,5,10

Integrating risk assessment for VTE into the routine process of care has been difficult. In order to improve rates of VTE prophylaxis, reliable and reproducible methods which identify patients at risk for VTE are needed. In addition, reliable methods to detect VTE events, especially following hospital discharge are needed in order to assess the effectiveness of VTE prevention programs.

Prior studies have reported results of manual identification of VTE risk during order set use, computer alerts for high-risk patients, and critiques and reminders during computerized physician order-entry (CPOE)^11–14. In addition, a computerized risk prediction model with alerts of patients at risk for VTE during CPOE use increased physician use of prophylaxis and reduced VTE¹⁵. However, the previous studies utilizing computer decision support have not reported on the sensitivity, specificity or positive predictive value of their methods for identifying patients at high risk for VTE.

This study reports an evaluation of a computer decision support tool which captures sufficient data to use a risk prediction model to identify high-risk patients and a complementary computer program able to detect VTE events in patients during hospitalization and following discharge.

Methods

Background:

Intermountain Medical Center is a Level-One trauma, 456-bed teaching hospital affiliated with the University of Utah School of Medicine and the largest of 22 hospitals owned by Intermountain Healthcare. The hospital information system utilizes an electronic medical record (EMR) that contains most clinical information. The majority of the information such as clinical pharmacy, laboratory and nurse bedside charting is stored as coded data while radiology and other dictated reports are stored as “free text” documents that can be analyzed with natural language processing (NLP). Patient information from the EMRs of all hospitals and over 150 clinics, insta-cares, doctor’s offices and home-care affiliated with Intermountain Healthcare is stored each night in the enterprise data warehouse (EDW).

VTE High Risk Surveillance:

To enhance and facilitate the identification of patients at high risk for VTE¹⁶, a new computer program was developed to monitor patients at hospital admission and during hospitalization. Patients with a score of 4 points or higher were identified as high risk for VTE. Cancer, previous VTE and hypercoagulability were scored as 3 points, surgery duration greater than one hour as 2 points and age > 70 years, bed rest, body mass index (BMI) > 29 kg/m2 and use of hormone-replacement therapy or oral contraceptives as 1 point each.

Each night, a program scans the EDW and looks for new laboratory test results indicative of hypercoagulability and stores them into a table and collects the most recent heights and weights and calculates and stores BMIs in another table. Each morning, another program uses the enterprise-wide encounter table to identify each hospital’s patients and determines if they have been previously identified as high risk. If not, the program uses the patient’s most recent BMI and hypercoagulability results and looks for ICD9 codes of cancer and previous VTE from the EDW, free-text admission diagnosis of cancer, surgeries lasting over one hour, hormone-replacement and oral contraceptives use, documented bed rest from nurse charting and age (calculated from the date of birth). Patients identified as high risk are identified through email reports sent to study personnel and inpatient pharmacists at 21 hospitals each day at 7:00am (Figure 1). The report includes patient identification, location, identifies each risk factor and total score. The program also screens the EMR for anticoagulation or compression prophylaxis use. Patients are assessed daily for new risk factors until identified as high risk or discharged.

Figure 1.

Example of computer alerts of patients at hight risk for VTE. (CA = cancer, VTE = previous VTE, Hyper = hypercoagulability, Surg = surgery duration > 1hr, BedRest = nurse documentation that patient is inmoble, Obese = BMI>29, HRT/OC = hormone replacement therapy/oral contraception.

VTE Detection:

In 2006, a computerized tool was created that used NLP for analysis of only venous duplex ultrasonography reports and was validated as a dependable and consistent method to identify peripherally inserted central catheter associated DVT¹⁷. A similar program was developed to make a sequential search of all upper and lower extremity venous duplex ultrasonography, CT angiography and ventilation perfusion scan dictation reports from the preceding 3 days. Detected reports are analyzed using a “keyword” driven procedure we developed to perform NLP of the free-text. Since venous duplex ultrasonography reports are just coded as “Vascular Study” in our EMR, all vascular reports, including arterial, have to be analyzed. DVTs are identified from the duplex ultrasonography reports and PE from CT angiography and ventilation perfusion scans. Currently, another email is sent to study personnel at two hospitals conducting VTE research at 7:00am each morning identifying patients who develop VTEs during hospitalization or readmission (Figure 2). The program identifies both the presence and location of the thrombus. Although both of these programs have been clinically used for two years, their sensitivity, specificity and positive predictive value have never been formally evaluated.

Figure 2.

Example from the VTE Monitor with alerts of patients who develop DVT or PEs.

Evaluation of VTE High Risk Surveillance:

A sample of 600 medicine patients admitted to four nursing divisions during May 11, 2009 to November 11, 2009 had their medical records prospectively and manually screened by an experienced study coordinator. The study coordinator applied the same criteria used by the automated tool to identify patients at high risk for VTE and not receiving VTE prophylaxis. Those patients were then retrospectively compared to the log of patients identified by the VTE high risk program from the same nursing divisions and time period. The medical records of patients not alerted as high risk by the computer program were examined to determine which risk factors were not identified. All patients alerted as high risk by the computer program and not receiving VTE prophylaxis at the time of the alert were also identified. The study coordinator then manually adjudicated each alert and validated each risk factor identified by the computer. Each risk factor that could not be verified by the study coordinator was further checked using direct data queries from the EMR and the EDW. Since these were all medicine patients, the surgery duration risk factor was not included in this evaluation.

Evaluation of VTE Detection:

To determine the sensitivity and specificity of the VTE monitoring program, a random sample of 100 venous duplex ultrasonography tests and 100 CT angiography or ventilation perfusion scans ordered for unique patients from the study population were manually examined by two authors for the diagnosis of DVT or PE. Dictated reports that were questionable for the presence of a DVT or PE were adjudicated by the study coordinator and physician experts in VTE.

To determine the positive predictive value of the VTE monitoring program, a random sample of 100 alerts from unique patients identified with DVTs and 100 alerts from unique patients identified with PEs by the computer were selected from the same study population. Each patient had their dictated venous duplex ultrasonography, CT angiography and ventilation perfusion scans manually examined as reported above. This study was approved by the Institutional Review Board at Intermountain Healthcare.

Results

VTE High Risk Surveillance:

There were 6,646 admissions for 5,601 unique patients to the four nursing divisions during the six month study. Manual chart review identified 109 of 600 (18%) patients screened as high risk for VTE and who were not receiving prophylaxis. The VTE High Risk computer program identified 107 of the 109 for a sensitivity of 98%. The two patients missed by the computer did not have previous VTE coded in the EMR or EDW. The study coordinator identified both patients previous VTE information from free-text history and physical reports.

Since the computer monitored all 6,646 admissions each day during the entire study period, it identified 131 patients in addition to the 107 for a total of 238 as high risk and not receiving VTE prophylaxis. For each of those patients, the study coordinator then manually checked each risk factor identified by the computer and could not find 79 risk factors from 74 different patients using any of the reporting functions on the EMR. Each risk factor not verified by the study coordinator was then checked with direct queries of the EMR and EDW (Table). Of the 79 risk factors not found by the study coordinator, 12 were found in the EMR and 66 were found only in the EDW. One cancer code could not be found in either database. Thus, the positive predictive value was 99%. The ability of the assessment program to query the EDW identified 60 patients at high VTE risk that were missed by manual EMR screening alone.

Table.

Validation of computer identified risk factors for VTE.

Risk Factor	Computer Alerts	Not Manually Found	Data Found in EMR	Data Found in EDW	Data Not Found (%)
Cancer	127	31	5	25	1 (<1)
Previous VTE	131	28	0	28	0
Hypercoagulability	37	16	7	9	0
Bed Rest	57	0	-	-	-
BMI > 29	106	0	-	-	-
HRT/OC	18	4	0	4	0
Age > 70	99	0	-	-	-
Total	575	79	12	66	1 (<1)

VTE Detection:

From the 6,646 admissions to the four nursing divisions during the study period, there were 751 vascular studies, including venous duplex ultrasound tests, ordered for 603 patients and 160 CT angiography or ventilation perfusion scans ordered for 154 patients. Manual review of each of the 100 random samples found a sensitivity of 92% and a specificity of 99% to detect DVT and a sensitivity of 100% and specificity of 98% to detect PE.

There were 225 (3.4%) patients from the study population identified by the computer as having DVT. Manual verification of the 100 random venous duplex ultrasonography reports from those patients identified a positive predictive value of 97%. Two of the false positive DVT alerts were found to be arterial thrombosis and one was an arteriovenous graft thrombosis. There were 158 (2.3%) patients identified by the computer as having PE. Manual review of the 100 random CT angiographies and ventilation perfusion scan reports identified a positive predictive value of 89%. Seven computer alerts contained the phrase “low probability” in the sentence used by NLP to identify the PE and were adjudicated as false positive by study personnel. The addition of that single phrase as a “negative” term could have increased the positive predictive value to 96%. Four additional false-positive cases were attributable to the phrases “septic emboli”, “thrombosed aortic aneurysms” or “pulmonary arterial thromboses”.

Discussion

VTE is common and may be the number one preventable cause of death in hospitalized patients¹⁸. The appropriate use of prophylaxis can reduce the number of VTE and direct medical costs in high-risk patients^18–20. However, the availability of VTE prophylaxis guidelines in the U.S and Europe has not resulted in increased guideline adherence, in part due to the absence of validated tools able to integrate risk assessment into the routine process of care for a very large number of patients⁵.

This study found that computerized surveillance combined with access to a large enterprise-wide electronic medical record can be a dependable and consistent method to identify patients at high risk of VTE. Manual screening of patients found two high risk patients not identified by the computer. However, computer access to enterprise-wide data found 66 risk factors in the EDW from previous admissions, including other hospitals, for 60 patients that were not manually identified.

In addition to facilitating VTE risk assessment, a consistent method to monitor patients for the development of VTE is essential to assess effectiveness of prophylaxis interventions. This study also found computer monitoring can be an effective method to identify patients with VTE.

Computer alerts of VTE high risk patients have been integrated into the routine process of care through a variety of methods. While one-screen CPOE alerts identifying high risk VTE patients increased prophylaxis and reduced VTE, the majority of alerts were ignored¹⁵. However, a multi-screen version of the same program resulted in prophylaxis orders for 58.4% of patients whose physicians ignored the first VTE alert²¹. Still, many CPOE alerts are ignored due to alert fatigue^13,21. Decision support programs that are used to monitor patients and report adverse events are designed to sacrifice specificity for high sensitivity. It is easier and requires less time and cost for the human recipient of the report to detect false positive alerts rather than have to manually screen each patient every day. However, as with alert fatigue during CPOE, the perceived value of patient monitors is diminished as the false positive rates increase. Our programs, which are capable of sending daily email alerts to specifically identified stakeholders, may reduce the impact of alert fatigue to due high reliability of the data.

Evaluation of decision support programs is essential for their improvement and maintenance. This study not only measured the ability of these programs to identify high risk patients and VTE, but identified how they could be improved. For example, the inclusion of “low probability” as a negative term in the NLP logic would have increased the positive predictive value of the PE alerts from 89 to 96% and reduced time spent checking those patients. The false positive PE alerts was also reflected in the 100% sensitivity, but at the cost of a lower specificity.

Limitations:

As reported above, our risk prediction model assigns 2 points for surgery durations longer than one hour. Since this study only included medicine patients, that risk factor could not be validated and these results may differ for surgical patients. However, this study does demonstrate that the risk prediction model used in this study can also be used for medicine patients

Our VTE High Risk Surveillance tool utilized a single, published risk assessment scheme¹⁶. Use of other scoring systems may provide different results.

Also, since the manual and computer screening for VTE high risk identified a different set of patients in this study, specificity could not be reported.

Many VTE in patients occur after hospital discharge. Most patients with PE would be re-admitted and could be detected by the computer methods reported in this paper. However, many post-discharge DVT may be diagnosed by independent radiology centers and may not have the results stored in our EMR or EDW. Thus, some post discharge DVT will not be identified with our current computer VTE alerts.

While the results of the VTE high-risk program clearly demonstrated the value of a large enterprise-wide EMR and EDW, many hospitals may not currently have this same capability. Thus, their results could be different. However, the “key-word” driven NLP approach applied to specific dictated reports evaluated in this study could be used by most text-based EMRs.

Future Plans:

Prompt delivery of alerts of VTE high risk patients to targeted health providers in a timely manner is the key to initiating appropriate thromboprophylaxis. In addition to currently sending the email alerts to study nurses and pharmacists who then notify physicians, we are testing a method to send the VTE high-risk alerts to physician’s email capable mobile devices. Many physicians keep a list of their current patients in the hospital information system. Thus, the new program would enable us to directly send physicians VTE high risk alerts for only their patients.

Conclusion:

The automated VTE High Risk Surveillance program provides an accurate and dependable method to identify medical patients at risk for VTE and who are not receiving prophylaxis. The VTE Detection program accurately identifies most patients with VTE during hospitalization or following hospital discharge and provides the ability to evaluate the effectiveness of prophylaxis interventions.