Using multiple sources of data for surveillance of postoperative venous thromboembolism among surgical patients treated in Department of Veterans Affairs hospitals, 2005–2010

Abstract

Background

There are limitations to using administrative data to identify postoperative venous thromboembolism (VTE). We used a novel approach to quantify postoperative VTE events among Department of Veterans Affairs (VA) surgical patients during 2005–2010.

Methods

We used VA administrative data to exclude patients with VTE during 12 months prior to surgery. We identified probable postoperative VTE events within 30 and 90 days post-surgery in three settings: 1) pre-discharge inpatient, using a VTE diagnosis code and a pharmacy record for anticoagulation; 2) post-discharge inpatient, using a VTE diagnosis code followed by a pharmacy record for anticoagulation within 7 days; and 3) outpatient, using a VTE diagnosis code and either anticoagulation or a therapeutic procedure code with natural language processing (NLP) to confirm acute VTE in clinical notes.

Results

Among 468,515 surgeries without prior VTE, probable VTEs were documented within 30 and 90 days in 3,931 (0.8%) and 5,904 (1.3%), respectively. Of probable VTEs within 30 or 90 days post-surgery, 47.8% and 62.9%, respectively, were diagnosed post-discharge. Among post-discharge VTE diagnoses, 86% resulted in a VA hospital readmission. Fewer than 25% of outpatient records with both VTE diagnoses and anticoagulation prescriptions were confirmed by NLP as acute VTE events.

Conclusion

More than half of postoperative VTE events were diagnosed post-discharge; analyses of surgical discharge records are inadequate to identify postoperative VTE. The NLP results demonstrate that the combination of VTE diagnoses and anticoagulation prescriptions in outpatient administrative records cannot be used to validly identify postoperative VTE events.

Introduction

Venous thromboembolism (VTE), which includes deep vein thrombosis (DVT) and pulmonary embolism (PE), may lead to 100,000 deaths annually in the US [1]. After myocardial infarction and stroke, VTE is the third most common cardiovascular disease [2]. VTE is often cited as one of the most preventable hospital-associated complications [3]. Approximately one-half of incident VTE events are associated with recent (within 90 days) acute care hospitalizations or surgeries [4,5]. Although a British study reported that VTE rates remained elevated above baseline population levels for 12 months postoperatively, most occur within 90 days of surgery [6].

Surveillance is needed to assess the preventable burden of hospital-associated VTE. Currently no reliable, nationwide surveillance system for VTE exists in the United States [1]. Key challenges to such a system include identifying probable or confirmed cases of VTE; distinguishing new from recurrent VTE; and identifying data from multiple healthcare settings where VTE is diagnosed and treated. In particular, since a large percentage of hospital-associated VTE, including post-surgical VTE, are diagnosed after discharge, data systems that are restricted to inpatient records are likely to significantly undercount VTE events [4,5,7].

Administrative healthcare data have advantages for VTE surveillance, including routine availability and large numbers of observations [8]. However, limited variables and coding on these datasets restrict definitive confirmation of diagnoses, and the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnosis codes contained in them are often not reliable [9–12]. In particular, the frequency of ICD-9 codes for DVT and PE in outpatient adult claims greatly exceeds validated rates of VTE [13]; in addition many patients with such claims have no record of treatment with anticoagulation or a relevant procedure [13,14].

In addition, because accurate assessment of the timing of VTE onset is often not possible, it can be difficult to distinguish between VTEs in hospital discharge records that were present on admission from those acquired after admission [15,16]. The Agency for Healthcare Research and Quality (AHRQ) Patient Safety Indicator 12 uses information on ICD-9-CM codes for DVT or PE in secondary diagnosis fields in patient records together with a “present-on-admission” (POA) indicator coded as N for not present on admission to identify presumed cases of pre-discharge postoperative VTE [9,10]. A recent validation study using reviews of medical charts in surgical admissions found a positive predictive value for the PSI 12 of 99% in one sample of hospitals and 81% in another sample [17].

Electronic health records (EHRs) can be superior to administrative data through extending access to narrative text detailing events surrounding diagnosis, information on prescribing and administering medications, and information on performance of medical procedures; abstracting such information manually, however, is resource intensive and difficult to sustain outside of a research setting. Natural language processing (NLP) covers a set of methods that can efficiently extract unstructured EHR data [18], including identification of VTEs [8,19].

The purpose of this study was to use a novel approach combining administrative data and NLP analysis of unstructured EHR data to describe the frequency of post-surgical VTE events among surgical inpatients treated in the Department of Veterans Affairs (VA) system, the largest integrated health care system in the United States [20]. Previous studies using VA data to document VTE events have been limited either to a small number of hospitals [8,19,21] or used data that are more than a decade old at this point [22]. We use both administrative and EHR data from 111 VA facilities throughout the United States to provide comprehensive estimates of the frequency of post-surgical VTE in a nationwide patient population of veterans. In particular, we include analysis of outpatient records to capture patients who are diagnosed and treated for VTE but who do not have a subsequent readmission to a VA medical center.

Methods

Datasets and patient selection

We conducted a retrospective cohort study of patients who underwent surgery as inpatients in VA hospitals between January 1, 2005 and December 31, 2010. The unit of analysis in our study was a surgery, with patients potentially contributing more than one surgery to the analysis. Surgical procedures were excluded from the analysis if the veteran: (1) did not have at least 365 days of observation in the VA data prior to surgery, (2) had a chronic or acute VTE recorded during the 365 days prior to inpatient admission for surgery or on the day of admission (Supplemental Table 1), (3) had a subsequent surgery within 90 days of the index surgery, or (4) the surgical hospital stay lasted longer than 90 days. In our analysis, we identified VTE events that occurred within both 30- and 90-day time periods after the surgery.

For each VTE, we distinguished whether it was diagnosed (1) post-surgery but pre-discharge, (2) post-discharge in a VA inpatient setting, or (3) post-discharge in a VA outpatient setting. If VTE events were found in more than 1 of these 3 settings following the same surgery, the event was counted only once and the setting in which it was identified was based on the order listed above. Additionally, if a patient had both DVT and PE diagnoses recorded within the same postoperative period, each event was reported separately for DVTs and PEs but only once when reporting VTEs.

We used ICD-9 procedure codes (Supplemental Table 2) to classify identified surgeries into 1 of 3 mutually exclusive types of surgery: (1) major orthopedic (total knee or hip replacement or hip fracture surgery), (2) abdominal-pelvic, and (3) all other. We identified inpatient surgeries and VTE events using VA Medical SAS datasets extracted from the VA National Patient Care Database, and clinical text extracted from the VA EHR. Approval was obtained through the University of Utah’s Institutional Review Board and the Salt Lake City VA’s Office of Research and Development.

Identification of acute VTE

We identified inpatient DVT and PE events using a variety of data elements available in the VA EHR which were combined sequentially. We present the events identified in each step in this sequential process as “levels” to indicate that as the data elements are combined, the level of confidence that the event identified is a true event increases. The ICD-9-CM diagnosis codes were used to identify inpatient Level 1 acute DVT or PE events (Supplemental Table 1) [6]. For inpatient Level 2 acute DVT or PE events, we required confirmation with a pharmacy record for anticoagulant therapy (Supplemental Table 3) either prior to discharge in the case of a pre-discharge event or within 7 days of a readmission in the case of a post-discharge event.

For outpatient events, we developed a pilot NLP system to identify clinical notes with mentions of PE, DVT, or VTE [23–25]. Each mention was associated with the date the clinic note was created. The date of the earliest mention was used as the date of a VTE event. The VTE event detection NLP system was built using the Unstructured Information Management Architecture Asynchronous Scaleout architecture [26] as a document processing pipeline, a common NLP approach that is characterized by several steps that are performed sequentially such that each step builds on the previous steps and overlays structured annotations on the text. The system performed three core steps: 1) VTE term identification, 2) VTE event disambiguation, and 3) context classification.

The VTE term identification step detected terms possibly used for PE, DVT, and VTE using contextual patterns and keywords initially determined by the project’s clinical team members and iteratively added through manual chart exploration and during the development of the NLP module. The sentence containing each mention of a possible term in a document was then reviewed using the Apache OpenNLP Sentence Detector using MaxEnt models trained on clinical data [27]. Many of the terms for VTE events are ambiguous and could actually refer to an unrelated concept (e.g., the acronym PE may mean pulmonary embolism, physical exam, pulmonary edema, peripheral edema, or pleural effusion depending on the context). The VTE event disambiguation step used word markers in the same sentence as the VTE term that assisted in making an accurate semantic interpretation of the way the term was used. In most cases, surrounding text provided clues to the correct meaning of the term. The context classification step was performed in two stages: (a) identifying mentions of PE, DVT, and VTE in the absence of specific language ruling out acute occurrence such as negation (e.g., “negative for PE”), historical aspect (e.g., “history of PE”), or family history (e.g., “mother had VTE”) using the ConText algorithm [28], and (b) identifying mentions of PE, DVT, and VTE with explicit language or additional rules that reflect expressions and templates commonly used by the VA clinicians (e.g., “PE protocol was positive” and “significant DVT”) to confirm an acute occurrence.

To identify an outpatient Level 1 acute DVT or PE event, we required an ICD-9-CM code in addition to either a pharmacy record for anticoagulant therapy or a Current Procedural Terminology (CPT) code for thrombectomy, embolectomy, vena cava filter placement, or thrombolysis. To confirm an outpatient acute DVT or PE event as Level 2, we required a mention of DVT, PE, or VTE in the patient’s narrative clinic notes identified using NLP without specific language ruling out acute occurrence such as negation, historical aspect, or family history (NLP context classification stage a). Finally, for Level 3, we required explicit confirmation in the clinic notes of diagnosis of acute DVT, PE, or VTE (NLP context classification stage b).

Possible VTE events were classified as Level 1 inpatient and Level 2 outpatient classifications. Probable VTE events were classified as Level 2 inpatient and Level 3 outpatient classifications. In the remainder of the paper, when we refer to estimates of VTE rates without qualification it is probable VTE that is meant.

Statistical methods

Kaplan-Meier curves were used to demonstrate the frequency of postoperative VTE events over time. In addition, we calculated the frequency of VTEs diagnosed in each setting (pre-discharge, post-discharge outpatient, or post-discharge inpatient) as well as overall during both a 30-day and 90-day time window. All data management and analyses were conducted using SAS 9.3 (SAS Institute, Cary, North Carolina) and Stata 12 (StataCorp LP, College Station, Texas).

To validate the NLP system, human annotators manually reviewed 6,000 randomly selected instances of PE, DVT, and VTE from 24,549 records from 7,673 individual patients with suspected VTE. This validation exercise was done prior to our decision to restrict suspected cases to those with pharmacy records for anticoagulant therapy or CPT codes and hence cannot be directly compared to the numbers of VTE cases in the present analysis. For each instance, the annotators compared their classification to that of the NLP system output. The performance of the NLP system was evaluated using positive predictive value (PPV), which was calculated using the annotators’ classification of acute VTE as the ‘gold standard’.

Results

Surgeries

A total of 3,493,378 inpatient admissions occurred at one of 111 VA facilities during our study period. Of these, 648,865 admissions included a surgical procedure and 468,515 (from 383,551 unique patients) met the inclusion criteria for our study (Fig. 1).

Fig. 1.

Cohort attrition summary.

Baseline demographic characteristics are summarized in Table 1. The majority of surgical procedures (n = 362,539, 77.3%) were classified as something other than orthopedic or abdominal.

Table 1.

Summary statistics of study sample.

Characteristic	Total (n = 468,515)		Major Orthopedic Surgery (n = 37,525)		Abdominal-Pelvic Surgery (n = 68,451)		Other Surgery (n = 362,539)
	n	%	n	%	n	%	n	%
Age
<30	3,422	0.7%	96	0.3%	546	0.8%	2,780	0.8%
30 – 39	9,372	2.0%	227	0.6%	2,147	3.1%	6,998	1.9%
40 – 49	32,123	6.9%	1,586	4.2%	6,287	9.2%	24,250	6.7%
50 – 59	120,802	25.8%	9,319	24.8%	17,419	25.4%	94,064	25.9%
60 – 64	98,028	20.9%	8,488	22.6%	13,189	19.3%	76,351	21.1%
65 – 69	57,690	12.3%	4,894	13.0%	7,783	11.4%	45,013	12.4%
70 – 79	96,337	20.6%	8,202	21.9%	13,725	20.1%	74,410	20.5%
80+	50,741	10.8%	4,713	12.6%	7,355	10.7%	38,673	10.7%
Gender
Female	25,139	5.4%	2,020	5.4%	6,962	10.2%	16,157	4.5%
Male	443,376	94.6%	35,505	94.6%	61,489	89.8%	346,382	95.5%
Race/Ethnicity
White non-Hispanic	340,629	72.7%	28,677	76.4%	49,003	71.6%	262,949	72.5%
Black non-Hispanic	77,277	16.5%	4,713	12.6%	11,220	16.4%	61,344	16.9%
Hispanic	24,979	5.3%	1,899	5.1%	4,207	6.1%	18,873	5.2%
Other/Missing	25,630	5.5%	2,236	6.0%	4,021	5.9%	19,373	5.3%
Surgery year
2005	71,362	15.2%	5,350	14.3%	10,220	14.9%	55,792	15.4%
2006	79,690	17.0%	5,982	15.9%	11,708	17.1%	62,000	17.1%
2007	80,355	17.2%	6,030	16.1%	11,736	17.1%	62,589	17.3%
2008	81,102	17.3%	6,787	18.1%	11,741	17.2%	62,574	17.3%
2009	83,010	17.7%	7,043	18.8%	12,059	17.6%	63,908	17.6%
2010	72,996	15.6%	6,333	16.9%	10,987	16.1%	55,676	15.4%

NLP classification validation

The positive predictive value (PPV) of the NLP context classification relative to manual chart abstraction was 0.78 for the review of 6,000 instances for individuals with suspected VTE after context classification stage b. In other words, 78% of records for individuals classified as having suspected VTE after stage b were confirmed as being records for VTE cases by manual chart review. Supplemental Table 4 contains examples of text that were identified as positive mentions of PE and DVT through each context classification stage.

VTE Events

Kaplan-Meier curves showed that postoperative DVT (Fig. 2) and PE (Fig. 2b) events were more common in orthopedic surgery patients than in patients having abdominal-pelvic or other surgeries for most of the post-surgery period. In both cases, the greatest incidence of events was in the first 20 days following surgery.

Fig. 2.

a: Kaplan-Meier curve for postoperative DVT events. b: Kaplan-Meier curve for postoperative PE events.

Overall, 6,509 (1.4%) and 9,720 (2.1%) surgical admissions were followed by a possible postoperative VTE within 30 and 90 days following surgery, respectively (Table 2). Confirmation with anticoagulant prescriptions for inpatient events (Level 2) and clinic note evidence consistent with outpatient events (Level 3) resulted in 3,931 (0.8%) probable VTEs within the 30-day window and 5,904 (1.3%) within the 90-day window. Postoperative VTEs occurred before discharge within 30 days following 2,052 (0.44%) surgeries.

Table 2.

Post-operative venous thromboembolism events within 30- and 90-day time window (N = 468,515).

	30-Day Outcomes						90-Day Outcomes
	VTE		DVT		PE		VTE		DVT		PE
	N	%	N	%	N	%	N	%	N	%	N	%
Pre-discharge
Possiblea	3,292	0.70%	2,369	0.51%	1,071	0.23%	3,583	0.76%	2,565	0.55%	1,186	0.25%
Probableb	2,052	0.44%	1,396	0.30%	743	0.16%	2,188	0.47%	1,491	0.32%	794	0.17%
Post-discharge outpatient
Possiblec	1,294	0.28%	1,008	0.22%	332	0.07%	2,294	0.49%	1,795	0.38%	586	0.13%
Probabled	271	0.06%	202	0.04%	72	0.02%	522	0.11%	386	0.08%	144	0.03%
Post-discharge readmission
Possiblea	1,923	0.41%	1,342	0.29%	769	0.16%	3,843	0.82%	2,836	0.61%	1,403	0.30%
Probableb	1,608	0.34%	1,086	0.23%	699	0.15%	3,194	0.68%	2,279	0.49%	1,279	0.27%
Total
Possible	6,509	1.39%	4,719	1.01%	2,172	0.46%	9,720	2.07%	7,196	1.54%	3,175	0.68%
Probable	3,931	0.84%	2,684	0.57%	1,514	0.32%	5,904	1.26%	4,156	0.89%	2,217	0.47%

Note: DVT and PE event counts do not sum to the VTE event count because patients may have both events.

^aPossible pre-discharge or post-discharge readmission events were those identified using ICD-9 codes only (inpatient Level 1).

^bProbable pre-discharge or post-discharge readmission events were defined as possible events confirmed with a pharmacy record for anticoagulant therapy (inpatient Level 2).

^cPossible post-discharge outpatient events were those identified using ICD-9 codes and confirmed with either a pharmacy record for anticoagulant therapy or a CPT code (outpatient Level 2).

^dProbable post-discharge outpatient events were defined as possible events confirmed with evidence of VTE without specific mention of negation, history, or family history and with explicit confirmation of acute VTE in clinic notes identified through NLP (outpatient Level 3).

Within the 30-day window, 49.4% of possible VTEs and 47.8% of probable VTEs were diagnosed post-discharge; within the 90-day window, 63.1% of possible VTEs and 62.9% of probable VTEs were diagnosed post-discharge (Table 2). Of the 3,716 probable VTEs diagnosed following discharge using a 90-day window, 522 (14.1%) were managed solely in an outpatient VA encounter and 3,194 (86.0%) resulted in a VTE-related VA readmission. The 3,194 VTE-related readmissions accounted for 3.4% of the 96,732 total 90-day readmissions. Using a 30-day window, there were 1,879 probable post-discharge VTEs (47.8% of 30-day postoperative VTEs). Of those, 1,608 (85.6% of post-discharge VTEs) resulted in a VTE-related VA readmission within 30 days, accounting for 3.3% of 57,064 30-day readmissions.

As shown in Table 3, probable postoperative VTEs were most common in orthopedic surgery (1.3% within 30 days, 1.7% within 90 days), followed by abdominal-pelvic (1.0% within 30 days, 1.4% within 90 days), and other procedures (0.8% within 30 days, 1.3% within 90 days).

Table 3.

Post-operative venous thromboembolism events within 30- and 90-day time window, by surgery type.

	30-Day Outcomes						90-Day Outcomes
	VTE		DVT		PE		VTE		DVT		PE
	N	%	N	%	N	%	N	%	N	%	N	%
Major Orthopedic Surgery (N = 37,525)
Pre-discharge	310	0.83%	201	0.54%	113	0.30%	320	0.85%	208	0.55%	116	0.31%
Post-discharge outpatient	37	0.10%	29	0.08%	8	0.02%	65	0.17%	51	0.14%	15	0.04%
Post-discharge readmission	155	0.41%	105	0.28%	64	0.17%	244	0.65%	175	0.47%	96	0.26%
Total	502	1.34%	335	0.89%	185	0.49%	629	1.68%	434	1.16%	227	0.60%
Abdominal-Pelvic Surgery (N = 68,451)
Pre-discharge	416	0.61%	234	0.34%	197	0.29%	446	0.65%	252	0.37%	211	0.31%
Post-discharge outpatient	31	0.05%	19	0.03%	12	0.02%	74	0.11%	51	0.07%	24	0.04%
Post-discharge readmission	206	0.30%	122	0.18%	116	0.17%	457	0.67%	293	0.43%	231	0.34%
Total	653	0.95%	375	0.55%	325	0.47%	977	1.43%	596	0.87%	466	0.68%
Other Surgery (N = 362,539)
Pre-discharge	1,326	0.39%	961	0.28%	433	0.13%	1,422	0.44%	1,031	0.32%	467	0.14%
Post-discharge outpatient	203	0.06%	154	0.04%	52	0.02%	383	0.12%	284	0.09%	105	0.03%
Post-discharge readmission	1,247	0.36%	859	0.25%	519	0.15%	2,493	0.77%	1,811	0.56%	952	0.29%
Total	2,776	0.81%	1,974	0.58%	1,004	0.29%	4,298	1.32%	3,126	0.96%	1,524	0.47%

Note: Event counts described here are probable VTE, DVT, and PE events. Probable pre-discharge or post-discharge readmission events were defined as possible events (ICD-9 codes only) confirmed with a pharmacy record for anticoagulant therapy (inpatient Level 2). Probable post-discharge outpatient events were defined as possible events (ICD-9 codes confirmed with either a pharmacy record for anticoagulant therapy or a CPT code) confirmed with evidence of VTE without specific mention of negation, history, or family history and with explicit confirmation of acute VTE in clinic notes identified through NLP (outpatient Level 3).

Discussion

Surveillance of potentially avoidable patient safety events such as HA-VTE is needed [29,30]. This paper presents estimates of the frequency of VTE events in a large sample of surgical inpatient admissions in the VA over a 6-year time period from 2005–2010 using data from 111 hospitals.

Our study improves on the postoperative VTE surveillance literature in at least three ways. First, by looking back 12 months prior to a surgical admission, we were able to exclude VTEs that were present on admission and thereby avoid false-positives. Although recently published data suggest that use of the POA indicator in hospital databases together with exclusion of VTE as a primary diagnosis can exclude almost all preexisting VTE diagnoses in surgical admissions [17], the VA National Patient Care Database does not include the POA indicator. Second, our longitudinal data allowed us to capture post-discharge VTEs up to 90 days after surgery. Third, we harnessed the valuable information in unstructured narrative text using NLP to verify an outpatient acute DVT or PE diagnosis that did not result in an inpatient admission at a VA medical center.

Using our algorithms, probable VTE occurred in 1.3% of surgical admissions for VA patients in the 90 days after surgery. More than one-half (63%) of VTE events within 90 days of a surgery occurred post-discharge, and 14% of those diagnoses were found only in outpatient VA records. The implication is that attempts to monitor postoperative VTE based on inpatient records alone will underestimate its occurrence, as has also been demonstrated by analysis of data from the National Surgical Quality Improvement Program (NSQIP) operated by the American College of Surgeons (ACS-NSQIP). According to one analysis of ACS-NSQIP data, 40% of postoperative VTE events are only recognized post-discharge [7]. That is slightly lower than the 48% figure found in our study with a 30-day window. However, our sample was restricted to inpatient admissions with surgical procedures whereas the ACS-NSQIP analysis included both inpatient and outpatient procedures. Also, as is explained below, the ACS-NSQIP data require validation by chart review.

Our estimates of 1.3% VTE within 90 days of surgery overall and 1.7% for major orthopedic surgery are lower compared with published estimates based on analyses of administrative data. For example, Amin et al. reported that the rate of symptomatic VTE within 180 days of inpatient surgeries in a sample of privately insured US adults was 1.9% among abdominal surgery patients and 3.2% among orthopedic surgery patients, roughly 80% of which were recorded within the first 90 days [31]. The investigators used outpatient claims accompanied by a prescription of an anticoagulant, a relevant procedure or an ED visit or inpatient admission within 2 days to identify symptomatic VTE. However, our NLP analysis found that the presence in outpatient administrative records of an ICD-9 code for DVT or PE together with a prescription of an anticoagulant does not reliably indicate acute VTE. Specifically, just 21–23% of possible VTEs in outpatient records were confirmed by NLP to be probable VTE events. An older study that conducted medical record reviews to validate outpatient diagnoses of DVT among pregnant women reported that 42% were true cases and that even after requiring the presence of a prescription for an anticoagulant just 65% of apparent DVT cases were confirmed [14].

To assess the accuracy of our estimates of postoperative VTE requires comparison with estimates based on the ‘gold standard’ of objective confirmation with imaging tests. Two US surveillance systems currently abstract samples of medical charts for recently hospitalized patients and record cases of DVT and PE. In hospitals participating in the ACS-NSQIP and a similar system operated by the VA (VASQIP), trained nurse abstractors abstract samples of patient charts, including from ambulatory surgical units and outpatient physician offices, using standardized case definitions, in order to identify events occurring up to 30 days post-surgery. For both systems, the diagnosis of DVT or PE requires positive results of imaging tests. In addition, DVT confirmation requires either anticoagulation therapy or a relevant procedure. However, coding errors still occur sometimes; a study using a NLP tool to audit VASQIP records from six VA hospitals identified cases of arterial thrombosis and hematoma incorrectly classified as DVT by the nurse chart abstractors [19].

Analyses of VASQIP data provide conflicting estimates of the sum of rates of postoperative DVT and PE within 30 days of surgery. Two analyses reported DVT or PE within 30 days in roughly 0.7% of major surgical patients [22,32]. One other study reported DVT or PE in roughly 2.0% of patients on the basis of VASQIP data, with almost no overlap between DVT and PE [8].

ACS-NSQIP analyses have reported rates separately for DVT and PE, with a total of 1.1–1.5% depending on which types of surgical patients were included. First, Bilimoria et al., using ACS-NSQIP data from 2006– 2007 on 329,951 inpatient or outpatient surgeries, report DVT and PE as having occurred within 30 days in 0.75% and 0.30%, respectively [7]. Second, an analysis by Gajdos et al. of ACS-NSQIP data for 165,600 nonemergent general major surgical inpatients (with procedures generally requiring more than an overnight stay) during 2005–2008 found a combined VTE rate of 1.5% [33]. The case definition in this latter study is closer to that used in the present study.

Our finding of 1.3% VTE diagnosed within 30 days following orthopedic surgery can be compared with ACS-NSQIP estimates of postoperative PE and DVT, but not for VTE as a group. One study, using ACS-NSQIP data from 2006–2010 on 21,434 patients who underwent primary total knee or hip arthroplasty, reported 30-day rates of PE and DVT of 0.62% and 1.06%, respectively [34]. Another study, using ACS-NSQIP data from 2005–2009 for 8950 patients undergoing primary total or partial knee arthroplasty, reported 30-day rates of PE and DVT requiring treatment of 0.76% and 1.46%, respectively [35].

Two previous studies from the same time period (2005–2007) that used chart reviews to validate presumed VTE diagnoses based on ICD-9 codes in inpatient records confirmed approximately 80% of presumed cases from administrative data [9,12]. In comparison, just 61–63% of possible pre-discharge diagnoses were classified in the present study as probable by the presence of a prescription for anticoagulation or an appropriate procedure in case anticoagulation might have been contraindicated. It is possible that charts may confirm diagnosis of DVT or PE but that clinicians had decided that the clots were asymptomatic and did not require treatment; our definition of probable VTE presumes treatment was ordered.

NLP offers the promise of accessing information previously only available through manual chart abstraction. The challenge is to develop NLP tools that perform as well as human experts. A previous study using NLP identified 59% of postoperative VTE events that were recognized by VASQIP-reviewed surgical patient charts from six VA medical centers during 1999–2006 [8]. A subsequent analysis indicated that the PPV for DVT was just 15% [19]. Our goal was not to use NLP on its own to identify postoperative VTE events but rather as a means to verify or reject suspected cases present only in outpatient records. Also, our NLP tool is limited in that it only applies to clinic notes, whereas the NLP tool in the VASQIP study was applied to all parts of the EHR, including imaging test results. Because we did not have a ‘gold standard’ review of imaging test results, we were not able to assess the sensitivity of the NLP tool; it is likely that some cases of treated VTE were not classified as “probable” as a result of missing information in the clinical notes. An implication is that our algorithm might have led us to understate the frequency of VTEs diagnosed and treated on an exclusively outpatient basis.

The current analysis has two other major limitations. First, many (28–47%) veterans receive at least some healthcare in non-VA facilities [36–38]. In our study, 14% and 10% of records with a probable post-discharge diagnosis of DVT and PE, respectively, within 90 days of an inpatient surgery lacked a VA record of a subsequent inpatient readmission. It is likely that many of these cases may have resulted in a readmission to a non-VA hospital, particularly among veterans dually enrolled in Medicare. Just over half (50.8%) of the probable post-discharge diagnoses of VTE occurred in patients over 65. Access to healthcare records at non-VA facilities would allow more complete assessment of hospital readmission and the number of post-discharge events. Some pre-admission VTE events might not be documented in VA data, which would also exclude post-discharge VTE events for which only non-VA care was received.

A second limitation is the lack of objective confirmation of VTE diagnoses through radiology records. First, certain ICD-9 codes used to ascertain DVT prior to October 2009 were not specific to DVT and included superficial thrombophlebitis [17], thus inflating our estimates to some extent. Second, the presence of an ICD-9 code for VTE in an outpatient record may be a rule-out diagnosis. To minimize that problem, we first required the presence of either a pharmacy record for anticoagulant therapy or a relevant CPT code. These items were not found in 58% of patients with at least one outpatient VTE diagnosis code (results available upon request), similar to the 71% reduction previously reported for outpatient VTE private insurance claims [13]. More work is needed to refine the preliminary NLP algorithm and apply it to radiology records and to records without prophylaxis prescriptions. We expect that an improved NLP algorithm will provide greater value. Without examining the results of imaging tests, we are not able to assess the sensitivity or specificity of our algorithms.

Among other limitations, the VA patient population is considerably older than the general adult male population. However, this limitation is not unique to our study. For example, studies that use insurance claims data to identify VTE may also suffer from this problem as people with employer-sponsored health insurance are also not representative of the general population. Finally, the date of the clinic note was used as the date of the VTE mention extracted by the NLP algorithm. However, clinic notes are not always entered on the date of the patient encounter. This may have resulted in misspecification of the event date.

In conclusion, VTE is an important, potentially preventable postoperative complication. Reliable monitoring of postoperative VTE events should incorporate two aspects. First, it should use data sources that allow for extraction of information from unstructured medical records. Second, it should use data sources that have the ability to follow patients after discharge from the hospital. Both data elements are essential for accurate surveillance of postoperative VTE.

Appendix A. Supplementary data

Supplementary data to this article can be found online at http://dx.doi.org/10.1016/j.thromres.2015.01.026.