Unintended Consequences of a Standard Admission Order Set on Venous Thromboembolism Prophylaxis and Patient Outcomes

ABSTRACT

BACKGROUND

Standard order sets often increase the use of desirable interventions for patients likely to benefit from them. Whether such order sets also increase misuse of these interventions in patients potentially harmed by them is unknown. We measured the association between a paper-based standard admission order set with a venous thromboembolism pharmacoprophylaxis (VTEP) module and use of VTEP for patients likely to benefit from it as well as patients with unclear benefit or potential harm from it.

METHODS

We conducted a retrospective cohort study using administrative and pharmacy charge data of patients admitted between 1 July 2005 and 31 December 2008 to two medical and three surgical services that implemented a standard admission order set in August 2006. The primary outcome was use of VTEP in patients with likely benefit, unclear benefit, and potential harm from VTEP prior to and following order set implementation.

KEY RESULTS

A total of 8,429 patients (32%) were admitted prior to and 17,635 (68%) following order set implementation. There was a small unadjusted rise in overall VTEP use after implementation (51% to 58%, p < 0.001). In multivariable models with interrupted time series analysis, patients with potential harm from VTEP had the largest increase in VTEP use at the time of implementation [adjusted odds ratio = 1.58; 95% confidence interval (CI), 1.12–2.22]. The increased likelihood of receiving VTEP in this subgroup gradually returned to baseline (adjusted odds ratio per month = 0.98; 95% CI, 0.96–0.99).

CONCLUSIONS

Implementation of a standard admission order set transiently increased VTEP in patients with potential harm from it. Order set and guideline success should be judged based on the degree to which they successfully target patients likely to benefit from the intervention without inadvertently targeting patients potentially harmed.

Electronic supplementary material

The online version of this article (doi:10.1007/s11606-011-1871-x) contains supplementary material, which is available to authorized users.

INTRODUCTION

As hospitals attempt to increase quality of care, order sets and pathways have become increasingly popular interventions for numerous problems. Order sets have successfully increased the provision of underused interventions such as venous thromboembolism pharmacoprophylaxis (VTEP),^1,2 basal insulin therapy,^3,4 and aggressive fluid resuscitation for sepsis;^5,6 many of these have been modularized and merged into comprehensive order sets¹ in the context of quality improvement initiatives.

Quality improvement initiatives have the potential for unintended consequences.⁷ Just as improved glycemic control can increase rates of hypoglycemia,⁸ improved door-to-balloon time can increase angiograms conducted for false-positive ECG findings,⁹ and improved adherence to an alcohol withdrawal protocol can even increase mortality,¹⁰ standardizing practice in the hospital through the use of order sets might result in overuse or even misuse of therapy in patients with unclear benefit or potential harm from it. Unfortunately, quality improvement initiatives often include no provision to measure overuse and misuse, and thus may overestimate any individual initiative’s efficacy.^11–13

We conducted a retrospective cohort study to evaluate the association between implementation of a standard admission order set at our hospital and VTEP use in patients with likely benefit, unclear benefit, or potential harm from VTEP. We also evaluated in-hospital VTE and hemorrhage prior to and following implementation.

METHODS

Setting

Our study took place at UCSF Medical Center, a 790-bed academic hospital. Adult patients admitted between 1 July 2005 and 31 December 2008 to the two medical (general medicine and cardiology) and three surgical (general surgery, gynecology, and gynecologic oncology) services that adopted the standard admission order set in August 2006 were included in the study. Other services such as neurosurgery adopted the order set at different times, and patients admitted to these services were excluded. The UCSF Institutional Review Board approved this study.

Data Sources

Through administrative billing data, we obtained for each patient the date of admission, age, gender, race, ethnicity, hospital admission source (emergency department or other), admitting service (medical or surgical), public vs. private insurance, and primary and secondary diagnoses and procedures based on International Classification of Diseases, 9^th Revision (ICD-9) codes as well as their “present on admission” designation (diagnoses) or administration date (procedures). We used ICD-9 diagnosis codes to determine patients’ Elixhauser comorbid measures, 30 conditions strongly associated with mortality and other adverse hospitalization-relevant outcomes.¹⁴ We also used a separate hospital pharmacy database to obtain the list of medications charged to each patient during the above admissions.

The Standard Admission Order Set

Prior to August 2006, UCSF providers handwrote unformatted admission orders; a VTEP guideline was available for reference, but there was no VTEP order set. After a 2-week orientation period beginning 1 August 2006, providers on the study services were required to use a preprinted order set for writing all admission orders.

Based on prior experiences with VTEP initiatives and an institutional goal of simplifying order sets, the forms committee introduced a pre-printed standard admission order set for all patients admitted to the medical center. The order set, and the VTEP module in particular (Appendix A, available online), was specifically designed with features critical to successful clinical decision support and guideline implementation as reviewed by Kawamoto et al.¹⁵: automatic provision of decision support within workflow, recommendations rather than risk assessment alone, at the time and place where decision-making occurs.

The VTEP module, based on then-current American College of Chest Physicians (ACCP) guidelines,¹⁶ was thus a required, integrated element of the decision-making process at admission. It included recommendations regarding VTEP use such as discouraging VTEP prescription for patients less than 40 years of age without other VTE risk factors or those expected to ambulate within 48 h. More importantly, it included the ability to easily prescribe (by checking a box) either enoxaparin 40 mg subcutaneously daily or unfractionated heparin 5,000 units subcutaneously three times daily (standard VTEP), or enoxaparin 30 mg subcutaneously twice daily (patients with very high VTE risk, such as those with hip fracture), with pneumatic compression devices recommended for all patients. We were unable to implement computerized decision support, the fourth and least clinically significant practice recommended by Kawamoto, in our (then) paper-based institution.

Definition of Patient Groups

Using a combination of primary and secondary diagnoses and procedures, we classified patients into three groups: (1) patients with likely benefit from VTEP; (2) patients with unclear benefit from VTEP; and (3) patients with potential harm from VTEP.

We classified patients as having likely benefit from VTEP if they first did not meet the criteria for potential harm below and also had diagnosis or procedure codes for conditions fulfilling the ACCP criteria for “high risk for in-hospital VTE,” including congestive heart failure, severe respiratory disease (pulmonary hypertension, obstructive airway disease with exacerbation, pneumonia, and/or mechanical ventilation within 2 days of admission), cancer, sepsis (present on admission), VTE history or thrombophilia, neurologically mediated immobility of any type, or inflammatory bowel disease.¹⁶

We classified patients as having unclear benefit from VTEP if they did not have diagnosis codes to support a classification of either potential harm or likely benefit.

We classified patients as having potential harm from VTEP if they had diagnostic codes, present on admission, in one of two categories (active hemorrhage or bleeding diathesis) that would place them at increased risk of complications from VTEP. Active hemorrhage included codes for intracranial or gastrointestinal hemorrhage, or hematuria, hemoptysis or epistaxis. Bleeding diathesis included any coagulation factor deficiency, thrombocytopenia (including heparin-induced), or liver disease with esophageal varices, portal hypertension, or cirrhosis with complications.

A list of ICD-9 codes used to classify patients and events is available as Appendix B (available online).

Definition of Outcomes—Receipt of VTEP, In-Hospital VTE, In-Hospital Hemorrhage

We classified patients as having received VTEP if they were charged for at least one dose of 40 mg or 30 mg of enoxaparin or 5,000 units of subcutaneously administered unfractionated heparin within the first 3 calendar days of admission. We used this time frame to account for patients admitted close to midnight and patients in whom prophylaxis was held for a procedure on the first 2 days of hospitalization.

We classified patients with ICD-9 codes for VTE not present on admission as having experienced in-hospital VTE, and patients with ICD-9 codes for intracranial or gastrointestinal hemorrhage not present on admission as having experienced in-hospital hemorrhage.

To verify the accuracy of ICD-9 codes and pharmacy charge data, one author (RK) reviewed 301 charts. Forty were selected from patients with diagnosis codes for gastrointestinal hemorrhage present on admission, while 226 were selected from patients with diagnosis codes for venous thromboembolism. The admission history and physical and progress notes were reviewed for evidence of gastrointestinal hemorrhage or VTE, respectively. An additional 35 charts were reviewed for medication administration records for evidence that our pharmacy charge data corresponded to actual VTEP receipt.

Gastrointestinal hemorrhage diagnosis codes present on admission corresponded to charted gastrointestinal hemorrhage on admission in 36/40 (90%) patients. VTE codes corresponded to definite or likely VTE in 212/226 (94%) patients. VTEP charge data were highly sensitive (100%; 15/15) and specific (85%; 17/20) for VTEP receipt by medication administration record.

Statistical Analysis

Since UCSF only began recording present on admission designations in January 2007, we used multiple imputation to predict present on admission designations for patients admitted to the hospital prior to this.^17–19 Using a stepwise procedure, we developed logistic models for present on admission designations—either “yes” or any other non-missing option (coded “no” to maximize positive predictive value of the “yes” designation)—for VTE, intracranial hemorrhage, gastrointestinal hemorrhage, other hemorrhage, liver disease, coagulopathy, and sepsis (the only contributor to the ACCP criteria for “likely benefit” that could plausibly develop in the hospital), based on data obtained in January 2007 and later. Each model contained different predictors to better approximate the diverse outcomes noted above, but in general included demographic and clinical variables such as sex, race, and variable Elixhauser comorbidity measures. Next, we used these models to impute present on admission designations five times for each missing value. In all analyses using these five completed data sets, we used standard methods to combine results, ensuring that overall inferences properly reflected the additional uncertainty induced by the missing present on admission designations.^17–20 To assess the accuracy of our imputation approach, we reviewed the 226 charts above, which were all missing the VTE present on admission designations. The expected present on admission designations by imputation were extremely well calibrated to observed presence on admission designation by chart review (Hosmer-Lemeshow test, p = 0.94).

Using our imputed data sets, we conducted univariate analyses of VTEP use in each subgroup as well as in-hospital VTE and hemorrhage event rates prior to and following the order set using χ² tests. To determine whether the effects of the order set differed in the three subgroups (likely benefit, unclear benefit, potential harm), we used a Mantel-Haenszel test of homogeneity to compare the relative risk for VTEP use prior to and following the order set in each subgroup.

In our multivariable analysis, we estimated the trajectory of VTEP use in each patient subgroup using interrupted time series (ITS) logistic models, clustered by patients to account for multiple admissions.^21,22 Within each of the three subgroups of the sample, the ITS logistic model specified three odds ratios. The first modeled the secular trend (in adjusted odds per month) in VTEP use in the time prior to the order set; the second modeled the change in VTEP use at the moment of order set implementation; the third modeled the secular trend (in adjusted odds per month) in VTEP use following implementation. The second odds ratio captured the early effects of the intervention, while the ratio of the third odds ratio to the first, a measure of change in the secular trend following implementation, captured the later effects of the intervention. We utilized ITS logistic modeling for in-hospital VTE and in-hospital hemorrhage as well. For the VTEP ITS model we adjusted for age, sex, race, ethnicity, admitting service, admission source, insurance status, and all Elixhauser comorbidity measures, all specified a priori. To ensure the robustness of the estimated ITS effects for the much less common in-hospital VTE and hemorrhage outcomes, we adjusted for age, sex, race, ethnicity, admitting service, admission source, and insurance status, but used backwards elimination of Elixhauser comorbidity measures with p values >0.20 to reduce the risk of overfitting.^23,24

All analyses were conducted using Stata Version 11 (College Station, TX).

RESULTS

Patient Characteristics (Table 1)

Table 1.

Patient Characteristics

Characteristic	Prior to order set N = 8,429 (32%)	Following order set N = 17,635 (68%)	Significance (p value)
Age, years
Mean (SD)	59 (19)	59 (19)	0.84
Male, n (%)	3,613 (42.9)	7,774 (44.1)	0.06
Race, n (%)			<0.001
White	4,601 (54.6)	9,372 (53.1)
Black	1,262 (15.0)	2,489 (14.1)
Asian	1,462 (17.3)	3,168 (18.0)
Other	1,104 (13.1)	2,606 (14.8)
Hispanic (any race), n (%)	708 (8.4)	1,580 (9.0)	0.14
Medicine or cardiology service, n (%)	5,910 (70.1)	12,403 (70.3)	0.72
Admission from ED, n (%)	4,984 (59.1)	10,630 (60.3)	0.08
Public or No Insurance, n (%)	4,982 (59.1)	10,237 (58.1)	0.11
Elixhauser comorbidities, n (%)*
Congestive heart failure	710 (8.4)	1,350 (7.7)	0.032
Cardiac arrhythmia	1,173 (13.9)	2,719 (15.4)	0.001
Valvular disease	373 (4.4)	1,015 (5.8)	<0.001
Other neurological disorders	507 (6.0)	1,021 (5.8)	0.47
Chronic pulmonary disease	1,303 (15.5)	2,805 (16.3)	0.095
Diabetes without complications	1,534 (18.2)	2,805 (15.9)	<0.001
Hypothyroidism	770 (9.1)	1,802 (10.2)	0.006
Renal failure	946 (11.2)	2,615 (14.8)	<0.001
Liver disease	554 (6.6)	1,335 (7.6)	0.004
Metastatic cancer	542 (6.4)	1,253 (7.1)	<0.001
Coagulopathy	371 (4.4)	1,245 (7.1)	<0.001
Weight loss	262 (3.1)	1,207 (6.8)	<0.001
Fluid and electrolyte disorders	1,511 (17.9)	4,401 (25.0)	<0.001
Deficiency anemias	1,248 (14.8)	3,569 (20.2)	<0.001
Drug abuse	496 (5.9)	989 (5.6)	0.37
Depression	715 (8.5)	1,884 (10.7)	<0.001
Hypertension	3,707 (44.0)	7,806 (44.3)	0.67
Patient subgroups, n (%)**			<0.001
Likely benefit from VTEP	3,865 (45.9)	8,365 (47.4)
Unclear benefit from VTEP	3,541 (42.0)	6,826 (38.7)
Potential Harm from VTEP	1,023 (12.1)	2,444 (13.9)

ED = emergency department. VTEP = venous thromboembolism pharmacoprophylaxis

*Only comorbidity measures with an overall prevalence of 5% or more are listed here (17/30)

**Average of the five imputed data sets. Ranges are 3,859–3,871 (likely benefit, prior to order set); 8,361–8,369 (likely benefit, following the order set); 3,536–3,546 (unclear benefit, prior to order set); 6,822–6,830 (unclear benefit, following the order set); 1,015–1,032 (potential harm, prior to order set); 2,438–2,452 (potential harm, following the order set)

Our data set included 26,064 patients; 8,429 (32%) were admitted prior to implementation of the order set and 17,635 (68%) afterwards. There were small but statistically significant differences in patient race and several comorbid diseases. After imputation of present on admission designations, there were fewer patients with unclear benefit from VTEP following implementation of the order set (p < 0.001).

Unadjusted Outcomes (Table 2)

Table 2.

Venous Thromboembolism Prophylaxis and In-Hospital VTE and Hemorrhage Events Before and After Order Set Implementation

Outcomes	Prior to order set N = 8,429 (32%)	Following order set N = 17,635 (68%)	Significance (p value)
VTEP rate (all patients), n (%)	4,261 (50.6%)	10,238 (58.1%)	p < 0.001
Patients with likely benefit	2,278 (58.9%)	5,466 (65.3%)	RR = 1.11*
Patients with unclear benefit	1,654 (46.8%)	3,788 (55.5%)	RR = 1.19*
Patients with potential harm	329 (32.0%)	984 (40.3%)	RR = 1.26*
In-hospital VTE, n (%)	47**(0.6%)	97**(0.6%)	p = 0.9
In-hospital hemorrhage, n (%)	37**(0.4%)	143**(0.8%)	p = 0.02

VTEP = Venous thromboembolism prophylaxis. VTE = venous thromboembolism. ED = emergency department. RR = relative risk of receiving VTEP following the intervention (as compared to prior to it)

*Mantel-Haenszel test for homogeneity between relative risk estimates conducted; p < 0.01

**Average of the five imputed data sets. Ranges are 35–62 (in-hospital VTE events prior to order set); 93–98 (in-hospital VTE events following the order set); 31–46 (in-hospital hemorrhage events prior to order set); 138–145 (in-hospital hemorrhage events following the order set)

Use of VTEP overall increased following the order set (51% to 58%, p < 0.001), with a significantly larger increase in the patients with potential harm (Mantel-Haenszel test for heterogeneity, p < 0.01). In-hospital VTE occurred in 0.6% of patients both prior to and following the order set (p = 0.9); unadjusted hemorrhage occurred in 0.4% prior to and 0.8% following (p = 0.02).

VTEP Use Over Time, Adjusted (Table 3, Fig. 1)

Table 3.

Adjusted Trends in Odds for Use of Venous Thromboembolism Prophylaxis by Patient Subgroup

	Secular trend in VTEP, months prior to order set, 7/2005–8/2006	Change at implementation of order set (omitting secular trends)	Secular trend in VTEP, months following order set, 8/2006–12/2008
	AOR-M (95% CI)	AOR (95% CI)	AOR-M (95% CI)
Patients with likely benefit	1.05* (1.03–1.07)	1.07 (0.89–1.28)	0.992* (0.986–0.999)
Patients with unclear benefit	1.04* (1.02–1.06)	1.22* (1.02–1.47)	1.00 (0.99–1.01)
Patients with potential harm	1.05* (1.01–1.10)	1.58* (1.12–2.23)	0.98* (0.96–0.99)

VTE P= Venous thromboembolism prophylaxis. AOR-M = adjusted odds ratio per month. AOR = adjusted odds ratio

Adjusted for age, sex, race, ethnicity, admitting service, admission source, insurance, and Elixhauser comorbidity measures

Odds ratios are in comparison to the baseline odds for each group

*p < 0.05

Figure 1.

Patients receiving venous thromboembolism prophylaxis, by subgroup. Lines indicate fitted rates in each group per quarter (solid, dashed, and dotted lines indicating patients with likely benefit, unclear benefit, and potential harm, respectively), assuming a linear trend prior to the intervention, a change at the moment of the intervention, and a linear trend following the intervention, adjusted for age, sex, race, ethnicity, admitting service, admission source, insurance, and Elixhauser comorbidity measures. The scatter plots indicate actual means per quarter (hollow circles, pluses, and solid triangles indicating patients with likely benefit, unclear benefit, and potential harm, respectively).

Among patients with likely benefit, VTEP use increased prior to the order set, plateaued at the moment of order set implementation, and fell slightly in the months following. Among patients with unclear benefit, VTEP use increased prior to order set implementation, increased at the moment of order set implementation, and remained constant in the months following. Among patients with potential harm, VTEP use increased prior to the order set, increased sharply at the moment of implementation, and fell significantly in the months following. In all three groups a trend toward increased use ended soon after implementation (p < 0.01 for all).

In-Hospital VTE and Hemorrhage Over Time, Adjusted (Table 4, Fig. 2)

Table 4.

Adjusted Trends in Odds for In-Hospital Venous Thromboembolism and Hemorrhage

	Secular trend in in-hospital events, months prior to order set, 7/2005–8/2006	Change at implementation of order set (omitting secular trends)	Secular trend in in-hospital events, months following order set, 8/2006–12/2008
	AOR-M (95% CI)	AOR (95% CI)	AOR-M (95% CI)
In-hospital VTE	0.98 (0.89–1.08)	0.52 (0.17–1.63)	1.04† (1.00–1.07)
In-hospital hemorrhage	0.98 (0.87–1.11)	0.82 (0.31–3.52)	1.05* (1.02–1.07)

VTE = Venous thromboembolism. AOR-M = adjusted odds ratio per month. AOR = adjusted odds ratio

Adjusted for age, sex, race, ethnicity, admitting service, admission source, insurance, and selected Elixhauser comorbidity measures (for VTE: diabetes, diabetes with complications, liver disease, peptic ulcer, coagulopathy, weight loss, and fluid and electrolyte disorders; for hemorrhage: arrhythmia, renal failure, peptic ulcer, coagulopathy, weight loss, chronic blood loss anemia, and deficiency anemias)

Odds ratios are in comparison to the baseline odds for each group

*p < 0.05

†p = 0.07

Figure 2.

In-hospital events. Lines indicate fitted events per 10,000 admissions per quarter (solid and dashed indicating in-hospital VTE and in-hospital hemorrhage, respectively), assuming a linear trend prior to the intervention, a change at the moment of the intervention, and a linear trend following the intervention (adjusted for age, sex, race, ethnicity, admitting service, admission source, insurance, and selected Elixhauser comorbidity measures—see Table 4.) The scatter plots indicate actual events per 10,000 admissions per quarter (solid and hollow circles indicating in-hospital VTE and in-hospital hemorrhage, respectively).

After adjustment, there was no statistically significant trend in in-hospital VTE prior to the order set, nor a change at the moment of implementation, but we did note a rise in VTE in the months following that did not reach statistical significance [adjusted odds ratio per month (AOR-M), 1.03; p = 0.07]. There was no statistically significant trend in in-hospital hemorrhage prior to the order set, nor at the moment of implementation; however, there was a significant rise in bleeding events following implementation (AOR-M, 1.05, p < 0.001).

Sensitivity Analyses

Stratifying the potential harm group into patients with active hemorrhage as compared to bleeding diathesis showed both groups following the same trend. We also redefined the time period following the order set into two stages (early vs. late); the overall trends did not change.

DISCUSSION

Implementation of a standard admission order set at our medical center was associated with a small increase in overall VTEP use. This increase, however, appeared to be driven by pre-implementation secular trends in all groups (which unexpectedly terminated after the order set, including, unfortunately, in the likely benefit group) and a transient increase in VTEP at the moment of implementation in patients with potential harm from it. Further, despite the intervention, rates of in-hospital hemorrhage and perhaps of in-hospital VTE both increased over time.

The transient increase in VTEP use in patients potentially harmed by it may have resulted from the following. First, institutional mandates to standardize care and pressure to improve VTEP rates, coupled with the new VTEP module, may have created the impression that VTEP prescription should be the default decision, overwhelming any module-based caveats or alternatives. Second, this impression may have had the maximum impact on patients with potential harm, because in these patients VTEP use was initially appropriately rare. Thankfully, VTEP use among this population quickly returned to baseline, indicating that perhaps early misuse was recognized and corrected. To our knowledge, at our hospital, there were no other specific interventions to explain this decline. Given that our intervention was explicitly modeled after successful decision support strategies,¹⁵ that our absolute increase in VTEP (7%) falls within the range seen in other studies (3.3%²⁵ to 24%²), and that our proportion of patients with a contraindication to VTEP due to potential harm (13%) is similar to that of other administrative data sets (20%²⁶), similar increases in potential misuse may well exist elsewhere and deserve urgent exploration.

Our study is the first, to our knowledge, to explicitly recognize VTEP use among patients with potential harm from it—potential misuse—as an important endpoint for quality improvement interventions. This proceeded from our observations that such misuse may serve as an early warning of unintended consequences and that many quality improvement initiatives do not currently report it.⁷ While the Society of Hospital Medicine comprehensive VTEP quality improvement toolkit does suggest risk-stratifying patients and measuring outcomes such as heparin-induced thrombocytopenia or hemorrhage, they also recommend omitting patients with contraindications from consideration within the initiative.²⁷ As a result, potential VTEP misuse in this group might never be discovered.

Our study supports the importance of measuring adverse events, as we saw a secular increase in in-hospital hemorrhage (and perhaps even in VTE) after implementation despite a raw increase in VTEP after the intervention. In-hospital hemorrhage and VTE would be the ideal efficacy endpoints for any VTEP intervention. However, they are very rare (approximately 0.5–1.0% each for our patients, similar to that seen in meta-analyses of VTEP efficacy²⁸) and should be measured in conjunction with, not as substitutes for, potentially beneficial and harmful use. Most single institutions will be underpowered to detect even clinically meaningful differences in actual benefit or harm.

Our study had important limitations. We used billing and diagnosis-code administrative data to identify patient groups, events, and VTEP use. ICD9 codes often describe severity of illness (e.g. degree of respiratory disease, gastrointestinal hemorrhage, or thrombocytopenia) or even category of illness poorly,²⁹ though our chart reviews suggest that our data are accurate for these uses. Though we were able to examine the effect of admission to a medical or a surgical service before, during, and after the order form, we were unable to identify individual providers who may have been more or less affected by it. We were unable to account for admission to the ICU (though we used mechanical ventilation as a proxy) as an ACCP criterion for likely benefit, and for impending surgical procedures as a criterion for potential harm (because there was no way to distinguish the hour of either VTEP or of surgery). We were forced to impute presence on admission for several diagnoses, a process that can introduce both imprecision and bias, especially if the imputation models are not optimized. However, our chart review indicated excellent calibration of the imputed data sets for VTE, we demonstrated that VTEP use increased in the potential harm group regardless of how potential harm was characterized, our imputed data sets were very similar to one another, and the trend (and change in trend) for each outcome was robust. The use of first prescription of VTEP is likely less specific for benefit and harm than overall time receiving VTEP, though we focused on the former because our order set centered on the admission. Information regarding pneumatic compression device use was also unavailable from our administrative data set, as was expectation of early patient mobility. The finding that increasing VTEP did not correlate with decreasing VTE is at odds with a recent review.³⁰ However, our study may have been different for two reasons. First, we were examining administrative data and thus clinical VTE events rather than radiological VTE events, which were the outcome of the reviewed studies. Second, while VTEP indisputably reduces VTE in a high-risk population, it may not be as effective in a more general population such as ours, which also appears to be at slightly higher risk of bleeding. Finally, the use of typewritten order sets and computerized ones may not correspond perfectly since handwriting and typewriting are different cognitive processes.³¹ It is unclear how our findings will generalize to the era of computerized physician order entry.

In conclusion, a standard admission order set at our institution was associated with a transient increase in VTEP use among patients with potential harm from it and increased in-hospital hemorrhage and potentially VTE as well. Quality improvement initiative assessment should measure targeted intervention use in all patients to ensure overuse and misuse are not overlooked, especially as drivers of overall use. This approach will provide a deeper understanding of the effects of these interventions and help us to avoid those in which the overall increase in use occurs primarily in the group most likely to be harmed by it.