Order Sets Utilization in a Clinical Order Entry System

Background

An order set is a predefined template that has been utilized in the standard care of hospitals for many years. While in the past, it took the form of pen and paper, today, it is, indeed, electronic. Within order sets are distinct ordering patterns that may yield fruitful results for clinicians and informaticians, alike. Protocols like there electronic counterpart, order sets, provide an ‘indication’ identifying the clinical scenario of the patient’s condition when the ordering event occurred. This ‘indication’ is rarely captured by individual orders, and provides difficult challenges to developers of information systems. While mandating an ‘indication’ be entered for every medication or lab order makes the job much more tasking on the physician provider, it is appealing to researchers and accountants. We have attempted to bypasses that consideration by identifying ordering patterns that predict diagnostic related codes (DRGs) and diagnostic codes which would greatly facilitate the information gathering process and still provide a flexible and user friendly physician interface.

Methods

Initial categorization grouped orders written within an order-set, and orders written without an order set. This grouping was done to establish the final cohorts that would permit comparison and allow determination of the utilization of order sets. Differences between individual orders in each order set were initially tested with a Yates corrected chi squared model. Each order set was then reanalyzed for patterns of orders nested within each individual order set. This was accomplished by coupling the subset of individual orders with the order that had the highest p-value from the previous round. Then, by comparing similar frequencies of DRGs, distinct ordering patterns were able to assign ambiguous orders (i.e. those placed without the aide of an order set) into the most likely order set. Based on this prediction, utilization of order sets was assessed (1).

Results

3.06 million orders were initially extracted from OSU's Medical Center Information Warehouse. The first round analysis resulted in a Yates corrected chi squared on 9762 orders of which 904 were statistically significant (x2 > 3.84). The second round resulted in more specific patterns and identified an ordering pattern in most order sets. Once these patterns were determined, they were correlated with known DRGs as well as with the detailed diagnostic codes. Over 2 million records were grouped by DRGs and diagnostic codes resulting in 516 DRG categories and 4729 diagnostic categories.

Our pattern of orders was modeled to profile an order set by combining two orders and evaluating their combined chi squared to predict what order set would corresponding to a seemingly uncharacteristic ordering pattern. The profile provides a powerful model not only to predict DRGs based on ordering patterns, but also to assess utilization and quality of the order sets, themselves.