The impact of computerized provider order entry systems on medical-imaging services: a systematic review

Abstract

Background

Computerized provider order entry (CPOE) systems have been strongly promoted as a means to improve the quality and efficiency of healthcare.

Methods

This systematic review aimed to assess the evidence of the impact of CPOE on medical-imaging services and patient outcomes.

Results

Fourteen studies met the inclusion criteria, most of which (10/14) used a pre-/postintervention comparison design. Eight studies demonstrated benefits, such as decreased test utilization, associated with decision-support systems promoting adherence to test ordering guidelines. Three studies evaluating medical-imaging ordering and reporting times showed statistically significant decreases in turnaround times.

Conclusions

The findings reveal the potential for CPOE to contribute to significant efficiency and effectiveness gains in imaging services. The diversity and scope of the research evidence can be strengthened through increased attention to the circumstances and mechanisms that contribute to the success (or otherwise) of CPOE and its contribution to the enhancement of patient care delivery.

Introduction

Medical-imaging modalities, including radiology, nuclear medicine, and ultrasound, allow clinicians to image the body for the examination, diagnosis, and increasingly treatment of medical conditions.¹ Medical imaging has progressed rapidly as a result of technological and computing advances,² and it is estimated to account for 5–10% of current healthcare expenditure.³ Factors driving the expansion of medical-imaging utilization include the increased availability and functionality of imaging modalities, patient demand for more examinations and an aging population dealing with complex conditions.³ As service use increases, concerns about the appropriate use of medical-imaging services, their costs, and their contribution to the quality of patient care have been raised. Some estimates have suggested that 30–40% of all imaging examinations in the USA may be inappropriate.⁴

While major advances have been made in the way that images are created, stored, and retrieved,⁵ they have not always been matched by the development of systems to manage workflow, ensure patient safety, or optimize device and modality utilization.⁶ The introduction of computerized provider order entry (CPOE) systems and associated clinical decision-support features such as structured order sets and automated feedback are promoted for their potential to reduce errors and improve conformity with evidence-based clinical practice.^{3 7} Reporting on the challenges facing medical imaging, Khorasani³ identified a number of key potential clinical and cost-saving benefits associated with the introduction of CPOE to medical-imaging departments. These included: (a) the seamless integration of information across the hospital to improve the accuracy and effectiveness of medical imaging and enhance its contribution to quality patient care; and (b) the use of decision support to address issues such as the appropriateness of medical-imaging orders and the problem of redundant or unnecessary testing.

In 1997, Bates et al⁸ and Harpole et al⁹ were among the first to report on the impact of CPOE on medical imaging. Bates et al⁸ examined the impact of computerized display of charges on test utilization but found no change in medical-imaging orders. Harpole et al⁹ reported positive effects of automated evidence-based critiquing, for abdominal radiograph orders, on physician decision-making. Aside from these early studies,^{8 9} attention to the impact of CPOE on medical imaging has not been extensive, and there has been no synthesis of the available evidence. Consequently, our aim was to undertake a systematic review to assess the evidence of the impact of CPOE on medical-imaging services and patient outcomes.

Methods

Search strategy

Our systematic review was based on Cochrane review principles¹⁰ and used the Critical Appraisal Skills Programme to guide and enhance the quality of its design.¹¹ We searched for relevant English language articles, published between January 1998 and December 2010, using Medline, Embase, Inspec (all via OvidSP), and CINAHL (via EBSCOhost). This period was used as a time span often associated with a dramatic increase in commercial CPOE packages.¹² To identify prospective articles, keywords and subject headings that refer to a CPOE system or ordering process were combined with keywords and subject headings related to hospital medical imaging (see figure 1, available as an online data supplement at www.jamia.org). Database searching was supplemented by hand-searching reference lists of relevant articles. As a means of checking that pertinent literature had not been missed, we examined the Oregon Health and Science University CPOE Bibliography, which contains an extensive listing of CPOE-related research papers.¹³

Study selection and inclusion criteria

Through the combined database search, 4129 articles were identified (excluding duplicates). Figure 1 illustrates the selection process. All citations were independently screened by four reviewers (AG, MP, AM, and EA). Only those that were clearly unrelated to the study were excluded based on their title—for example, opinion piece, editorial, commentary. Abstracts of citations that appeared relevant, or those where the titles were ambiguous, were assessed by all four reviewers to determine inclusion. Variance between reviewers was resolved by discussion, requiring consensus from all four reviewers. The full text of 72 articles was reviewed and resulted in a final sample of 14 articles which met the inclusion criteria.

Figure 1.

Article search and selection process.

We included experimental, evaluative or observational studies which reported quantitative measures of the impact of CPOE on medical imaging involving the modalities of radiology. Studies of picture archiving and communication systems (PACS), decision-support systems (DSS), radiology information systems, hospital information systems, and other information systems were included only if they were linked to a CPOE system. Studies were excluded if they were not explicitly about a CPOE system and medical imaging within a hospital setting.

Data abstraction and analysis

Relevant indicators of impact, as a result of CPOE implementation, were identified from the studies, including: ordering behavior; efficiency of test ordering and turnaround times; and patient outcome indicators (eg, readmission rates; length of stay; mortalities). Additionally, we extracted the reported technical features—for example, whether the CPOE systems in the studies were homegrown (developed internally by the institutions in which they were used) or commercial applications (systems purchased from software vendors that may have been modified for use in a particular clinical setting).¹⁴ Information about technical integration with other information systems, such as if the CPOE system was integrated with PACS, was abstracted where available.

The heterogeneous character of the studies, and the multifaceted medical-imaging process, precluded formal meta-analyses. As a means of evaluating the effect of CPOE, a temporal breakdown of the imaging process was developed, based on processes outlined by Mekhjian et al,¹⁵ which distinguished between the following components: (a) practitioner ordering and scheduling—the commencement of the order request using the CPOE application, or paper request, and scheduling of the test; (b) medical-imaging department procedures up to and including the delivery of completed results and reports; and (c) the application of test results to diagnose and support patient-management decisions.

Findings

We identified 14 studies that assessed the impact of CPOE systems on hospital medical-imaging services. The key elements of these studies are summarized in table 1 (available as an online data supplement at www.jamia.org). The majority (n=11) were conducted in the USA, with the remaining studies conducted in France (n=1), South Korea (n=1), and Canada (n=1). Eight of the studies were conducted at one of three US study sites; three at the Vanderbilt University Medical Center (21%), three at Massachusetts General Hospital (21%), and two at the Ohio State University Medical Center (14%). Twelve studies were conducted in academic hospitals, one in a veterans affairs hospital, and one at a Kaiser Permanente hospital. Six studies evaluated CPOE systems in critical care settings; three within emergency departments (EDs) and three within intensive care units (ICUs). A further three studies were carried out within outpatient departments. The study designs were predominantly pre-/postintervention comparisons (n=10), one of which provided a control, and four were time-series studies. There were no randomized controlled trials.

Table 1 depicts the aspects of the medical-imaging process investigated within the included studies and the associated outcomes measured. Ten studies measured the impact of CPOE on the ordering of medical-imaging examinations by practitioners;^16–25 five measured the impact of CPOE on ordering efficiency and result turnaround times;^{15 22 26–28} and five measured the impact of CPOE on patient outcome indicators, such as length of stay, mortality, and readmission rates.^{15 19–21 27}

Table 1.

Measures of outcome assessed in the medical-imaging process

Medical-imaging process	Outcome measures
Ordering of medical-imaging examinations by the practitioner	Ordering behavior16–25—impact of computerized provider order-entry systems with and without decision-support features on practitioners' test-ordering practices and choices Test-ordering times26—time required to complete test-order entry Turnaround times15 22 27 28—time taken from entering a test order to results availability
Medical-imaging department procedures	Turnaround times15 22 27 28—time taken from entering a test order to results availability
Application of results—patient-outcome indicators	Length of stay15 19–21 27 Mortality19 20 27 Readmission rates19 20

Ordering behavior

Of the 10 studies that evaluated the impact of CPOE systems on practitioners' test ordering behavior, seven^{16–20 24 25} assessed the effect of decision-support features that promoted the use of guidelines in the test ordering process. The other three studies^21–23 did not specify the presence of a decision-support feature.

CPOE systems with decision support

Carton et al¹⁶ conducted a time-series study, where computerized guidelines were made available to practitioners on alternating months during a 6-month study period in two EDs. When practitioners placed an order, the CPOE system provided a list of possible clinical contexts relating to the requested examination. During intervention months, on-screen reminders displayed the appropriate recommendations concerning the clinical context, and practitioners were alerted if requests did not conform to guidelines. The availability of guidelines in the CPOE system decreased the percentage of radiology orders that did not conform to guidelines from 33.2% to 26.9% (p=0.0001). The proportion of orders that did not conform to guidelines during the study period was significantly different across the two EDs (17.6% in site A and 34.8% in site B (p=0.0001)). A large portion of this difference was attributed to systematic attitudes instilled in site B, where chest radiographs (CXRs) were ordered for all hospitalized patients.

Chin¹⁷ demonstrated improvements in the percentage of upper gastrointestinal radiography orders conforming to guidelines following their implementation in the CPOE system (from 55% to 86%, 6 months post implementation). Guidelines were embedded into the ordering process, and practitioners could choose whether to read or ignore the guidelines. The number of upper-gastrointestinal orders also decreased from 10.6 per 1000 patients to 5.6 across a 5-year period. The authors did not report statistical significance testing.

Sanders and Miller¹⁸ used a similar decision-support feature to Carton et al¹⁶ where only practitioners who selected a patient's clinical context from a list generated in the CPOE system, as opposed to those entering free text, were provided with guidelines (n=551; 78%). The majority of practitioners ordered the examination recommended by the guidelines (n=328; 60% (p=0.001)). Practitioners who chose to over-ride the examination suggested by the guidelines (n=223; 40%) were required to give a reason for doing so. Reasons for not conforming to guidelines included: a patient-specific context (n=176; 79%); influence or verbal order by another person (n=34; 15%); and the recommended test had already been performed (n=18; 8%).

A time-series study, conducted by Neilson et al¹⁹ across 35 inpatient care wards, assessed the impact of two decision-support features on practitioners' ordering of portable CXRs. The first intervention, aimed at reducing test ordering beyond 72 h, prompted practitioners to elect whether to continue or discontinue requested examinations. This intervention did not result in a statistically significant change in ordering of portable CXRs. The second intervention, which limited orders to one portable CXR per fixed period of time, resulted in a reduction in the average number of daily orders by 18.6 orders per day (p=0.03). This decrease was attributed to restricting the ability of practitioners to place recurring orders, as opposed to allowing practitioners the choice of conforming to guidelines.

Portable CXRs were also assessed by Wang et al,²⁰ who implemented a three-part intervention, consisting of practitioner education, guidelines, and order templates, within one inpatient ward. Ordering of examinations was compared pre- and postimplementation in the intervention ward, and with a control ward. Practitioners within the intervention site were educated about the guidelines and were encouraged to use the order templates in the CPOE system. The use of order templates was not mandatory. The average daily test utilization of portable CXRs per ICU day decreased in the intervention ward from 0.97 to 0.88 (p=0.10), while in the control ward, test utilization increased significantly from 0.75 to 0.96 (p<0.001).

Two studies,^{24 25} from the Massachusetts General Hospital in the USA, examined the effect of decision support for ordering MRI, CT, and ultrasound examinations in the outpatient department. Based on the American College of Radiology Appropriateness Criteria,²⁹ the decision support provides an appropriateness score once a practitioner has chosen clinical indications and submitted a request. Sistrom et al²⁴ used the feature to perform a time-series analysis before, during, and after CPOE to reveal decreases of 2.75% (p<0.001), 1.2% (p=0.016), and 1.3% (p=0.001) respectively in the growth rate for CT, MRI, and ultrasound examinations. In 2010, Vartanians et al²⁵ reported on the impact of a modification to the system that required all examinations with low-yield scores to be personally authorized by a responsible clinician. This intervention resulted in a decrease of 5.43% (2106/38 801) to 1.92% (1261/65 765) (p<0.001) in low-yield CT, MR, and nuclear medicine examinations.

CPOE systems without decision support

Three studies investigated the impact of CPOE without decision support. Hwang et al²¹ reported on the effect of a CPOE system on the number of radiography orders placed per day for patients with liver or renal disease, or those undergoing simple mastectomy or gastrectomy. Data were collected in three intervals: prior to CPOE (73 patient records); 3 months post-CPOE (60 patient records); and 6 months post-CPOE (38 patient records). The findings indicated no significant changes in the average number of daily orders pre- (0.4; SD 0.6) and post-CPOE implementation; either at 3 months (0.4; SD 1.0) or at 6 months (0.1; SD 0.4) (p>0.05). Adam et al²² investigated the impact of a CPOE system on the number of CXRs ordered for adult patients with a primary complaint of chest pain during a 3-month period prior to CPOE and during the same 3-month period post-CPOE implementation in the subsequent year. Records for 150 randomly selected patients from each data-collection period were reviewed. Results demonstrated a significant increase in ordering volumes for CXRs, from 18 orders pre-CPOE to 135 orders post-CPOE (p<0.01). Alkasab et al²³ compared the level of information provided by ED practitioners for abdominal CT examinations before and after the introduction of CPOE. The authors reported positive findings, including that computerized requests were more likely to contain clinical questions (52.6% vs 34.8% (p<0.001)) and information on prior diagnoses (71.1% vs 51.1% (p=0.0027)).

Test ordering and turnaround times

Five studies^{15 22 26–28} assessed the impact of CPOE systems on test ordering and result turnaround times in medical imaging. One study²⁶ focused on the time required to complete order entry, while four studies^{15 22 27 28} evaluated the time taken from order entry to the time a result was available.

Schuster et al²⁶ compared ordering times using a commercial CPOE system menu and a customized CPOE system menu. A task force, made up of a radiologist and practitioner, modified the commercial CPOE system menu to reflect orthopedic practitioners' imaging practices. The commercial and customized CPOE menus were tested by five orthopedic practitioners who were required to complete two tasks: (1) order five separate examinations; and (2) order a myelogram and hip aspiration with laboratory, preparation, and scheduling. Task 1 resulted in an average time saving of 43 s, decreasing from an average of 3 min 23 s to 2 min 40 s. Task 2 completion times decreased by an average of 4 min 31 s, from an average of 6 min 8 s to 1 min 37 s. No statistical significance testing was reported by the authors.

Adam et al²² assessed the time interval from entering a CXR order to the availability of test results. Despite reporting a significant increase in the number of CXR orders placed during the study period, no change was found with turnaround time, which remained 80 min pre- and post-CPOE (p=0.49).

The time interval from entering plain radiography and CT orders to the availability of a result was assessed by Thompson et al.²⁷ The study focused on urgent or ‘STAT’ orders within an ICU, which included 29 orders pre-CPOE and 46 orders post-CPOE. Turnaround times decreased significantly from a median time of 96.5 min to 29.5 min (p<0.001) following the implementation of a commercial CPOE system.

Cordero et al²⁸ measured the turnaround time for the first CXR and abdominal x-ray ordered for very-low-birth-weight neonates within a neonatal ICU, where timeliness is considered critical. The turnaround time decreased significantly from an average of 42 min (SD 12 min) to 32 min (SD 16 min) (p<0.001). The greatest reduction was identified in the time interval from order entry to radiologist arrival, which decreased from 28 min (SD 13 min) to 17 min (SD 12 min) (p<0.001). The time from radiologist arrival to results availability remained relatively constant (14 min pre-CPOE (SD 18 min) compared with 15 min post-CPOE (SD 12 min)). Within this study site, portable radiographs were utilized, which may account for the shorter turnaround time intervals compared with those reported by other studies.

Turnaround times for CXR, abdominal x-ray, and abdominal ultrasound orders were assessed within one inpatient care ward by Mekhjian et al.¹⁵ The 11 orders evaluated pre-CPOE had a turnaround time of 7 h 37 min (mean or median not specified). This significantly decreased to 4 h 21 min for the 54 orders evaluated post-CPOE (p<0.05). Additionally, the authors outlined the work processes associated with test ordering and results turnaround pre- and post-CPOE. A number of tasks in the process were eliminated post-CPOE, such as delivering the examination request to the radiology department, while others became automated, such as scheduling the examination. Tasks related to conducting the examination procedure and results preparation remained the same pre- and post-CPOE.

Patient outcome indicators

Five studies assessed the impact of CPOE systems on one or more patient outcome indicators including length of stay (LOS),^{15 19–21 27} mortality,^{19 20 27} and hospital readmission rates.^{19 20} Two studies demonstrated significant changes in patient LOS. Hwang et al²¹ analyzed the average LOS for a cohort of patients with liver or renal disease, or those undergoing simple mastectomy or gastrectomy (n=171). Prior to the implementation of CPOE, the average LOS was 11.4 days (SD 7.4). This duration decreased significantly to 8.2 days (SD 6.0) 6 months after CPOE implementation (p=0.049). Mekhjian et al¹⁵ investigated casemix-adjusted LOS across a period of 10 to 12 months, pre- and post-CPOE implementation, within two hospital sites. Site A showed a significant decrease in average LOS (3.91 days pre-CPOE compared with 3.71 days post-CPOE (p=0.002)). Site B, however, showed no significant change in average LOS (3.68 days pre-CPOE compared with 3.61 days post-CPOE (p=0.356)). The three other studies assessing patient outcome indicators reported no significant change in LOS,^{19 20 27} mortalities,^{19 20 27} or readmission rates.^{19 20}

Technical features

Descriptions regarding the technical features of the CPOE systems utilized within the included studies were not extensive. Some studies provided brief descriptions of the functionality of the systems used, but descriptions of any existing CPOE integration with other systems, such as PACS, were rare. One study¹⁸ reported using a homegrown CPOE system; one study¹⁷ used a commercially available CPOE system; while five studies^{15 19 26–28} reported the use of commercial systems that had been ‘modified’ or ‘extensively modified.’ Seven^{16 20–25} did not report whether their system was commercial or homegrown. The majority (n=9) of the studies^{15–20 24 25 28} reported that the CPOE system utilized in their study included decision-support features, while the remaining studies did not specify whether decision support was available.

Discussion

This systematic review examined the impact of CPOE on medical-imaging services and patient-outcome indicators. The existing evidence suggests that significant imaging-department efficiency and effectiveness gains associated with CPOE may be achieved. Most of these benefits were associated with DSS promoting adherence to test ordering guidelines. This included electronic reminders displaying recommendations,^16–19 electronic advice about test orders that fell outside recommended boundaries,^{19 24 25} and decision support linked to the provision of education and ordering templates.²⁰ Studies which reported on the impact of CPOE without decision support were not as conclusive, with one study showing no change in the number of radiography orders²¹ and another showing a significant increase in CXR test volumes in an ED setting.²² Three of the five studies which investigated medical-imaging ordering and reporting times revealed statistically significant decreases in turnaround time.^{15 27 28} The body of evidence about the impact of CPOE on medical-imaging services remains small. Factors for this may include the continued work required to develop sophisticated evaluation models which take account of the ways in which such technologies impact upon multiple aspects of health-service delivery.³⁰ Clinical-decision support in this area is relatively underdeveloped, which may result from failures in both demand and supply. As such, the potential gains for dramatic effects on care delivery may yet to be measured or realized.

The evidence regarding patient outcome indicators such as LOS, mortalities, and hospital readmission rates mostly showed no change following CPOE, with a couple of exceptions that reported a decrease in LOS.^{15 21} The variable evidence in this area is indicative of the complexities involved in measuring and attributing patient outcomes to an intervention given the large number of variables (eg, case mix, clinical environment, and variations over time),¹⁵ which may influence these measures.³¹

Taken as a whole, this systematic review highlights a number of important findings which are relevant, not only to the practice of medical imaging and its contribution to patient care, but also to understanding the challenge of realizing the benefits of CPOE systems more generally.³² The review has identified a useful set of indicators applicable to the medical-imaging field (table 1), which, while not as comprehensive as those emanating from other ancillary departments such as pathology,³¹ nevertheless provide a foundation to build upon. A number of implications for future work in this area may be explicated from the findings of existing studies. These are discussed below under the headings: (a) Quality of the research evidence; (b) Relevance and generalizability of the research evidence; and (c) Decision-support systems.

Quality of the research evidence

The absence of any randomized controlled trials in the evidence base invites concern about the strength of the inferences that can be drawn from the results.³³ This does not imply a denigration of the existing evidence. Rather, it points to the need to critically appraise the findings through consideration of such things as the value of the description of the technical features of the system that is provided and the need to identify the limitations of the study and its design.²⁸ Judged by these criteria, we can conclude that there is a need for improvement to be made in the existing evidence base about the impact of CPOE on medical-imaging services. Many descriptions of the systems' functionality were scant. Some studies did not mention if the systems were or were not integrated, the degree of adoption across the hospital or even the underlying technology infrastructure,³³ thus making it difficult for interested parties to make comparisons and draw appropriate lessons. Future research in this area needs to take into account how CPOE systems integrate with other major systems as integral parts of their contribution to the whole hospital environment.³⁴

Relevance and generalizability of the research evidence

Six of the studies identified in this review were located in a critical care setting (ICU or ED) where the delivery of patient care requires ready access to patient information and procedures that optimize efficiency and enhance the decision-making process.^{35 36} While this provides important evidence in an area of major concern, it should be noted that results reported from critical care units may not be applicable to other hospital wards or locations. For instance, Khorasani³ highlights the outpatient area, which accounts for 60 to 70% of all imaging tests performed, as a key area for the utilization of CPOE and decision support. Only three^{17 24 25} of the medical-imaging studies identified in this systematic review dealt with outpatients, two of which were from the same hospital.^{24 25} The impact of CPOE in the outpatients area remains an area in need of greater research attention.³⁷

Although the studies in this systematic review were chosen because they involved an investigation of the use of CPOE for medical-imaging services, it was rare to find any consideration of the effects of the system on the medical-imaging department, its management or the way in which imaging services related to other departments.³⁸ Some of the key findings from the systematic review, particularly those related to efficiency of test ordering and turnaround times, are likely to have a major impact on medical-imaging service and the way it functions and communicates with clinical professionals.³⁹ It is noteworthy that the main focus of studies in this review tended to be on general radiology procedures. Yet, as Hara et al⁴⁰ have pointed out in relation to nuclear medicine, each modality has its own information and integration requirements related to the conditions necessary to set up their tests (eg, radiopharmaceuticals and contrast agents for nuclear medicine).

Decision-support systems

Half of the studies concentrated on decision-support features of CPOE providing evidence of their ability to increase adherence to evidence-based guidelines. The findings suggest that decision support for medical imaging can be effective in reducing unnecessary and inappropriate testing leading to reduced health costs and preventing unnecessary and potentially harmful radiation exposure for patients.^{24 25 41–43} However, the challenging task of high-quality decision-support implementation should not be underplayed. Recent research has demonstrated many of the difficulties and complexities involved with decision-support implementation, including the organizational challenges of reaching agreement among diverse professional groups¹⁷ and the problem of ensuring their usability by clinicians across a range of hospital settings.⁴⁴ According to Miller et al,⁴⁵ the implementation of decision support must involve multiple mechanisms including a detailed knowledge of the proposed intervention, its clinical significance, and the organizational workflows of those affected. Evidence about how implementation teams meet these challenges can be of enormous value to others. For instance, from this review, we were able to highlight: (a) the rich description provided by Wang et al²⁰ whose test utilization intervention in a coronary care unit involved a variety of strategies and stages of development including multidisciplinary teams, consensus methods, and practitioner education; (b) Mekhjian et al's¹⁵ identification of the changed work procedures caused by the CPOE which led to decreased turnaround times; (c) Vartanian et al's.²⁵ description of the alteration of the business logic in CPOE which resulted in a decrease in the rate of low-yield imaging examinations; and (d) Sander's and Miller's¹⁸ identification of the reasons why clinicians did not conform to test-ordering guidelines. This information is valuable because it helps to identify and explain the factors that contribute to the functioning and usability of CPOE systems.

Conclusion

In 2001, Khorasani³ highlighted the following potential benefits of CPOE: efficiency gains, improved test selection, and test appropriateness along with cost-savings. The evidence outlined in this review suggests that nearly a decade on, there has been very slow but positive progress. However, this systematic review also shows that the diversity and scope of the research evidence can be improved through increased attention to the circumstances and mechanisms that may lead to the success (or otherwise) of the system and its contribution to the enhancement of patient care delivery and outcomes.⁴⁶