Abstract
The purpose of this study was to objectively quantify the impact of implementing picture archiving and communication system-electronic medical record (PACS-EMR) integration on the time required to access data in the EMR and the frequency with which data are accessed by radiologists. Time to access a clinic note in the EMR was measured before and after integration with a stopwatch and compared by t test. An IRB-approved, HIPAA-compliant retrospective review of EMR access data from security audit logs was conducted for a 14-month period spanning the integration. Correlation of these data with report signatures identified the studies in which the radiologist accessed the EMR to obtain additional clinical data. Proportions of studies with EMR access were plotted and compared before and after integration using a chi-square test. Time to access the EMR decreased from 52 to 6 s (p < 0.001). Proportion of studies with EMR access increased from 36.7% (10,175/27,773) to 44.9% (10,843/24,153) after integration (p < 0.001). Integrating PACS and the EMR substantially decreases the time to access the EMR and is associated with a significant increase in the proportion of studies for which radiologists obtain additional clinical data.
Keywords: PACS, EMR, Context integration, Clinical history
Introduction
Radiologic interpretation and diagnosis is widely accepted to be more effective and accurate when performed with knowledge of and reference to the patient’s clinical context and history. Unfortunately, the clinical history provided along with the imaging requisition is often inadequate [1, 2], and thus optimal interpretation may require obtaining additional clinical data.
An effort by Franczak et al. [3] to quantify the importance of additional clinical data found that clinical data beyond what was available in the requisition would be very likely to have changed radiologic interpretation in 6.1% of a sample of 2000 emergency department head CTs.
One of the driving forces behind the widespread implementation of electronic medical records (EMRs) has been the desire to make clinical data more widely available to a patient’s entire healthcare team, including radiology. Although an EMR represents an enormously valuable source of data to radiologists, it also presents a challenge to the radiologist’s workflow. Radiologists typically spend the majority of their time working in a picture archiving and communication system (PACS). Since PACS have generally been developed and implemented separately from (and often prior to) EMRs, the EMR has historically been an additional system that exists outside of PACS. Thus, to obtain clinical data from the EMR, a radiologist has generally had to turn away from the PACS, launch a separate EMR application (possibly on a separate workstation), log in to the EMR, and then manually transcribe the medical record number of the patient of interest.
More recently, increasing levels of integration between PACS and EMRs have become available, often via a radiology information system (RIS) embedded in the EMR [4]. The most tightly integrated of these solutions offer patient context integration: when a study is opened for viewing in the PACS, that patient’s record is simultaneously opened in the EMR without need for further intervention on the part of the radiologist. This integration often comes at a cost, as it may require purchasing new software and maintaining more complex configurations. Having integration as a requirement may also limit the range of software options.
Radiology informatics experts have advocated for PACS-EMR integration for several years [5], but adoption has been slow. A recent survey of academic radiology departments found that less than half have PACS-EMR integration. [6] We believe that an important factor in this slow uptake is an absence of published data establishing a quantifiable benefit to integration. Such data are needed to support the business case for integration by estimating return on the investment required for implementation.
To better understand the value of integrating the PACS and EMR, we quantified whether implementing PACS-EMR patient context integration at our medical center decreased the time required to access data in the EMR and/or increased the proportion of imaging studies for which the radiologist obtained additional clinical data from the EMR.
Methods
EMR-PACS Integration
EMR-PACS integration was accomplished using file-based communication between the PACS client (Agfa IMPAX ver 6.5.1, Mortsel, Belgium) and the EMR client (Epic HyperSpace ver 2013, Verona, WI) on the PACS workstation.
EMR Access Time
To determine the direct impact of PACS-EMR integration on efficiency of radiologist access of the EMR, one of the authors used a stopwatch to measure the time for him to open the most recent clinic note for five arbitrarily selected patients both with and without integration. The timeout period of the non-integrated EMR is less than the typical time between radiologist EMR accesses, typically requiring the radiologist to log in to the EMR for each access. Therefore, time to log in to the EMR was included in the non-integrated times. Mean times were calculated for each group and compared with a t test.
EMR Access Proportion Data Collection
Data on EMR usage and study volume were collected from February 5, 2014 to April 1, 2015, bracketing the patient context-level integration of the PACS and EMR systems that occurred beginning August 4, 2014. At our institution, the majority of studies are read initially by a trainee alone and then later over-read by an attending sitting next to the trainee on the trainee’s PACS station under the trainee’s login. Since attendings almost always do their diagnostic reads on a station that is logged in as a trainee, data collection was limited to trainee accounts, which reflects both trainee and attending usage of the EMR. The data collection period spanned two academic years, so only trainees who were clinical (i.e., non-research year) residents during the entire data collection period were included.
EMR usage data were obtained from security audit logs. These data included the radiologist accessing the record, time stamp, patient medical record number (MRN), and portion of the record accessed. In the post-integration period, the EMR automatically presented a summary report (including items such as vitals, relevant recent labs, and lines and tubes) for each patient viewed in PACS. Since it was not possible to determine whether the radiologist consulted this summary report, entries representing the automatic opening of the summary report were filtered out of the logs and not counted. EMR access within the CT and MR protocoling module was also filtered out.
Study volume data were obtained for each trainee by searching for reports they approved in a report database (Illuminate ver 3.5, Softek, Prairie Village, KS). These data included radiologist, dictation time stamp, and patient MRN.
EMR Access Proportion Analysis
Both datasets were imported into a SQL database (SQLite ver 3.12.1, Hwaci, Charlotte, NC). A query was performed to generate a summary dataset representing the study volume (number of reports approved) and number of patient records accessed by each radiologist during each day of the sample period. The query filtered EMR accesses to include only patient record accesses where the radiologist had approved a report on that patient that day.
The summary dataset was imported into R (ver 3.2.1) [7] for plotting and statistical analysis. Proportion of studies for which the radiologist accessed the EMR was plotted for each day of the sample period. To better visualize trends, these data were smoothed by calculating the proportion for overlapping 28-day windows centered on each day of the sample period. The proportion of studies with EMR accesses was calculated pre- and post-PACS-EMR integration, and these proportions were compared using a chi-square test. Because there were serious stability issues with the integration in the period immediately following implementation, the first 60 days were excluded from analysis of the post-integration period. To assess whether differences in the pre- and post-integration EMR access proportions could be explained by increased training and experience level of the resident cohort as time passed, the total EMR access proportion was calculated for each residency class year, and these proportions were tested for trend using a Cochran-Armitage test. Linear regression of EMR access proportion against day of the study period was performed separately for the pre- and post-integration periods to identify chronologic trends in EMR access proportion. 0.05 was used as the threshold for statistical significance.
Results
Mean time to access the most recent clinic note in the EMR was 52 (95% CI 48–56) seconds without integration, decreasing to 6 (95% CI 5–8) seconds with integration (p < 0.001).
EMR usage and study volume data was obtained for 37 users spanning a period from 180 days prior to the implementation of integration to 240 days after. During this period, excluding the 60-day run-in after implementation, there were a total of 51,926 studies interpreted. Twenty-one thousand eighteen of these studies had associated radiologist-initiated EMR accesses (40.5%, 95% CI 40.1–40.9%).
A plot of the proportion of studies with EMR accesses (Fig. 1) demonstrates a significant increase in EMR usage after integration. The total proportion of studies with EMR access increased from 36.7% (10,175/27,773; 95% CI 36.1–37.2%) prior to integration to 44.9% (10,843/24,153; 95% CI 44.3–45.5%) afterward (p < 0.001).
Fig. 1.
Proportion of exams for which the radiologist accessed additional clinical information from the EMR, before and after implementation of context integration. Plot represents proportions for overlapping 28-day periods; gray lines represent overall proportions for pre- and post-integration periods
Linear regression of the pre- and post-integration data demonstrated a significant trend for increasing use of the EMR with time in the post-integration period: the fitted increase was 0.05 percentage points per day (p < 0.001). There was no significant trend over time in the pre-integration period (p = 0.79).
The proportion of studies for which the EMR was accessed was similar across the three residency classes from most junior to most senior: 43.2% (95% CI 42.6–43.9%), 41.2% (95% CI 40.6–41.8%), and 43.1% (95% CI 42.4–43.8%). No significant trend was detected by Cochran-Armitage test (p = 0.39), suggesting that the increases seen in EMR usage could not be explained by increasing training level of residents with the passage of time.
Discussion
This study showed that patient context integration of the PACS with the EMR significantly reduced the time for a radiologist to access clinical data in the EMR, and that this increased efficiency was associated with a substantial increase in EMR utilization by radiologists. EMR usage continued to increase throughout the post-integration period.
Although the average of 46 s saved per EMR access may at first glance seem like a relatively trivial period of time, short delays for frequent tasks become large in aggregate. For instance, based on the number of EMR accesses in the pre-integration period, the time saved by this 46-s reduction amounts to an average of more than a working day (8.26 h) per radiologist per year. This increase in efficiency alone supports the importance of PACS-EMR integration. Errors in interpretation were not assessed as part of this study, but it has been shown that interruptions of complex cognitive tasks increase errors [8], so there may also be some quality benefit to reducing the cognitive task switching and extent of the interruption caused by interacting with the EMR.
The increased efficiency afforded by PACS-EMR integration was also associated with a substantial increase in EMR usage. The 8.2 percentage point increase in proportion of studies for which the EMR was consulted represents a 22% increase in the likelihood that a radiologist would seek additional clinical data for a given study. The increase in EMR usage accounted for the 37 radiologists obtaining additional clinical data on approximately 2500 additional cases during the 180-day post-integration period that they otherwise would not have, based on pre-integration EMR usage proportions. Although this study did not directly assess quality of interpretation, if Franczak et al.’s [3] estimate of additional clinical data altering the radiological interpretation in 6.1% of cases is used, then the increase in EMR usage seen in this investigation would be expected to result in an alternate, more accurate diagnosis in an average of 8.1 cases per radiologist per year.
An interesting aspect of the behavioral change associated with the integration is that the effect increased as time passed. The best-fit line to proportion of studies with EMR access demonstrated a 9.4 percentage point increase over the 180-day post-integration sample period, with approximately 50% of studies associated with EMR access by the end of the data collection period. This may be because the experience of efficiently accessing useful information in the EMR creates a positive feedback loop that makes radiologists more likely to access the EMR in future cases.
This investigation had multiple strengths. It was based on objective data collected on all studies read by multiple radiologists over a period of approximately 1 year. The data collection process was transparent to the radiologists involved, and thus would not be expected to bias the frequency with which they accessed the EMR. EMR accesses were matched with studies read using timestamps and patient medical record numbers. This matching process allowed calculation of the proportion of studies for which the EMR was accessed, which eliminates the biasing effect of fluctuating study volume that may be present in the raw rate of EMR accesses per day. Additionally, the matching process excludes EMR accesses unrelated to interpretation of clinical studies (e.g., research or clinical follow-up). Finally, this quantitative analysis of the benefits of PACS-EMR integration provides a foundation for cost/benefit calculation to support decisions on whether to implement PACS-EMR integration.
There were also several limitations to the approach taken here. There was no way to tell whether or not the radiologist viewed or obtained additional clinical data from the automatically opened summary report in the post-integration period. To be conservative, accesses of the summary report were not counted. It is likely that there were some cases in which the radiologist obtained additional clinical data from the summary report without accessing other parts of the EMR; to the extent that this occurred, the true effect of the integration would be underestimated. Because the integration had to be implemented on a medical-center-wide basis, this study had to employ a historical control. Although this leaves open the possibility that other factors besides the implementation of integration affected EMR usage, there were no other significant changes to the clinical systems used by radiologists during this period, and the timing of the change in proportion of EMR usage was well correlated with the timing of the integration. The investigation was limited to a single academic medical center. Measurement of mean time to access a clinic note may have been affected by radiologist awareness of being timed, though given that the mean times varied by a factor of almost 9, this seems unlikely to have substantially affected results. Medical centers with faster login and access times for non-integrated EMRs might see less benefit to integration than was realized in this study. Additionally, the EMR access patterns of academic radiologists and trainees may not be representative of private practice radiologists, who read the majority of radiology studies in America. Nevertheless, while the magnitude of the results would likely vary in different practice environments, it seems reasonable to expect that the overall effect of increased EMR usage as a result of integration would be consistent.
The results presented here illustrate that patient context integration between the PACS and the EMR is not merely a matter of convenience. There is substantial aggregate time savings provided by integration. More importantly, these time savings organically drive changes in radiologist practice projected to change diagnosis in 1 out of every 200 cases read. We believe that in light of these results, PACS-EMR patient context integration should be considered an essential component of every PACS environment.
Compliance with Ethical Standards
This HIPAA-compliant retrospective study was approved by the local Institutional Review Board.
Conflict of Interest
The authors declare that they have no conflict of interest.
References
- 1.Gunderman RB, Phillips MD, Cohen MD. Improving clinical histories on radiology requisitions. Acad Radiol. 2001;8(4):299–303. doi: 10.1016/S1076-6332(03)80498-1. [DOI] [PubMed] [Google Scholar]
- 2.Obara P, Sevenster M, Travis A, Qian Y, Westin C, Chang PJ. Evaluating the referring physician’s clinical history and indication as a means for communicating chronic conditions that are pertinent at the point of radiologic interpretation. J Digit Imaging. 2015;28(3):272–282. doi: 10.1007/s10278-014-9751-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Franczak Michael J., Klein Madeline, Raslau Flavius, Bergholte Jo, Mark Leighton P., Ulmer John L. In Emergency Departments, Radiologists’ Access To EHRs May Influence Interpretations And Medical Management. Health Affairs. 2014;33(5):800–806. doi: 10.1377/hlthaff.2013.0171. [DOI] [PubMed] [Google Scholar]
- 4.Geeslin MG, Gaskin CM. Electronic health record-driven workflow for diagnostic radiologists. J Am Coll Radiol. 2016;13(1):45–53. doi: 10.1016/j.jacr.2015.08.008. [DOI] [PubMed] [Google Scholar]
- 5.Khorasani R. Can you efficiently incorporate patient-specific electronic medical record data into radiology workflow? J Am Coll Radiol. 2012;9(12):862–863. doi: 10.1016/j.jacr.2012.09.019. [DOI] [PubMed] [Google Scholar]
- 6.Forsberg D, Rosipko B, Sunshine JL, Ros PR. State of integration between PACS and other IT systems: A national survey of academic radiology departments. J Am Coll Radiol. 2016;13:812–818.e2. doi: 10.1016/j.jacr.2016.01.018. [DOI] [PubMed] [Google Scholar]
- 7.R Core Team. R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing; 2015.
- 8.Einstein GO, McDaniel MA, Williford CL, Pagan JL, Dismukes RK. Forgetting of intentions in demanding situations is rapid. J Exp Psychol Appl. 2003;9(3):147–162. doi: 10.1037/1076-898X.9.3.147. [DOI] [PubMed] [Google Scholar]