The impact of transitioning from availability of outside records within electronic health records to integration of local and outside records within electronic health records

Abstract

Objective

While there has been a substantial increase in health information exchange, levels of outside records use by frontline providers are low. We assessed whether integration between outside data and local data results in increased viewing of outside records, overall and by encounter, provider, and patient type.

Materials and Methods

Using data from UCSF Health, we measured change in outside record views after integrating the list of local (UCSF) and outside (other health systems on Epic [Epic Systems, Verona, WI]) encounters on the Chart Review tab. Previously, providers only viewed records from outside encounters on a separate tab. We used an interrupted time series design (with outside record viewing event counts aggregated to the week level) to measure changes in the level and trend over a 1-year period.

Results

There was a large increase in the level of outside record views of 22 920 per week (P < .001). The change in trend went from a weekly increase of 116 (P < .05) to a decrease of 402 (P = .08), reflecting a small effect decay. There were increases in the level of views for all provider and encounter types: attendings (n = 3675), residents (n = 3277), and nurses (n = 914); and inpatient (n = 1676), emergency (n = 487), and outpatient (n = 7228) (P < .001 for all). Results persisted when adjusted for total encounter volume.

Discussion

While outside records were readily available before the encounter integration, the simple step of clicking on a separate tab appears to have depressed use.

Conclusions

User interface designs that comingle local and outside data result in higher levels of viewing and should be more broadly pursued.

INTRODUCTION

BACKGROUND and SIGNIFICANCE

Substantial effort over the past decade has resulted in greater electronic availability of outside records for frontline clinicians.¹ However, most approaches to enabling such availability require clinicians to go outside of their local electronic health record (EHR) (eg, by logging in to a community longitudinal record) or to go to a separate tab in their local EHR that houses outside records.² Under either approach, clinician workflow is interrupted and cognitive effort is required to marry local EHR data with data available in outside records. This could explain why recent evidence reviews have found low levels of use of electronically exchanged information by frontline providers.^3,4 Newer approaches seek to comingle local EHR data with data from outside records, such that clinicians stay within their workflow and are presented with an integrated and logically organized (eg, chronological) list of encounters, lab results, problems, medications, etc. Given the substantial work required to achieve this final step of integrating data from local and outside records, it is critical to assess the impact of such integration.

It is possible that easier access to outside records could have a differential impact by provider type, or by encounter type. For example, residents may be more likely than attending providers are to view outside records because residents are often more likely to perform detailed chart review for patients.⁵ In terms of encounter type, the value of outside records is substantially higher in the emergency department setting, where providers are more likely to care for patients who have received care elsewhere and there are time pressures to quickly diagnose and execute appropriate treatment.^6–9 Finally, it is also possible that easier access to outside records could tip providers towards viewing outside records for patients who are healthier (ie, in which the value of doing so is more marginal). It is therefore valuable to assess differential impact not only by provider and encounter type, but also by key patient characteristics.

OBJECTIVE

We therefore undertook the first-ever study to specifically investigate whether the volume of outside record viewing increases after such integration as well as characterize for which types of providers, encounters, and patients any increases accrue. We took advantage of a natural experiment in which an EHR upgrade on July 11, 2018, resulted in a change to the user interface in which outside records were newly available as part of a comingled, chronologically ordered list of encounters on the Chart Review (CR) tab that displayed local (UCSF Health) and outside (other health systems using Epic [Epic Systems, Verona, WI]) encounters. Both before and after the change, providers could access outside encounters and associated records on a separate tab within the EHR (Care Everywhere [CE]). Our study setting therefore allowed us to isolate the effect of the user interface design change, rather than a change in the breadth or depth of available outside records. Our study expands on prior work examining the benefits of clinical data integration in other contexts, such as the integration of ambulatory and inpatient EHRs within single institutions.¹⁰

MATERIALS AND METHODS

Setting

UCSF Health is a large academic medical center with more than 3000 clinicians across 2 hospitals and 175 ambulatory practices. The health system implemented Epic in 2012, and in mid-2018 as part of upgrading to the 2017 version, we turned on a new feature that created a single, integrated list of local (from UCSF Health) and external (from other Epic-based health systems in California automatically as well as for individual patients linked manually) encounters presented in the CR tab (see Supplementary Figure A1 for a screenshot). Before this switch, outside records were only available through the CE tab, a section of the EHR exclusive to outside records (see Supplementary Figure A2 for a screenshot).¹¹ The new feature was supplementary, such that there was no change to user ability to view outside records via the CE tab.

Data and sample construction

To study the impact of the user interface change on patterns of frontline clinician viewing of outside records, we studied a 12-month period of CE events (January–December 2018) centered on the new feature go-live date. We constructed 2 analytic datasets (Supplementary Figure A3, left-hand side). Our first analytic dataset was at the outside record viewing event level and captured all instances in which the audit log indicated that an outside record had been viewed, along with a timestamp and a user ID (Supplementary Figure A3). To trigger an outside record viewing event, the user has to actively click on the document and have it appear in the sidebar or a report viewer window; simply viewing the list of encounters, either in the CE tab or in CR, would not trigger this event. We then limited analysis to events performed by clinical users and mapped clinical user IDs to 4 role categories (attending physician, resident physician, nurse, and other). As outside record viewing events were not explicitly linked to patient encounters, we created this link by first capturing all audit log events in the 2 hours before the outside record viewing event and then assigning the encounter of the temporally closest event. Via this approach, 1.30 million of 1.56 million outside record views could be linked to an encounter; those remaining did not have an encounter-linked audit log event within the 2-hour window.

Finally, to more precisely establish that any observed changes resulted from the change in the user interface, we used the granular nature of the audit log to assess whether the prior log entries indicated that the user was in the CE tab (ie, used the “old” workflow) or was in the CR tab (ie, used the “new” workflow) before the outside record viewing event. Specifically, we linked each outside record viewing event to the temporally closest prior event that was not an outside record viewing event up to 2 hours from the outside record viewing event. We labeled this event the entry point event. Based on the screens associated with these entry point events, we manually mapped entry point event types to the CE tab, the CR tab, or another tab based on metric descriptions and spot evaluation to confirm that the events corresponded to the expected screens. Supplementary Table A1 lists the mapping of entry point events associated with the user being in the CE tab or in the CR tab. In a small number of instances, we observed the CR entry point before the new feature go-live date; we identified these cases as occurring when the user was in CR (and would not have seen the outside encounters there but knew about them via other means) and then accessed the outside record via another (non CE) channel to which they navigated from CR.

Our second analytic dataset was at the encounter level to allow us to examine patient characteristics (Supplementary Figure A3, right-hand side). The inclusion criterion was any encounter with at least 1 outside record view by a clinical user. Outside record viewing events do not have linked encounter identifiers. As a result, we associated any given outside record viewing event with the encounter of the nearest preceding audit log event that had an encounter identifier and shared the same user and patient. For these encounters, we pulled in the following patient characteristics: gender, age, race, insurance class, and number of UCSF Health encounters in the 12 months before the encounter with the outside record viewing. We also included all the outside record event views and viewer types to allow us to calculate encounter-level summary statistics of view volume and viewer characteristics, including total distinct user counts in the study period with at least 1 outside record view, total outside record views in the study period by all users, average distinct viewers per encounter, and average number of outside record views per encounter. We extended this dataset back to August 2017 in order to capture the same calendar months as the postimplementation period in a robustness test described subsequently.

Analytic approach

Using 52 weeks of data (calendar year 2018), we used our first analytic dataset (at the view event level) to assess whether there were changes in the level (1-time) and trend (pre-implementation vs post-implementation linear slope) of weekly outside record views, using the week containing July 11 (week 28) as the interruption date. Specifically, we conducted an interrupted time series analysis (ITSA)¹² for the overall number of outside record views, as well as for outside record views by user type (attending, resident, nurse) and by encounter type (inpatient, outpatient, emergency). We repeated this analysis using our measure of whether an outside record viewing event was preceded by being in the CR tab or the CE tab because we hypothesized that most or all of any increases would come from the new CR pathway.

Because we would expect the number of outside record views to increase simply as a result of greater encounter volume, and it is possible that encounter volume could be a time-varying confounder (ie, increase in parallel with the timing of the user interface design change), we adjusted for the weekly encounter volume. Specifically, we created a rate by dividing the weekly number of record views by the weekly encounter volume for each type of encounter and then repeated our ITSA on these weekly rate measures.

Finally, we used the encounter-level dataset to assess any differences in encounters with outside record views (based on their associated viewer and patient characteristics) before and after the change. We therefore constructed our primary sample of encounters with outside records views in the preimplementation period (January–May 2018) and encounters with outside record views in the postimplementation period (August–December 2018). These were both 5-month periods and factored in a washout period in the month before and during the upgrade. We created an alternative preimplementation period containing encounters with outside record views for August–December 2017 (the same calendar months as the postimplementation period) to address concerns about seasonality impacting any differences observed in the primary comparison. We then ran summary statistics and chi-square tests to assess whether total and per-encounter viewer and patient characteristics differed for encounters in the preimplementation vs postimplementation periods for both our primary and alternative samples. Analyses were performed in Stata 16.0 (StataCorp, College Station, TX) and R 3.5.0 (R Foundation for Statistical Computing, Vienna, Austria).

RESULTS

Changes in volume of outside record viewing: ITSA

In our ITSA of total weekly volume of outside record views, there was a statistically significant increase in level of 22 920 views (P < .001). Before the change in level, weekly views had been increasing by 116 (P < .05), and in the postimplementation period, weekly views decreased by 402 per week, but this was not statistically significant (P = .08).

As shown in Figure 1, when we performed the ITSA for the count of outside record viewing events preceded by an event indicating the user was in the CR tab, we similarly saw a large increase in level of 16 600 (P < .001). For the count of outside record viewing events preceded by an event indicating the user was in the CE tab, we saw a small but still significant increase in level of 2773 (P = .013), suggesting a small spillover effect from the new feature onto views initiated from within the CE tab.

Figure 1.

Number of outside record views by entry point (existing vs new) before and after the switch to integrated local or outside records.

For each user type and for each encounter type, we found a significant increase in level of viewing volume, revealing that increases were not limited to specific user or encounter types. Specifically, as shown in Figure 2, all provider types experienced significant 1-time increases in the level of views: attendings (n = 3675), residents (n = 3277), and nurses (n = 914) (P < .001 for all). There was a small effect decay (statistically significant reduction in trend in the postimplementation period compared with the preimplementation period) for residents (−67 per week; P < .05) and nurses (−31 per week; P < .01) but not for attendings (−57 per week; P = .16).

Figure 2.

Number of outside record views before and after the switch to integrated local or outside records, by provider type.

Similarly, as shown in Figure 3, there were significant 1-time level increases for all 3 types of encounters: inpatient (n = 1676), emergency (n = 487), and outpatient (n = 7228) (P < .001 for all). There were again small effect decays for all 3 settings: inpatient (−51 per week; P < .05), emergency (−7 per week; P < .05), and outpatient (−183 per week; P < .05). When we repeated these analyses adjusted for encounter volume (ie, the number of views divided by the number of encounters), the pattern of results for these rate measures was identical. There were significant 1-time level increases for all 3 types of encounters: inpatient (2.16 views/encounter), emergency (0.38 views/encounter), and outpatient (0.22 views/encounter) (P < .001 for all). There were small effect decays for all 3 settings: inpatient (−0.07 per week), emergency (−0.01 per week), and outpatient (−0.005 per week) (P < .01 for all).

Figure 3.

Number of outside record views before and after the switch to integrated local or outside records, by encounter type.

Changes in patient and viewer characteristics for encounters with outside record views: Preimplementation vs post implementation analyses

Table 1 presents the characteristics of encounters with outside records views in the preimplementation vs postimplementation periods. Across encounter types, there were consistent increases in the total number of outside record viewing events (consistent with our ITSA results) as well as the total number of unique viewers (Table 1). Comparing preimplementation and postimplementation periods, the total number of outside record views increased by 30.8%, 44.9%, and 91.7% for inpatient, outpatient, and emergency encounter types, respectively. This was accompanied by an overall 27.0%, 15.7%, and 46.5% increase in the total number of distinct users performing 1 or more outside record views for inpatient, outpatient, and emergency encounter types, respectively. When we examined differences by user type, we found that nurses represented the user group with the greatest increase in number of distinct users—increases of 46.2%, 43.9%, and 72.1% for inpatient, outpatient, and emergency encounters, respectively. For emergency encounters, attendings and residents also had large increases in the number of distinct users—increasing by 52.7% and 24.9%, respectively.

Table 1.

Viewer characteristics before and after the switch to integrated local or outside records (January–May 2018 vs August–December 2018)

	Inpatient			Outpatient			Emergency
	Pre	Post	% Change	Pre	Post	% Change	Pre	Post	% Change
Total encounters	7601	8208	7.0	38 312	53 083	38.6	2630	4335	64.8
Total outside record view events	75 669	98 957	30.8	124 696	180 688	44.9	9617	18 433	91.7
Total distinct users with outside record view events	3780	4800	27.0	4312	4989	15.7	995	1458	46.5
Attending	626	712	13.7	989	1079	8.0	186	284	52.7
Resident	1017	1087	6.9	1015	1017	0.2	386	482	24.9
Nurse	798	1167	46.2	312	449	43.9	122	210	72.1
Other	1339	1834	37.0	1986	2444	23.1	301	482	60.1
Per-encounter unique user statistics, mean
Total unique users	3.1	3.8c	18.8	1.2	1.2c	3.0	1.5	1.7c	15.2
Unique attendings	0.47	0.52c	9.7	0.40	0.41b	2.4	0.31	0.30	−3.6
Unique residents	1.1	1.3c	18.7	0.19	0.21c	8.9	0.58	0.65c	10.4
Unique nurses	0.36	0.48c	25.6	0.045	0.049b	7.8	0.26	0.44c	40.6
Unique other	1.2	1.5c	20.0	0.55	0.55	0.85	0.32	0.35a	8.3
Per-encounter view events, mean
Total view events	9.96	12.06c	17.4	3.25	3.40c	4.4	3.66	4.25c	14.0
Attending view events	1.53	1.72b	11.0	1.16	1.24c	6.9	0.71	0.68	−5.2
Resident view events	4.02	4.97c	19.0	0.63	0.70c	10.6	1.68	1.84	8.6
Nurse view events	0.91	1.22c	25.6	0.11	0.12	1.3	0.52	0.92c	43.2
Other view events	3.49	4.15c	15.8	1.35	1.34	−0.90	0.74	0.82	9.2

^a P < .05.

^b P < .01.

^c P < .001.

On a per-encounter basis, overall and for almost all user types, there were statistically significant increases in the number of unique users with outside record views and in the number of view events (Table 1). Overall, total unique users increased by 18.8% for inpatient encounters, 3.0% for outpatient encounters, and 15.2% for emergency encounters. The number of view events per encounter increased by 17.4%, 4.4%, and 14% for inpatient, outpatient, and emergency encounters, respectively. Results were very similar when we substituted the alternative preimplementation period (August–December 2017), as shown in Supplementary Table A2.

Table 2 presents the patient characteristics for encounters with outside record views in the preimplementation and postimplementation periods. Gender, age, race, insurance class, and prior utilization were generally similar, though small-magnitude, statistically significant differences were detected in age for the outpatient setting (50.3 years vs 49.6 years; P < .001), insurance class for both inpatient and outpatient settings (slightly higher percent commercially insured and slightly lower percent with Medicare in the postimplementation period), and prior inpatient utilization in the outpatient setting (0.18 vs 0.19 encounters in the prior 12 months; P = .025). Results were similar when we substituted the alternative preimplementation period (August–December 2017), as shown in Supplementary Table A3. We did observe a small-magnitude difference in race (slightly lower percent Asian; P = .033) and some newly significant differences in prior utilization.

Table 2.

Patient characteristics before and after the switch to integrated local or outside records

	Inpatient			Outpatient			Emergency
	Pre	Post	P value	Pre	Post	P value	Pre	Post	P value
Female, %	51.8	51.1	.322	56.7	56.8	.827	48.8	49.65	.468
Mean age, y	51.8	51.4	.305	50.3	49.6	<.001a	46.9	45.9	.072
Race, %
African American	10	9	.184	7	7	.276	19	20	.968
Asian	14	15		14	14		14	13
Caucasian	51	52		55	56		45	44
Other/declined	26	25		24	23		23	23
Insurance class, %
Commercial	33	34	.0381a	48	49	.0012a	30	32	.264
Medicaid	24	25		15	15		34	34
Medicare	37	36		31	30		27	26
Other	0	0		0	0		0	0
Self-pay	3	2		3	3		6	6
Prior 12-mo utilization, mean
Inpatient encounters	1.71	1.74	.357	0.18	0.19	.025a	0.36	0.33	.198
Outpatient encounters	8.99	8.88	.592	8.70	8.79	.215	4.08	3.99	.708
Emergency encounters	1.29	1.20	.051	0.29	0.29	.586	5.04	4.80	.357

^a P < .05.

DISCUSSION

We conducted a novel study that found that a user interface change that presented integrated local and outside records in chronological order in the CR tab resulted in a large increase in the level of outside record viewing. Increases were spread across user and encounter types, indicating the widespread impact of this change on ensuring that frontline clinicians have ready access to outside records. Given the substantial federal investment in interoperability, our results indicate that the benefits will not be maximized unless we specifically focus attention on last-mile issues of integrating external data, rather than simply ensuring it is electronically available.

We were not surprised to find that a relatively small change in user interface design had such a large impact on behavior, given the growing body of literature on EHR nudges. Prior evidence has shown that changing default drug dosing, default selection of generic drugs, and shifting from opt-out to opt-in for referrals have all resulted in large changes to provider and clinical process behaviors.¹³ Our study further bolsters this conclusion and the evidence base for designing choice architecture in ways that make the right thing to do the easy thing to do.

Our study extends this literature by examining variation in impact by user and encounter types. While there were increases in outside record viewing from all categories, we were interested to see larger gains for nurses—both in terms of the number of unique nurses viewing outside records and the number of outside record viewing events by nurses per encounter. We hypothesize that this may be because review of past medical history is a foundational activity for attendings and residents, such that these groups more consistently sought out outside records as compared with nurses. This is supported by the higher baseline levels of outside record viewing events among these 2 groups as compared with nurses. As such, the more visible availability of outside records may have prompted nurses to view them when they would not otherwise have had a compelling motivation to go outside their workflow to do so.

Less surprising were the larger relative increases in viewing for inpatient and emergency encounters as compared with outpatient encounters; these 2 settings focus on patients with more urgent and higher-acuity needs, such that the value of outside records is greater, and we would expect to see greater consumption of outside records once they are made more easily accessible. We were encouraged to see that there were not meaningful differences by patient characteristics, and where there were small differences, they tended to suggest increased viewing for healthier patients—younger, commercially insured, less prior utilization. This could be explained by the fact that users always took the time to find outside records for patients when they were critical (ie, for sicker patients), even if doing so required extra workflow steps. After the change, when it was quicker and easier to view outside records, they may have been more likely to view outside records for patients in which the additional information was of lower marginal value (ie, for slightly less sick patients).

Our study has important policy implications. Federal interoperability policy has largely focused on ensuring that information can be sent and received to support care transitions, and has not had any programmatic requirements under the Meaningful Use or Promoting Interoperability programs around data integration or use. ONC does report data on the level of integration and use of outside data from the American Heart Association IT Supplement survey, and in light of our findings, it is worrisome that the levels of these 2 interoperability dimensions are low. In the most recent data, 53% of hospitals indicated that they integrate data and 23% reported that they often used data from outside sources.¹⁴ Efforts under 21st Century Cures may partially help promote improved data integration via the real-world performance assessments in the interoperability domain under the EHR Reporting Program. Our findings suggest that integration of local and outside records, perhaps by domain (eg, encounters, test results, medications, problems), should be a focal concept in the reporting approach, alongside efforts to promote interoperability and integration more broadly, including for patient-mediated access that was also included in the legislation. In the interim, it is incumbent on health systems and EHR vendors to push for integration of outside data into clinician workflows in order to mobilize data to improve patient care.

Limitations

The generalizability of our findings may be limited by a single-site, single EHR vendor setting. However, the concept of a single list of encounters presented in chronological order is not highly specific to our organization or EHR; therefore, we believe that our overall conclusions are generalizable. Nonetheless, the magnitude of effect sizes may differ and, in particular, be lower in settings that are not referral-heavy, academic medical centers. Our ITSA results also feature an effect decay—a small-magnitude but statistically significant weekly decline in outside record views after the large, 1-time level increase. We were not able to observe a sufficiently long postimplementation period to know if levels will slowly creep back down to baseline or level off at a higher level than in the preimplementation period. More broadly, the effect decay suggests that not all outside record views were valuable, and there is need for future work to assess what features of outside records make them useful and under what circumstances. Finally, there were possible inaccuracies in linking outside record viewing events to encounters, as we had to develop logic that assumed the relevant encounter was the temporally preceding one. However, this would not have introduced systematic bias into the results.

CONCLUSION

We took advantage of a natural experiment in which an EHR upgrade resulted in a change to the user interface in which outside records were newly available as part of a comingled, chronologically ordered list of encounters on the CR tab while the same records were historically and continued to be available on a stand-alone tab. The change resulted in large-magnitude increases in the level of outside record views, suggesting that even small workflow barriers (clicking onto a separate tab) may be impeding optimal use of outside records. Given the substantial, ongoing investments in improving interoperability, our results point to the need to invest in the last-mile efforts of making data readily available within clinical workflows.

AUTHOR CONTRIBUTIONS

Both authors contributed to the design, analyses, and writing of the article. MDW is responsible for the accuracy of the data. JA-M is responsible for the accuracy of the analyses and the final contents of the manuscript.

CONFLICT OF INTEREST STATEMENT

None declared.