Use of Simulated Physician Handoffs to Study Cross-cover Chart Biopsy in the Electronic Medical Record

Abstract

Clinical handoffs involve the rapid transfer of patient information from one provider or team to another, through activities which may introduce errors and affect care delivery. “Cross-coverage” requires quickly familiarizing oneself with unfamiliar patients whose management plans were established by another provider or team. Through this work, we describe physicians’ information seeking approaches within an electronic medical record (EMR) during physician handoff and chart biopsy at a major academic medical center. We conducted simulated handoff sessions and interviews with 21 physicians using standardized patient cases and we analyzed screen capture data, and video and audio recordings of interactions with the EMR and handoff printouts. We found highly variable navigation of the EMR but greater similarity in physicians’ EMR navigation behavior when the chart review was prompted by simulated interruptions. Understanding how physicians seek and assimilate patient data can inform handoff tool design and suggest strategies for explicitly supporting EMR chart biopsies.

Introduction

Hospitalized patients typically require care from multiple teams of providers whose work shifts align to provide continuous 24-hour care. To manage this process, providers conduct clinical handoffs that involve the transfer of information and accountability from one provider or team to another across shifts¹. At academic hospitals, rules governing consecutive resident physician work hours have increased the frequency of handoffs to at least twice daily². Based on recent length-of-stay data, physicians hand off the care of the average U.S. inpatient at least nine times per admission, introducing the potential for harmful communication errors at each transition³. Communication failure has been shown to be the key preventable contributing factor for adverse events, reported in 60% of sentinel events^{4, 5}. A growing body of literature documents widespread efforts to improve the quality of handoffs through standardization of care transitions^{1, 6}. However, handoff norms and practices often develop informally, and physicians learn them from peers or attendings⁷. As a result, handoff processes may vary widely, and formal training in handoffs is uncommon⁸.

Evening handoffs are unique as they often involve “cross-covering.” A cross-covering physician is responsible for a patient when the primary team is absent from the hospital⁹. These physicians typically manage a larger number of patients at night, most of whom they have never met or examined. Clinical encounters allow physicians to collect large amounts of medical, social and administrative data and organize those data into a mental model of the patient^{10, 11}. These mental models allow them to integrate new information and make inferences about changes in a patient’s condition, based on prior knowledge of the patient’s situation. To maintain continuity of care, cross-covering physicians must build accurate mental models of patients whose care they are assuming. They do so through the information exchanged during the handoff and through a quick and efficient chart review. This process of reviewing the chart to create a mental snapshot of the patient has been called “chart biopsy” by some authors¹².

The growing use of electronic medical records (EMRs) has facilitated the development of electronic tools to support handoffs. These electronic tools combine patient data abstracted from the EMR and free-text boxes allowing for a narrative record. Pre-populated data typically include demographics, room number, vital signs, laboratory data, and current medications. Free text boxes are often used for an informal summary of the patient’s history of present illness, problem list, treatment plans, and to-do list for the oncoming provider. Handoff tools display this patient information in a structured format,¹³ and physicians often print it out to use as a reference and they annotate it during their shift. The use of such handoff tools data may aid the chart biopsy process.

Although workflow studies show the benefits of using a handoff tool integrated with an EMR,¹⁴ most recommendations for the content of verbal physician handoffs and the manner in which such content is acquired and communicated are not well supported by evidence^{15, 16}. Observational studies, surveys and recall studies have generated suggestions for improving handoff quality, including face-to-face communication, interactive questioning, and standardization through mnemonics and checklists^{17, 18}. However, these studies do not describe the process of physician data collection or provide an understanding of how physicians use the EMR when cross-covering to develop mental models of patients. While some patterns of EMR navigation patterns have been described,¹⁹ repeated switching between EMR and paper resources, and between structured and less structured data suggest that the level of experience may affect the way and effectiveness of data gathered to develop these mental models²⁰.

In this study, we studied physicians’ reasoning and use of the EMR after receiving the handoff of four simulated, standardized clinical scenarios to uncover opportunities for redesign of electronic clinical tools to better support data collection (or chart biopsy). Specifically, our analysis for this paper seeks to accomplish the following:

1. Describe the chart review process through measurement of user time spent within various EMR sections and sequence of pathways between sections. We hypothesized that the habitual order of review of EMR sections would trump the order motivated by clinical reasoning (for example early diagnostic hypothesis generation which drives data collection).

2. Describe the effect of a clinical cue on the chart review process. We simulated text pages from nursing staff to create interruptive cues, which are a common form of work interruption. We hypothesized that the chart review pathway for a patient would be more similar among participants responding to a cue, than during “naïve,” unprompted chart reviews.

3. Evaluate the impact of physician experience on chart biopsy behavior. We hypothesized that attendings would follow a shorter, more purposeful pathway compared with residents, as studies show that they tend to have a more holistic, less detailed approach²¹. We also predicted that attending pathways would vary extensively based on prior clinical reasoning studies showing small overlap of pertinent features used by experts for problem solving²².

Methods

Case preparation:

From a population of patients managed by two investigators (JB, AW) between Jan 22, 2011 and Feb 21, 2012 at two academic medical centers, investigators (KB, AW, JB) identified 6 internal medicine admissions and 2 consultations. Selected cases represented a spectrum of clinical complexity and available medical information in the EMR. We chose to use real patient charts rather than to create fictitious cases. Due to the complexity of the EMR system, creating the wide range of clinical notes, laboratory data and imaging results in a realistic manner would not have been feasible. We generated a back-dated case summary page in place of the actual EMR page as this section is dynamically linked to an active admission, and therefore would not show the correct data for the selected patient at the time of the study sessions. Cases are listed in Table 1. For each case, we created a nursing page event (interruptive cue), prompting the participant to review the medical chart and reflect on next steps. These fictitious events were chosen with varying degrees of urgency (hemodynamically stable fever, for example, or acute shortness of breath) and represent common concerns for which an internal medicine physician would be paged. The original electronic handoff printouts, containing auto populated medical data from the EMR and direct physician input, are deleted every 24 hours. We therefore also re-created back-dated handoff printout summaries. Two investigators (JB, AW) also prepared and audio-recorded handoffs for each case.

Table 1.

Overview of patient cases and content of interruptive cues used during simulation.

Case	Main diagnosis	Page event
A	Pancytopenia with perirectal erythema	“Pt shivering, should we take blood cultures?”
B	Acute pancreatitis	“Pt tremulous and agitated, can we give Ativan?”
C	Sepsis in HIV patient	“Vancomycin tough 24, should we give next dose
D	Cecal bascule	“FYI - Pt BP is 182/108, asymptomatic”
E	Esophagitis and diabetic ketoacidosis	“PT had 10 beats of V-tach”
F	Leg hematoma and panhypopituitarism	“FYI - Pt BP 90/50, asymptomatic”
G*	Dyspnea in a polytrauma patient	“Pt would like an update on his condition”
H*	Perioperative care (ankle surgery)	“Pt is having chest pain 3/10, ECG pending”

^*Consultation cases

Participants:

We recruited volunteer residents and attendings in Internal Medicine from two academic hospitals, and in Family Medicine (one site) between August 2012 and January 2013. All sites share a Cerner Millennium-based EMR system with an integrated electronic handoff tool (CORES).¹³ As compensation, all participants were entered into a drawing for two gift cards. The study design was reviewed by the university’s Human Subjects Division.

Simulation:

To understand physician interactions with an EMR and an electronic-based handoff tool, we simulated an evening handoff in which the participant begins a cross-cover night shift. Study participants began their simulated evening shift by receiving a standardized, pre-recorded audio handoff of four patients. Cases were block randomized by type (admissions and consultations), with three admission cases and one consultation assigned to each participant. Participants were given the opportunity to ask questions after each recording to maintain some of the interaction that occurs during actual handoffs. Subjects were also given printouts from the electronic handoff tool for each of the four cases and were invited to annotate these as they might during an actual handoff. Following the handoff, subjects further assumed patient care through a medical chart review. During this 25-minute EMR review, participants received an interruptive cue once for each patient about a new clinical concern such as low blood pressure. Participants verbally described their planned response to the page (for example, calling the nurse back to ask for a new set of vitals). The order of interruptive cues for each participant was predetermined prior to data collection and was not related to the actual order of chart review chosen by the participant. Cues occurred at approximate 4-minute intervals. EMR interactions were captured using a screen capture tool and interactions with handoff printouts were video-recorded. At the conclusion of the EMR review, we staged an urgent departure from the hospital, requiring participants to give a verbal sign-out of the four cases to a new provider. Following the simulation, participants were asked to comment on their EMR use for two of the four patients using a screen capture recording of their chart review in order to probe their clinical reasoning, EMR use, and use of handoff printouts for patient assessment (stimulated recall). All sessions were conducted using a portable handoff simulation lab at clinical sites where participants worked. In this study, we use the term “handoff” when the participant is the receiver, and “sign-out” when the participant is providing information at the end of the simulation.

Data collection:

For each subject, we collected the following data: (1) demographics, medical specialty, and duration of medical practice; (2) screen capture of all EMR interactions using Morae 3, a usability assessment software; (3) annotated printouts or other notes taken during the session and video recordings of all annotations synched with the screen capture software; (4) timing of pages relative to the chart review of the case, (5) field notes taken by investigators during the session. For the pathway analysis, we grouped EMR pages with similar or overlapping content (e.g., vital signs, nursing codes and basic labs are available in both the “IView note” and “Result Review,” and were grouped together as “Results” section for this part of the analysis).

Data Analysis

Using Atlas.ti 7.0, three of the authors (LK, SK, CA) coded the navigation behavior of the participants using a video export of the screen capture data recorded by the Morae software during each simulation session. We created a preliminary codebook consisting of different pages in the EMR and notable events such as annotations, interactions with researchers, and interruptive cues sent to participants during the simulation. The coding team iteratively revised the codebook through several rounds of reviews of the coded data to more concretely define the observed behavior. We then performed a descriptive analysis of the chart review process following the verbal handoff: we studied which EMR pages were visited and the time spent in each of these pages, comparing the use of pages with structured information (e.g., labs and medications), less structured information (e.g., medical notes) and image information (ECG and radiology images). We refer to structured data as sources where information is generated from specific fields or parameters (e.g. blood pressure) in contrast to less structured information such as the free text found in clinical notes. We performed stratified analyses by level of experience (attending versus resident), by case and by cued or un-cued review (whether the interruptive cue was received after or before the review). This allowed assessment of the degree of overlap in the type of EMR pages visited. Finally, we analyzed overall time spent by the participants in reviewing the four charts (absolute time and relative percentage). We also examined time spent per case to assess the impact of individual differences and case specificity.

Results

We recruited a total of 21 participants, 12 residents and nine attendings (Table 2). While attendings had more years of clinical experience than residents (7.1 vs. 2.3 years), a greater proportion of residents reported previous handoff training (67% vs. 13%).

Table 2.

Study participant characteristics.

	Resident	Attending
Site A (Internal Medicine)	4	3
Site B (Internal Medicine)	4	4
Site C (Family Medicine)	4	2
Female, n(%)	6 (50%)	5 (55%)
ICU exper. (>6 months), n(%)	2 (17%)	3 (33%)
Handoff training, n(%)	8 (67%)	1 (13%)
Mean years experience (SD)	2.3 (1.1)	7.1 (6.1)
Years of EMR use (SD)	1.8 (1.3)	3.9 (2.0)

Time spent on the EMR review

Overall, participants spent just over 6 minutes on average for each case (range: 4:48 to 8:15) and almost 25 minutes in total (range: 19:12 to 25:35) reviewing patient charts (Table 3). We conducted a one-way analysis of variance test (p < 0.05) to compare if the there was any significantly different time spent between the cases used for the simulation. While there was no significant difference in average time spent per case (p=0.07), participants spent slightly more time on cases that investigators had judged during the study design to be slightly more complex (e.g., hematoma and panhypopituitarism diagnosis in Case F) or cases that required a more complete chart review such as a new consultation (Case G). Within cases, time spent varied widely by participant. Participant P14, for example, spent over 70% of the entire chart review (∼17 minutes) on one case (Case F), while the other participants who reviewed the same case spent 12% to 40% of their total chart review time on this case. We observed similar variability with Case G where one participant spent 62% (∼12 minutes) of their time reviewing this patient’s chart, almost twice the overall average of 34% among study participants.

Table 3.

Time spent (mm:ss) on chart reviews across cases sorted by median time.

Cases	N	Min	Max	Median	Mean	SD
All	21	19:12	33:03	25:35	24:51	03:55
A	11	02:06	10:06	05:41	05:19	02:13
B	11	02:36	07:39	04:06	04:41	02:03
C	12	01:03	12:10	06:11	06:23	03:53
D	9	02:32	07:26	04:16	04:33	01:46
E	9	03:44	13:00	06:12	06:49	02:52
F	11	02:49	17:00	06:29	07:47	04:08
G	10	04:55	12:05	07:20	07:42	01:58
H	11	01:57	09:23	05:53	06:20	02:23

When we analyzed where in the EMR study participants spent their time on chart review, we observed an interesting split between time spent on structured (e.g., labs) and less structured (e.g., clinical notes) information sources. As shown in Table 4, clinicians spent the most time on progress notes (19%), followed by various structured resources such as labs (17%) and vitals (12%). Across cases, participants spent 40% of their total chart review time in structured data sources, 52% of their time in less structured data sources, and approximately 8% of their time in imaging. As with the overall trends, the absolute amount of time in each section varied from case to case based on complexity and length of hospitalization—and, consequently, the number of clinical notes for that patient. For example, the review of ECG imaging data appeared to be common only to two of the cases (Case E and H).

Table 4.

Time spent across participants within types of EMR data sections by case (as a % of total time for a case). EMR sections are sorted in descending order of overall time spent and grouped by structured or less structured data.

EMR Section (n=21)	% of total time spent (all cases)	by case
		A	B	C	D	E	F	G	H
Structured Data	%	%	%	%	%	%	%	%	%
Lab Results	17	25	27	9	7	45	11	14	2
Other Results†	12	6	6	10	27	7	23	15	5
Medication Admin	8	4	8	18	21	2	6	5	6
Lab Culture	3	6	1	12		<1		4
Less Structured Data
Progress Note	19	32		19	23	20	33	1	23
Admit Note	12	8	52		3	8	15	2	16
Consult Note	11	13		22	11	1	3	20	9
Other Clinical Note	7	1	4	2	1	1	2	30	11
Other EMR Screen*	2	2		3	1	1	2	1	4
Web Reference	1	2		2			1		3
Imaging
ECG Results	5			<1		14	4	<1	20
Radiology Results	3		2	3	6	1		8	2
Total Time (h:mm:ss)	8:15:24	0:56:47	0:48:34	1:18:14	0:39:25	0:54:45	1:20:20	1:11:56	1:05:23

^†Other Results is an implementation-specific EMR source that primarily includes vitals as well as some lab test information.

^*Includes implementation-specific EMR sections such as a summary page, care team, medication profile summaries, and computerized physician order entry (CPOE).

Navigation patterns within the EMR

We were interested in understanding if physicians employed similar EMR navigation strategies to gather information about patients. On average, subjects accessed seven different EMR sections in their review of a single case (range: 1 to 16 sections). Many participants returned to a given section type multiple times during their chart review. For example, P06 went to information on the lab results six times while reviewing Case E and four times for Case A. To underscore the variation in navigation patterns, we illustrated the order of EMR sections that physicians went to during their chart review for a given case (Figure 1). Few physicians started their analysis in the same EMR section and the amount of sections viewed varied widely (range: 3 to 12 sections). While there is considerable diversity overall in the order of review of EMR sections, in the instance of Case A, we did observed an emphasis on less structured pages (e.g. consultation and progress notes) that likely reflect case-specific medical concerns.

Figure 1.

Sample sequence of differing navigation patterns for case A, with cued and un-cued EMR review. While there some similarity in common EMR sections, the sequence of navigation and time spent for a given case varies substantially between participants.

Analyzing the data across all participants regardless of case, 100% of participants viewed the Labs sections, followed by Progress Notes (90%) and Consult Notes (90%). Fewer participants viewed other areas such as Radiology (52%) and ECG results (48%), which are more case-specific. In fact, study participants only viewed these sections in the cases related to chest pain (case H) and arrhythmia (case E). Between physicians, the overlap in EMR sections accessed ranged from as low as 33% (p13 vs. p17) to as high as 92% (e.g. p07 vs. p20). Even within individual cases, where we expected greater similarity due to case standardization, we observed similarly diverse information-seeking strategies. Table 5 shows the percentage of participants that viewed an EMR section for each case. For Case A, for example, only 2 of the 11 participants who reviewed that case went to the Admit Notes, while 10 of the 11 went into the Progress Notes. Within cases, we saw more consistent behavior in the use of the Lab Results, Progress notes, and Results Review (100% for Case D). The Medication administration section was less frequently viewed, as participants seemed to rely more on their handoff printout for this information. We note that the patient in case B had just been admitted, and therefore the Admit note was the most recent note (no progress note available). Overall, however, physicians employed dissimilar strategies for seeking information about a particular patient case.

Table 5.

% of study participants that viewed a particular EMR section, by case.

EMR Section (n=21)	A	B	C	D	E	F	G	H
Structured Data	%	%	%	%	%	%	%	%
Lab Results	73		67	78	56	82	20	55
Other Results	36	55	50	100	44	82	70	45
Medication Admin	27	27	75	78	22	27	30	45
Lab Culture	45	18	67		11		40
Less-Structured Data
Progress Note	91		67	78	56	82	20	55
Consult Note	91		67	56	11	9	80	36
Admit Note	18	82		11	22	18	20	55
Other Clinical Note	9	18	17	11	11	27	100	27
Other EMR Screen*	36		42	11	22	27	10	18
Web Reference	18		8			9		27
Imaging
ECG Results			8		56	9	10	55
Radiology Results		9	33	11	22		70	18

^*includes implementation-specific EMR sections such as a summary page, care team, medication profile summaries, and CPOE.

Effects of cue events on EMR navigation

While there are a variety of factors that might explain which EMR sections our study participants chose to view, the interruptive cues appear to have some influence on the navigation behavior. As we discussed previously, there is a wide variation in which sections participants chose to view, both across and within cases. In charts reviewed prior to receiving the cue prompt, participants’ navigation is variable. However, after receiving an interruptive cue for a case, participants demonstrated greater similarity in navigation patterns. Namely, the first two EMR sections viewed by our study participants were typically structured test results such as vitals and labs found within the Other Results section. In Figure 2, we have highlighted the increase in structured data, broken out across vitals, labs, and medications. While the use of less structured information remained consistent pre and post cue event, we observed a tendency for a large number of participants to go to structured data like vitals across all of the cases.

Figure 2.

Count of EMR sections that participants viewed as their first or second EMR page before and after receiving an interruptive cue (a text page). Structured information has been separated out into vitals (referred to as Other Results within the EMR system), labs, and medications in order to demonstrate the change in structured information after a physician received the cue event. While the number of times that physicians navigated to less structured information remained similar pre and post cue, we observed an increase in views of structured data sources.

In addition, when comparing between cases, there was not a single section viewed by all participants within their initial purview of the EMR. But all participants, regardless of case, went into the Other Results section after receiving an interruptive cue. We note, however, that the delay until EMR use after being cued was not taken into account, and therefore the first two sections that a physician participant viewed after receiving the cue may be not necessarily be triggered or influenced by the information from the cue event.

Physicians experience and information seeking behavior

We recruited roughly an even number of residents and attendings for this study in order to explore differences in their information seeking approaches to understanding and building mental models of patients. Overall, the level of experience was not associated with the amount of time spent reviewing the EMR charts (p=0.35, see Table 6). While not significant, attending physicians tended to review a greater number of EMR sections on average than residents (p=0.12). This difference is more pronounced in less typical (acute pancreatitis (case B)) or more complex scenarios (tumor-related esophagitis and diabetes mellitus (case E)). Although there was no significant difference in the type of information viewed (structured vs. less structured) by level of experience, residents did tend to view images more often than attendings (ECG and radiology). In general, we did not observe a significant difference between residents and attendings, and in fact observed a continued trend of diverse approaches to navigating the EMR even within similar levels of care experience.

Table 6.

Comparison of the number of EMR sections viewed by residents and attendings overall and by case.

	Resident	Attending
Average	6.3	7.6
Max	16.0	15.0
Min	1.0	2.0
By Case
A	7.3	6.8
B	3.1	6.5
C	8.8	8.8
D	6.3	5.0
E	4.5	10.3
F	6.0	6.3
G	10.0	10.3
H	6.0	5.0

All participants found our simulation session realistic, with some degree of time-pressure to review charts and the interruptive cues. They described the experience as amusing and pleasant, despite possibly increased anxiety due to the two observers and the presence of recording devices. None of the participants had taken part in the management of the patient cases they were assigned during the study session.

Discussion

Overall, our data suggest both a need for better information architecture for EMR systems and for better ways to collect and summarize clinical information, as the data physicians needed to fully understand a new case were scattered across the EMR. Currently, physicians spend a lot of time as information curators, pulling information from different parts of the system and putting it together to gather the necessary data for each patient. Although our participants tended to gather some data from a core set of EMR sections for each case (progress note, lab results and vitals)—two of which were described by Zheng et al in residents’ navigation patterns¹⁹—great variation characterized the navigation of the other EMR sections (Table 4). Furthermore, the order in which they viewed the EMR pages differed, particularly if the review began before the participant received the interruptive cue (Figure 2). To aid their data collection, clinicians annotated the handoff printout, both to improve recall and to make a note of the latest available results, which were not on the printout.

EMR systems should support data collection and summarization much more effectively. Some ways to achieve better data access include: (1) creating views that unify data about a single patient that is stored in different EMR systems (e.g., if a different system is used for inpatient and outpatient care); (2) providing views for accessing data not by type (labs vs. radiology reports) but by problem or by a diagnostic hypothesis. One could imagine being able to tag different types of EMR data, and then having views that display in one place all data with the same tag. Alternately, the system could enable embedding and linking of data so that, for instance, a progress note could hyperlink to the relevant lab results, radiology reports, and so on; (3) EMR systems could provide a temporary storage location (a scratchpad) where physicians could drag relevant information as they traverse the system, so that they can bring together the information they need to understand the patient. Such a scratchpad would not change what is in the EMR system itself, but would just provide a way to visually bring together pieces of information that the physician needs to consider, regardless where that information is actually located. An advantage of such a scratchpad would be that it would contain just the information a particular physician needs, summarizing the case more effectively—for that physician—than standardized chart summaries are able to do. Indeed, very few of our participants used the existing chart summary page, suggesting the need for improved summary views, views that are personalized, and more easily accessible within the system.

The navigation patterns we observed suggest opportunities for creating catered EMR presentations based on certain triggers like an interruptive cue. The overlap in EMR sections that participants visited was considerably greater after an interruptive cue (Figures 2 and 3), suggesting that participants interpreted the events and required information similarly. A page sent through the EMR and automatically analyzed by a cue-event interpretation module, or an abnormal result flagged in the medical chart, would enable the physician receiving the cue to be presented with a targeted, customized representation of information related to the cue as soon as the physician accesses the EMR. Although such targeted representations might not be perfect, the significant overlap of content that our participants accessed after a prompt suggests that they could be made good enough to require only occasional navigation to seek additional information. Although we only created eight interruptive cues for this study, these cues were chosen for their common occurrence in clinical settings. Our results show that it is possible to predict which type of EMR section participants will tend to visit after receiving certain types of cues (for example, viewing the vital signs for trends after being paged for new hypo- or hypertension). Such predictions could be used to ease cued chart review.

EMR navigation patterns were affected by the complexity of the clinical cases, but may also vary with the level of experience. Mamede et al found that complexity made physicians take more time in each case²³. Although our study may have lacked power to show significant differences, attendings tended to navigate between more EMR sections than residents, particularly for less typical or more complex cases (Table 6). They tended to view less EMR sections than residents, however, in cases where the interruptive cues were common (e.g, chest pain). The attendings’ shorter navigation pattern in cases with the more typical cues could reflect the use of pattern recognition generated by higher experience, compared to the slower hypothetico-deductive reasoning used typically by more novice physicians, or in more complex cases^{23, 24}. Although future studies are needed to confirm these hypotheses, further analysis of the clinical reasoning during the talk-aloud part of our study, should allow more in-depth understanding of this difference in navigation patterns by level of experience, and will be presented in a future report. The use of pattern recognition to manage common cues could reinforce the ability to predict EMR use when cued, with potential implications for new EMR designs as mentioned above or physician training.

Reliance on clinical notes suggests that a system that automatically populates relevant information in a handoff tool could provide valuable support for the handoff process. One initial target for this work should be generation of an accurate clinical problem list. The American Recovery and Reinvestment Act of 2009 authorizes the Centers for Medicare and Medicaid Services (CMS) to provide electronic health record incentives to providers or health systems who demonstrate so-called “meaningful use” of health information technology, in part, by maintaining an up-to-date list of current and active diagnoses for at least 80% of patients²⁵. Treatment plan would be another target, and our analyses may identify other relevant content.

Finally, while our sample is limited and the difference did not reach statistical significance, we observed that attending physicians accessed more sections of the EMR per case than residents. This finding could be interpreted in several ways: although attending physicians’ chart review might be more governed by specific hypotheses they are trying to confirm or disconfirm, they may generate more hypotheses than residents, and residents may be more likely to do a more general, systematic case overview. Conversely, due to their age and familiarity with technology, residents might be more efficient at using the EMR, reflected in a smaller number of section jumps they have to make. Whatever the case might be, our data do seem to suggest that in spite of greater experience, attending physicians are unable to access all needed information from the EMR significantly more efficiently than the residents can. The large number of sections and page switches observed in both groups might be primarily a function of how the data are organized in the EMR systems and how the systems themselves are designed. Clinical experience does not seem to be able to overcome such technical and design limitations. In cases with common interruptive cues (possible sepsis in case A, high blood pressure in case D and chest pain in case G), experts actually viewed less EMR sections than residents.

Study strengths and limitations

Our use of a simulation study design with real patient cases and fictitious interruptive cues enabled participants to experience as real of an EMR environment as possible while also allowing us to compare participant performances both across cases and for each individual case. Using real medical charts provided us with a choice of rich, complex cases and it helped address case-specificity issues. Although we have a relatively small number of participants, our simulation environment offers high comparability among participants. We would like to note that while approximately half the physician participants were residents, we recruited few novice residents (1^st years) because of their lack of familiarity with the local EMR system.

Our simulation approach does have limitations, however. When we created typical interruptive cues to prompt the chart review, we restricted those events to sub-acute events, which would not incite the participant to immediately go see the patient. Also, the researchers leading the simulations (KB, LK) limited or provided vague responses to participant questions during the initial handoff within the simulation in order to incite the participant to navigate in the medical chart and retrieve the needed data themselves. It’s possible that in their everyday experience, physicians might get more detailed information verbally from their colleagues during the handoff.

The EMR system used in the study presents similar data in different EMR sections, which complicated our analysis. For instance, summary lab results in the clinical notes section contain similar information to the chart summary page. Due to this overlap, content seen on one page may not need to be collected from another page. To address this limitation, we will need to perform more granular analysis taking into account the specific content and data that physicians collected within a given note or page. This will be presented in a future paper.

Finally, participants’ expertise in cross-covering and case complexity may have an effect on the results. Although all participants had performed cross-covering and nightshift work, the daytime hospitalists/attendings are not actively providing cross-cover during their shifts anymore. Moreover, the Internal Medicine and Family Medicine participants may have differently levels of familiarity with the complexity of the chosen cases in this study which may have affected their performance.

Conclusion

Our study describes the EMR content that clinicians use during chart reviews to collect pertinent data and develop initial impressions for new patients when cross-covering. We found that the data physicians sought during the review was spread throughout the EMR, and that physicians’ navigation through the EMR was highly variable but that the variability was reduced by clinical interruptive cues. We propose designs to improve the data review process, especially when the chart review is prompted by the occurrence of an interruptive cue. In these cases, clinicians commonly reviewed structured data like vital signs and lab test results, suggesting the possibility for contextual EMR views that are automatically initiated by an interruptive cue. When chart review is not guided by an event, however, clinicians show less similarity in their navigation patterns. In spite of the diversity of navigation practices, our findings indicate that there is a clear need to better consolidate information in the EMR for the chart biopsy process during cross-cover. Our work provides pointers for how such consolidation of information could take place, enhancing our ability to build tools that can support an effective and safe handoff process.

Finally, data mining methodologies like those employed in our study are commonly used to analyze the use of web applications. We suggest that such methodologies should be a standard part of EMR implementations to help healthcare organizations to better understand how their providers use EMR systems and to more effective user interfaces that can help providers deliver safer, more efficient care.