Uncertainty, Case Complexity and the Content of Verbal Handoffs at the Emergency Department

Abstract

Handoffs are known to increase the risk of medical error and adverse events. Few electronic tools can support this process effectively, however. Our objective was to describe the relationship between clinical complexity, diagnostic uncertainty, fit with illness script and the content of case presentations by physicians. We observed the handoff of care for150 patients during eleven shift changes at a large urban emergency department (ED). Results indicate that as uncertainty about diagnosis and perceived illness script increased, more descriptive detail was conveyed to the incoming physicians. Physicians were concerned primarily with creating a shared mental model of a patient’s clinical state and with describing the expected path to disposition rather than simply passing on data and findings. Electronic tools for ED handoffs should allow adjustment of structure and content to capture complexity and uncertainty appropriately without requiring extra effort for more routine cases that better fit to more standard narratives.

Introduction

Physicians transferring the care of patients to another team at the end of a shift have the challenging task of collecting sets of clinical data with laboratory results, test findings, significant events and gathering other sometimes complex evidence that collectively characterize a patient’s course from complaints, problems and clinical decisions to disposition plans. All information needs to be presented to incoming clinicians as a succinct narrative of interconnected events that can be accurately interpreted in order to allow them to continue care without delay. Research evidence suggests that handoffs engender misinterpretations and omissions that may lead to medical errors and adverse events^{1, 2} but that these discussions also afford the opportunity to consider alternative viewpoints, correct mistakes and propose new tests or interventions.³ Efforts to analyze and improve the process generally assume that its primary goal is a unidirectional and complete transfer of information between individuals and teams on adjoining shifts. However, it is also an opportunity for a dialog in which participants co-construct and negotiate a shared understanding of existing clinical issues and check the accuracy of medical reasoning and prioritize next steps.⁴

Developing a shared mental model of illness and care by the active involvement of both parties during a handoff is particularly relevant to emergency medicine where adequate and reliable information may not be available and where uncertainty about symptoms, presentation and several possible diagnoses create ambiguities that allow different interpretations.^{2, 5} Interventions and strategies intended to make handoffs more robust and resilient toward misinterpretation and loss of important facts usually emphasize training, standardization and the use of checklists and structured forms as best practices.⁶ This approach has clearly reduced adverse events in some settings⁷ and has appropriate application in specific areas of care.

Intrinsic to emergency care, however, are frequently changing priorities, time limitations, the need to treat multiple patients concurrently and large variations in problem complexity and acuity that are encountered and treated on a single shift. When the incoming team takes over, patients are at very different points in their care trajectory. Uniform and inflexible handoff structure is therefore not likely to provide support to clinicians effectively and may even introduce costly unintended consequences.⁸ For example, in an environment where time and attention needs to be allocated to many patients simultaneously, detailed, exhaustive documentation of routine problems or standard treatments may in effect reduce the opportunity to discuss complex and uncertain conditions or unfolding events.

Theories of cognition distinguish between reasoning strategies that people use in ambiguous, complex and arbitrary situations to develop insight and understanding and those used in predictable or familiar conditions. Problems that can be articulated within clearly defined categories, relationships and hierarchical structures, such as a relatively uncomplicated or routine illness and care history, are generally best evaluated and resolved by the use of paradigmatic reasoning. Standardized handoff tools and templates will likely improve human performance and reduce the occurrence of errors for these cases. By contrast, a narrative mode of reasoning becomes more effective for complex problems as it organizes knowledge into temporal plots, linking specific events into causal chains and emphasizes consequential connections among them.⁹ Highly structured, static forms of support will not align well with uncertain diagnoses and indeterminate illness scripts. Modern handoff tools will need to support both cognitive strategies equally well.

Current methods of coordinating handoff procedures and organizing communication provide an important foundation for tool development but may be conceptually limiting.¹⁰ Market innovation has not yet adequately addressed existing design challenges and electronic handoff modules often resemble or mimic paper forms,¹¹ providing little meaningful support for diverse cognitive tasks. Studying handoffs from the perspective of uncertainty, complexity and perceived fit to illness script is in our opinion a productive approach to identifying and describing cognitive strategies that are used in different clinical situations and that may need different kinds of support. For example, new insights may inform the design of electronic tools that will be appropriate for the volume and character of information discussed between outgoing and incoming teams for patients with high and low levels of diagnostic certainty or who are at a different stage in their emergency department (ED) trajectory. The goal is to give clinicians the right information in a form that will allow them to make timely and effective interventions when needed and to sustain the care process uninterrupted across shift and unit boundaries.

Our objective was to observe discussions of physicians during handoffs and to analyze how case complexity, uncertainty and fit to illness script affect the content of their patient case presentations. This predominantly descriptive and exploratory study was intended to report findings from a research perspective not frequently present in publications and also to test our assumptions about the relationship between uncertainty levels, clinical complexity and different needs of cognitive support.

Background

There seems to be little convergence in clinical research literature on the characterization of a good handoff¹² and little published evidence on what constitutes best practices.¹³ Many quality measures have been proposed although there is a general lack of consensus on the primary purpose of handoffs and how to best improve the process.¹⁴ For example, one rating scale assessed information transfer, shared understanding, working atmosphere, and overall quality¹⁵ and another rated, on a nine-point scale, performance in interviewing, physical examination, humanistic qualities and professionalism, clinical judgment counseling, and organization and efficiency.¹⁶ Recognizing that there are multiple purposes for handoffs that need to be addressed simultaneously is a critical precursor to quality improvement.¹⁷

Studies of uncertainty in clinical work also give a somewhat fragmented and incomplete account of its effects and important insights have not been translated to practice.¹⁸ One taxonomy conceptualizes uncertainty as a multidimensional phenomenon with theoretically distinct domains and constructs that are potentially measurable and related to different outcomes, mechanisms of action, and management strategies.¹⁹ A measure developed to study clinical reasoning strategies during patient visits and derived in part from a cognitive engineering framework includes assessment of uncertainty that refers to how well limitations of available information are recognized, explained and solutions are planned to adjust to the current situation.²⁰ Our current framework uses uncertainty for this study in a similar way to analyze its relationship to reasoning and the amount of detail conveyed in narrative accounts of care during handoffs.

Cognitive and human-computer interaction studies in healthcare have shown that team and system interaction,²¹ staff workload,²² clinical workflow and the effects of technology on decision making and cognition²³ need to be investigated if new information technology is to be meaningfully integrated into the process of care. A design paradigm that embodies this comprehensive approach and is employed routinely for the design of safety-critical systems (although still somewhat sporadically for health information technology) is User-Centered Design.²⁴ The most common method for gaining insights and understanding of tasks and the environment is direct observation of work and ethnographic studies. Our investigation contributes to the body of knowledge about the handoff process in emergency care and can be used as a starting point in the design of new technology to support clinicians.

Methods

We observed clinicians during rounds in the ED and recorded their conversations concerning patient care handoffs. Eleven scheduled shift changes were studied on ten days over a four-month period. On each occasion, a group of two to three outgoing ED residents presented between seven and twenty-two patient cases to an incoming attending physician. The aggregated time of all recordings was 190 minutes (17 minutes on average) and contained conversations about 168 patients. Recordings were transcribed, de-identified (when names were audible) and reviewed by two physicians to correct errors and misinterpretations. Eighteen patients were excluded either because they had not yet been formally seen or because of poor recording quality. The remaining 150 cases were analyzed in this study.

Setting

The study was conducted in the main adult ED of a large academic hospital in an urban area. The annual census is approximately 88,000 in three clinical districts and a separate fast-track area. Two of the districts have shift changes for all providers twice a day, at 8am and 8pm. The attending physicians and senior residents in the third district have three shifts a day, ending at 8am, 4pm, and at midnight. Emergency medicine residents, those rotating from different services and attending physicians will “round” as a group on all patients in the district. These rounds constitute the usual handoff process between the teams. At the time of data collection, no program for improvement of handoffs had been implemented. Rounds were primarily recorded at 8am and 4pm when the census tends to be significantly lower than when shift change occurs at 8pm and midnight. A fully integrated EHR system was used for virtually all clinical activity.

Analytical constructs and coding scale development

We operationalized the assessment of uncertainty along three axes with 3- and 2-point scales. The constructs were derived from established concepts in medical education and clinical care that are well described in literature and have generally accepted meanings. They were developed and refined iteratively in a series of meetings and conversations. For example, scales were initially proposed to have 5-points but were subsequently simplified so that they were more appropriate for describing variations encountered in the analyzed sample.

Apparent Diagnostic Uncertainty estimates the amount of confidence physicians seemed to have in their understanding of the disease process that lead to the acute presentation. The concept may be defined in ways that are specific to clinical context.^{25, 26} In our interpretation, uncertainty about a diagnosis may originate from the lack of expert knowledge, level of training, experience with a particular disease and its presentation, and also from information conveyed by patients whose clinical history may contain ambiguity.

Apparent Fit with Illness Script to typicality of the presentation for a particular complaint or diagnosis is based on a theoretical framework describing how experts prioritize syndrome recognition through comparing and contrasting key clinical features in making a diagnosis.²⁷ Expert clinicians use illness scripts most of the time in their clinical reasoning since it involves a highly efficient knowledge-driven model of pattern recognition.²⁸

Clinical Complexity was an assessment regarding the number of complicated elements and the overall extent of problem complexity classified on a 2-point scale. The complex patient has been described as “one for whom clinical decision-making and required care processes are not routine or standard”^{29, 30} Our characterization of the concept is somewhat qualitative as the definition is difficult to translate into quantitative terms. Definitions of each scale level are in Table 1. The acuity of each case and disposition status were also assessed as a secondary descriptive measure.

Table 1.

Definition of assessment measures and scale levels

Measure	Scale and definition
Apparent diagnostic uncertainty	0 Unable to determine.
	1 Specific presumptive diagnosis (e.g., “appendicitis on CT, UTI, pneumonia”).
	2 Diagnosis fits into a narrow differential (e.g., sepsis, vaginal bleeding). If unclear, process is well defined for a workup (e.g., chest pain).
	3 Diagnosis appears to be unclear.
Apparent fit with illness script	0 Unable to determine.
	1 Typical presentation, fits well illness script.
	2 Atypical presentation, but seems a variant of a typical problem.
	3 Does not easily fit into a pattern that is readily recognized.
Apparent clinical complexity	0 Unable to determine.
	1 Simple problem with few or no complicating features (e.g., isolated laceration).
	2 Moderately complicated problem but not complex (e.g., pneumonia)
	3 Complex elements or a highly complicated problem (e.g., undifferentiated shock).

The amount of descriptive detail in communications about each patient as presented by the outgoing physician was classified into three categories: History of Present Illness and Physical Examination, Diagnostic Findings and Course of Treatment. The categories correspond to standard components of evaluation practice in the ED. They were aggregated into the Informational Content measure and scored on a three-point scale (Table 2). Assessment and classification definitions were developed by two physicians (authors) based on their clinical expertise and detailed knowledge of emergency care and further refined (originally from a 5-point scale) through consensus in a series of meetings and conversations. Audio recordings and their transcripts were scored by an ED physician (author) and results reviewed and revised in collaboration with another physician (author) to assure appropriate coding and classification.

Table 2.

Definition of informational content measures and scale levels

Informational Content measure	Scale and definition
History of present illness / Physical examination	0 None present.
	1 Minimal or brief (e.g., chief complaint or a brief history of symptoms only).
	2 Moderate (e.g., linear history or physical findings, recent history, some comorbid conditions).
	3 Detailed (e.g., specific history and physical findings, description extends beyond the present illness).
Diagnostic findings	0 None present.
	1 Minimal or brief (e.g., only qualitative descriptions such as “labs were ok”, “appendicitis on CT”).
	2 Moderate (e.g., quantitative descriptions of selected tests).
	3 Detailed (e.g., quantitative descriptions of several tests).
Detail of ED course	0 None communicated, only disposition
	1 Minimal or brief (e.g., status before disposition “couldn’t endorse.” definitive treatment given related to disposition or diagnosis, “Lasix for CHF”).
	2 Moderate (e.g., highlights events and treatments influencing current status).3 Detailed (e.g., provides thorough detail about ED events and treatments).

The dataset was uploaded to SAS 9.3 statistical software³¹ and tests were performed on the Confidence and Informational Content sets of variables. Non-parametric analysis of variance for ordered values (Jonckheere–Terpstra Trend Test) was used as a measure of difference between groups with three levels of a predictor variable. The levels of Clinical complexity were analyzed using the Wilcoxon Score (Rank Sums) test.

Results

The assessed acuity of most cases (67%) was found to be of small to moderate likelihood of morbidity and mortality. Nine cases (6%) were considered more serious and for twenty-nine (19%) acuity was not possible to determine. Disposition status at the point of a handoff was to admit (including boarding) for 86 (58%), discharge for 13 (9%) and under evaluation for 51 (33%) patients. Informational detail conveyed by the presenting to the incoming physician increased with rising diagnostic uncertainty and case complexity. Results are provided below. Measures are compared to the History of Present Illness and Physical Exam, Diagnostic Findings and Detail of ED course.

Apparent diagnostic uncertainty

Two patients did not have any history or physical examination discussed and for 25 (17%) diagnostic uncertainty could not be determined from the recordings. The remaining sample of 123 patients was analyzed (Figure 1). Most patients (76, or 62%) either had a narrow differential diagnosis or the process for a workup was well defined. The extent of communicated history and physical exam detail was mostly minimal, with a smaller proportion of patients with a moderate or detailed description. Similar pattern was observed for patients with a presumptive diagnosis. This trend was reversed when the diagnosis was uncertain where most patients (42%) had a detailed description. We found the statistical difference between groups with Presumptive, Differential and Uncertain diagnoses for levels of informational detail about the history of present illness and physical examination to be significant (p=0.0009).

Figure 1.

Description of history and physical examination by diagnostic certainty

The quantity of Diagnostic Findings in the presentations was mostly minimal for all groups (Figure 2). Equal proportions of cases had no or moderate levels of descriptive detail, and patients with the most detailed explanations represented the smallest proportion in each group. This trend was most pronounced for differential diagnoses that were described only in minimal descriptive detail, for 43 patients (35%). Patients with uncertain diagnoses tended to have larger proportions of detailed description than other groups. Difference between groups with Presumptive, Differential and Uncertain diagnoses was statistically significant for levels of detail about diagnostic findings (p=0.024).

Figure 2.

Descriptions of findings by diagnostic certainty

The Detail of ED course was described for all groups in a pattern similar to Diagnostic Findings although differences between the levels of detail were not statistically significant. The sample consisted of 125 patients: 25 (17%) were excluded because the ED course could not be determined from the recordings.

Fit with illness script

Two patients did not have any history or physical exam discussed and for 44 patients (29%) the fit to illness script could not be determined from the recordings. The remaining sample of 106 patients was analyzed (Figure 3). Most patients had a typical presentation of an illness (61, or 58% of total) and were most likely to have a minimal description of history and physical (29 or 48% of their group). Those with a variant or atypical presentation were most likely to have a moderately detailed description (16, 55% in group) and those with no apparent fit to an illness script a detailed description (8, 50% in group). Statistical difference between groups with a Typical Presentation, Variant and No Fit with illness script was significant for levels of informational detail about the history of present illness and physical exam (p=0.01).

Figure 3.

History and physical by fit to illness script

Diagnostic findings were communicated mostly at a minimal level of description and with a higher level of detail for only three patients (2%) (Figure 4). However, patients with no apparent fit of presentation to an illness script tended to have more information conveyed during handoff. Statistical difference between groups with a Typical Presentation, Variant and No Fit with illness script was significant for levels of informational detail about Diagnostic Findings (p=0.02).

Figure 4.

Description of findings by fit to illness script

The Course of Treatment in the ED was communicated in a pattern similar to Diagnostic Findings with a slightly larger proportion detailed descriptions for all groups. Differences in amount of descriptive detail between groups with different uncertainty levels were not statistically significant, however. The sample consisted of 106 patients as 44 (30%) were excluded because fit with illness script could not be determined from the recordings.

Clinical complexity

Two patients did not have any history or physical exam discussed, for 30 patients (20%) clinical complexity could not be determined from the recordings and five patients with simple problems were excluded as the group was too small for analysis. The remaining sample of 115 patients was analyzed (Figure 5). Clinical complexity was moderate for 53 patients (46%) and 62 (54%) had more complicated problems. Both groups had about the same proportion of minimal and moderately detailed descriptions of history and physical exam but complex patients had more detailed narratives. Observed statistical difference between groups of clinically moderate and complex patients in levels of informational detail about the history of present illness and physical exam approached significance (p=0.058).

Figure 5.

History and physical by clinical complexity

Trend patterns and the distribution of descriptive detail in communication was similar for diagnostic findings and course of ED treatment, with higher proportions of detailed descriptions for more complex cases. The difference between groups of clinically moderate and complex patients in levels of informational detail about diagnostic findings was significant (p=0.001) and approached significance for levels of detail about course of ED treatment (p=0.051).

Discussion

This preliminary investigation shows an association between the extent of descriptive content in verbal communication during handoffs and the level of confidence in diagnosis and probable course of further care that the outgoing physician has. As diagnostic uncertainty rises and the fit of the illness to a recognized script is less typical, the amount of detail in the history of present illness, physical examination, diagnostic findings and the course of care increases.

We have observed several trends in the way diagnostic uncertainty changed during a typical ED course. For many patients, there was initially a high degree of uncertainty about the cause of the presentation and about best ways to evaluate before deciding on disposition. As information was gradually gathered a corresponding reduction in the degree of uncertainty usually ensued but could also rise again when new data did not confirm expectations and hypotheses or new complications arose. A scheduled handoff could happen anywhere on this trajectory and clinicians would need to reason about the same patient differently. An innovative handoff tool would allow them to select the right modality to support the creation of their report or guide their discourse. Patients can be signed out from one provider to the next at virtually any point along the course of evaluation and treatment, at points when uncertainty may be at its peak, its lowest level or anywhere in between. Our analysis has demonstrated that there is a tendency towards giving fuller and more complete accounts of information when the degree of uncertainty is higher.

Diagnosing a patient with a high degree of confidence is generally of less concern to ED clinicians than the process of reaching a disposition. In the context of emergency care, a definitive diagnosis is often elusive, and the constant influx of potentially critically ill patients makes time and cognitive resources precious commodities. There is rarely a complicated plan for treatment that needs to be conveyed, as priorities are in addressing immediate, most critical and time-sensitive needs of new patients seeking care. Incomplete information is a routinely expected condition as are rapid changes and new developments and physicians typically view their role as stabilization and disposition rather than securing definitive diagnosis and management.³² It is therefore important to emphasize in the handoff process the aspect of collaborative discussions and review. Our contention is that ED physicians are not primarily concerned with transmitting sets of data about patients from one to another in handoffs. Rather, they engage in the process of developing a shared mental model of each patient’s current state and expected path that can be quickly and accurately adopted for care. These mental models function similarly to individual illness scripts, but encapsulate the entire process of evaluation and disposition, including the prospects of multiple possible outcomes.

When the presenting physician feels uncertain about which pre-defined ED “process script” is appropriate for a patient, the handoff cannot consist only of heuristics but is necessarily more detailed. This allows the incoming team to more explicitly contribute to the understanding of the patient’s state and invites their input into the current plan of management. The narrative of the patient encounter becomes much more valuable to the incoming physicians in constructing their own understanding of the situation, especially where there is asymmetry between the roles of those giving and receiving a report, as it exists for example in teaching hospitals.³³ A form of a handoff decision support would help both parties to navigate through this situation and perhaps provide patient-specific advice to guide their discussion. However, when patients do fit well onto an established process script and expected trajectory, the value of adding additional detail to the handoff is questionable. Required data are usually available in the EHR already.

Rounds were conducted by walking from patient to patient, carrying printouts of the ED status board that includes the patient name, age, and chief complaint as documented in triage and boxes that indicate if laboratory tests, radiology studies and nursing orders are completed or pending. However, these indicators were quickly out of date and were considered unreliable. There were also columns that show the current disposition, and if planned for admission, the assigned admitting team. This information is also not reliable and is usually verbally reviewed. Only one ED attending routinely conducted rounds with a mobile computer workstation. Mobile technology such as tablet computers may serve as a convenient technology platform for handoff tolls so that all parties have access to updated, shared information that is unobtrusively present during discussions.

Our emphasis on the changing rather than stable components of a handoff process is not contrary to the current effort to standardize handoffs in order to achieve more successful transfer of information. Consistency and predictability are core design principles that have been known improve communication speed, clarity and to reduce errors of omission. Checklists or other structural constants would be essential in the ED to quickly and accurately document routine cases with low complexity and high levels of certainty in diagnosis and disposition plans. We are proposing that often the ebb and flow of information is a meaningful variation and when the reasoning of clinicians follows a strategy that needs to account for ambiguous, missing or uncertain information, standardization would not adequately fit the model. Designers would need to include this source of variation into their tools to make them more flexible in supporting the sharing of mental models in authentic conditions of emergency care delivery.

We are not aware of any published tools or programs that explicitly address this understanding of the handoff and believe that a valuable direction for future investigation would be continued research into the nature of the types of information conveyed in the handoff and their usefulness. However, it seems apparent that one size does not fit all, as it were, when sign-out tools and processes are concerned.^{8, 32}

Limitations

We did not have access to written and electronic information that was used many physicians during rounds and for documentation of care. Verification and comparison of verbal and written communication was therefore not possible which sometimes limited our insight into the clinical cases that were discussed. The study design was somewhat limited by our focus on the verbal report of the outgoing physician and did not capture the understanding of the incoming physician. This would be an important aspect of the mental model sharing description and would need to include interviews with physicians and other forms of data collection.

Observations were limited to one institutions so the findings may not be directly relatable to other emergency departments, particularly in non-teaching hospitals. Our exclusive priority was on physicians and relevant communication between nurses and other clinical staff in the ED that is also critical to the process was not captured. Our estimates of the stage of the ED process care in which a particular patient was at the time of handoff are often not accurate. This knowledge would be important to validate our assumption about variations in uncertainty along the course of care.

Conclusion

Evidence supporting medical decisions is often shared among clinicians in a written form (electronic or otherwise) as a part of diagnostic and care planning documentation. In the time-constrained environment of emergency care, verbal presentations and discussions during rounds and handoffs are essential for developing a more comprehensive understanding of findings and symptoms and for increasing confidence and certainty through consensus, especially in the absence of complete information or distinct manifestations of known illness trajectories. Both forms of communication are necessary as they seem to have complementary roles in refining and sharing knowledge that will be the basis for further treatment decisions and interventions. Electronic handoff tools that could direct and focus verbal discourse by allowing clinicians to identify and point out missed or possibly misinterpreted evidence from notes or to elicit opinions and clarifications from other parties may be more effective in safeguarding the integrity of communicated information than a simple demand for completeness regardless of case complexity or apparent good understanding of the patient case by those involved. This view is framed in theories of cognition and management of uncertainty under time-constrained conditions and situated in a larger socio-technical context. Better understanding of the process will have positive implications for both training and design of rounding and handoff tools specific for emergency care physicians.

The importance of understanding handoffs is well recognized by the clinical research community. A well-executed handoff, in our view, is not necessarily one with the most details or high levels of completeness. Rather, the ability of the incoming clinician to make well informed and timely interventions when a patient’s state suddenly changes is seen as a high benchmark of safe and quality handoff process. We hope that our work will provide insight and guidance for the development of modern electronic support tools and best practices for effective and safe handoffs of care in the ED.