Using *ORA, a Network Analysis Tool, to Assess the Relationship of Handoffs to Quality and Safety Outcomes

Abstract

Communication during patient handoffs has been widely implicated in patient safety issues. However, few studies have actually been able to quantify the relationship between handoffs and patient outcomes. We used *ORA, a dynamic network analysis tool, to examine handoffs between day and night shifts on seven units in three hospitals in the Southwest. Using *ORA’s visualization and analysis capabilities we examined the relationships between the handoff communication network metrics and a variety of patient safety quality and satisfaction outcomes. Unique network patterns were observed for different types of outcome variable (e.g., safety, symptom management, self care, and patient satisfaction). This exploratory project demonstrates the power of *ORA to identify communication patterns for large groups, such as patient care units. *ORA’s network metrics can then be related to specific patient outcomes.

Introduction

Communication, particularly related to handoffs, has been widely implicated in patient safety issues (e.g., errors). Most research has focused on the quality of communication between individuals (e.g., erroneous, missing, or irrelevant information). Little attention has been paid to the quality of the overall communication network on the patient care unit and how it may relate to patient quality and safety outcomes, in part because of a lack of tools to do so.

We used *ORA, a dynamic network analysis tool, to examine the characteristics of communication networks on seven medical surgical units in three acute care hospitals in the Southwestern United States and how those characteristics relate to a variety of patient safety and quality outcomes. Each of the hospitals had achieved Magnet® recognition from the American Nurses’ Credentialing Center for the quality of care, excellence of nursing, and professional practice innovation. Therefore, it is not surprising there were a number of commonalities in the safety and quality goals on which staff were focused, as well as various outcome reporting mechanisms. Here we report the results of our analysis of communication patterns involved in the handoff that occurs between the day and night shift.

Background

Communication breakdowns during the transfer of care from one provider to another have been implicated as a significant cause of medical errors.^1–5 It has been estimated that nearly 2/3 of sentinel events follow communication breakdowns.⁶ Although hospitals are being challenged to develop and maintain a structured handoff procedure, a recent systematic review of the literature between 2006 and 2008 found little evidence to support any specific protocol or method.⁷

Handoffs are a standard part of nursing communication between nurses as care is transferred from one to another when nurses change shifts or patients change locations. The handoff process has been the topic of increasing scrutiny with the prioritization of an organization-specific structured handoff in 2006 by the Joint Commission as a National Patient Safety Goal.⁸ Handoffs affect every area of the acute care environment and work is ongoing to strengthen handoff procedures along the surgical continuum of care,⁹ as well as in medical step-down units,¹⁰ surgical intensive care units,¹¹ community hospitals,¹² perinatal inpatient care,^13–14 rural nurse managed clinics ¹⁵ and emergency departments. ¹⁶

In an effort to strengthen handoffs between hospitalists, Arora and colleagues¹⁷ conducted a systematic review to identify methodologically rigorous studies examining patient-level, staff-level or system-level outcomes. The review provided strong support for handoffs that integrated verbal and written components that were supplemented, but not replaced by, technology.¹⁷

Risenberg and colleagues⁷ identified multiple barriers to effective handoffs through their systematic review of literature spanning 1987–2008. Human factors found to hinder effective handoffs included communication barriers, lack of standardization, equipment issues, environmental issues, poor use of time, complex cases and high caseloads. Nursing care requires effective handoffs; but high turnover of nurses, too few nurses on a shift, rushed reports, poor team dynamics and poor cohesion within the team all hinder effective transfer of information at change of shift.⁷ Facilitators of effective handoffs with minimal information loss include: using structured checklists,¹¹ limiting comments to pertinent information, conveying information in a consistent way using standard communication structure, allowing time for questions, and incorporating face to face interaction.¹⁸ Thus, although it is clear from the literature that handoffs are important to patient safety and that multiple opportunities exist for communication breakdowns during the transfer of care, little is known about the relationship of handoffs to specific patient outcomes.

The purpose of this study was to compare the characteristics of nursing unit handoff communication from the day to night shifts and how those characteristics might relate to patient safety and quality outcomes on the units. To do so, we used *ORA, a network analysis tool that was developed at Carnegie Mellon University.¹⁹ (*ORA is available at www.casos.cs.cmu.edu/projects/ORA.) *ORA’s theoretical underpinnings are derived from social network and organizational theory, as well as operations research. Most readers will be familiar with social network theory, which has been used widely to describe small group interaction (e.g., families, school classes, etc.), but has also been applied in health care as Social Network Analysis. For example, a number of researchers have used social network analysis to examine the effects of information technology on communication patterns.^20–23 Creswick and Westbrook ²⁴ used social network analysis to examine the effect of an electronic medication management system on staff interaction. Netdraw was used to visualize the network, comprised of those individuals who sought information about medication decisions; measures of density were calculated and compared pre and post implementation.

In all of these studies, data to create the network were collected via self report. Despite its obvious limitations, self report continues to be the most common way to collect data for social network analysis. Each of the alternatives to self report (e.g., direct observation and technology such as phone or GPS) has its own limitations. Direct observation is prohibitively expensive for large networks such as measured in this study; and technologies are not always available and, when they are, are restricted to capturing the communications they track.

Still, social network analysis is inherently a static tool and its use is limited to relatively small groups and usually a single network. *ORA, by contrast, is a dynamic tool that can be used to analyze organizations and other large groups, including those comprised of multiple networks. *ORA can identify risky behavior in organizations that is likely to lead to undesirable outcomes, such as “groupthink” tendencies, ignoring available information, communication barriers, cliques, and key performers (both from a positive and a negative point of view).

Organizational network analysis is a method designed to examine the relationships between organizations or groups within organizations. Knowledge of inter-organizational networks can enable managers to control or exploit access to resources and information. Further, managers’ understanding of inter-organizational networks can influence mergers and joint ventures. Networks affect the flow of information and rumors, the adoption of new technology, and the spread of diseases. Managers can use their knowledge of social networks to accumulate information, gain early warning of potential risks, plan, and make decisions. Network knowledge can also be used to identify critical performers (either those who are central to a process, such as a handoff; or those whose absence would hinder effective information transfer because of their particular knowledge or resources).

For these reasons, *ORA has begun to be used in a variety of military, police, and business settings. ^25–28 In healthcare, *ORA has been less widely used. However, Merrill and colleagues used *ORA to examine the impact of an impending merger of public health programs; ^29–30 and Benham-Hutchins and Effken used *ORA to examine handoffs from the Emergency Department Staff to other medical-surgical unit staff in one hospital. ¹⁶

Using *ORA, researchers can visualize the networks for better understanding. Users can not only see the number and types of links (connections) between individuals (nodes), but can also see characteristics of each individual (e.g., in our study, we focused on the shift worked, but could also see the individual’s education, time worked in the hospital, and time worked on the unit). Qualities, such as frequency or quality of communication or access to information or resources, can also be visualized if those data are collected and of interest.

Elsewhere ³¹ we have reported the results of our analysis of the communication networks on day and night shifts on seven medical-surgical patient care units in three hospitals in the Southwest. Here we focus only on handoff communication from the day to the night shift on those units and the relationship of the handoff communication network characteristics to patient safety, quality, and satisfaction outcome measures.

Methods

Setting and Sample

The sample was comprised of seven medical-surgical patient care units from three acute care urban hospitals in the Southwest. Unit demographics are shown in Table 1.

Table 1.

Nursing Unit Demographics

Characteristic	Nursing Unit
Unit number (respondents)	1 (n=20)	2 (n=29)	3 (n=30)	4 (n=30)	5 (n=35)	6 (n=42)	7 (n=40)
No. of Beds	28	28	32	28	36	42	36
Skill Mix (% RN)	75	79	73	73	66	60	80
% Female	95	69	97	90	86	83	83
Age (Mean)	39.5(14.6)	37.8(12.9)	35.4(11.5)	34.3(10.3)	44.9(11.7)	39.4(13.4)	39.2(10.9)
Hospital Experience (years)	5.0(8.6)	5.0(6.3)	2.0(1.8)	3.0(2.6)	8.5(9.8)	6.7(9.3)	4.2(5.4)
Unit Experience (years)	3.6(4.0)	4.4(6.3)	1.6(1.0)	2.3(2.5)	6.0(7.2)	5.6(6.7)	2.7(2.7)
Education (%)
High school	5.9	-	7.7	7.7	23.3	10.3	2.6
Diploma	29.4	8.7	19.2	3.8	13.3	17.9	23.7
Associate	41.2	30.4	26.9	38.5	20.0	43.6	39.5
Bachelor	23.5	56.5	42.3	42.3	30.0	23.1	28.9
Master	-	4.3	3.8	7.7	13.3	2.6	5.3
Time Since Completed Education (years)	12.5(13.6)	4.5(6.3)	6.3(7.9)	4.2(4.5)	14.4(13.0)	9.2(11.4)	8.2(8.2)

Note. Values in parentheses = standard deviation.

One hospital was an academic medical center. All unit nursing staff scheduled to work on two days (in the same week) chosen to minimize staff overlap, and thus reduce respondent burden, were invited to complete communication surveys through fliers and an information session during a staff meeting on each unit. Both day and night shift staff members were included in the sample.

Discharged patients on the sample nursing units were surveyed just prior to their departure from the hospital to collect information about their satisfaction with care, changes in their ability to do complex and simple self care, and changes in their ability to manage their conditions since admission. For each unit, all patients being discharged on 2–3 randomly selected days were potential respondents. Our previous research had demonstrated that this was an effective way to obtain a sufficient and representative sample of patients. After being introduced by a staff member, research assistants informed patients about the purpose of the project and invited the patients to complete the questionnaire while waiting for their discharge process to be completed. Patients unable to complete the survey themselves were read the survey by the research assistant. Surveys were available in both Spanish and English.

Human subjects’ protection procedures for the study were approved by the University of Arizona Investigational Review Board. Site approval was obtained from each participating hospital.

Instruments

Outcomes data

Patient satisfaction data were collected via a Well-Cared-For questionnaire ³² from patients on the day of discharge. Scales in that questionnaire relate to: (a) general satisfaction with the quality of staff expertise, as well as with various services; (b) satisfaction that their individual needs were met, and (c) satisfaction with caring. The self-reported ability of patients to manage simple self care needs (e.g., carrying out activities of daily living such as dressing) and more complex self care needs (e.g., adjusting their treatment regimes to changes in symptoms) were measured by a scale adapted from Lorig and colleagues.³³ Symptom management difference (the percentage of patients on the unit who met the researcher-defined goal of a 1-point increase in their ability to manage their symptoms from admission to discharge); and symptom management capacity (the percentage of patients on the unit who met the research-defined goal of a 1-point increase (from admission to discharge) in the ratio of symptom management to symptom distress, as calculated from their responses to two different scales in the survey.³² All patient scales showed satisfactory validity as measures of the nursing unit, with rWG(j), an indicator of group consensus, values of .75–.89. The number of patient falls and adverse drug events (ADEs) per thousand patient days was obtained via survey from the quality management departments for a three-month period.

Communication data

The staff communication questionnaire was adapted from Merrill and colleagues.²⁹ Nursing staff were asked to identify, from a list of staff members assigned to their unit on the day of the survey, those with whom they interacted. They simply placed a check mark beside the name of each staff member with whom they had interacted. For those individuals whose names they had checked, they were asked to indicate how often they: (a) got information from the individual, (b) gave information to the individual, or (b) discussed patient care with the individual using a 5-point rating scale where 0 = not at all, 1 = rarely, 2 = some, 3 = a lot, and 4 = constantly. Because the questionnaires necessarily included staff names, we listed those names and titles on a tear-off overlay that respondents were asked to remove before returning the survey. The completed questionnaire showed only the under-page, which listed staff only as RN-1, UC23, etc. Staff also rated the frequency of their interaction with other professionals such as physicians, pharmacists, and dietitians (although these data were not used for the current analysis) and responded to several demographic questions (e.g., years worked on the unit; years worked in the hospital; age; education; shift).

Design and Procedures

The study utilized an exploratory, descriptive correlational design. We used a survey methodology to collect data on staff communication and several safety and quality outcomes (falls, medication errors, as well as patients’ self-reported improvement in simple and complex self care and symptom management over the course of their hospitalization, and satisfaction with care). Outcomes and communication data were collected approximately 6 months apart. This was necessary because, at the time of the outcome data collection, we were also collecting survey data from staff for the larger study, of which the network analysis is just one portion. Because both questionnaires were lengthy, we deemed it necessary to collect the data at two different times to reduce respondent burden. Completed questionnaires were either placed into a closed box on the unit or given to a Research Associate. For each unit, data were collected on two weekdays in the same week, preselected such that there would be minimal overlap of staff, as well as minimal conflict with other unit surveys or external reviews. Participants received a $20 Barnes & Noble gift card for completing the questionnaire.

Data Analysis

Data from the surveys were entered into an Excel (network data) spreadsheet or SPSS (IBM) (outcomes data) file. For this analysis, only results from the day with the higher response rate were used because network analyses are highly sensitive to response rate and require a very high response rate to be representative of the unit. Using the day with the higher response rate results in fewer holes in the network and therefore provides a more accurate representation of the unit communication network. For five of the units, this was the first day of data collection. The analysis of handoffs was limited to two specific data sets for each patient care unit: (a) data from the Day shift reflecting the patient information staff GAVE to the oncoming shift and (b) data from the Night shift reflecting the information they GOT from the previous shift. Excel spreadsheets for each unit listed staff members with each of their contacts. Each staff-contact pair required a different line such that an individual staff member’s data often required multiple lines. For the questions that required communication frequency ratings (e.g., how often they got or gave information), the third column listed the rating score (0–4). The 0–4 scale ranged from never (0) to constantly (4). Demographic data were entered into a single spreadsheet for each unit with the staff ID code in the first column and demographic data elements in subsequent columns.

The network data were entered into *ORA as comma separated values (.csv) files to create a communication network (metamatrix) for each unit by shift. Metamatrix is a term used to describe the interrelated networks (people, knowledge, resources, or tasks, or groups) that define an organization. In our case, the metamatrix for each unit’s day to night handoff was defined by the overlap of two networks—the day and night shifts (specifically, day shift staff who reported that they gave information to staff on the night shift and night shift staff who reported that they got information from staff on the day shift). Demographic data for unit staff were also entered into *ORA as “attributes” of the individual staff (“agents” in network terminology) within the metamatrix.

*ORA contains over 80 metrics that users can use to analyze network characteristics quantitatively. The metrics fall into several categories addressing: quantity of connections between people (e.g, density, or the percentage of actual to possible connections), quality of those connections (strong vs. weak), the percentage of clusters or small groups within the network, the degree of hierarchical tendencies, the speed of information diffusion, number of individuals who are central to the network, and variation in the paths along which information flows. Rather than using all 80 metrics, some of which measure the same properties and differ only in their mathematical calculations, we focused on 12. These 12 were selected to provide an adequate, but concise, representation of the various metric categories most likely to reflect the communication patterns of nurses in inpatient medical-surgical units.

Once the metamatrix was created, we used *ORA’s visualizer component, a software feature that produces a graphical depiction of the network, to ensure that the resulting network was adequately represented, and that the visualization contained the features that we wanted. For example, we used color to indicate shift, and node size to highlight those individuals who were central to the network (larger nodes equal greater centrality). Communication frequency ratings were mapped onto link color or link width (for gray-scale visualizations). Arrows were used to indicate the direction of the communication.

We then used *ORA’s analysis tools to compute network metrics for each handoff by shift within unit. Network metrics (those 12 that we had initially identified as relevant to our study) for the seven units were entered into a spreadsheet along with the eight outcome variables described earlier. Finally, Spearman rank order correlations were calculated between the seven units’ network metrics and outcome variables using SPSS Version 18.

Results and Discussion

The Well Cared For survey was completed by 256 patients. The number of patient respondents on each unit ranged from 29 to 45 (M = 37). Patient response rates ranged from 58% to 90% (M = 73%).

As noted earlier, to minimize holes in the network, we used data from the day with the higher response rate for each unit (a response rate of over 90% is desirable for social network analysis). Response rates to the communication survey for that day ranged, by unit, from 88–100% of the nursing staff working on that day (M = 94%).

The results of correlating *ORA metrics with patient outcomes are summarized in Table 2. Only those outcomes with statistically significant correlations (p < .05) are shown to simplify the table. Three outcomes (Symptom Difference, Well Cared for-Caring, and Well Cared for-General) did not correlate significantly with any *ORA metric in this analysis and are not shown in the table.

Table 2.

Statistically Significant Correlations of *ORA Metrics with Outcome Variables (p < .05)

*ORA Metric	Falls	ADEs	Symptom Capacity	Simple Self Care	Complex Self Care	Well Cared For – Individual Care
Clustering Coefficient		−0.80
Component Count
Strong	−0.82		−0.86
Component Count
Weak			−0.84
Density			1.00
Diffusion			0.86	−0.86	−0.79
Fragmentation			−0.82
Hierarchy			−0.86	0.86	0.86	−0.82
Isolates			−0.86
Centrality In Degree			0.93
Eigenvector Centrality				−0.82	−0.89	0.87
Cliques		−0.80
Triads		−0.80

Inspection of Table 2 reveals that the majority of statistically significant correlations were negative. For the two safety outcomes, Falls and ADEs, these negative correlations indicate a facilitative relationship. That is, more strong connections among handoff staff (Component Count Strong) were associated with fewer falls. Similarly, more clusters or small groups, cliques and triads were associated with fewer ADEs.

For the patient satisfaction, symptom management, and self-care outcomes, negative correlations indicate network characteristics associated with poorer results. These differ by type of outcome. For example, the percentage of patients whose symptom capacity to symptom difference ratio increased by more than 1 point during their hospitalization was higher on units with denser communication networks (r_s = 1.0), with faster diffusion of information (r_s = .86), and on units when there were more communication links coming into an individual (node) (r_s = .93). By contrast, fewer patients achieved this goal on units when there were higher numbers of strong and weak communication links, when there was more hierarchical (top-down) communication, and when there were more disconnected individuals (fragmentation and isolates).

Self care showed similar correlations for simple and complex self care. An example of simple self care is being able to independently carry out activities of daily living. Complex self care is exemplified by being able to adjust a treatment regimen based on changing symptoms. Self care was positively correlated with higher Hierarchy scores; but negatively correlated with faster information Diffusion and Eigenvector Centrality (units with more individual staff who are central to the network and therefore more influential).

Patients’ satisfaction with individual care was positively related to Eigenvector Centrality and negatively to Hierarchy. This is precisely the opposite relationship from that observed with patients’ self-reported abilities to perform complex and simple Self Care.

It is worth noting that patient care unit rankings varied dramatically by outcome measure. For example, Unit 1 ranged from best of the group for Patients’ Perceptions of Being Well Cared For –Individual Care; to worst of the group for Falls and for Simple and Complex Self Care. (Supplemental Digital Content 2, Handoff Visualization Unit 1) Visualizations can illuminate the differences in units’ communication patterns. Consider Units 2 (Figure 1) and 5 (Figure 2). Unit 2 ranked highest of the seven units on Symptom Difference, Symptom Capacity, and Caring; while Unit 5 ranked lowest on those outcomes.

Figure 1.

The Handoff Network for Unit 2. Gray nodes = Day Shift; Black nodes = Night Shift. Link width is scaled to communication frequency (thicker links = higher frequency. Larger node size represents higher Eigenvector Centrality values. Several subgroups are evident (e.g., one centered around PCT29 and another centered around RN30). Several PCT pendula (single links) can be seen at the bottom left of the figure. See text for further details.

Figure 2.

The Handoff Network for Unit 5. Gray nodes = Day Shift; Black nodes = Evening or Night Shift. Communication frequency is mapped onto link width (more frequent communication = wider links). Larger node size represents higher Eigenvector Centrality values. Seven isolates (three new graduates, Two RNs, 1 Floated RN, and a PCT) are shown along the left border. See text for details.

[NOTE: Supplemental Digital Content 1 was deleted by editorial office as it is not appropriate for inclusion. Please renumber the following if necessary.]

Figure 1 shows clearly how information might diffuse quickly across the network of Unit 2 and, indeed, Unit 2’s diffusion metric is nearly twice as high (.43) as that for the much larger Unit 5 (.28) (Figure 2). In *ORA all metrics are scaled from 0.00 to 1.00. (One caveat for interpreting the figures: in *ORA, the length of links do not represent communication distance, but are created arbitrarily by the application to make as many connections visible as possible.) A number of small groups are evident within both networks. The average Eigenvector Centrality value for Unit 2 is .45, compared with that of Unit 5 (.20). This suggests that on the smaller unit, more individuals are connected to highly connected staff. Worth noting is the cluster of PCTs at the left of both figures; although the precise impact of this on patient outcomes remains unclear. The communication network for Unit 5 has seven isolates. In addition, two of the more influential RNs (high Eigenvector Centrality) are not communicating with staff not on their shift, and there are a number of “pendants” (people with single links). The pendants are usually PCTs. Readers who wish to examine the visualizations for the other 5 nursing units may do so online. (Supplemental Digital Content 2–6, Handoff Visualizations)

General Discussion

This handoff study differs substantially from Benham-Hutchins and Effken,¹⁶ which used an ego-centric approach wherein the patient was at the center of the handoff and participants were identified by the investigator/observer. Questionnaires were distributed to those staff who were observed participating in handoffs as well as others who were identified by the participants. Even so, some participants either were not identified or did not complete questionnaires. In this study, all unit nursing staff were asked to complete questionnaires and we selected the day with the highest response rate to analyze so that each network had the least possible holes and therefore offered the best representation of the unit. Ideally, 100% of staff would have responded; but that goal was reached on only one of the seven units. Although the Benham-Hutchins study focused on the communication among multiple professions during the handoff, this study involved only nursing personnel. Finally, none of the prior healthcare studies using *ORA ^{16, 29–30} associated handoff network characteristics with patient outcomes.

Although there were significant correlations with patient satisfaction outcomes in both the overall networks and the night shift networks, they were with different variables (e.g., Individual Needs Met was significantly correlated with *ORA metrics in the overall network and Caring exhibited the only significant relationship for the night shift). That these are different seems problematic until one realizes that the networks that are considered in each case are very different. The overall handoff network (including both shifts) is not simply the addition of the two shift networks. The handoff network emerges from the connections among staff when the two networks are combined. The result is somewhat analogous to Venn diagrams. If each network were a circle in a Venn diagram, the overall network for the shift handoff is that common area in which they overlap.

The unique correlation patterns obtained for falls, adverse drug events, symptom capacity, self care and patient satisfaction are striking in terms of the specific metrics that were correlated with each outcome and suggest that a particular communication pattern may have very different effects on different types of patient outcomes. Further, the ideal communication pattern for preventing falls may be very different for reducing medication errors. This cannot be good news to managers who might expect to improve all nursing-sensitive patient outcomes by changing the nursing communication patterns on their unit.

The results of this study differ significantly from other analyses our team has done that examine different partitioning of the data by other groups (shift, day plus night shift) and other questions (e.g., how often did you discuss patient care with other staff?). This provides further support for the “emergent” nature of networks; but it raises the issue of network stability and context sensitivity. Are there unit communication characteristics that persist, despite changes in personnel; or do different personnel self organize to create unique network patterns? Those of us who have worked in hospitals for much of our careers learned to quickly recognize the difference in the “tone” of certain nursing units when specific staff or groups of staff were working. In this project, we collected data from staff on two different days, intentionally selected to involve as few of the same staff as possible. When we compared the data from the two days, four of the units were similar (suggesting that the pattern of communication persisted, despite different staff (agents); but three were not. To what degree this disparity was simply a function of disparate response rates or the presence or absence of key actors is not known. Future studies are needed that explicitly focus on network stability in organizations such as these. To our knowledge, only one such study has been done--and that study was in Antarctica and not in healthcare. ³⁴ However, in that study, the nature and number of informal and formal roles affected both the evolution and stability of the social networks.

The fact that no single unit was the best or worst for all outcome variables may be, in part, due to their homogeneity. Each of these Magnet® units was very focused on improving quality and safety outcomes. Still, the results suggest the different network variables that seem to facilitate or hinder good outcomes—for each type of quality and safety outcome. Nurse managers will need to identify the specific outcomes they want to improve and tailor interventions to facilitate the communication patterns that facilitate their achievement.

This exploratory study has several limitations. First, the sample is small and homogeneous, limiting its generalizability. For that reason, the study should be replicated in a larger, more diverse sample of units. Second, the communication and outcomes data were collected within the same year, but roughly 6 months apart. Although this is a potential bias, we demonstrated that the trend of outcomes data was consistent within each unit, by showing in an earlier study that data collected over a 3 month period (e.g., June – Aug) were not significantly different from data collected over a 6 month period (e.g., April-Sept). Third, stability of observed network communication patterns over time and including different staff has not been sufficiently tested. We selected the higher response day to better represent the unit but did not determine possible reasons for the difference in response rate between the two days (e.g. one day might have been less busy) that might bias these results. However, to minimize such differences we did not collect data on weekends or holidays because staffing is generally less on those days. Fourth, the number of pairwise comparisons in the analysis is high, given the number of units (seven). Fifth, differing bed size, patient populations, and staffing characteristics could be confounds. Finally, we relied on self-report to collect data about the communication networks. Self report has the potential for error due to memory lapses or respondent fatigue. However, questionnaires remain the most common way to collect these data because direct observation is not feasible for a large group (particularly since the data we collected were being used for a number of different analyses in addition to handoffs); and technology such as phone communication would miss any face-to-face interaction.

Conclusion

This exploratory study demonstrates the power and utility of *ORA for examining handoffs and their relationship to quality and safety outcomes, which has not been well documented in the literature. Network differences were apparent, not only in the *ORA metrics and their correlations with safety and quality outcomes; but also in the network visualizations. Network metrics showed unique patterns for different types of outcome variable (e.g., safety, symptom management, self care, and patient satisfaction). In a number of instances, the correlation patterns between variables within a category were quite different as well. These results suggest that a staff communication pattern that facilitates improvement in patients’ management of their symptoms may not be one that also prevents falls or improves their satisfaction with their care.

Nurse managers need to better understand the relationship between staff communication (particularly during change of shift) and specific patient outcomes. Doing so will require that managers have access to the data that simulation and decision support tools such as *ORA can provide. Such tools can help nurse managers better understand their unit’s communication patterns and how those patterns may be affecting, not only the currently targeted safety or quality outcomes, but other outcomes as well. For example, reducing medication errors at the expense of increasing patient falls may not be a satisfactory solution. Without these kinds of network analysis and decision support tools, nurse managers can only guess at the effects their interventions may have on patient outcomes.

Supplementary Material

1

Supplemental Digital Content 2. *ORA Visualization for Unit 1. Node color indicates shift (Green = day shift, Blue = night shift). Node size corresponds to Eigenvector Centrality (frequency the individual is connected to other highly connected people). Link color corresponds to communication frequency (red = “constantly”, yellow = “a lot”, green = “some” and blue = “rarely”). Arrows denote the direction of communication. On the night shift, a Unit Clerk and two RNs are the most highly connected persons—and therefore highly influential. On the day shift, 3 RNs and a PCT are the most influential. The number of bi-directional and strong (red or yellow) connections on this unit is worth noting since strong links were associated with more falls (see text for details). Three isolates (non-connected individuals) are shown at the left margin for convenience. This unit had the highest fall rate.

Supplemental Digital Content 3. *ORA Visualization for Unit 3. Although there are a number of bidirectional links, these are all blue or green (low frequency). The most influential (central) staff member on the day shift is a Unit Clerk; on the night shift it is an RN. This unit has 5 isolates. This unit had the highest rate of medication errors among the seven units.

Supplemental Digital Content 4. *ORA Visualization for Unit 4. The visualized communication pattern is striking for the long chains, few cliques, and the centrality of the Day Shift PCT17, through whom nearly all communication must run. This unit has 4 isolates (mainly PCTs). The unit had the second highest rates for both falls and medication errors.

Supplemental Digital Content 5. *ORA Visualization for Unit 6. A prominent feature of this communication pattern is the cluster of individuals surrounding PCT29, whose shift assignment is unknown (indicated by the red color of the node). There is also a large grouping of PCTs at the right of the visualization, who seem to interact largely with each other. There are a number of high frequency (red or yellow) links. Day shift in this visualization are shown as either blue or aqua nodes. This unit has 3 isolates, one of whom is an RN floated to the unit from another area. This unit had the lowest fall rate.

Supplemental Digital Content 6. *ORA Visualization for Unit 7. Prominent features are the clusters around RN15 and US (Unit Secretary) 10. Also worth noting are the number of high frequency (red and yellow) links. Day and night shift RNs are the most connected (influential). The unit has two isolates, one of whom is a nurse extern (student nurse). This unit had the second lowest medication error rate and the third lowest fall rate.