Correlations of Nursing Communication Network Metrics with Patient Outcomes

Abstract

Communication problems have been implicated in many safety and quality issues, but tools to examine communication networks and their impact on patient outcomes are only beginning to become available. We used *ORA, an organizational risk analyzer that allows the dynamic analysis of organizational networks to explore the communication networks among staff on seven nursing units in three Arizona hospitals. The results showed correlations between a number of *ORA metrics and patient safety and quality outcomes. *ORA provides researchers another way to study the influence of communication among staff on patient outcomes.

Introduction

U.S. healthcare is complex, expensive, and frequently unsafe. The 2000 IOM report ¹ and subsequent IOM reports² attribute the dismal patient quality and safety outcomes in the U.S. primarily to system problems, rather than to individual human error or negligence. Many system problems are due to either the lack of essential clinical information or to information overload. Up to 75% of clinical decisions are made with missing pertinent clinical information.³

Other system problems are related to failures in communication and coordination, particularly at handoffs (e.g., shift changes, or when patients transition from one level of care to another). ^{2, 4} Today’s patients are cared for by a bevy of professionals, including nurses, physicians, pharmacists, social workers, dietitians, and unlicensed personnel. Often, nurses interact with providers that change from day to day—or even shift to shift. Resident physicians rotate, pharmacists and dietitians may cover multiple units, and temporary nurses fill vacancies.² This constant churning precludes effective teamwork and interferes with the organization’s ability to build core knowledge.⁵

Social Network Analysis (SNA) is becoming increasingly popular, particularly for studying communication. When SNA is done to address organizational concerns it is sometimes called Organizational Network Analysis. SNA typically studies the patterns of relationships among people, or organizations.⁶ SNA can provide a visual map of the connections between and among individuals, or groups, or organizations, as well as quantitative metrics to clarify the communication patterns and communication-related roles (e.g., gatekeeper, star, or isolate) in functional groups. SNA also allows researchers to investigate the communication load and the effectiveness of communication within an organization. SNA has been used to study processes such as the effect of competition,⁷ the effect of centrality on perceived power,⁸ turnover,⁹ and social interaction after technology change. ¹⁰

Carley and her team recently extended SNA using a meta-matrix approach derived from knowledge management, operations research and social networks techniques that formalizes the interdependencies between agents in an organization and the knowledge and resources they bring to their work (or tasks).^11,12 This “dynamic network analysis” (DNA) provides a unique way to represent an organization in terms of a set of relations connecting people, knowledge, resources and tasks and the changes in those relations over time, as well as a set of measures for assessing the structure or health of the organization and analyzing the performance data for that organization.¹³ In contrast to SNA, which focuses on small, well-bounded networks with only two to three types of links for which there is complete information at one point in time, DNA can handle large dynamic, multi-mode, multi-link networks with varying degrees of uncertainty and can use that information to assess the current state of the organization and forecast the potential impact of changes in those networks on organizational performance. SNA takes a fairly static, position-based view of agents, but DNA treats agents as actively involved in communication, storing information, and learning. In DNA both networks and agents change dynamically and can learn.

One of the DNA tools, *ORA, was designed by the center for Computational Analysis of Social and Organizational Systems (CASOS) at Carnegie Mellon University as a risk assessment tool for locating individuals or groups that are potential risks based on social, knowledge, resource and task networks. *ORA contains hundreds of social network, dynamic network metrics, trail metrics, and procedures for grouping nodes, identifying local patterns, and comparing and contrasting networks, groups, and individuals from a dynamic metanetwork perspective. *ORA has been applied both to traditional organizations and to covert networks ^14,15 and can be used to examine how networks change through space and time. *ORA contains procedures for moving back and forth between trail data (e.g. who was where when) and network data (who is connected to whom, who is connected to where), and has a variety of geo-spatial network metrics, and change detection techniques. *ORA can handle multi-mode, multi-plex, multi-level networks. It can identify key players, groups and vulnerabilities. Distance based, algorithmic, and statistical procedures for comparing and contrasting networks are included in the application.¹⁶

In healthcare, *ORA has been used to analyze information use in a public health organization.¹⁷ That analysis produced graphical representations of the organization’s structure and statistical reports on quality of the information network, employees in key positions, status of experienced staff, potential impact of a planned merger, and strengths and weaknesses of the organization. *ORA has also been used to explore multidisciplinary handoffs from the emergency department to the nursing unit.¹⁸

Methods

Setting and Sample:

After obtaining approval from the University of Arizona Investigational Review Board and site approval from the three hospitals, we collected organizational network data from nursing staff on seven units on two different days, chosen to have the least overlap of staff possible, via an Organizational Network Analysis survey adapted from Merrill et al. ¹⁷ Response rates ranged from 70–100%, with Day 1 ranging from 85–100% and Day 2 from 70–100%. Because accurate network analysis depends on high response rates, we report here only the results for Day 1. Patient falls and medication errors (ADEs) data were obtained from Quality Management. Other outcome data were collected from 256 patients via questionnaire at the time of discharge on 2–3 randomly chosen days. Mean respondents per unit were 37; mean response rate per unit was 73%. Outcomes are defined in Table 1.

Table 1.

Patient Outcomes Defined

Outcome	Definition
Total Falls	Falls (with and without injury) per 1000 patient days, averaged over the 3 months for which data were collected
Total ADEs	Total adverse drug events (with and without injury) per 1000 patient days/averaged over 3 months for which data were collected
Symptom Management Difference	% of patients meeting a goal of a 1 point increase in their ability to manage their symptoms from admission to discharge
Symptom Management Capacity	% of patients meeting a goal of a 1 point increase (from admission to discharge) in the ratio of symptom management to symptom distress.
Simple Self Care Management	% of patients meeting a goal of 7.5 or higher on a Self Care scale (6 items related to simple self care (e.g., activities of daily living)
Complex Self Care Management	% of patients meeting a goal of 7.5 or higher on a Self Care scale (4 items related to complex self care; e.g., adjusting care regime to symptom changes)
General Satisfaction with Nursing Care	% of patients meeting goal of score > 3 on Well Cared For Scale (6 items) that measures perception of staff competency & knowledge, also appropriateness of care
Satisfaction with Caring	% of patients meeting goal of score > 3 on Well Cared For Scale (5 items) that measures listening & understanding
Satisfaction with Individual Needs being met	% of patients meeting goal of score > 3 on Well Cared For Scale (4 items) in terms of individual needs being met

Design and Procedures

The study used a descriptive design. With their nurse manager’s permission, we contacted staff via email, flyers and staff meetings and informed them of the study. Questionnaires were distributed to nursing staff on day and night shifts. The questionnaires asked staff to put a check mark by each staff member they interacted with on the last shift they worked. They then were asked to record how frequently they discussed patient care, got information from or gave information to each of the individuals they checked using a scale from 0–4 (never to constantly). The questionnaire listed only the names of staff on their current shifts and the previous and subsequent shift (to cover handoffs). The names were listed on a tear-off sheet so that researchers did not see the names, but only codes such as RN23, or PCT10 when the form was returned. Questionnaires were collected on the units by research staff. Nursing staff received a $20 Barnes & Noble gift card for completing the survey, which also included several demographic questions related to shift, education, etc.

Analysis

Data were entered into an excel spreadsheet and converted to .CSV format for use in *ORA. Separate files were created for communication and demographics. When the files were entered into *ORA, they were converted by *ORA into meta-matrices for analysis purposes.

For this study, we used only 12 of *ORA’s 80+ metrics because many of the metrics are similar and therefore redundant. The metrics we used are shown in Table 2 with their definitions. We computed Spearman rank order correlations between *ORA metrics and the outcome variables. Correlations were done separately for frequency ratings of: (1) getting information from (got), (2) giving information to (gave) and (3) discussing patient care (discuss) with other staff. We set p < .10 as an acceptable level of statistical significance because of the small sample size (n = 7) and the exploratory nature of the study. Specifically, we were looking for identifiable communication patterns and this level of significance was determined to be most useful for that purpose.

Table 2.

*ORA Metrics Defined

*ORA Metric	Definition
Clustering Coefficient (Clust. .Coef.)	Extent to which there are small clusters (cliques).
Component Count Strong (Comp. Ct. .Strong)	Number of strongly connected components in a network
Component Count Weak	Number of weakly connected components in a network
Density	Ratio of actual connections between individuals to possible connections for a network.
Diffusion	How fast information travels through the network.
Hierarchy	Degree to which the network has a purely hierarchical structure.
Isolates	Number of nodes (individuals) that have no connections or links.
Network Centralization In Degree (Cent. In.)	Number of connections coming into individual nodes
Network Centralization Out Degree (Cent. Out)	Number of connections out of individual nodes (if higher, more individual connections)
Betweenness Centrality (Between)	Number of times that connections must pass through a single individual to be connected, (i.e., which person is the most central to the network as a whole).
Eigenvector Centrality (Eigenvector)	Measure of node connections to highly connected people
Simmelian Ties (Simmelian)	Number of strong ties embedded in cliques

Results

Table 3 displays only the statistically significant correlations. Metrics that did not have significant correlations with any outcome variable are not shown. No significant correlations were obtained with either of the self care outcomes.

Table 3.

Significant Non-parametric Correlations of *ORA Metrics with Patient Outcomes (p < .10). (See Tables 1 and 2 for definitions of variables and abbreviations.)

Outcome/					*ORA Metric
Question	Clust Coef	Comp Ct Strong	Density	Diffusion	Hierarchy	Between	Cent In	Cent Out	Eigenvector Simmelian
Symptom Difference - % Patients with 1 point increase from Admission to Discharge
Discuss		−0.74	0.70	0.72			0.73	0.72
Got		−0.84	0.74	0.78			0.76	0.78
Gave		−0.81		0.79	−0.81
Symptom Capacity - % Patients with 1 point increase from Admission to Discharge
Discuss
Got				0.68			0.68
Gave
Well Cared for - General
Discuss		−0.89		0.82	−0.86			0.75	0.71
Got
Gave
Well Cared For - Caring
Discuss	0.68	−0.68	0.71		−0.68		0.75	0.82			0.82
Got	0.71		0.75				0.71	0.75
Gave			0.82				0.82	0.71
Well Cared For - Individual Care
Discuss
Got
Gave						−0.70
Total ADEs per 1000 Patient Days
Discuss						0.69
Got
Gave
ADEs w/injury
Discuss		0.70	−0.93	−0.85	0.85		−0.93	−0.85	−0.85		−0.82
Got		0.75	−0.93		0.78				−0.96
Gave		0.78	−0.82	−0.90	0.89		−0.82	−0.78	−0.96
Total Falls per 1000 Patient Days
Discuss	0.86
Got	0.71	−0.74		0.71	−0.68		0.75	0.82			0.93
Gave		−0.75					0.79				0.89

Higher network density, diffusion, and more links (centrality in and out) were positively associated with a greater symptom management difference from admission to discharge; but more strongly connected people were negatively correlated with this variable, perhaps suggesting that too much communication among staff be a deterrent to teaching patients self management. Symptom management capacity was positively related to diffusion and the number of connections coming to individual staff, but only for “got information”.

Well cared for – general showed negative correlations with component count strong and hierarchy and positive correlations with diffusion, centrality out degree and eigenvector centrality—but, only for the discussing patient care question.

Patients’ perceptions of a caring staff were positively associated with clustering coefficient, density, and centrality in and out degree, suggesting that more staff communication, as viewed by patients, made a difference. Clustering coefficients are high when small groups tend to merge into one cohesive group Patients may detect this.

More ADEs with injury were associated with more strong links among staff and with more hierarchical (top-down) communication. Fewer ADEs with injury were associated with higher density, diffusion, eigenvector centrality and centrality in and out degree (i.e., with more links). This may be due to better staffing or more effective team communication. However, the number of ADEs with injury is very low, so we are reluctant to speculate further.

Falls showed a very different pattern. Higher clustering coefficients, faster diffusion of information, and more links among staff were associated with more falls; while greater component strength (stronger links) and more hierarchical (unidirectional) communication were associated with fewer falls. Perhaps too much staff communication detracts from individual patient observation.

Conclusion

We found *ORA to be a valuable tool for identifying and analyzing communication network patterns on a nursing unit. Using *ORA, we were able to identify statistically significant relationships between *ORA metrics and the majority of our key safety, satisfaction, and quality outcomes measures. Self care was an exception in this study. Perhaps “self care” is more an individual patient function and less sensitive to staff communication patterns.

Distinct communication patterns were obtained for different outcome variables, although there was considerable similarity across the three questions. That similarity may be partly a function of responder fatigue because the questionnaire was quite lengthy (taking about 45 minutes to complete). The distinct patterns for the various outcomes suggests that there may be no single communication pattern that will facilitate all patient outcomes, which may not come as good news to managers.

This study focused solely on nursing communication. However, we did collect some data on the interaction of nursing staff with other professionals; therefore, our future studies will explore some aspects of multiprofessional communication as well.

The extent to which our results, gleaned from data collected on a single day in a homogeneous sample (7 medical-surgical units in 3 magnet hospitals), can be generalized remains to be determined (especially given the number of number of pairwise comparisons conducted without statistical correction). Still the results of this exploratory study highlight the importance of organizational communication data as a source of information for improving outcomes.