Quantifying the Qualitative with Epistemic Network Analysis: A Human Factors Case Study of Task-Allocation Communication in a Primary Care Team

Abstract

Health care is fundamentally about people, and therefore, engineering approaches for studying healthcare systems must consider the perspective, concepts and methods offered by the human factors and ergonomics (HFE) discipline. HFE analysis is often qualitative to provide in-depth description of work systems and processes. To deepen our understanding of care processes, we propose the next level of analysis, i.e. quantification of qualitative data. Here, we describe epistemic network analysis (ENA) as a novel method to quantify qualitative data and present a case study applying ENA to assess communication in a primary care team. One high-performing primary care team consisting of a physician, nurse, medical assistant and unit clerk was observed for 15 hours. We analyzed task-allocation communications and identified the sender, receiver, synchronicity and acceptance. We used logistic regression and ENA to evaluate sender, receiver and synchronicity impact on task acceptance. The physician and unit clerk were most successful allocating tasks. Future work should consider the role of synchronous, interruptive communication as potentially useful in time-critical tasks and further investigate the role of the unit clerk. HFE researchers should consider ENA as a tool to expand and deepen their understanding of care processes by quantifying qualitative data.

Introduction

Health care is fundamentally about people (Carayon, Alyousef & Xie, 2012), and has many quality, efficiency and safety issues (Institute of Medicine Committee on Quality of Health Care in America, 2001; Kohn, Corrigan & Donaldson, 1999; Reid et al., 2005). Therefore, engineering approaches to analyzing and redesigning care processes to improve healthcare quality and patient safety must consider the human factors and ergonomics (HFE) perspective (Gurses, Ozok & Pronovost, 2011). Wilson (2014) notes that HFE research studying complex sociotechnical systems should be conducted in the field. As such, HFE research frequently utilizes qualitative methods and data (Åsberg, Hummerdal & Dekker, 2011). In order to proceed to the next level of analysis and achieve a deeper understanding of care processes, it is beneficial to quantify qualitative data (Carayon et al., 2015; Creswell et al., 2011; Hignett & Wilson, 2004).

Quantification of qualitative data is one type of mixed methods research, which Johnson et al. (2007) defined as the combination of qualitative and quantitative research elements “for the broad purposes of breadth and depth of understanding and corroboration” (p. 123). Carayon and colleagues’ (2015) systematic literature review found 58 mixed methods studies on HFE in health care between 2002 and 2013. While only twenty of the 58 studies (34%) mixed qualitative and quantitative data at the data analysis phase, Carayon et al. (2015) call for continued investigation into how to integrate quantitative and qualitative data and ensure consideration for the HFE perspective in healthcare system improvement. Ten of the studies reviewed by Carayon and colleagues accomplished mixing at the data analysis stage by quantifying qualitative data. In this paper, we describe epistemic network analysis (ENA) – a novel, useful method to quantify qualitative data – and demonstrate its application in a human factors case study on communication in a primary care team.

Epistemic Network Analysis

ENA is a new method for the quantification of qualitative data by combining principles from social network analysis (SNA) and discourse analysis to look for relationships between and patterns in discourse elements (i.e. what people say and do). Researchers can apply ENA to a variety of data types, such as transcripts (e.g. interviews, focus groups and online chats), and observational data. Researchers primarily apply SNA to survey data and typically do not focus on relationships and patterns between discourse elements identified in qualitative analysis. Additionally, the networks produced by ENA typically have fewer nodes than SNA networks, which can simplify the visual interpretation of the networks. See Andrist et al. (2015) for further discussion of the similarities and differences between ENA and SNA.

ENA is based on high-quality qualitative data that are collected using rigorous research methods, which ensures that the researcher can generate thick description and minimize bias (Glesne, 2016). These data are segmented according to principles from discourse analysis: first into lines, the smallest unit of data, which are then grouped into stanzas, or related lines (Gee, 2014). Each line of data is coded according to a codebook; the codebook can be developed using grounded theory and inductive thematic analysis, a framework established a priori, et cetera, as long as it is developed using rigorous qualitative research methods. Then, using mathematics similar to SNA, singular value decomposition (SVD) and principal component analysis (PCA), the relationships between coded elements are calculated and depicted graphically. For a detailed description of the mathematics of ENA, please see Shaffer et al. (2009), Shaffer (2014), Andrist et al. (2015), Shaffer et al. (2016) and Shaffer (2017). The graphical depiction of the relationships is a network graph, with node size representing frequency of code occurrence and the thickness of lines depicting strength of relationship indicated by code co-occurrence. Both individual and mean networks can be used descriptively and compared to identify difference with or without further statistical analysis (Orrill & Shaffer, 2012; Rupp et al., 2009; Rupp et al., 2010; Shaffer, 2014; Shaffer et al., 2016; Shaffer et al., 2009).

ENA was originally developed in the learning environment to measure relationships between elements of professional expertise, such as in urban and regional planning and engineering (Bagley & Shaffer, 2015a, 2015b; Chesler et al., 2015; Shaffer et al., 2016; Siebert-Evenstone et al., 2016). It has been subsequently used in a variety of fields and settings; for example, to model surgeons’ communication in the operating room (D’Angelo et al., In preparation), neuronal coactivation in functional MRI data (Collier, 2015), social gaze coordination (Andrist et al., 2015), surgical error management (Ruis et al., Under review) and semantics in historical documents (Ruis, 2016). Some of these applications, such as the modeling of gaze behaviors between two conversation participants (Andrist et al., 2015) and neuronal coactivation (Collier, 2015), demonstrate that ENA can focus on network structure, without examining the content of qualitative data. In this sense, ENA can be a statistical analysis technique. However, ENA can also be used for qualitative content analysis; for example, D’Angelo and colleagues (D’Angelo et al., In preparation) use ENA to analyze conversation of surgical trainees during simulation of a laparoscopic surgery, which allowed them to model and evaluate how surgical residents integrate elements important for successful operative performance. Surgical error identification and management was also studied by conducting a qualitative content analysis of intraoperative speech in a simulated laparoscopic hernia repair to show that integrated error management skills predicted outcome scores (Ruis et al., Under review). Additionally, ENA has recently been used to provide quantitative models of group learning in an engineering virtual internship, and was able to identify individual contributions to team conversations (Siebert-Evenstone et al., 2016), demonstrating potential to study teams and team interactions.

Background for Case Study on Primary Care Teamwork

Effective teams can improve patient safety (Baker et al., 2012). Team leadership, including the management and distribution of team tasks, is key to effective team performance and, consequently, to patient safety (Thomas, Sexton & Helmreich, 2004; Zaccaro, Rittman & Marks, 2002). In this case study, we focus on how primary care teams communicate to allocate tasks, with the Shannon-Weaver model of 1948 as our framework, as it is the most impactful communication model (Hollnagel & Woods, 2005). In this framework, a sender encodes a message that is transmitted to a receiver (Shannon, 1948). The Shannon-Weaver model is often extended to include a response from the receiver to the sender, forming a complete, dyadic communication (Dayton & Henriksen, 2007; Hollnagel & Woods, 2005). This study focuses on dyadic communication between two parties (Panko & Kinney, 1992) to assign tasks in a primary care team, which we refer to as task-allocation communication. Although the Shannon-Weaver model focused on telephonic communication, message transmission can occur via other modes as well, such as face-to-face speech, email and written note. Researchers categorize these communication modes as synchronous or asynchronous. Synchronous communication requires simultaneous interaction between two parties, and asynchronous communication allows the recipient to handle the message when it is convenient (Parker & Coiera, 2000).

Several studies have focused on communication in primary care (Mundt et al., 2016; Mundt et al., 2015; O’Malley et al., 2015). In a study of team communication networks and quality of primary care, Mundt and colleagues (2015) asked team members with whom and how frequently they communicate to associate these measures with quality and cost of care of patients with cardiovascular disease. Using SNA, they found teams demonstrating dense communication networks are associated with better care provided at lower cost. Mundt et al. (2016) incorporated additional data on team climate and distinguished between synchronous and asynchronous communication in a follow-up analysis focusing on patients with diabetes. They found that primary care teams with more frequent face-to-face communication were associated with fewer urgent care visits and hospitalizations, as well as lower healthcare costs; a measure of team climate mediated this relationship. More frequent electronic health record (EHR) communication was not associated with any of those quality measures, but was associated with a lower likelihood of optimal blood pressure control. Further, in a study focusing on how EHRs can support teamwork in primary care, O’Malley and colleagues (2015) found that EHRs facilitated task allocation and communication. While EHRs were found to support asynchronous, written communication via instant messaging, notes and routing functionality to assign tasks, study participants did indicate that these functions could replace synchronous, in-person communication during the day. These findings beg the question of whether synchronous or asynchronous communication is more effective for task allocation.

Synchronous communication can be interruptive since the receiver must stop their task to participate in the communication (Parker & Coiera, 2000). Researchers often view interruptions negatively and associate them with errors (Brixey et al., 2010; Coiera & Tombs, 1998; Institute of Medicine Committee on Quality of Health Care in America, 2001; Werner & Holden, 2015). However, this research considers only the viewpoint of the receiver and focuses on eliminating interruptions without considering the complexities of team communication (Rivera, 2014), including the perspective of the interruption sender.

When considering the perspective of the interruption sender, Rivera (2014) found the sender considers functional role of both sender and receiver before interrupting, in addition to other factors. This correlates with Magee’s (2009) finding that organizational rank, often indicated by job title, is correlated with power, or the ability to influence the behavior of others (Edmondson, 2003). Magee (2009) argues the more power one possesses relative to others, the more influence one has over them. Therefore, higher-ranking individuals in an organization typically have power over lower-ranking employees, and hence more influence over their behavior and actions, e.g., the ability to interrupt. This power hierarchy is pervasive in health care, with major power differential between physician and nonclinical staff (Miller & Cohen-Katz, 2010; Nutting, Crabtree & McDaniel, 2012). While current research fails to investigate the power differential between physician and nonclinical staff, we have a clearer understanding of the physician-nurse power differential (Manias & Street, 2001; McDonald, Jayasuriya & Harris, 2012; Stein, 1967). Further, the medical-dominance paradigm characterizes primary care in particular, which remains exceptionally physician centric (Nutting et al., 2012), with many physicians demonstrating emotional reactivity and abrasiveness (Miller & Cohen-Katz, 2010). This combination results in authoritative leadership behavior and an immense power differential between physician and nonclinical staff, negatively impacting team performance as respectful interactions are key elements of effective, efficient teams (Ghorob & Bodenheimer, 2015; Lanham et al., 2009; Miller & Cohen-Katz, 2010; Nutting et al., 2012). For example, Solimeo and colleagues (2015) report that primary care teams were unable to share tasks beyond traditional role structures in the Veteran’s Health Administration because they remained so fragmented by hierarchy. Therefore, it is especially important to consider organizational rank, hierarchy and power when studying teamwork in primary care.

Objective

The objective of this study is to demonstrate the application of ENA to quantify qualitative data in an HFE study on communication in primary care teams. Our case study integrates the concept of power when investigating communication and asks how communication synchronicity, and sender and receiver roles impact successful task allocation in a primary care team.

Methods

Setting

The study site was an internal medicine clinic, recognized as a high performing patient-centered medical home (PCMH), which serves approximately 14,000 patients in a medium-sized Midwestern city. The PCMH model is the predominant primary care team model, complete with national recognition programs. PCMH consists of a team providing safe, high-quality, evidence-based medicine while coordinating care and promoting patient engagement (Wetterneck et al., 2012). PCMH teams typically include a provider (physician, nurse practitioner, or physician assistant), 1–2 medical assistants (or similar), and sometimes a registered nurse or scheduler at check-in (Ghorob & Bodenheimer, 2015). This study focused on one four-person team in an internal medicine primary care clinic. This team included a physician, registered nurse, medical assistant (MA) and unit clerk, and all team members had been in their role at least two years.

Data Collection

We employed ethnographic observation methods. One researcher (ARW) observed the team for 15 hours over 7 workdays between July and October 2015 and took detailed, written notes focused on capturing work system elements (Carayon et al., 2006) and communication interactions. During the observations, the researcher sat or stood behind the participants as they completed their work process as usual, asking clarifying questions as possible without interrupting patient interactions or care. The researcher transcribed observation notes to electronic form immediately following the observation. The health system IRB deferred to university IRB, which approved this study.

Data Analysis

First, each transcription was segmented into lines, the smallest unit of analysis. We considered each complete sender/receiver communication and response pair as a single line (see table 1 for examples). Each line was then coded to characterize the structure of task-allocation communication; the coding structure was refined iteratively with input from all researchers. We identified the job role of the sender and receiver of the task-allocation communication, and whether that communication was synchronous or asynchronous. Finally, we identified if the task allocation was immediately accepted or not, which could indicate the task was rejected, ignored or accepted later, perhaps unobserved. The code definitions follow:

Sender: The role initiating the task-allocation communication.

Receiver: The role responding to the task-allocation communication.

Synchronicity: If the communication involved simultaneous interaction between parties (or not).

Response: If the receiver responded to the task-allocation communication by beginning to do what was requested (or not).

The sender and receiver role codes had four possible values: physician, nurse, MA and unit clerk. The synchronicity code could be synchronous or asynchronous (1 or 0, respectively). The response was coded as accepted (1) or not accepted (0). Table 1 shows an example of two subsequent coded utterances. We considered the response as the dependent variable and the sender, receiver and synchronicity as independent variables.

Table 1.

Example coding of observed task-allocation communications

Task Allocations	Sender	Receiver	Synchronicity	Response
[Physician] sent a patient’s message to [nurse] with a note to put a prior authorization in for 3 pills this month.	Physician	Nurse	Asynchronous	Not accepted
[Nurse] sent this to [unit clerk], who started entering it.	Nurse	Unit clerk	Asynchronous	Accepted

Note: This example shows the application of codes to two complete utterances.

To ensure the rigor of the qualitative analysis, therefore meeting the criteria of credibility, dependability and confirmability (Devers, 1999), we used the following strategies:

• Soliciting feedback from two group members (one physician and one physician assistant trainee) on the credibility of interpretation and findings (credibility)

• Asking three other researchers with training in qualitative research and analysis to do a skeptical review of our analysis (dependability, confirmability);

• Keeping a reflective journal of both data collection (e.g., personal characteristics, feelings and biases) and analysis (e.g., changes to coding structure, decisions made during analysis), documenting how researchers influenced data collection and analysis (dependability, confirmability) (Devers, 1999; Mays & Pope, 2000).

In addition, we also present direct quotations, i.e., original evidence, to show the reader the relationship between evidence and interpretation (Mays & Pope, 1995).

We then conducted a series of regression analyses to develop a model predicting the response to each task allocation (dependent variable) based on synchronicity, sender role and receiver role (independent variables) using RStudio© (RStudio Team, 2015). We used logistic regression to account for the binary response variable (immediately accepted or not) and a link logit function for all predictors due to nonnormally distributed variables. We represented the sender and receiver roles as categorical variables, with the physician as the reference role because all primary care teams include a physician, who is considered to be the highest-ranking member and leader of the team. The synchronicity variable was also categorical, with the asynchronous mode as the reference. We developed the reported logistic regression model by iteratively including independent variables in subsequent models after individual significance. In other words, we first developed models with only one independent variable to verify the significance. As all variables were significant on their own, our reported model includes all three independent variables; however, interactions are not included because they resulted in over specified models.

Subsequently, ENA (Epistemic Games Group, 2016) was used to analyze and visualize the communication network representing each combination of accepted versus unaccepted and synchronous versus asynchronous task assignments, for a total of four networks, within the team. The data, including each coded line, were uploaded to ENA in a .csv file. In our analysis, each line was a complete stanza. The ENA software develops an adjacency matrix for each stanza with the codes co-occurring in that stanza. The adjacency networks for all the stanzas are summed, and the resulting vector is plotted in a high-dimensional space. These vectors are normalized, and then a dimensional reduction is performed, typically a single value decomposition, which is similar to a principal components analysis (PCA). ENA uses a rigid body rotation to avoid rescaling the data and preserve meaningful relationships between the codes. The first (x) axis shown in an ENA network graph is the dimension that accounts for the highest percent of variance explained by a dimension, and the second (y) axis is a dimension orthogonal to the first that explains the next highest percentage of variance; these percentages are shown by each axis. For a detailed description of the ENA mathematics, please see Shaffer et al. (2009), Shaffer (2014) and Andrist et al. (2015).

The analysis ultimately produces a graphical representation of the relationships between codes by projecting the network. On a network graph, the node size represents the frequency that code occurred in the dataset. The thickness of the lines between the nodes shows the weights of those connections, where thicker indicates stronger relationships indicated by more frequent co-occurrence. Individuals’ networks can be compared and contrasted. These networks can be further described quantitatively, facilitating the comparison of more than two or three individual networks: the mean network of several individuals can be shown, (mean) networks can be subtracted to highlight differences between networks, the centroid can be calculated to summarize the network into a single point, confidence intervals of the centroid can be developed and t-tests between networks (i.e. to compare the networks of individuals or groups) can be conducted (Orrill & Shaffer, 2012; Rupp et al., 2009; Rupp et al., 2010; Shaffer, 2014; Shaffer et al., 2009). On these centroid graphs, the larger, square points are the centroid of the network, with the thin black lines representing the 95% confidence interval on the location of the centroid. We developed centroid graphs, showing the centroid for each sender role, for each of the four networks as well. All of these analyses provide further, quantitative evidence to support claims based in qualitative data, which is often expected, or even required, to substantiate and disseminate important findings. Additionally, the ENA software makes it easy to find the data associated with connections between nodes – when a line is selected, a pop-up window with the data resulting in that line, i.e. the stanzas those codes co-occur in, appears; this facilitates qualitative interpretation of the quantitative data.

Results

Description and Examples of Task-Allocation Communication

The observing researcher documented 83 task-allocation communications. An example of synchronous task-allocation communication is this face-to-face request from the nurse to the unit clerk:

Nurse: “Can I see the protocols?”looking at the unit clerk, who had sent lab orders for her signature

Unit Clerk: nods and begins sorting through stacks of papers looking for protocols

The nurse’s request arises in response to an asynchronous message the unit clerk sent the nurse. Although the nurse phrases it as a question, here she assigns the unit clerk the task of finding paper copies of protocols for her review. The unit clerk affirms nonverbally by nodding, indicating her acceptance, and immediately begins looking for the protocols to give the nurse.

The observing researcher witnessed numerous asynchronous electronic messages requesting permission to order tests or treatment, indicating next steps and drawing attention to important information, ultimately allocating tasks. She noted asynchronous task assignments when one team member placed paperwork at the desk of another to indicate they need to do something, e.g., complete and/or submit that paperwork. Both the unit clerk and MA frequently used this method to assign tasks to the physician.

Sometimes task assignments were not immediately accepted, such this asynchronous task-allocation communication from the MA to the physician:

MA: sends physician an electronic message letting him know first patient is ready and waiting in exam room

Physician: sees message, minimizes it and opens details of another patient’s EKG results

In this instance, the physician read the relatively unassertive task-allocation communication from the MA, informing him that a patient was ready instead of explicitly asking him to leave his office to begin a patient visit. Instead of moving to immediately accept this task, the physician opens another patient chart to review test results. There may be reasons for this delay, such as the physician needing to review the EKG results of the other patient in order to provide safe, high quality care; nonetheless, the physician did not immediately move to see his first patient.

Each member of the team assigned tasks to every other team member, synchronously and asynchronously, and sometimes unsuccessfully. Table 2 shows the percentage of accepted and total task assignments among each pair. The physician accepted 54% task allocations from the unit clerk, but only accepted 14% from the nurse and accepted none from the MA.

Table 2.

Observed Task-Allocation Communication

	Receiver
Sender	Physician	Nurse	MA	Unit Clerk
Physician	-	70% (10)	75% (4)	82% (11)
Nurse	14% (7)	-	100% (3)	83% (18)
MA	0% (2)	0% (2)	-	50% (4)
Unit Clerk	54% (13)	83% (6)	100% (3)	-

Notes: Values displayed as percentage accepted (total observed task-allocation communications)

Figure 1 shows the percentage of accepted task allocations by sender/receiver pair and the synchronicity of the communication. Assignments made by the MA were mostly unaccepted, while assignments from the physician, even asynchronous, were generally accepted. Additionally, synchronous task assignments were more frequently accepted. These initial findings demonstrate emergent patterns prior to regression analysis.

Figure 1.

Percentage of accepted task assignments between sender and receiver by communication mode.

Note: The bottom, horizontal text indicates the sender, and the rotated, vertical text indicates the receiver. Task assignments between specific sender/receiver pairs seem to be more frequently accepted. Further, synchronously communicated task assignments seem to be more frequently accepted.

Regression Analysis Results

All of the independent variables were significant predictors of the acceptance of task assignments and were all included in the reported logistic regression model (shown in table 3). The AIC of this model is 86.702 and McFadden’s R² estimate is 0.3337. Receivers were 7.63 times more likely to accept task assignments when the communication was synchronous instead of asynchronous. Receivers were about 2 times less likely to accept an assignment from the nurse than the physician (0.45 OR) and even less likely to accept an assignment from the MA than the physician (0.04 OR), but were approximately twice as likely to accept a task from the unit clerk than the physician. The nurse was approximately 4.5 times more likely, the MA 14 times more likely and the unit clerk to nearly 17 times more likely to accept an assignment than the physician.

Table 3.

OR and 95% CI for the associations between task acceptance, communication synchronicity, sender and receiver (n=83)^*

	Task Allocation Acceptance
	Model I	Model II	Model III	Model IV	Model IV	Model IV	Model IV
Model fit (AIC)	95.717	107.24	100.36	90.63	92.431	95.038	86.702
Communication synchronicity
Asynchronous	Referent	-	-	Referent	Referent	-	Referent
Synchronous	6.45 (2.42 to 18.29)	-	-	11.42 (0.53 to 4.31)	5.42 (1.88 to 16.77)	-	7.63 (2.17 to 31.17)
Sender
Physician	-	Referent	-	Referent	-	Referent	Referent
Nurse	-	0.67 (0.19 to 2.22)	-	0.30 (0.06 to 1.23)	-	1.01 (0.19 to 5.28)	0.45 (0.07 to 2.68)
MA	-	0.11 (0.01 to 0.59)	-	0.04 (0.00 to 0.27)	-	0.13 (0.01 to 0.87)	0.04 (0.00 to 0.38)
Unit clerk	-	0.68 (0.18 to 2.45)	-	0.29 (0.06 to 1.32)	-	5.06 (0.80 to 48.15)	1.91 (0.23 to 20.01)
Receiver
Physician	-	-	Referent	-	Referent	Referent	Referent
Nurse	-	-	3.50 (0.98 to 13.75)	-	2.93 (0.73 to 12.77)	7.99 (1.31 to 76.02)	4.50 (0.64 to 45.15)
MA	-	-	15.75 (2.34 to 319.10)	-	9.21 (1.21 to 196.16)	31.41 (3.37 to 820.19)	14.08 (1.17 to 431.69)
Unit clerk	-	-	6.50 (2.03 to 23.03)	-	5.97 (1.70 to 23.48)	24.96 (4.63 to 209.19)	16.95 (2.56 to 164.82)

Notes: Results are presented as OR (95% CI)

^*Italicized values are significant at p<0.05.

Epistemic Network Analysis Results

Figure 2 shows the communication networks representing the four combinations of synchronicity and acceptance, which graphically displays the connections between sender/receiver pairs. In the synchronous, accepted network (Figure 2a), we see the physician, nurse and unit clerk are more frequent, successful senders of task allocations than the MA (larger nodes). The physician receives task assignments less frequently (smaller node of physician receiver as compared to receiver nodes of nurse, MA and unit clerk). We also see that the unit clerk is the most successful assigning tasks to the physician than other team members, as the line connecting unit clerk sender to physician receiver is the darkest and thickest. In the asynchronous accepted task allocation network (Figure 2b), the unit clerk is the only team member to successfully allocate tasks asynchronously to the physician, while she accepts tasks from both the physician and nurse. The nurse successfully assigns tasks asynchronously to the unit clerk more frequently than the MA (that line is darker and thicker), but the nurse accepts asynchronous task allocations only from the physician. The line connecting physician sender to unit clerk receiver is darker and thicker than the line from physician sender to nurse receiver, indicating the physician successfully allocates tasks asynchronously to the unit clerk more often than the nurse. The MA does not successfully assign any tasks asynchronously, and so the MA sender node does not appear on this graph. Further, when comparing the synchronous (2a) and asynchronous (2b) accepted networks, the asynchronous network has fewer lines between nodes, reflecting fewer sender/receiver pairs with accepted, asynchronous task assignments than accepted, synchronous task assignments.

Figure 2.

ENA communication networks.

Note: Abbreviations in this figure are MD for physician, RN for nurse and UC for unit clerk. Line color and sender node indicate the sender role; receiver node indicates the receiving role. Line thickness and darkness indicates connection strength, i.e. relative frequency of accepted task assignments from that sender to that receiver. Node size indicates frequency the role is a sender or receiver, respectively.

Likewise, Figures 2c and 2d show the synchronous and asynchronous rejected task networks, respectively. The similar thickness of the lines between the unit clerk sender and physician receiver, as well as physician sender and MA receiver in Figure 2c, show that the physician rejects synchronous task allocations from the unit clerk similarly to how the MA rejects tasks from the physician. In Figure 2d, we see that the line connecting physician receiver with nurse sender is the thickest, so the nurse is the least successful assigning tasks asynchronously. When we compare Figures 2c and 2d there are more lines from the physician sender node, indicating that asynchronous task allocations from the physician are rejected more than synchronous ones.

ENA can also be used to summarize networks in single points, or centroids. The location of one of these single point has meaning based on the layout of the network nodes across the x and y dimensions in ENA space. Therefore, these points can be particularly useful when comparing multiple networks at once, with comparisons based simply on the location along the ENA dimensions. For example, Figure 3 shows the centroids of the network of each sender, organized by synchronicity and acceptance as in Figure 2. In each of the four graphs, the unit clerk is located higher on the positive y-axis than the other roles. However, the nurse and MA roles’ centroids are nearer the zero point of the y-axis and the physician centroids are furthest down the y-axis. Combined with the qualitative data, we can interpret the y-axis to represent administrative responsibilities in the positive direction, and clinical responsibilities in the negative direction. Similar logic can then be applied to the x-axis. The physician tends to be the furthest in the positive direction x-axis direction, while the nurse role tends to be the most negative. The MA role falls near the zero point again, while the unit clerk tends toward the positive end with the physician. We could interpret this axis to have some relationship with the difference between nursing and medical knowledge.

Figure 3.

ENA communication network centroids.

Note: Centroid and node color indicate the sender role. The black boxes around each centroid represent 95% confidence intervals.

The ENA software allows for further understanding of quantitative data by easily linking them to the qualitative data. For example, we can use this feature to investigate the types of tasks the unit clerk is able to successfully assign. Some of these include having the physician sign paperwork and the nurse sign orders she had already entered. Figure 4 shows the partial pop-up from the ENA software that links to the qualitative data for the unit clerk assigning the task of signing orders for the nurse. In this instance, the unit clerk asked the nurse to sign orders for a patient (synchronous communication); the nurse said okay, and in order to start completing that task, asked the unit clerk to find the order protocol.

Figure 4.

ENA link to qualitative data.

Note: The ENA software converts special characters, in this case an apostrophe (‘), to <d5> in order to handle them without error.

We can also use the software to easily look at the tasks the physician successfully assigned to the unit clerk synchronously and asynchronously, but for clarity’s sake, we return to showing the observed actions and conversations as earlier in the results section. When the physician wanted to know if the insurance prior authorization for a special treatment for a patient he was seeing then, he chose to communicate synchronously with the unit clerk:

Physician: “Has the [treatment] been approved?”

Unit Clerk: switches screens in EHR and searches for prior authorization in patient record“Yes it has been.”

In this case, the unit clerk immediately switched tasks in the EHR to find the status of the prior authorization, while the physician was still standing at her desk. However, when he wanted to add appointment for a patient later in the week, he sent a secure asynchronous message, which she still immediately started working on.

Unit Clerk: opens new message from physician asking her to fit a patient in later in the week, switches to screen showing physician’s appointment schedule“His schedule is all double booked, so I want to talk to [MA]…if there were openings I would book the appointment without checking with [MA]”walks to MA’s desk and discusses appointment time

Here, the unit clerk opened the message from the physician, in which he assigned her the task of scheduling an appointment. Immediately, she reviewed his schedule for the week and noticed it was already full. So, she decides she needs to talk to the MA about when that appointment could be scheduled.

Discussion

In total, we observed 83 synchronous and asynchronous task-allocation communications, some of which were not accepted. All team members attempted to assign a task to every other team member. The logistic regression analysis showed that sender role, receiver role and communication synchronicity were significant predictors of task acceptance. We then compared ENA network graphs of accepted synchronous and asynchronous communication, which demonstrated the application of ENA to study communication in a primary care team; we also described how these graphs can be interpreted. These graphs show the differences in the accepted synchronous and asynchronous communicate networks. Finally, we showed how the software facilitates returning to the qualitative data in order to interpret ENA graphs .

Communication Implications

Synchronous task assignments were more likely than asynchronous to be accepted. Synchronous communication requires the receiver to halt their current task to respond (Parker & Coiera, 2000), so they have already switched focus and incurred the psychological cost of interruptions. Consequently, it is easier to deal with the interruption instead of resuming the task that was just interrupted (Brixey et al., 2010; Coiera & Tombs, 1998). Rivera (2014) found that, in addition to the role of the interruption sender and receiver, the sender considers the nature of the interruption content, i.e. the urgency, or time-sensitivity and importance, of the content. We see similar behavior in this study, providing evidence from another context to support Rivera’s claim. For example, the physician chose to tell the unit clerk to schedule a special appointment on the same day verbally; however, when an appointment needed to be scheduled later that week, the physician used an asynchronous communication mode. Therefore, interruptions may be more acceptable to assign time-sensitive tasks. Future work studying interruptions should further investigate and incorporate the interrupter’s (i.e. sender) perspective.

In addition to communication synchronicity, sender and receiver roles also influenced task-allocation acceptance. Tasks assigned by the physician were more likely to be accepted, which fits with reports that the medical profession (i.e. the physician) has more power than other professions and is considered the leader of the team (Ghorob & Bodenheimer, 2015; Solimeo et al., 2015). However, the unit clerk was successful at assigning tasks, particularly synchronously. Further, the unit clerk was the most successful assigning tasks to the physician, even though the unit clerk was seemingly lowest in the team hierarchy. The types of tasks she assigned could perhaps explain this, as they were often relatively simple and quick to complete, such as signing forms. Future work should investigate the unit clerk’s role on the team, why her task assignments were so frequently accepted and how this role could be leveraged to improve team performance.

Methodological Implications

This case study demonstrates a successful application of ENA to an HFE research question, thus paving the way for further applications. We used ENA to visualize networks of task-allocation communications by synchronicity and acceptance, demonstrating the viability of using observation data collected from the HFE perspective in ENA. We then showed that these graphs can be used in at least two ways. First, these ENA network graphs can be used to triangulate results from other analyses (e.g. logistic regression) to corroborate previous findings, as we draw similar conclusions based on the ENA networks as the logistic regression. The graphs also contain embedded evidence that can provide further insights beyond those findings. They were used to support findings from logistic regression, simply by providing a visual representation of the data. Additionally, we were able to draw some comparisons based solely on the graphs, e.g., the MA was a less frequent sender of task-allocation communications, while the unit clerk was more successful allocating tasks to the physician, etc. Finally, we demonstrated the use of the ENA software to quickly find qualitative evidence of the quantitative findings, by returning to the qualitative data. Figure 4 showed the way the ENA tool presents the qualitative data. We used this feature to quickly find and compare two observed task allocations from the physician to the unit clerk, one synchronous and one asynchronous. In both instances, the unit clerk immediately started completing the task the physician gave her, despite the different communication modes. The easy linkage between the quantitative and qualitative data helps with data interpretation.

Despite this success, we would be remiss if we did not note that this application of ENA to structural communication analysis is relatively simple and does not showcase all of the potential of ENA as a conceptual tool. In the case study, we essentially developed our coding structure a priori based on a conceptual model of communication, although we did refine it after reviewing the data. We also only considered communication structure, and used ENA as a statistical analysis technique while ignoring communication content. The ENA algorithm was then applied to visualize the structural communication models, without considering the content of the communication.

However, this is not the only way to apply ENA. Researchers can utilize coding schemes, developed inductively via thematic content analysis and a grounded theory approach (Robson, 2011) or theoretical models instead, to study the content of interactions; essentially, ENA can be a tool to apply network analysis techniques to content analysis. This could be particularly useful when considering transcripts of interviews, focus groups and/or audio- or video-recorded interactions. In this type of application, the dimensions of the network graph may further aid interpretation, with the placement of nodes along the x- and y-axis being of more interest, such as done in the interpretation of ENA results in the coordination study by Andrist et al. (2015) and in our Figure 3. Another possibility is to compare individual networks and/or mean networks representing groups. For example, an extension of this study could develop and compare the communication networks of several primary care teams. Similarly, researchers can compare networks based on a coding structure emerging from a grounded theory approach across groups of individuals (e.g. members of different professions). Further, ENA has been used to understand group interactions as well as individuals’ contributions to those group interactions, as described by Siebert-Evenstone et al. (2016), showing promise for understanding team interaction. In this type of application, the content of team interactions is analyzed; one example of an extension of this study taking this approach would be to note the kinds of tasks that were assigned (and perhaps the tasks those assignments interrupted), which would allow the incorporation of clinical priority in analysis. As noted previously, ENA has been applied in a variety of fields to study a variety of topics, indicating its flexibility and applicability to a variety of problems. Many opportunities for future application in healthcare systems engineering exist beyond those mentioned here.

Because ENA is still relatively cutting-edge in the field it was developed, we have an opportunity as healthcare systems engineers to further contribute to the development of this method by incorporating our knowledge and experience in mixing qualitative and quantitative data to expand its usefulness and applicability to fields outside the learning sciences. However, ENA is just one possible method to quantify qualitative data. For example, Jylhä et al. (2011) quantified qualitative data when they reported the frequencies of adverse drug events from patient complaints and statements. Johnson and Turley (2006) coded transcripts physicians’ and nurses’ case summaries after they reviewed a medical record, and then used ANOVA to compare physicians’ and nurses’ clinical understanding of the cases. Wu et al. (2008) quantified video recorded activities of usability testing of a mobile electronic health record. These are all useful and meaningful ways to quantify qualitative data. However, interdisciplinary approaches where HFE researchers work with other disciplines, in our case game scientists, can result in developing and/or introducing new methods. Going forward, HFE researchers and analysts should work with other disciplines (e.g., statistics, operations research, simulation, modeling, etc.) to create novel methods to model healthcare systems and processes.

Limitations

This study focused on a single team, necessarily limiting generalizability of the results beyond this team, and the data collection timeframe was short with a relatively small number of observed task-allocation communications. Our focus on immediate acceptance or rejection does not consider the nuance of delayed acceptance, but was necessary because eventual acceptance of an assignment could occur after the observation period concluded. There may be additional variables influencing task acceptance that we did not account for in our model, such as team member tenure; future work should identify and explore these predictors. This application of ENA to a structural communication analysis does not fully display the utility of this method; however, we purposefully selected a relatively simple application to facilitate the introduction and demonstrate the application of the method. Future work should include more complex analyses to make use of the power of the ENA method.

Conclusion

This study demonstrates a successful, if somewhat limited, application of ENA to an HFE research question. By doing so, it provides empirical, observation-based evidence that synchronicity, sender and receiver were important factors in task allocation acceptance. By quantifying qualitative observation data, we deliver evidence that interruptive, synchronous communication may have a place if used judiciously, providing confirmatory evidence that the physician is the team leader while the unit clerk emerged as an influential team member as well. Providing evidence based on observed communication in the wild rather than sociometric survey data is novel to communication network analysis in health care (Salas, Cooke & Rosen, 2008). ENA expands the repertoire of methods researchers may draw on to quantify qualitative data, and therefore increases our ability to gain deep, meaningful understandings of care processes and shift qualitative analyses to the next level of engineering analysis.

Highlights.

• Engineering healthcare redesign approaches must include human factors perspective.

• Quantifying qualitative data can deepen care process understanding and assessment.

• Epistemic network analysis is a novel, useful method to quantify qualitative data.

• Synchronicity, sender role and receiver role impact task allocation acceptance.