Quantifying the Qualitative: Exploring Epistemic Network Analysis as a Method to Study Work System Interactions

Abstract

Studying interactions faces methodological challenges and existing methods, such as configural diagramming, have limitations. This work demonstrates Epistemic Network Analysis (ENA) as an analytical method to construct configural diagrams. We demonstrated ENA as an analytical tool by applying this method to study dementia caregiver work systems. We conducted 20 semi-structured interviews with caregivers to collect caregiving experiences. Guided by the Patient Work System model, we conducted a directed content analysis to identify work system components and used ENA to study interactions between components. By using ENA to create configural diagrams, we identified five frequently occurring interactions, compared work system configurations of caregivers providing care at home and away from home. Although we were underpowered to determine statistically significant differences, we identified visual and qualitative differences. Our results demonstrate the capability of ENA as an analytical method for studying work system interactions through configural diagramming.

Practitioner Summary

A new methodology, Epistemic Network Analysis (ENA), was presented to better support the study of work system interactions through configural diagramming. ENA was applied to qualitative data to demonstrate the capabilities of this method to construct configural diagrams of the work system. This study successfully demonstrated that ENA can visually represent and describe work system configurations.

1. Introduction

System interactions are a cornerstone of Human Factors/Ergonomics (HF/E) [1–3]. As Wilson (2014) stated “the basic nature of a system is that it consists of interacting parts. This very fundamental view lies at the heart of HF/E approaches and concepts (p.8)” [1]. Interactions involve one component that either influences, reinforces, or exists in presence of either another or multiple other components [1, 4–6]. Interactions can be purposeful, occur simultaneously, and be modified when a change occurs to one of the components in the interaction [4, 7]. The healthcare domain, characterized by fragmentation, the proliferation of new technologies, and the integration of emerging roles of patients and their family caregivers in care processes [3, 6], requires an HF/E approach that accounts for complex system interactions [1, 2, 8–10]. A review of 360 studies on HF/E in the healthcare domain found that most studies focused on individuals [9] rather than system interactions.

A potential contributing factor to the dearth of interaction-focused HF/E studies in the healthcare domain is the methodological challenges associated with analyzing system interactions. For example, studying system interactions usually requires using multiple data collection methods and specialized statistical techniques [9] as well as conducting conceptual and empirical analyses of constructs that exist at the intersection of two or more system levels [11, 12]. The longitudinal and complex nature of this particular method has made it challenging to apply in the dynamic healthcare domain [9].

System models have been proposed that provide a framework for interaction investigation in healthcare [13–15]. One of these models, the Systems Engineering Initiative for Patient Safety (SEIPS), has been used extensively to study healthcare work systems, [8, 16–20]. SEIPS incorporates a person-centered macroergonomic work system model [4, 5, 21] into the Structure-Process-Outcome (SPO) model of healthcare quality [22] to produce a more systematic analysis of the healthcare structure [4, 8, 13]. SEIPS depicts a work system of five interacting components – person(s), tasks, tools and technology, organization, and environment. SEIPS also considers that element interactions can be optimized to balance the limitations of one another [4, 5]. The interacting components produce processes that lead to organizational, healthcare provider, and patient outcomes that feed back into the work system. [4, 13, 23].

More recently, SEIPS was adapted to address the increasing role of patients and caregivers in healthcare delivery and to provide guidance for healthcare-related work system analysis resulting in SEIPS 2.0 [7]. SEIPS 2.0 presents the concept of configuration, which assumes that all system components are networked, that each can interact with one another, and that the focus on interactions is central to understanding the system [7]. Moreover, the identification, description, and modelling of interactions among work system components can be accomplished through configural diagramming. Configural diagramming is the modelling of the active work system components to diagram active interactions among work system components [7, 24–27]. An illustration of configural diagramming can be found in Figure 2 of Holden et al., (2013) [7]. Configural diagramming has several potentially beneficial uses for work system analysis such as the identification of relevant work system components, their interactions, and the associated facilitators and barriers within and across individual work systems [28]. As such, configural diagramming can also be used to identify interactions across multiple work systems and boundaries [28]. Additionally, configural diagramming can be used to compare work system interactions for two different processes, assess differences within systems, and examine changes in system configurations over time [7].

Figure 2:

ENA summary diagram for all 20 caregivers interviewed

Figure 2 Alt-text: ENA summary diagram for all twenty caregivers. The average point (centroid) is located just slightly to the right and up of the origin.

*Note: Green square is the plotted point mean, green circles are plotted points for the participant models, and black circle dots represent the codes

To realize the benefits of configural diagramming, there is a need for analytical tools that can facilitate the construction of configural diagrams [7, 27, 28]. Useful analytical tools would guide the identification of interactions between work system components as well as facilitate the quantitative representation of the influence of interactions on the process being studied. For example, Holden et al (2013) suggest that the spheres in a configural diagram represent the work system component, the size of the spheres reflect the level of influence of the component (i.e., larger spheres indicate more influence), and the arrows represent the interactions among the spheres [7]. An analytical tool that can support the identification of the level of influence of individual work system components (i.e., size of sphere) and the existence of component interactions would be critical in diagramming these configurations. Further, Hay et al (2020) created configural diagrams using a thematic analysis, guided by SEIPS 2.0, to draw the configuration of a system to facilitate studying a work system re-design for diagnosing rare diseases. Based on their results, the authors highlighted a remaining need for analytical tools that could quantify the level of influence of each component in the work system [27]. Thus, to have maximum utility, an analytical tool for configural diagramming should provide a visual diagram of system interactions that represents the level of influence of components on interactions while simultaneously offering qualitative and quantitative avenues to analyze the nature of the interactions and their potential influence on the process being studied. Therefore, we propose Epistemic Network Analysis (ENA) as an analytical tool to perform configural diagramming.

Table 1 is provided below to summarize the key concepts described in this introduction and their role in the present study.

Table 1:

Summative table describing key concepts and their role within this study

Concept	Description	Role in the study	Relevance to study
Systems Engineering Initiative for Patient Safety (SEIPS) [13]	Macroergnomic work system model designed for healthcare systems	Theoretical Framework	Representation of work system components and their structure informed by interactions between components
SEIPS 2.0 [7]	Adaptation of SEIPS to capture advancements in the field of macroergonomics	Theoretical Framework	The concept of configuration is applied to the healthcare domain to inform interactions between work system components
Patient work system model (PWS) [6, 47]	Adaptation of SEIPS 2.0 to incorporate the role of the patient and informal caregivers into the care process	Theoretical Framework	Application of work systems analysis to informal caregiving for people with dementia
Configural Diagramming [7, 24–27]	A method used to identify, describe, and model the active work system components and diagram interactions among those components	Analysis Method	This method has documented limitations that can be addressed to more effectively study interactions within the work system
Epistemic Network Analysis (ENA) [29–35]	A qualitative data analysis tool that quantifies interactions, generates visual representations of those interactions, and offers the ability to quantitatively and visually compare visual representations of interactions.	Analysis Method	Our proposed method that can model interactions similar to configural diagramming, while offering capabilities that can address the limitations posed by configural diagramming

1.1. Epistemic Network Analysis

Epistemic Network Analysis (ENA) is a qualitative data analysis tool that builds from social network analysis techniques to quantify interactions in qualitative data [29–32]. ENA generates network graphs which are visual representations of the interactions among codes found in qualitative data to depict the structure and strength of those interactions. The network graphs allow interactions to be visually analyzed and compared to identify and describe differences without the use of quantitative measures [29, 30, 32–34]. ENA also offers the functionality of statistically comparing network graphs through summary statistics to quantitatively describe differences [29, 35]. Finally, ENA provides the qualitative data that generated the connections among codes to facilitate additional and more in-depth qualitative analysis [32]. ENA was developed in the education psychology field and has recently been used in the field of HF/E to study communication structures within primary care teams [32].

The ENA visual representation of interactions as a network graph provides an analytical tool for conducting configural diagramming through the following mechanisms: 1) providing a network graph that represents components and their interactions; 2) potential for inferences based on element size and locations in the network graph; 2) ability to understand the context of these interactions by referencing the qualitative data used to construct the network graph; and 3) ability to perform quantitative comparisons between network graphs.

1.2. Dementia Caregiving

We explored the context of informal caregiving for people living with dementia (PLWD) as a case study for demonstrating the utility of ENA as a tool to conduct configural diagramming for work system interaction analysis. Informal caregivers provide care for an estimated 43.8 million PLWD globally [36]. Informal caregivers (henceforth: caregivers) are defined as unpaid, non-professional individuals (family, friends) who voluntarily provide care to the PLWD [37, 38]. Caregiving for PLWD is complex due to prevalence of co-morbidities [39], limited caregiver support [38], and the challenges associated with the management of behavioral and psychological symptoms [40–42]. As a result, caregivers may experience negative outcomes such as burden, stress, and burnout [39, 40], all of which suggest that the work system is not designed to support dementia caregiving.

Caregivers perform work-like tasks in caring for PLWD, [43] that have been conceptualized as patient work [44–46]. Patient work is performed in and influenced by the Patient Work System (PWS). The PWS model adapted SEIPS 2.0 to focus on patient work performed outside of traditional clinical settings (e.g., the home) [6, 47]. Similar to SEIPS 2.0, the PWS model depicts a structured work system of interacting components (Person(s), Task(s), Tools, and Physical-Spatial, Social-Cultural, and Organizational Contexts) that comprise processes and produce outcomes.

1.3. Research Objectives

Thus, our objective was to use the domain of dementia caregiving as a case study to examine the usefulness of ENA as an analytical tool for conducting configural diagramming. To achieve this objective, we used ENA to conduct configural diagramming with the following objectives:

1. Identify and visually represent the interactions among PWS components for caregiving for PLWD;

2. Describe interactions among PWS components for caregiving for PLWD; and

3. Determine if there were differences in PWS interactions across caregiver work systems

2. Methodology

2.1. Design and Data Collection

We conducted a mixed-methods work system analysis where we integrated the PWS model with ENA to create a configural diagram of caregiver work system interactions. We used semi-structured interviews to obtain qualitative data about caregivers’ experiences [48–52]. We developed the semi-structured interview guide (see Appendix) based on the work system model [4, 5] to elicit caregivers’ positive and negative experiences while providing care; the strategies, tools, and resources used to provide care; and the context in which care activities occurred. We conducted interviews at a mutually agreed-upon location. Each interview lasted approximately one hour. We audio-recorded interviews and sent our audio recordings to a professional transcription service to be transcribed and de-identified prior to analysis. Participants received a $25 honorarium. We collected data between 2017 and 2018. This study was approved by university IRB.

2.2. Participants

We interviewed 20 caregivers (female =12) of PLWD. Caregivers were between the ages of 49–82, provided care to either a parent (N=11) or spouse (N=9), and lived within a 90-mile radius of a midwestern city. Participants were self-identified primary caregivers, persons who considered themselves as providing the majority of informal care for the PLWD. All caregivers spoke and understood English. We used a convenience sampling approach, recruiting participants through a hospital-based recruitment mechanism and a community agency.

2.3. Data Analysis

We used a four-step process to perform the ENA: 1) data segmentation, 2) directed content analysis, 3) network analysis, and 4) work system interaction analysis.

Step 1: Data Segmentation

We segmented data into sentences and added meta-data to the transcripts to provide structure for using ENA software [35, 53]. Meta-data are additional information that facilitate data segmentation by explaining where content came from and where in the data set the content belongs. When using ENA software, meta-data organizes and the transcripts into sections and provides identification to these sections when selecting sections for analysis. Meta-data included: 1) a running count of turn-of-talk and total lines; 2) interview number; 3) participant number; 4) a running number of lines to be coded; 5) speaker (interviewer/respondent); 6) transcript (the sentence to be coded); and 7) codes.

Step 2: Directed Content Analysis

We then conducted a directed content analysis guided by the PWS components defined in the PWS model [6, 47]. The codebook included all PWS components. The directed content analysis identified PWS components that exist within the caregiver’s description of their dementia caregiving experience. We dual coded each line using a binary coding structure [35], which involved coding binarily “1” if the code exists, or “0” if the code does not exist per each segment of text. Prior to the analysis process, coders met to discuss and become familiarized with the codebook. Then, coders coded a single transcript and met to discuss differences and made necessary changes to the codebook. Next, coders coded two transcripts and then met to identify discrepancies and discuss until consensus on final binary codes for each line in every transcript. Due to the focus of this work on caregiving from a caregiver’s perspective, we only included descriptions of care where the caregiver was directly involved. We created a final binary code sheet and uploaded it to the ENA software for analysis.

Step 3: Network Analysis

We applied ENA to visually represent, qualitatively describe, and quantify interactions among PWS components. ENA software [54] is a commercially available software (https://www.epistemicanalytics.org/) that uses a moving stanza window method to draw connections among codes to create configural diagrams. A moving stanza window is a fixed number of lines that slide along the coded data file that defines a stanza for the referring line (i.e., the line being analyzed). The moving stanza window models connections within that stanza among existing codes (codes with a “1”) and the codes existing in the referring line [29, 35]. A visual description of a moving stanza window is presented in Figure 1. We set the moving stanza window size at 5 lines because the caregivers took an average of 4.6 sentences to respond to interviewer questions. For more information on functionality of a moving stanza window, see Siebert-Evenstone et al., 2017 [55].

Figure 1:

Example of Moving Stanza Window Method generating connections in data. Coded data is fictitious in nature and not pulled from the study. Highlighted text is the text that is coded. Moving stanza window size is 5 lines with red lines indicating connections between codes that would be drawn by ENA software.

Figure 1 Alt-text: Two sections with one section listing an example of coded text (not from the present study) and the other section showing a moving stanza window, with a size of four lines, moving down the coded data. This illustrates the connections between codes as the moving stanza moves down the coded data.

ENA software generated a configural diagram for each transcript (n=20 individual graphs) and a summary diagram (1 diagram to summarize the individual diagrams). These diagrams present the location and size of nodes, edges, plotted points of each individual transcript and the centroid (i.e., summary point or plotted points mean) informs initial interpretations of the data during visual inspection of the diagrams. In Table 2, we describe each feature and the effect the feature has on interpretation. For purposes of this study, we primarily focused on the edge weight and the centroid location to generate initial interpretations of the diagrams. These are the indicators of the level of influences of components and component interactions on work system processes.

Table 2:

Description of ENA network graph features and their implication on interpretations of the ENA network graphs

Graph Feature	Description [29, 66]	Implication on Interpretation [29, 66]
Node	Node(s) are points in the graph that represent the codes in the directed content analysis. The larger the size of the node, the more frequently that code was assigned a “1” in the analysis.	For this study, the size and location of the node did not have significant implications on our interpretations. However, the size of the node could serve as a predicate to thicker edges connecting that node to other nodes.
Edge	Edge(s) are lines that connect two nodes. For this study, edges represent an interaction between the nodes being connected. The thicker the edge the more frequent these nodes are connected in the data.	For this study, edges represent the frequency interactions between the nodes occur. The thicker the edges the more influence (or “pull”) the edge has on the centroid location.
Plotted Point(s)	Summary statistic of an individual network graph (i.e., indicated by a circular dot) which is impacted by the thickness of the edges and where the edges are located. In the context of this study, one plotted point represents a single transcript.	Plotted point location represents the weighted average of the individual model, so if the plotted point is on one side of an axis that has thicker edges, then those edges can be interpreted as having an impact on the plotted point location.
Summary Centroid (i.e., Plotted Points Mean)	Summary statistic of all individual network graphs (i.e., indicated by a square) which is impacted by the thickness and locations of the edges and the location of the individual plotted point(s).	Location of the plotted points mean offers a high-level visual summary of all the individual plotted point(s). The interpretation is the same as individual plotted point(s), but is an summary representation of all network graphs rather than individual graphs, by taking averages of the individual graphs.

Finally, to determine differences among the caregiver work systems, we grouped ENA data into two sample sets based on where the PLWD receives care, which generated two configural diagrams. Note that care location was not a variable we recruited for but informed our grouping after determining that we had the proper data for each participant to create those sample sets. One group consisted of caregivers who provide care to a PLWD (N=10) in a home setting, whereas the other group consisted of caregivers who provide care to a PLWD who lives away from home (i.e., living in a nursing home, memory care facility, or senior apartment community) (N=10). We created these groups to determine if differences in work system interactions existed based upon where a PLWD receives care.

Step 4: Work System Interaction Analysis

As part of the work system interaction analysis, we conducted visual inspection, qualitative analysis, and statistical testing of the configural diagrams produced by the ENA. We used ENA to identify and model the co-occurrence of PWS components in the data and the resulting configural diagram. Then, we conducted an in-depth qualitative analysis to verify if co-occurrences were representative of interactions between PWS components and generated a conceptualization of these interactions for dementia caregiving. We defined work system component interactions as instances where more than one PWS components occur within the caregiver’s description. Participants explicitly described interactions in response to our interview questions. To ensure what was coded was representative of explicitly described interactions, we reviewed coded co-occurrences to determine if they were explicitly described as interactions or if they were two work system components mentioned in the same sentence rather than described as an interaction.

Visual Inspection of Configural Diagrams

We conducted a visual inspection of the summary configural diagram to identify work system interactions across caregivers. Potential interactions are visually represented by the co-occurrence of codes in the diagram. We later verified the identified potential interactions in the subsequent qualitative analysis step. The process of visual inspection required coders to focus on the edge thickness and plotted point location. The thicker the edge, the more frequent the nodes co-occurred in the analysis. Plotted point locations are directly related to thickness and location of the edges. If a plotted point is located close to a particular edge, then it may be interpreted as the edge being close the center of mass of that network, or that the weight of that is ‘pulling’ the plotted point towards it. Coders inspected the summary diagram and documented the location of the thickest edges, the two nodes being connected by those edges, and the location of the centroid (represented by a square marker). We used the documentations to guide the subsequent qualitative analysis for objective 2.

We then identified differences among home and away caregiver groups by visually comparing the diagrams for home caregivers and away caregivers using difference, or subtracted, diagram created by the ENA software. This difference diagram is generated by overlaying the two summary diagrams to produce a new diagram consisting of edges generated by the differences in edge weights. In other words, the thicker an edge is in the difference diagram, the greater the difference is between that edge in the diagrams being compared. Coders recorded the resulting edges and differences in centroid locations.

Qualitative Analysis

We conducted a qualitative analysis of the data used by ENA software to create the edges in the diagrams. This additional analysis involved a detailed review of the identified co-occurrences to determine if they could be characterized as work system interactions. We defined interactions as the involvement of one PWS component influencing, reinforcing, or existing in the presence of one or more other PWS components. To perform this analysis, coders reviewed these co-occurrences by re-reading the interview transcripts before and after the coded occurrences to determine if it could be interpreted as a work system interaction. Coders identified ‘false’ interactions, or co-occurrences generated by ENA that did not fit our definition of interaction. We excluded false co-occurrences from the analysis. Next, two coders reviewed the remaining co-occurrences and generated descriptions. These descriptions were defined by our previous interaction conceptualization and direct quotes. Coders identified exemplary quotes of their descriptions and organized them based on which interaction they described. The research team met to discuss and refine the interaction descriptions until a consensus was met [56].

Quantitative Analysis

To demonstrate the quantitative capabilities of ENA to test for potential differences between models (i.e., work system configurations), we wanted to determine if there was a difference between the PWS configuration of caregivers providing care at home and away from home (i.e., when the PwD lived in a long-term care facility). Given the recognized differences between the people, tasks, tools and technology, organization, and physical environment between the work system of the home and the work system in formal healthcare settings [6, 47], the hypothesis that work system interaction were different for PWS of caregivers providing care at home and away from home is appropriate. We recognize that we were likely underpowered to find statistically significant differences between our models. This comparison is primarily being performed to demonstrate this capability of ENA to statistically compare configural diagrams, which is not feasible with currently described configural diagramming methods. We used ENA software to conduct comparative t-tests between the graphs of the two samples. These t-tests use the individual plotted point values for all samples within the groups being compared to determine if the models are statistically different along either the x- or y-axis.

3. Results

3.1. Objective 1 – Identify and Visually Represent Interactions

Guided by the thickest edges among the nodes (Figure 2), we identified five interactions among work system components that occurred most frequently in the configural diagram: 1) task and person(s), 2) task and organizational context, 3) task and physical context, 4) person(s) and organizational context, and 5) person(s) and physical context. While there were other edges present in the configural diagram, those edges were visibly thinner which, when considered with the plotted point location, the coders determined those connections likely did not have as much of an influence on the plotted point mean location as the five interactions discussed above. The location of the plotted point mean suggests that the tasks and person component interaction edge had the greatest influence on the location thus meaning this interaction was the most frequently described by caregivers. Specifically, the plotted point mean is located to the left of the y-axis and slightly below the x-axis indicating that the interaction among the task and person nodes had the most influence on caregiving processes.

3.2. Objective 2 – Describe Interactions

The five interactions identified in objective 1 occurred across caregivers but are described differently based on if the PLWD lived at home or in a long-term care facility. Table 2 provides a description of the interactions with illustrative quotations from the data.

1. Interaction between care tasks and patient characteristics.

Fourteen (70%) caregivers described how changes in mental and physical status of the PLWD made care tasks such as medication management, bathing, and meal preparation continuously challenging. For example, one caregiver described the PLWD getting confused with taking medications on certain days of the week, so the caregiver prepared medications for the PLWD. In addition, three caregivers explained that care tasks challenges were a major contributing factor to transition the PLWD to a long term care facility.

2. Interaction between tasks and organizational context.

Interactions included changes in how tasks were distributed across caregiving roles over time and the use of care routines. Nine (45%) caregivers described sharing care tasks with formal caregivers. Eight of these were caregivers to a PLWD living in a long-term care facility. For example, one caregiver recently placed the PLWD into a formal care facility and described the facility taking on a majority of daily care activities. One caregiver of a PLWD receiving care at home described sharing care tasks with outside care organizations they hired. Ten (50%) caregivers described implementing, or attempting to implement, a care routine. For example, one caregiver described creating and implementing a morning routine to ensure the PLWD ate and showered each day.

3. Interaction between care tasks and physical context.

Eight caregivers (40%) described arranging the physical environment to ensure the safety of the PLWD. For example, one caregiver installed motion detectors that would alert if the PLWD wandered out of the home. Four (20%) caregivers identified proximity to the PLWD as an advantage in conducting their care tasks. For example, one caregiver’s employment was near the PLWD allowing the caregiver to more easily provide daily care.

4. Interaction between the person and organizational context.

Seventeen (85%) caregivers described relying on local research centers and social workers to assist with providing care either in the home or away from home. Specifically, caregivers used these resources to help them overcome various knowledge or capability limitations by either arranging for paid/unpaid care resources or gaining access to sources of information to better understand dementia care. For example, one caregiver mentioned using a social worker to become aware of available care resources and arrange for a volunteer to check in on the PLWD.

5. Interaction between person and physical context.

Eighteen (90%) caregivers described their ability (or inability) to provide care either based on the physical location of the PLWD or the limitations imposed by the layout of the physical environment. For example, one caregiver discussed closing off the basement to prevent the PLWD from having to traverse down the stairs as the PLWD became less physically able to navigate stairs.

3.3. Objective 3 – Determine Differences

The third research objective was to determine if there were differences in the interactions of PWS components among caregivers who provide care to a PLWD in the home and caregivers providing care to a PLWD away from home. Initially we could not find clear visual differences in the summary diagrams between the two caregiver groups. Figure 3 is the summary diagram for PLWD living at home and Figure 4 is the summary diagram for PLWD living away from home. We conducted quantitative comparative testing and found that there was no statistically significant difference between the two group summary diagrams along either the x- or y-axis. For the x-axis, a two sample t-test assuming unequal variance showed the Home diagrammodel (mean=0.24, SD=0.26, N=10 was not statistically significantly different at the alpha=0.05 level from the Away model (mean=0.01, SD=0.56, N=10; t(12.72)= 1.20, p=0.25, Cohen’s d=0.54). Along the Y axis, a two sample t-test assuming unequal variance showed the Home model (mean=−0.05, SD=0.36, N=10 was not statistically significantly different at the alpha=0.05 level from the Away model (mean=0.18, SD=0.24, N=10; t(15.92)= −1.67, p=0.11, Cohen’s d=0.75).

Figure 3:

ENA Summary diagram for caregivers providing care to a PLWD at Home (n=10)

Figure 3 Alt-text: ENA summary diagram of the caregivers who provide care at home. The average point (centroid) is to the right and below the origin.

*Note: Red square is the plotted point mean, red circles are plotted points for the participant models, and black circle dots represent the codes

Figure 4:

ENA summary diagram for caregivers providing care to a PLWD Away from home (n=10)

Figure 4 Alt-text: ENA summary diagram for caregivers providing care to a person living with dementia living away from home. The average point (centroid) is located above and just slightly to the right of the origin.

*Note: Blue square is the plotted point mean, blue circles are plotted points for the participant models, and black circle dots represent the codes

4. Discussion

The results of this study demonstrate the utility of ENA as a useful analytical method to perform configural diagramming by providing a visual diagram of system interactions that represent the level of influence of components and interactions on the system that can be analyzed using qualitative and quantitative methods. We were able to use the ENA-generated configural diagrams to identify, visually represent, describe, and explore differences in work system interactions across caregivers. Further, we were able to identify which interactions had the greatest influence on the work system. Finally, we visually and statistically compared diagrams to identify and describe differences among interactions across caregiver work systems. The differences identified were qualitatively described, but we found no statistically significant differences.

We found that ENA can be useful as an analytical method by providing meaning to the visual representations of interactions and the subsequent quantitative analysis results. In doing so, ENA addresses the current need for configural diagramming tools that facilitate interpretation of the sizes and locations of components [28], frequency of the lines representing interactions, and the lack of a diagram summary point. ENA addresses this need by assigning qualitative meaning to the plotted point locations and the edge thickness to guide inferences and provide interpretive meaning to the physical structure of the diagrams created. The qualitative meaning assigned by ENA is the physical location of the plotted points in the projection space, which are then used to conduct statistical tests to identify differences among summary diagrams. Future research could look to integrate the contextual details uncovered in configural diagramming into the summative nature of a plotted point mean used in ENA.

By using ENA to conduct configural diagramming, we were able to gain a deeper understanding of the PWS. Initial research of the macroergonomic factors that influence patient work provided solid theoretical groundwork for this study [6, 7, 44, 47]. However, this study expanded on this research by focusing on the interactions between these macroergonomic factors. Specifically, ENA allowed us to identify those system interactions that were most affecting caregiving processes. The visual representations provided by ENA facilitated the identification of these interactions based on the node size and edge thickness that connected these two components to the other work system components. We were able to describe the most frequent interactions by using the ENA’s ability to create the network model-based configural diagram, which guided a secondary qualitative analysis of that data [32]. Specifically, complex tasks such as information sharing or providing care-related tasks may involve multiple individuals with varying roles, which aligns with prior research on dementia caregiving [43, 57–60]. Future research can look to explore the potential to use ENA to study the relationships between work system component interactions, processes, and outcomes. This would facilitate understanding how the macroergonomic factors and their interactions influence the entire PWS.

We also examined whether there were differences in work system interactions between the work system configurations of caregivers providing care to a PLWD at home versus those providing care away from home. Although we were able to identify both visual and qualitative differences in work system interactions between the configural diagrams, the differences were not statistically significant. This demonstrates the importance of the mixed methods approach of ENA because while there was no statistically significant difference between the models, qualitative analysis identified potentially key contextual differences between the work systems. For example, caregivers described the role and level of involvement of formal healthcare providers differently based on the location of the PLWD. This example demonstrates a qualitative difference between the organizational context and person(s) and task(s) that were not clearly represented visually or quantitatively. It is possible that the lack of statistical difference in the model could be a result of the data collection method, which was focused on a single caregiver perspective.

Building the potential for ENA as an analytical method for configural diagramming

To date, research that used configural diagramming to examine work system interactions has been primarily qualitative, and findings indicated challenges in determining which interactions were more or less influential to the work system [27, 28]. ENA introduces a method for quantifying qualitative data to produce configural diagrams, which can further expand our understanding of configuration. However, to fully demonstrate the potential of ENA for configural diagramming, future work is needed to refine what is considered an interaction and if this is accurately represented through ENA modeling. Secondly, it is unclear how human factors performance measures such as workload or stress can be incorporated into ENA to help explain the outcomes of system interactions. For example, caregivers could complete self-report surveys and the responses can be analyzed in parallel to the ENA analysis with the goal of identifying which ENA model is associated with manageable levels of stress or workload.

Considerations for ENA use in future research

There are certain considerations to take into account when planning to use ENA as an analytical method for configural diagramming. First, ENA is time and labor intensive. Following the standard requirements for ENA, we performed a binary coding process to produce the network models. We then combined that with a more traditional qualitative analysis to better understand the context underlying the co-occurrences found in the models. This two-step analysis process may not be necessary in all studies using ENA but is a consideration when using ENA to conduct configural diagramming.

Second, it is beneficial to have someone with quantitative ethnographic background to help guide the interpretation of the ENA models. Formal training in quantitative ethnography is not required, but certainly helpful when making decisions on data segmentation (turn-of-talk vs. sentences), determining the types of meta-data needed, and analysis types (moving stanza method vs. strope method) [30, 55]. These decisions require knowledge of how they would influence how the software interprets the data to create ENA models.

Third, as with many approaches, the goals of the research should guide sample size determination. For example, this study would have benefitted from a larger sample size to more adequately test for statistically significant differences and demonstrate the statistical comparison capabilities of ENA. However, if statistical comparison between configural diagrams is not the goal, ENA is appropriate for a range of sample sizes depending on the purpose of the research. Prior research using ENA has used sample sizes ranging from less than 20 individuals participating in a discussion to over 2.4 million tweets all analyzed for purposes of ENA [61, 62].

Finally, as with any study, the data collection methods should be aligned with the goal of the research, and potential limitations should be adequately addressed. There is no ‘ideal’ data collection method for ENA, with current studies using a range of data sources including group discussion activities, a case study, and evaluation of datasets pulled from social media posts [32, 61–63]. The present study used semi-structured interviews, which is an acceptable collection method for ENA [35]. However, we encountered some methodological challenges related to capturing only the respondent and not the interviewer in the ENA models. As ENA is used for configural diagramming in the future, it will be important to report any challenges with data sources and the methods used to overcome those challenges.

Limitations

Our findings and demonstration of ENA should be interpreted with certain limitations in mind. First, this study used a retrospective interview technique with caregivers located in a single metropolitan area in the Midwest. Retrospective interview techniques have limitations related to interviewee recollection [64] and the experiences discussed may not generalizable across all caregivers. Alternative data collection methods such as contextual inquiry or diary studies, and interviewing multiple perspectives are important next steps for future research.

Second, we used ENA to create a visual representation of the configural diagram and maximized the software’s ability to present qualitative data as a network graph. However, the quantitative potential of ENA was not fully used. ENA can also provide numerical values to represent edge weights, node sizes, and other numerical values of interest. Moreover, ENA offers various data segmentation and analysis techniques [29, 30, 35, 55]. We used a moving stanza window as our analysis protocol, which means that co-occurrences were established within a specific number of lines (i.e., 5 lines: the 1 line being analyzed, and the previous 4 lines) of the transcript. A limitation of using a moving stanza window is the assumption that the context for every line can be understood within a preset number of lines. This assumption creates the possibility for connections outside of the set window size to be missed. To address this limitation, we conducted a secondary qualitative review of the co-occurrences, but it is important to note that this process can be time-consuming. Additionally, we did not record the number of ‘false’ interactions that were identified during the secondary qualitative review. Recording this data would have provided us with additional information about how well our coding structure performed within ENA.

Finally, we decided not to code descriptions of healthcare professional delivered care in acute care settings. For example, if the PWLD was hospitalized. This was done to place an emphasis in the configural diagrams on caregiver process.

5. Conclusion

Configural diagramming is a beneficial method of work system analysis, and ENA provides an analytical method for producing and analyzing configural diagrams [31, 32, 65]. We successfully applied ENA to produce and interpret configural diagrams of PWS interactions using semi-structured interviews. ENA identified interactions of PWS components and through a qualitative analysis, we were able to describe the context of these interactions. While further research is needed to refine and validate ENA for configural diagramming, this study demonstrates an important first step in constructing and quantitatively and qualitatively analyzing configural diagrams using ENA. Finally, this study enhances the understanding of interactions between work system components and their influence on the dementia caregiving process from the caregiver’s perspective.

Table 3:

Direct quotes for each of the five primary PWS interactions identified through ENA modelling

Direct Quotes of PWS Component Interactions
Task-Person(s) Interaction
“[When taking medications] he’d get Tuesday confused with Thursday, so that’s why I leave one [pillbox for that day of the week] out in daytime” (P418)
Task – Organizational Context Interaction
“I’m turning over my [care] responsibility to the PTs and the OTs, which, like I said is a load of my mind for a while” (P516)
“We had our routine down in the morning where I’d go over there and we’d have coffee, and I’d fix her, just cereal or oatmeal … give her showers, because I did it every day” (P903)
“A doctor at the Memory Clinic, her internist, sets aside an extra 15 minutes to spend with PLWD on every visit.” (P251)
Task – Physical Context Interaction
“We got motion detectors too after that [the wandering event] so we would know if she went out the front door.” (P735)
“So, at this point in time, he is still home by himself while I’m at work, but I work really close to home. And I’m able to go home at lunch to see him every day, and also if he needs some help with something, he's got me on speed dial at home, and I can run home and help him take care of something and then come back to work” (P216)
Person – Organizational Context Interaction
“Well, of course [the Alzheimer’s Disease Research Center has been helpful], they stay in touch with you. The social worker there, has been really helpful. … They connected us with a volunteer who comes to see my husband once a week, and they kind of struck gold with that. And, they made sure I know the resources.” (P326)
Person – Physical Context Interaction
“I wanted [the PLWD] to have as high a quality of life as possible, but at the same time they had to be safe, and as their physical health started going downhill, I realized that [the PLWD] should not be going [down stairs] to the basement” (P619)

Appendix

HelpCare Connect Interview Guide

Administrative notes:

• Thank you for agreeing to participate in this interview. The purpose of today’s interview is to find out more about:

  • What your caregiving experience is like now and before hospitalization
  • Strategies and resources you use in your caregiving and how these are working well for you as well as ways they could be improved
  • What you feel your greatest areas of need are as a caregiver

• Explain purpose of recording, remind participant that no names, should be mentioned and no identifiable info etc. is ever used in the future

• The audio recorder is now recording and for the purposes of the recording, this is interviewer [INTERVIEWER INITIALS], [DATE], [TIME] and I am interviewing [PARTICIPANT ID].

Interview prompts:

Current Caregiving Experience

Transition

First, can you tell me (without using any names) who you’ve been caring for and how long you’ve been providing care? Do you share your caregiving responsibilities with anyone else?

Can you tell me about how things have been going since the hospital?

• 1
  Can you tell me a little bit about what that transition was like for you as a caregiver?

  1. Were there things that went particularly well during the transition?
  2. Were there things that were more problematic?
  3. If you were to make these transitions as smooth as possible, can you tell me what that would be?

Pre-Hospitalization

Now I’d like to shift gears a bit, and talk about what things were like before the hospital stay.

• 2
  Can you tell me what a typical day was like for you before the hospital stay?

  1. What did a usual day consist of?
  2. How would you describe your caregiving activities during a typical day?
• 3
  Prior to the hospital stay, can you think of a particularly “good caregiving day” that you had recently and tell me what that was like?

  1. What happened?
  2. What went well? (did you talk to anyone else about it? Was anyone else present that day?)
  3. How common is a day like this?
• 4
  Prior to the hospital stay, can you think of a particularly “challenging caregiving day” that you had recently and tell me what that was like?

  1. Can you tell me what that day was like? What happened?
  2. What made that day challenging?
  3. How common is a day like this?
  4. What are some of the differences between the good day and the challenging day? (did you talk to anyone else about it? Was anyone else present that day?)
• 5Sometimes people living with dementia have [behavioral or emotional*] changes that some caregivers may find challenging?

*** Pay attention to how caregiver describes “behavior” ➔ try to use their language

1. Have you experienced any of these changes with [person you provide care for]?

2. What are your thoughts about why these [changes] occur?

3. What do you do when [change] occurs?

Strategies and Resources

• 6Do you have any strategies you use to help you carry out your caregiving activities, m12age challenging [behavioral or emotional*] changes or challenging situations?

*** Pay attention to how caregiver describes “behavior” ➔ try to use their

1. Prior to the hospital stay, can you think of a time when you used a strategy that helped you manage a challenging caregiving situation?

   1. How did you identify the strategy?
   2. What worked well?
   3. Did you tell others involved in [CRs] care about the strategy?
2. Can you think of a time when you used a strategy that didn’t work?

   1. How did you identify the strategy?
   2. What do you think didn’t work?
   3. Did you tell others involved in [CRs] care about the strategy?

3. What were the differences between the strategies that worked and the strategies that didn’t work?

4. Can you think of a time when you didn’t know how to handle a caregiving situation? What did you do?

5. If you think about the situations you just described, what do you think would have been helpful to you when confronted with a caregiving situation you are unsure of or struggling to manage?

• 7
  Is there anyone you talk to about caregiving or who may be important to you in your caregiving role?

  1. Are there other family members nearby?
  2. Is there anyone else who is involved with providing care?

Probes

• How do you share information with them?

• What do you use to share information with them?

• What type of information do you share?

  • 8
    People have shared with us in the past that there are things about the physical environment of the home that can make caregiving more challenging or easier. Are there certain things about the physical environment of your home that make caregiving easier or more challenging?
  • 9
    We are also interested in learning about how caregivers prefer to access resources.

What types of tools or resources are you currently using to help you with your caregiving?

• aWhat about these has worked well for you?
• bWhat about these resources has not been helpful or presented barriers?
• cWhat could make these resources more useful for you?
• dDo you access any resources on your computer phone or tablet?

Type of access

• e
  Would you use a device such as resources on a phone or tablet? If you could have support for caregiving on a phone, etc. which of the following would be helpful to you?

  • Timeliness/Responsiveness of resource (i.e., at your fingertips vs. call back later)
  • Hands-on/demonstration?
  • Guidance for specific challenges
  • A chance to learn from and interact with other caregivers
  • Help with tracking care/behaviors
  • Help you connect and share information with your own caregiving teams?

  • 10
    Is there anything else you think it is important for me to know about the types of resources caregivers like you could benefit from?