Skip to main content
NCBI home page
HHS Author Manuscripts logoLink to HHS Author Manuscripts
. Author manuscript; available in PMC: 2022 Apr 4.
Published in final edited form as: Proc Int Symp Hum Factors Ergon Healthc. 2021 Jul 22;10(1):211–216. doi: 10.1177/2327857921101138

Symbiotic Design Application in Healthcare: Preventing Hospital Acquired Infections

Mengyun Li 1, Dane A Morey 1, Michael F Rayo 1
PMCID: PMC8979277  NIHMSID: NIHMS1783902  PMID: 35381999

Abstract

Any clinical decision support (CDS) design project integrating computational technologies with clinician workflows will require the merging of multiple perspectives and fields of expertise in multidisciplinary teams. Much like the tools these teams aim to create, the team itself will need to continuously build, monitor, and repair a mutually beneficial relationship between each of its members. From our experience during the early development stages of an AI-enabled CDS tool for hospital-acquired infection (HAI) prevention, we abstract three central tenets of a symbiotic design process we have found to be vital for aligning goals, priorities, mental models, and techniques among a multidisciplinary team: (1) recurrent bottom-up feedback, (2) continual model (re-)alignment, and (3) openness to co-direction. With regards to these tenets, we discuss the successes and challenges our team has faced during the symbiotic design process through a series of vignettes and how these experiences coalescing diverse human design teams can influence the design of human-machine teams.

INTRODUCTION

The increasing digitization and prevalence of healthcare data have opened new possibilities for clinical decision support (CDS) tools driven by powerful computational technologies (Sheikh et al., 2017), including artificial intelligence (AI). However, these new tools pose new challenges as well. The ways in which human decision-makers and machine decision-aides interact can produce outsized positive (Corcoran et al., 1972; Parasuraman, 1987; Thackray & Touchstone, 1989; Woods & Hollnagel, 2006) or negative (Robinson & Sorkin, 1985; Sorkin & Woods, 1985; Dalal & Kasper, 1994; Smith et al., 1997; Wickens & Dixon, 2007) effects on the system as a whole. As machines behave increasingly more like active cognitive teammates than passive tools, the relationship between humans and machines can no longer be thought to be unidirectional; each can affect and be affected by the other. Therefore, it is useful to conceptualize this relationship as a human-machine symbiosis, where humans and machines are analogous to organisms in a working relationship. As in nature, this relationship can be mutually beneficial, beneficial to only one party while doing no harm to the other, or beneficial to one party while harming the other (i.e., mutualism, commensalism, or parasitism). The goal of CDS is to design for mutualism, and mitigate all possible risks of degrading to parasitism.

This symbiotic design process requires a coalescence of diverse expertise and techniques, which suggests symbiosis is also needed within the teams of people designing symbiotic human-machine teams. Multidisciplinary teams must include experts in interaction design, cognitive agent design, computer science, and data science as well as subject matter experts in the domain of interest. This diversity of expertise will bring with it diverse, yet not always complementary, perspectives on the goals, problems, and solutions that they are meant to jointly address. These teams must simultaneously cooperate while also continually cross-checking one another (Watts-Perotti & Woods, 2009). Teams must develop a shared set of techniques through a fusion of user-centered design, cognitive systems engineering, and cognitive agent design (Rayo, 2017), yet remain active individuals that continually act and react to the rest of the team (rather than passive receivers of information and instructions). We have found that it takes symbiosis (in the team) to achieve symbiosis (in the resultant solution), and that the study of symbiosis in either human-machine or human-human teams directly informs the other.

We have begun to explore the process of symbiotic design in creating an AI-enabled CDS tool for hospital-acquired infection (HAI) prevention. HAIs contribute to increased mortality, morbidity, and length of hospitalization in patients (Voidazan et al., 2020) which costs US hospitals $28–45 billion annually (Stone, 2009). To mitigate the spread of HAIs, infection preventionists (IPs) must identify and anticipate a variety of different disease-specific transmission routes among patients separated by time and space (i.e., clusters). The complexity and scale of this task makes symbiotic design a particularly attractive approach in creating a CDS tool which amplifies human sensemaking with computational modeling. The multidisciplinary team we have assembled to design this tool includes epidemiologists, infectious disease physicians, infection preventionists, clinical informaticists, data scientists, computer scientists, geographers, human factors researchers, and visualization designers. In this paper, we discuss our experiences throughout our ongoing symbiotic design process, including our central tenets, the known pitfalls we have avoided, and the challenges we have faced along the way as a symbiotic team designing a symbiotic team.

CENTRAL TENETS OF SYMBIOTIC DESIGN

All adaptive systems, including our own design team, risk falling into maladaptive patterns, two of which are: (1) getting stuck in outdated behaviors and (2) working at cross-purposes (Woods & Branlat, 2011). Motivated by these known shortcomings, we explicitly configured our symbiotic design process with three central tenets.

Recurrent Bottom-Up Feedback

Technology-driven approaches, where design and development is primarily guided by technological limitations, are particularly vulnerable to solving the wrong problem (Woods & Roth, 1988), ultimately creating a tool that is not usable, useful, and desirable (Sanders, 1992). To help ensure our symbiotic design process would instead be problem-driven, we incorporated recurrent feedback sessions, bringing in subject matter experts (SMEs, i.e., epidemiologists, infectious disease physicians) as full members of the team, and continued to engage with frontline operators throughout the entire design process, even after we felt we fully understood the domain. This was challenging because the more we understood about the domain and problems, the more these feedback sessions could appear unnecessary; however, we continued to discover key insights long after our initial domain research finished, which we likely would not have discovered if these sessions had not been fixed activities in our project timeline. Each of our process artifacts, including abstraction networks, user personas, scenarios, and wireframe designs, elicited different aspects of the problem domain beyond what was possible from initial user research alone. By tethering our design team to a continual source of bottom-up research, we enabled our team to explore and adjust to what was important in the domain as it became apparent over time.

Continual Model (Re-)Alignment

The differing perspectives and responsibilities of our hierarchically-organized team increased the risk of developing misaligned goals. Solutions advantageous to one sub-team’s responsibilities could hinder other sub-teams’ abilities to meet their responsibilities or undermine the long-term shared goals of the larger team (Woods & Branlat, 2011). To mitigate these risks, we continuously shared and revised our process artifacts in recurrent full-team meetings. These artifacts and mechanisms helped detect misalignments by eliciting each others’ understanding (i.e., mental model) of the project. However, we soon discovered these standard procedures for aligning hierarchical, multidisciplinary teams were necessary but not sufficient to detect when our team was working at cross-purposes. The abstracted information communicated at the highest levels of the organization (i.e., team leads) obscured some of the ways in which small misalignments were causing the overall team to work in ways that were not synchronized to the larger shared goals of the project. These misalignments were only revealed through horizontal communication at the lower levels of the organization, where information was typically conveyed in more granular detail. Therefore, we needed a regular, structured communication mechanism between the people who did work beyond what was available between the people who coordinated work. Again, these meetings could appear unnecessary at times, but proved vital in detecting when our team was not well-coordinated and needed realignment.

Openness to Co-Direction

Mutual directability, or deliberate bi-directional efforts to modify the actions of others as conditions change, is a key aspect to successful coordination in cooperative teamwork (Klein, Feltovich, et al., 2004). However, co-direction is costly. It requires prolonged, synchronous engagement of multiple team members, which is too inefficient to be the predominant modality of work. Therefore, a signaling mechanism is needed to decide when and how to increase or decrease participation (Maguire, 2020; Reynolds, 2020). Cross-checking, where two or more members of the team with different perspectives examine each other’s assumptions and actions, can functionally supply this signal; however, the cross-checks must be carefully architected to produce the intended diversity instead of a routinized redundancy (Patterson et al., 2007).

As a two-way relationship between agents, effective collaborative cross-checking must be both well-delivered and well-received to beneficially impact the system. To deliver cross-checks, personnel must have spare capacity to engage in challenging sensemaking functions (Patterson et al., 2007) and possess deep, overlapping competencies (with the receiver) that surpass their own individual stated responsibilities. To receive cross-checks, personnel must explicitly communicate the rationale and intent behind actions and decisions (Patterson et al., 2007), make their actions and decisions openly observable to others (Klein, Feltovich, et al., 2004; Patterson et al., 2007), and actively invite others to cross-check (Rayo et al., 2013).

Co-direction retains all the same requirements as cross-checking with one addition: the ability to resolve conflicting perspectives. These multiple, contrasting perspectives are critical to avoid working at cross-purposes and getting stuck in stale behaviors (Woods & Branlat, 2011); however, multiple perspectives will inevitably (and necessarily) conflict. The inherent confrontational nature of cross-checking and co-direction could easily become antagonistic if coordinated poorly. Therefore, effective teams must have strategies for managing and resolving conflicts, including building rapport, tempering reactions, de-escalating situations, and reinforcing common ground. The ability to embrace but control these conflicts is a sign of a healthy symbiotic relationship.

VIGNETTES

Though our current work is ongoing, we have already seen value in symbiotic design as a process to align our multidisciplinary team. With the coordination mechanisms we have implemented to operationalize our three tenets, we have been able to detect, address, and resolve misalignments (ranging from benign to potentially disastrous) early in the design process and long before becoming costly to the project. We discuss some of the benefits and challenges we have experienced while pioneering a symbiotic design process through a series of three vignettes.

Vignette 1: process artifacts to sustain shared common ground on project priorities

After initial user research, the human factors sub-team had developed a mental model of IPs’ intervention mechanisms that was not well-aligned with the priorities and processes of their current work. The human factors sub-team initially envisioned preemptive interventions (e.g., identifying potential infections and ordering tests prior to patients showing symptoms) as one of the primary functions of the tool. In other settings, this early detection of potential hazards is highly advantageous (Horwood et. al., 2018). However, in this case, there was a strong perception among leadership that earlier initial detection of infection would be extremely difficult and not extremely valuable due to the transmission characteristics of many of the infectious organisms. This combination of factors and the degree of misalignment across the team indicated that the team would not be willing to invest heavily in this feature or support it post-implementation unless it proved its value quickly and unambiguously (Fitzgerald, 2019). Therefore, the near-term priorities were shifted towards a CDS tool focused on supporting how IPs identity and clean contaminated rooms based on patterns of already-identified cases.

This gap between work-as-imagined and work-as-done (Hollnagel & Braithwaite, 2021) was discovered by our SMEs through shared process artifacts. The initial abstraction network and user scenarios created by the human factors sub-team, when shared continuously and iteratively among all team members, was sufficient for our SMEs to quickly recognize the misalignment between theirs and the human factors sub-team’s mental models and recalibrate the team’s priorities before incurring significant project costs. Therefore, these internally created and shared artifacts in this example successfully functioned as coordination devices to build, maintain, and repair shared common ground across the team on the knowledge, assumptions, and priorities of the project (Klein, Feltovich, et al., 2004).

Vignette 2: horizontal communication to continuously minimize gaps across the team

This project presented several promising yet competing opportunities that risked our team working at cross-purposes (Woods & Branlat, 2011). Our data science sub-team had rich expertise in spatio-temporal modeling and access to an extensive dataset on the duration of time infected patients had occupied hospital rooms. The advantages and limitations of this dataset naturally led to the creation of a spatio-temporal model of room contamination likelihood. While this model was highly advantageous from the perspective of the data science sub-team, a model of room contamination by itself was not alone sufficient to support the highest-priority objective for the project: identifying potential clusters of infections (i.e., patient disease cases linked by likely transmission routes) that deserved additional monitoring. However, the high-level information communicated between team leaders at full-team meetings obscured the gap between the data available for modeling (room contamination) and the information valuable to IPs (transmissible connections between patients).

This widening gap among sub-teams was only revealed through the weekly briefings among the graduate students of each sub-team. Coordinating the lower-level details from each sub-team in these graduate student meetings uncovered that teams had conflicting expectations about the deliverables of others. Because of these meetings, these misalignments were identified and resolved early and with minimal consequence to the project timeline. Consistent with what researchers have observed in tightly-coupled work, we found horizontal communication mechanisms among lower echelons was essential for coordinating across our team and avoiding working at cross-purposes (Branlat & Woods, 2010).

Although our extra meetings to facilitate horizontal communication proved to be valuable in this example, the time and resource costs of these meetings had to be low enough to be sustainable (Fitzgerald, 2019). This meant that, even when it turned out that there wasn’t a pressing need to meet (i.e., all sub-teams were well-aligned), everyone still reserved a small amount of time to commit to the subsequent meeting, not allowing other obligations to encroach on that reserved time. A frequently cancelled meeting would greatly reduce the potential to identify misalignments. For our team, reserving 10–15 minutes weekly was a small enough commitment to be perceived beneficial even when it did not yield tangible benefits. However, we have found even this short amount of time is sufficient to monitor each other’s work, resolve gaps, and identify when a more in-depth conversation is needed to align work.

Vignette 3: leveraging overlapping competencies beyond project responsibilities

As mentioned above, one of the most challenging goals of our CDS tool was to aid IPs in detecting clusters. IPs evaluate the likelihood of clusters among a set of disease cases by considering spatial proximity, temporal closeness, intermediary connections (common room, equipment, or personnel), room risk, and disease onset, among other factors. The combination and sequence with which these factors arise greatly influences whether IPs determine a grouping of cases to be coincidence or evidence of an emerging outbreak. However, without direct access to how and when disease spreads among patients, differentiating clusters from non-clusters is inherently uncertain.

As expected in complex and highly uncertain domains (Woods, 2018), no single member of our team had sufficient expertise, perspective, or domain understanding to independently envision a viable solution. Because the room contamination model created by our data science sub-team was not alone sufficient to help IPs identify clusters (see vignette 2), cluster detection necessarily had to emerge from the interactions between model outputs, additional contextual data, and IP expertise. This in turn required our design sub-team to be able to interrogate the outputs of the model, connect the strengths and limitations of these outputs to domain-specific cognition, and visually communicate these outputs in an understandable but not oversimplified way. Fortunately, our design sub-team had considerable experience visually integrating computational inferences of both transparent (Morey, 2021) and opaque (Rayo et al., 2021) algorithms with additional context. The results from our tool are preliminary but show promise that this visual analytics solution improves cluster detection by facilitating human decision-makers and machine decision-aides functioning as joint contributors to team cognition (Woods, 1985).

Leveraging the complementary perspectives and expertise of this multidisciplinary team required a breadth and depth of competencies across the team outside of what would be typically assumed from each group’s defined role. For example, to visually integrate the outputs of the room contamination model in a way that contributed to joint sensemaking, our visualization design sub-team needed to converse in technical detail with our data science sub-team. This required an understanding of Bayesian statistics, signal detection theory, and the specific details of how features were computed from the data - competencies which are not typically expected of a design team and far outside their project responsibilities. However, our design sub-team was able to engage deeply with these technical details, in part because the entire team had formal training in both engineering and psychology in addition to over ten years of experience working with clinicians. All of the sub-teams exhibited similarly overlapping and extended capabilities. The leader of the data sub-team was a practicing infectious disease physician with extensive biomedical informatics experience. The leader of the modeling sub-team had extensive experience in geography, mathematical modeling, and epidemiology. For designing an integrated CDS tool, these unusual competencies and backgrounds were the most critical factor enabling the team to cross-check, co-direct, and co-design with others.

Equally important as possessing the overlapping competencies necessary to cross-check, each sub-team must also continuously and enthusiastically invite cross-checking. Our team employed several strategies to ensure cross-checking would be openly invited and mutualistic. First, our team made intentional efforts to establish and maintain good rapport. This initially was helped by the fact that most of the team leads had worked with each other on prior projects, but we also continued to make deliberate efforts throughout the project. Sometimes, this was as simple as taking a moment to acknowledge and appreciate the work others had done. Second, our team explicitly invited dialogue and cross-checking. In the above example, the data science sub-team initiated a separate meeting to explain to and receive feedback from the design sub-team about the initial model. With an established precedent that even contentious questions would be openly received, these conversations remained easy to have when perspectives conflicted. Third, we actively sought ways to relieve pressure from these cross-checking conversations. For example, we found alternative mechanisms to full-team meetings (e.g., graduate student meetings, ad hoc meetings) to facilitate these conversations in lower-stakes environments. Together, the rapport, enthusiastic invitations, and low-stakes environments made these conversations enjoyable and mutually beneficial for the team.

DISCUSSION

Throughout our ongoing work, we continue to see the benefits of a symbiotic design approach. In our experience, symbiosis has been a useful analogy to encourage diverse, multidisciplinary teams to invest in the coordinative activities (and associated costs) necessary for mutually beneficial joint activity (Klein, Feltovich, et al., 2004). Maintaining low-cost mechanisms to continually elicit bottom-up feedback, realign mental models, and co-direct teammates has paid dividends in both the efficiency and effectiveness of our team’s solutions; however, they are not without their own set of challenges. Most notably, sustaining these mechanisms even when they do not yield tangible benefits will continue to require a deliberate investment of time and resources. Well-adapted teams can hide the benefits of being well-adapted (Woods & Hollnagel, 2006); therefore, sustaining the tenets of symbiotic design will require sustained commitment, especially from team leaders.

Building Competencies for Co-Design

Of the three tenets of symbiotic design, openness to co-direction is perhaps the most critical and the most challenging. Overlapping competencies are a key coordination component enabling a team to leverage multiple perspectives; however, building the skills necessary to converse in-depth with other team members requires foresight and substantial time and resource investments in capabilities that may not currently (or ever) appear obviously relevant. Some of these competencies will develop over time scales longer than the course of the project itself; so, they will need to be present (at least in part) prior to start of the project. Within our team, this process began years before the team first convened and long before any of us realized how valuable those competencies would be. Realistically, project teams will not be able to definitively know which overlapping competencies their team will need early enough to develop them from nothing. Therefore, the most pragmatic approach would be to encourage experts to develop a wide breadth of expertise at a mid-depth understanding. Then, as the requisite competencies become clearer over the course of the project, each team member can focus on building additional depth in specific areas with lower time and energy costs. Notably, mutual observability becomes all the more critical for teammates to recognize early which competencies they need to develop in order to augment common ground among the team (Klein, Feltovich, et al., 2004).

To be better situated for co-design, teams need to more highly value a breadth of expertise. Especially when designing cognitive technologies, a highly specialized team (with few overlapping competencies) is not likely to be a good symbiotic team because they will be limited in their ability to leverage multiple perspectives in a mutually beneficial way. Teams should therefore incentivize developing overlapping competencies, even when they are not immediately or obviously useful. Further, the degree to which team members possess overlapping competencies should be considered when choosing the project team. In complex domains where no singular perspective is sufficient, our experiences suggest these overlapping competencies are a prerequisite to team success. Developing and encouraging these competencies beyond the responsibilities of team members is therefore a necessary investment of time and resources, although not always one that yields tangibly obvious returns.

Implications for Human-Machine Teaming

The symbiotic perspective of human design teams can also inspire a new way of thinking about the relationship between clinicians and CDS tools. As these CDS systems become increasingly cognitive, we find value in conceptualizing the relationship between clinician and decision-aide not as a human wielding a tool, but as two cognitive teammates collaborating in a joint decision-making process (Woods, 1985). We therefore expect the insights generated from using a symbiotic design process with teams of people to have direct implications for human-machine teams. We hypothesize a mutually beneficial human-machine symbiosis will require mechanisms functionally analogous to the bottom-up feedback, horizontal communication, and overlapping competencies that were necessary within our own design team. Human-machine symbiosis may prove even more challenging to achieve than it is for teams of people because of the fundamental asymmetry between human and machine coordinative competencies (Klein, Woods, et al., 2004); however, we believe first having a symbiotic design team is both a prerequisite and a template for discovering mutually beneficial human-machine teaming relationships.

ACKNOWLEDGEMENTS

This project was supported by grant number R01HS027200 from the Agency for Healthcare Research and Quality. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality.

We also want to thank all the members of the GeoHAI research team at The Ohio State University and Wexner Medical Center for their collaboration on this project, including Courtney Hebert, Elisabeth Root, Marie Reid, Justin Smyer, Jennifer Flaherty, Adam Porr, Megan Gregory, David Kline, James Odei, Joshua Radack, Kaiting Lang, Phani Atyam, Varun Dhanvanth, Luyu Liu, Aaron Cochran, and Ethan Timko.

REFERENCES

  1. Branlat M, & Woods DD (2010). How do systems manage their adaptive capacity to successfully handle disruptions? A resilience engineering perspective. In 2010 AAAI fall symposium series. [Google Scholar]
  2. Corcoran DWJ, Dennett JL, & Carpenter A (1972). Cooperation of Listener and Computer in a Recognition Task. II. Effects of Computer Reliability and “Dependent” versus “Independent” Conditions. The Journal of the Acoustical Society of America, 52(6B), 1613–1619. [DOI] [PubMed] [Google Scholar]
  3. Dalal NP, & Kasper GM (1994). The design of joint cognitive systems: the effect of cognitive coupling on performance. International Journal of Human-Computer Studies, 40(4), 677–702. [Google Scholar]
  4. Fitzgerald MC (2019). The IMPActS Framework: the necessary requirements for making science-based organizational impact. (Master’s thesis, The Ohio State University; ). [Google Scholar]
  5. Hollnagel E, & Braithwaite J (2021). Resilient Health Care. Farnham, UK: Ashgate. 10.1201/9781315605722. [DOI] [Google Scholar]
  6. Horwood CR, Moffatt-Bruce SD, Fitzgerald M, & Rayo MF (2018). A qualitative analysis of clinical decompensation in the surgical patient: Perceptions of nurses and physicians. Surgery, 164(6), 1311–1315. 10.1016/j.surg.2018.06.006. [DOI] [PubMed] [Google Scholar]
  7. Klein G, Feltovich PJ, Bradshaw JM, & Woods DD (2004). Common ground and coordination in joint activity. Organizational simulation, 53, 139–184. [Google Scholar]
  8. Klien G, Woods DD, Bradshaw JM, Hoffman RR, & Feltovich PJ (2004). Ten challenges for making automation a” team player” in joint human-agent activity. IEEE Intelligent Systems, 19(6), 91–95. [Google Scholar]
  9. Maguire LMD (2020). Controlling the Costs of Coordination in Large-scale Distributed Software Systems. (Doctoral dissertation, The Ohio State University; ). [Google Scholar]
  10. Morey DA (2021). Breaking away from brittle machines: Evaluating simultaneous inference and data (SID) displays to facilitate machine fitness assessment. (Master’s thesis, The Ohio State University; ). [Google Scholar]
  11. Parasuraman R (1987). Human-computer monitoring. Human Factors, 29(6), 695–706. [Google Scholar]
  12. Patterson ES, Woods DD, Cook RI, & Render ML (2007). Collaborative cross-checking to enhance resilience. Cognition, Technology & Work, 9(3), 155–162. [Google Scholar]
  13. Rayo MF (2017). Designing for collaborative autonomy: updating user-centered design heuristics and evaluation methods. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 61, No. 1, pp. 1091–1095). Sage CA: Los Angeles, CA: SAGE Publications. [Google Scholar]
  14. Rayo MF, Horwood CR, Fitzgerald MC, Grayson MR, Abdel-Rasoul M, & Moffatt-Bruce SD (2021). Using visual integration of threshold alarm data to support machine fitness assessment and mitigate the risks of imperfect machine inferences. Manuscript in preparation.
  15. Rayo MF, Mount-Campbell AF, O’Brien JM, White SE, Butz A, Evans K, & Patterson ES (2013). Interactive questioning in critical care during handovers: a transcript analysis of communication behaviours by physicians, nurses and nurse practitioners. BMJ quality & safety, 23(6), 483–489. [DOI] [PubMed] [Google Scholar]
  16. Reynolds ME (2020). It takes two to un-tango: Modulating continuous participation in joint activity. (Master’s thesis, The Ohio State University; ). [Google Scholar]
  17. Robinson DE, & Sorkin RD (1985). A contingent criterion model of computer assisted detection. Trends in ergonomics/human factors, 2, 75–82. [Google Scholar]
  18. Sanders EBN (1992). Converging perspectives: product development research for the 1990s. Design Management Journal (Former Series), 3(4), 49–54. [Google Scholar]
  19. Sheikh A, Bates DW, Wright A, & Cresswell K (Eds.). (2017). Key Advances in Clinical Informatics: Transforming Health Care through Health Information Technology. Academic Press. [Google Scholar]
  20. Smith PJ, McCoy CE, & Layton C (1997). Brittleness in the design of cooperative problem-solving systems: The effects on user performance. IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, 27(3), 360–371. [Google Scholar]
  21. Sorkin RD, & Woods DD (1985). Systems with human monitors: A signal detection analysis. Human-computer interaction, 1(1), 49–75. [Google Scholar]
  22. Stone PW (2009). Economic burden of healthcare-associated infections: an American perspective. Expert review of pharmacoeconomics & outcomes research, 9(5), 417–422. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Thackray RI, & Touchstone RM (1989). Detection efficiency on an air traffic control monitoring task with and without computer aiding. Aviation, Space, and Environmental Medicine. [PubMed] [Google Scholar]
  24. Voidazan S, Albu S, Toth R, Grigorescu B, Rachita A, & Moldovan I (2020). Healthcare Associated Infections—A New Pathology in Medical Practice?. International journal of environmental research and public health, 17(3), 760. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Watts-Perotti J, & Woods DD (2009). Cooperative advocacy: an approach for integrating diverse perspectives in anomaly response. Computer Supported Cooperative Work (CSCW), 18(2–3), 175–198. [Google Scholar]
  26. Wickens CD, & Dixon SR (2007). The benefits of imperfect diagnostic automation: A synthesis of the literature. Theoretical Issues in Ergonomics Science, 8(3), 201–212. [Google Scholar]
  27. Woods DD (1985). Cognitive technologies: The design of joint human-machine cognitive systems. AI magazine, 6(4), 86–92. [Google Scholar]
  28. Woods DD (2018). The theory of graceful extensibility: basic rules that govern adaptive systems. Environment Systems and Decisions, 38(4), 433–457. [Google Scholar]
  29. Woods DD, & Branlat M (2011). Basic patterns in how adaptive systems fail. Resilience engineering in practice, 2, 1–21. [Google Scholar]
  30. Woods DD, & Hollnagel E (2006). Joint cognitive systems: Patterns in cognitive systems engineering. CRC Press. [Google Scholar]
  31. Woods DD, & Roth EM (1988). Cognitive Systems Engineering. In Helander M (Ed.), Handbook of Human-Computer Interaction. Elsevier. [Google Scholar]

RESOURCES