Toward Joint Activity Design: Augmenting User-Centered Design with Heuristics for Supporting Joint Activity

Abstract

From their common roots in Human Factors Engineering, Human-Centered Design and Cognitive Systems Engineering have drifted into distinct fields over the past three decades, each developing beneficial heuristics, design patterns, and evaluation methods for designing for individuals and teams, respectively. GeoHAI, a clinical decision support application for preventing hospital-acquired infection, has yielded positive results in early usability testing and is expected to test positively in supporting joint activity, which will be measured through the novel implementation of Joint Activity Monitoring . The design and implementation of this application provide a demonstration of the possibilities and necessities to unify the work of Human-Centered Design and Cognitive Systems Engineering when designing technologies that are usable and useful to individuals engaged in joint activity with machine counterparts and other people. We are calling this unified process Joint Activity Design, which supports designing for machines to be good team players.

INTRODUCTION

Cognitive Systems Engineering and Human-centered Design, though they share common roots in Human Factors Engineering, evolved over the last three decades as distinct practices with their own set of methods, competencies and perspectives, heuristics, design patterns, and evaluation methods. This divergence mirrored the differences in the technologies they were intended to design for. Human-centered design cohered as a response to digital technologies proliferating across a wider proportion of the population, ultimately reaching virtually everyone on the planet and becoming central to virtually all aspects of modern life (Cooper et al 2014). Cognitive Systems Engineering was founded as a response to the increasing complexity in sociotechnical systems, which began accelerating with the digitalization of high-stakes, high-uncertainty settings (Hollnagel and Woods 1982). We introduce Joint Activity Design (JAD) as a new design framework that combines the methods, perspectives, and deliverables of these two design traditions. JAD is uniquely suited to the design of solutions in which joint activity is critically important, including high-stakes, high-complexity, and high-uncertainty settings.

JAD emerges as we enter a paradigm shift in which highly automated, and therefore highly complex, systems are not contained to controlled industrial settings which are managed by professionals with high levels of expertise. This complexity is creeping into the work and lives of nearly all people. Automation highly influences the information that we read and important decisions in our lives, as well as the work that we do. Technologies that were once firmly in the jurisdiction of human-centered design are now behaving in similarly unanticipated ways as they once did only in “high-technology” fields such as commercial aviation (Smith et al 1997), power generation (Mumaw et al 2000), healthcare (Christian et al 2006), and, more recently, critical digital services (Woods 2020), which CSE sprang up to address. The mass distribution and increased reach of these automation technologies also expands the reach of the risks that they bring with them.

This transition to highly automated technologies with broad reach is becoming a central focus of thought leaders in the HCD space. Since 2010, Don Norman, author of The Design of Everyday Things, has shifted (or re-shifted) his focus from designing individual products to designing systems. Living with Complexity (2010) focused on dealing with the complexity of reality and course correcting from a focus on simplicity. His forthcoming book, Design for a Better World (2023), takes the scale even larger to focus on how individual design choices are coupled with larger systemic effects. John Maeda, author of The Laws of Simplicity and Vice President of Design and Artificial Intelligence at Microsoft, has transitioned his popular Design in Tech Report (2015–2020) to the Computational Experience (CX) Report (2021) to the Resilience Tech Report (2022–). His recent book How to Speak Machine (2019) was written in response to how the “new kinds of interactions with our increasingly intelligent devices and surroundings require a fundamental understanding of how computing works to maximize what we can make,” (Maeda 2019, p. xiii). The design is calling for new techniques and competencies. The merged thinking in JAD is a response to this call.

In this paper, we highlight the unique merging of the disciplines of Human-Centered Design and Cognitive Systems Engineering in the design of a clinical decision support (CDS) application for minimizing the spread of hospital-acquired infection (HAI). The development of this application, GeoHAI, represents one of the first executions of the emerging practice of Joint Activity Design.

GeoHAI has yielded positive results in early usability testing and is expected to effectively support joint activity, which will be measured through the implementation of Joint Activity Monitoring (Morey et al 2022), an in-situ approach to evaluating post-implementation performance. The design and implementation of this application will provide a demonstration of the potential to merge the work of Human-Centered Design and Cognitive Systems Engineering toward technologies that are usable and useful to individuals while supporting their ability to work as teams and maintain awareness of their position within a larger joint cognitive system composed of human and machine agents.

BACKGROUND

Human-Centered Design

Human-Centered Design (HCD), popularized by Don Norman in The Design of Everyday Things (Norman 2013), has served as the foundational framework for interface design since the late 1980s. Later work from Alan Cooper and others built upon these frameworks to demonstrate HCD in practice as Interaction Design (IxD) (Cooper 2014). User Experience Design (UX) has evolved to wrap these two frameworks into one practice and profession.

Heuristics for HCD. Norman’s seven principles for HCD—discoverability, feedback, conceptual model, affordances, signifiers, mappings, and constraints—serve as foundations for evaluating useful and usable designs (Norman 2013). These are also the foundation for Nielsen’s 10 usability heuristics, which are the standard usability heuristics used by professionals worldwide (Nielsen 1990). GeoHAI was designed with these heuristics, and tested positively in two rounds of usability testing in July and December 2021 with infection preventionists (IPs) at The Ohio State University Wexner Medical Center.

Cognitive Systems Engineering

Cognitive Systems Engineering (CSE), begun by David Woods and Erik Hollnagel with their 1983 paper “New Wine in New Bottles,” has developed an understanding of how people, technology, and work environments function as a joint cognitive system. Resilience Engineering (RE) has built on CSE to understand how these systems adapt in the face of change and surprise.

Heuristics for CSE. Rayo’s five heuristics for supporting macrocognitive functions in joint activity—support event detection, support sensemaking, support replanning, make the basic compact explicit, and design for shared common ground—serve as the foundation for supporting macrocognitive functions across teaming agents. (Rayo 2017). The GeoHAI design process included the use of Rayo’s heuristics, providing a tangible demonstration of their application and the need to couple these heuristics alongside HCD principles. We argue that, in practice, this coupling is necessary to design for joint activity.

Merging Cognitive Systems Engineering and User-Centered Design

Joint activity is where more than one agent contributes to the completion of a task or activity (Woods and Hollnagel 2006). Designing for joint activity focuses on explicitly supporting the inherent interdependence and facilitating the necessary coordination and collaboration of agents engaged in joint activity. In the language of Joint Activity Design, the concept of usability and the practices detailed in HCD are translated into how technology supports joint human-machine action. How a technology supports the remaining macrocognitive functions is equally, if not more, important and constitutes the majority of how JAD merges CSE and HCD.

METHODS

Techniques from CSE: Knowledge Elicitation, Cognitive Task Analyses, and Abstraction Networks

Since January 2020, the project team developing GeoHAI has been conducting interviews and tests with the Infection Prevention (IP) team at The Ohio State University Wexner Medical Center. Our team conducted initial cognitive task analyses (CTAs) based on preliminary secondary research. Interview questions were derived from these CTAs and interview results led to iterative refinements of the CTAs. An abstraction network (Figure 1) was created at the conclusion of the interviews by the research team while the design team was simultaneously beginning early design iterations (Tewani et al 2023, submitted). The designs were refined based on the CTAs and abstraction network. The first round of usability testing was conducted in July 2021 and designs continued to be iteratively refined through the second round of usability testing in December 2021. This work demonstrated the need for mutualistic teaming within the design team to support applications that yield the same benefit for other teams (Li et al 2021). Finally, the introduction of joint activity monitoring—the real-time continuous monitoring of performance over time and over varying challenges—is currently requiring further iteration in the interface design that was not anticipated. The requirement to support the ability to test joint performance in a team has helped refine the design to support such teaming.

Figure 1.

Post-interview Abstraction Network describing infection prevention.

By using a combination of user interviews (used by both CSE and HCD) and artifacts for communicating analysis at a system level from CSE, the team was able to develop design recommendations that expanded beyond features requested by the IPs. The cognitive systems engineer focused on representation design and the designer focused on user interaction and experience were able to use the abstraction of the system in combination with the scenario to begin to develop their ideas in the context of the IPs work and lived experience.

Design Patterns from CSE

A primary design output that came from the contribution of CSE was a persistent representation of HAI across the whole hospital (see Floor Overview in Figure 2). This was not aligned with the interview responses directly, but was vital to uphold the CSE’s responsibility to make the basic compact explicit and design for shared common ground. The representations designed by the CSEs were grounded in improving an understanding of HAI across units, a capability lacking but less prioritized by the IP team.

Figure 2.

An early iteration of GeoHAI. Usability testing led to more standard navigation elements for switching between MDROs and a new layout.

The cognitive systems engineering team developed animocks (Anders et al 2007) of representations for the application to meet the goals and functions outlined in the abstraction network. The initial representations were not interactive and primarily consisted of a series of dashboards displaying.

Design Patterns from HCD

The design of GeoHAI’s initial low-fidelity iterations was performed with multiple tools. Balsamiq was used to design and communicate the early, low-fidelity iterations of user interface, which included how to navigate across screens and how to interact with each screen. Early iterations of the data visualizations were constructed with Microsoft Powerpoint, Balsamiq’s limited palette of UI elements required an initial grounding in basic web design patterns derived from HCD, which remained for general navigation and basic controls but were replaced by bespoke interactive data visualizations for most screens.

As the team entered the later, high-fidelity iterations, designing for joint activity became a higher priority, the application designs were moved to Adobe XD to incorporate CSE-inspired design patterns. The team realized that compromises would have to be made both from a CSE perspective and an HCD perspective to meet the goals of the end users. Navigation of the application became a primary emphasis. Many of the ideas from early design work had focused on reducing data filtering by implementing analog longshot displays (Woods 1984) wherever possible to funnel viewers’ attention and facilitate navigation to important areas of the application and. Compromises had to be made to ensure that features were discoverable and recognizable within the pragmatic constraints of modern web applications. The number of primary modes were halved in order to develop a more robust conceptual model of the interface.

RESULTS

The final design consisted of four primary modes:

• Timeline Page: A timeline visualization of all HAI cases over a period of time across all floors in the hospital (Figure 2).

• Floor Map Page: A mapped visualization of all HAI cases over a period of time across up to two selected floors.

• Floor Overview Sidebar: An omnipresent sidebar displaying an abbreviated picture of all positive cases, the highest risk rooms, and connections between floors across all floors.

• Common Factors View: A summary of the rooms a patient traveled through that may have been contaminated. This view was later reduced to a pop-up accessed through the former views.

First Usability Test

The first usability test was conducted in July 2021 with five IPs. The test consisted of a combination of semi-structured interviews and participant observations. IPs were presented with an interactive design mockup of the application similar to the one shown in Figure 2 Participants were asked to carry out a series of tasks using the prototype while their actions were observed. During subsequent semi-structured interviews, participants provided feedback on the usability, usefulness, and desirability of the design. Several design features were not tested at this stage due to the constraints of the prototyping software, but participants were allowed to speculate on their merit.

Participants all responded positively to the interface and provided anecdotes of ways it could improve their workflow. The Floor Map Page posed the largest challenge as the multiple levels of encoding were not intuitive.

As a result of this test, the contrast between user interface elements was increased and navigation was streamlined to follow more common web application conventions.

Second Usability Test

The second usability test was conducted in December 2021 with three IPs. In addition to the mockup of the Overview Page, this test added a training handout, 10-minute training session, and an interactive prototype with a functional Floor Map page.

Participants all responded positively to the interface and expressed enthusiasm at its potential to improve their workflows. Participants were able to navigate tasks directly related to problem-solving to identify clusters of infections sharing a common source. Feedback during this test was more granular and led to improvements in navigation of the app and the elimination of the Common Factors Page. Many of the features of this page were integrated into expandable pop-ups that allowed the IPs to maintain an awareness of their place in the application while viewing more detailed data. This final split-screen design can be seen in Figure 3.

Figure 3.

The final design for the application removed the Common Factors page, embedding its features into expandable pop-ups that allowed IPs to maintain an awareness of their place in the application while viewing more detailed data.

DISCUSSION

The work on GeoHAI provides a glimpse of a path toward Joint Activity Design: a new discipline of design focused on explicitly supporting the inherent interdependence and facilitating the necessary coordination and collaboration of agents engaged in joint activity. Early interviews following the tradition of Human-Centered Design were augmented by analysis techniques from Cognitive Systems Engineering, yielding an application that not only can support the work of individual operators, but can also support coordination and collaboration across the organization and between operators and their machine counterparts. The application still needed to meet the needs of individuals and was analyzed through usability testing to positive reception from the IP management and selected interviewees. Subsequent design has focused on further enhancing the usability of the application and bringing it in line with modern web interface patterns, both for easier use and development. After implementation, the application will be tested for supporting joint activity through the implementation of Joint Activity Monitoring (Morey et al 2022).

This paper does not provide a comparison whether an HCD approach alone could have yielded the same or similar results. The authors assume that without the consideration of factors related to joint activity, the application would most likely effectively support the ability of IPs to observe their own floors in isolation. Further, the high cost of development of the bespoke analogical representations (Timeline, Floor Overview, and Map View) would most likely lead to the elimination of these features under a standard HCD model. These elements are supported by the heuristics for designing for joint activity, but are not required for enhancing the usability of the application. Arguably, these features pose a detriment to overall usability, as they are not standard web widgets. A more simple approach could have used simple numerical scores and foregone a map entirely, creating a dashboard of the number of patients per floor with an MDRO-related infection.

Together, HCD and CSE are a more robust team and support more resilient teams (Li et al 2022). This fusion is approached through teaching designers to support event detection, support sensemaking, support replanning, make the basic compact explicit, and design for shared common ground. Conversely, this relationship requires cognitive systems engineers to understand how to make features discoverable, provide feedback, support a conceptual model, and provide intuitive affordances, signifiers, mappings, and constraints, all within an understanding of modern user interface patterns and innovations.

CONCLUSION

In this paper, we have recounted how the heuristics for designing for joint activity (Rayo et al 2017) were used alongside processes, techniques and artifacts from CSE and HCD (Norman 2013) in the design of a novel clinical decision support application, GeoHAI. The unique process, artifacts, and heuristics of CSE have yielded distinctly different results than HCD methods and heuristics alone could. This has led to the creation of a novel user interface that supports work not only for individual IPs, but enables new collaboration and creation of common ground across the IP team and provides a solid foundation for integrating advanced analytics. For the first time, IPs can see how movement across the hospital affects their unit and call out concerns across units to prevent the spread of infection into or out of their unit. This affordance has the potential to increase reciprocity across the team, further supporting team resilience. Further evaluation results will be provided from final usability testing and joint activity testing after implementation in early 2023.