Bridging HCI and Implementation Science for Innovation Adoption and Public Health Impact

Abstract

Human computer interaction (HCI) and implementation science (IS) each have been applied to improve the adoption and delivery of innovative health interventions, and the two fields have complementary goals, foci, and methods. While the IS community increasingly draws on methods from HCI, there are many unrealized opportunities for HCI to draw from IS and to catalyze bidirectional collaborations. This workshop will explore similarities and differences between fields, with a goal of articulating a research agenda at their intersection.

1. BACKGROUND

Human-Computer Interaction (HCI) has changed how people engage with technologies and with each other, and many other fields now draw on HCI’s methods and processes to strengthen their own contributions and impact. This is particularly true across the allied health disciplines, which have increasingly incorporated HCI approaches to improve the accessibility and quality of healthcare services. Most notably, this includes combining approaches from human-centered design (HCD) and HCI with implementation science (IS), a field that examines the methods and processes that support real-world adoption and use of evidence-based innovations [6].

We propose this workshop to systematically discuss ways that HCI and IS can come together to improve the design and adoption of technology-mediated health interventions. A recent concept mapping study [5] revealed that experts in implementation and HCI had perspectives that both converged (e.g., trans-discipline clusters, which were all rated as high-priority) and diverged (e.g., in importance/feasibility ratings). These results provide a shared understanding of the alignment between implementation science and HCI and provide a key starting point for how HCI and IS can work together to increase the adoption and impact of new innovations. We believe continuing and broadening these conversations is an important step for broadly increasing the impact and reach of HCI research, and we outline key research challenges on the way to that vision. In our own work, for example, we have seen how HCI is transforming mental healthcare and support, with the potential to create more engaging, accessible, scalable, and timely interventions. Many HCI researchers are making important contributions to the mental health space, through innovative and low-burden techniques for detecting mental health challenges, new ways of connecting patients with clinicians and peer support, and in-the-moment automated support for both clinicians and patients. HCI methods also offer approaches for understanding people and their contexts and innovating to meet the unique needs and opportunities of each.

Among the organizers, we have drawn on these strengths to include HCI and design as foundational components of mental health research centers we have launched at the University of Washington and at Northwestern University, and we have worked to translate approaches from HCI and HCD to mental health [1, 2, 7, 9, 10, 13, 23] and to healthcare more broadly (e.g., cancer treatment [8]; management of chronic conditions [21]), including adapting and developing methods to address clinical environments [15, 17, 20]. In this work, HCI and HCD have been a strong complement to techniques and perspectives from IS, and teams including some of the organizers have developed frameworks for integrating HCD and implementation science [14, 16].

As we combined these fields in our work, we also started noticing ways in which the strengths of IS could advance HCI. In particular, while HCI excels at innovating and designing compelling and engaging experiences, as a field, we can sometimes lose perspective on innovation and engagement toward what end? Additionally, the HCI field has struggled with its ability to consistently steward its innovations into real-world practice, spurring a need to develop a more robust translational science of HCI [3].

These weaknesses of HCI–emphasis on novelty over impact and inconsistent translation to the real world–are strengths of IS. IS focuses on the measurement of outcomes that quantify how much and how well evidence-based innovations are used in the real-world, as well as various proximal measures (e.g., shifts in organizational climate) that support that use and enhance impact. It also focuses on ensuring that the continued process of adapting an innovation to specific users or settings does not inadvertently weaken or remove the core evidence-based components. Over the past two decades, IS has also developed a range of strategies for engaging with the diverse individuals and groups needed to support effective uptake and sustained use of an intervention across multiple system levels. These include (but are not limited to): training and support for organizational leaders to create climates conducive to innovation adoption, strategic communications targeting policymakers, coaching for frontline practitioners to promote new skill development, and empowering service recipients to play active roles in innovation rollouts.

While we see strong potential for bidirectional exchange and collaboration among HCI and implementation science, we thus far have seen more transfer from HCI to IS than vice-versa. Consequently, we believe that CHI 2023 is an opportune time for conversations that assess shared and divergent research interests, to articulate a research agenda at the intersection of HCI and IS, and to catalyze collaborations.

1.1. Two synergistic fields: HCI and Implementation Science

As background for this workshop, we briefly outline what we see as the strengths of HCI and implementation science. We then discuss where we see these strengths aligning and some preliminary work to integrate methods.

1.1.1. Contemporary Human Computer Interaction.

HCI brings together expertise from various fields to understand and improve problems in sociotechnical systems. While we anticipate that HCI will be familiar to most readers of this article, we describe how we–a set of HCI, IS, and health researchers–see contemporary HCI, so that readers can situate our perspectives on HCI relative to their own experiences.

We see HCI as combining strength in design process, theory (both its own and borrowed), understanding, and technology to analyze combinations of interested or affected parties and context to describe or prescribe sociotechnical systems. While HCI has increasingly discussed when technology might not be part of a solution, the field overall remains oriented toward technology. With that orientation, HCI focuses on both problem discovery and solving problems. HCI’s historic strengths include focusing on novelty, productivity, safety and risk management, usability, (computer) ethics, and, increasingly, engagement [4].

A core strength of HCI is its ability to explore and to do so rapidly: to understand different contexts, what people want to do or can do, and to experiment with different design ideas through a range of probes and prototypes.

1.1.2. Contemporary Implementation Science.

IS is a multidisciplinary field focused on promoting the quality and effectiveness of services through methods that support the uptake of research evidence into routine practice [6]. Although implementation problems occur across a wide range of domains, contemporary IS is most advanced within healthcare, such as in public health, cancer control, mental health, primary care, and global health. Implementation processes tend to address on the core objectives of (1) using multilevel frameworks to identify explicit barriers and facilitators (i.e., determinants) to successfully implement innovations into practice, (2) developing methods or techniques (i.e., implementation strategies) to address determinants, (3) uncovering the mechanisms (i.e., mediating or explanatory processes) through which strategies operate [12], and (4) evaluating outcomes that describe how well or how much an innovation is being used (e.g., adoption, sustainment) [19]. To realize these goals, implementation studies range from exploratory to experimental and commonly apply a range of qualitative, quantitative, and mixed methods.

Although formative studies are critical to successful implementation processes [11], the ultimate goal of implementation research is frequently hypothesis-driven evaluations of the outcomes produced by specific implementation strategies [22]. This focus on real-world impact and large-scale improvement (i.e., how well something works and for how many people) is a major strength of implementation science. Among the core strengths of implementation science is its dedication to using relatively comprehensive, multilevel frameworks to understand individuals, settings, and implementation processes as well as drive intervention and measurement approaches. There has also been considerable recent attention on ensuring the pragmatism and methodological rigor of assessment instruments through efforts to (1) develop high-quality instruments and (2) harmonize measurement approaches across projects. Quality measurement has also become increasingly critical as the implementation field has moved toward identifying and testing the mechanisms and causal pathways through which implementation strategies impact implementation outcomes [12].

1.2. Potential bridges and intersections

HCI and IS have many similarities, including a focus not just on content of health interventions but also on their delivery mechanisms, engagement with end-users and other closely affected people, and recognition of the importance of context. They also share many methods. Beyond these similarities, the two fields also have many complementary strengths (Table 1).

Table 1:

Complementary foci of contemporary human-computer interaction and implementation science, with the caveat that many exceptions exist in each field.

Dimensions	Human Computer Interaction	Implementation Science
Scale	Single system/platform as element of sociotechnical systems	Multi-system/multi-level
Focus	Technology innovation & impact, including adaptation and scale	Strategies to facilitate translation to practice; Adaptation and tailoring of evidence-based approaches to context
Participants	Primary users, secondary users, other immediately affected people	Multi-level, including policymakers, organizational leaders, service providers, and/or service recipients
Goals	Increase engagement & proximal effects	Implementation and service outcomes
Duration	Short to intermediate	Long term (sustainment)
Research approach	Discovery, process oriented with a focus on user-centered and participatory methods	Ultimately hypothesis driven (often following early formative work), outcomes oriented, validation
Theory	Bricolage of theory from many fields as lens into a topic or setting. Model-building to explain use and context; design patterns for describing transferable approaches to addressing problems	“Classic” theories drawn from psychology, sociology, and organizational theory [18]; Frameworks for understanding possible factors affecting implementation and strategies for addressing those; Development of implementation-specific theories

Despite this potential, collaboration between the two fields has been sparse. A team including three of the organizers (Dopp, Lyon, and Munson) has gathered qualitative data from HCI and IS experts about challenges to their collaboration. Preliminary content analysis identified ten themes. Five related to differences between implementation and UCD experts (i.e., 1. different disciplinary and training backgrounds, 2. disagreement on prioritizing design versus implementation, 3. selection of appropriate research methods, 4. orientation toward innovation rigidity versus adaptability, and 5. different requirements, expectations, and incentives). An additional five were common challenges to collaboration shared across disciplines (1. barriers to interdisciplinary networking, 2. difficulties obtaining funding that combines HCI and IS, 3. a lack of sustained collaboration across design and implementation phases, 4. perspectives that design’s complexity is underappreciated, and 5. challenges working to effectively engage interested and affected people and groups). A summary of these themes will be distributed prior to the workshop and each represents a potential discussion point. In particular, the common challenges represent potential leverage points for group problem solving.

1.3. Toward a research agenda at the intersection of HCI and IS

The objective of this workshop is to map commonalities, unique contributions, and untapped opportunities for collaboration between HCI and IS with the goal of moving innovations into real-world service settings and articulating a research agenda at the intersection of HCI and IS. While we propose that this agenda be developed through position papers and conversations with workshop participants, we offer the following directions for conversation:

• Approaches balancing innovations that facilitate engagement while preserving intervention fidelity (e.g., maintaining the mechanisms that make the intervention work) in design and evaluation.

• Approaches for information gathering and assessment of proximal and distal outcomes throughout design and implementation processes.

• Approaches for enhancing innovation application and impact (e.g, best practices for communicating prescriptive guidance for IS in design; moving beyond frameworks – of which IS already has 150+)

• Translating HCI and IS knowledge to non-researchers (e.g., those in industry, organizational administrators, practitioners), including dispelling common myths

We also invite conversations about how to best organize this work (e.g., when do fields need to collaborate, versus learn from each other?) and the role of funding and incentive structures (current or alternative) in shaping and supporting the needed research.

2. ORGANIZERS

Aaron Lyon, PhD (contact) – Professor, University of Washington. Improving the accessibility and effectiveness of community-based health services, redesigning mental health interventions (e.g., psychotherapies) to improve their adoption potential, and developing implementation strategies to support innovation adoption and use.

Sean Munson, PhD – Associate Professor, University of Washington. Bridging IS with HCI and HCD to design and evaluate health interventions, with a focus on supporting collaboration.

Madhu Reddy, PhD – Professor, UC Irvine. Designing and using health information technologies in clinical settings to enhance clinical collaboration. Understanding and addressing organizational and design challenges related to digital health applications.

Stephen Schueller, PhD - Associate Professor, UC Irvine. Improving mental health services by expanding access and improving accessibility. Development, evaluation, and implementation of digital mental health products in diverse settings and populations.

Elena Agapie, PhD – Assistant Professor, UC Irvine. She studies and designs technologies that draw on health evidence-based interventions. Her research also identifies challenges and opportunities for collaboration at the intersection of HCI and Health.

Lana Yarosh, PhD – Associate Professor, Computer Science & Engineering, University of Minnesota. Lana has designed and deployed computational interventions for mental health and substance use disorders. Her work identifies IS as an important future direction to advance the potential positive impact of computing in health.

Alex Dopp, PhD – Behavioral/Social Scientist, RAND Corporation. An implementation scientist and child clinical psychologist, he studies the use of research evidence, and related policy implications, for improving youth mental health and substance use services. His research is grounded in an interdisciplinary team science approach.

Ulrica von Thiele Schwarz, PhD - Professor in Psychology, Mälardalen University and Karolinska Institutet, Sweden. Studies the design, implementation and evaluation of changes taking place in the workplace, including both evidence-based interventions and technical innovations, from a working life perspective.

Gavin Doherty, D.Phil. - Associate Professor, School of Computer Science & Statistics, Trinity College Dublin. Design of technologies for healthcare. Developing systems to increase access, increase engagement, and improve the outcomes of mental health interventions.

Andrea Graham, PhD – Assistant Professor, Northwestern University. Designing, optimizing, and implementing digital health interventions. Understanding the costs of treatment that impact adoption of interventions in practice.

Kaylee Kruzan, PhD – Research Assistant Professor, Northwestern University. Leveraging ubiquitous technologies to design theory-informed, accessible, and scalable digital mental health interventions to improve health outcomes for young people who are under-represented in traditional treatment settings.

Rachel Kornfeld, PhD – Research Assistant Professor, Northwestern University. Supportive communication within digital mental health interventions, including communication with peers, mental health professionals, and automated messaging systems.

3. PRE-WORKSHOP PLANS

3.1. Participant and Recruitment

We will recruit participants who have experience in both IS and HCI as well as those who have only experience in one domain or the other (but who have interest in, or at least curiosity about, both). We will work to ensure that our workshop includes perspectives from people who have experience and interest in working with historically marginalized populations and addressing disparities in healthcare access and quality.

We plan to recruit participants via ACM and field-specific list-servs, social media channels (institutional social media presences, the Society for Implementation Research Collaboration, Global Implementation Society, etc.), as well as via the organizers’ professional networks to reach potential participants from related fields such as improvement science.

3.2. Submissions

Prospective participants will be asked to submit a 1000-word position statement based on one of two prompts, designed to evoke opportunities for collaboration between HCI and IS to address health service impact:

• Share a case study that encountered substantial barriers to scale up or barriers to scale out (i.e., translating to another setting) for an innovation, including how these barriers were successfully (or unsuccessfully) addressed.

• Share a critical reflection on how one’s current work or research interests relate to the intersection of HCI and implementation science and what they think this can contribute to the discussion.

Regardless of prompt, participants will be instructed to write their position papers for a broad audience, avoiding field-specific jargon, to allow participants from diverse disciplines to understand and engage with their position.

3.3. Selection Process

Submissions will be reviewed by an interdisciplinary panel of organizers according to the following criteria: (1) Given the prominence of implementation science in healthcare, we will focus on researchers and industry leaders who work in the health space. (2) In selecting participants, we will aim for a diverse group that includes faculty, graduate students, practitioners, researchers from multiple relevant fields, and those who have done substantial work in the field alongside those interested in becoming involved but who have not yet made a significant contribution. (3) We will also strive to recruit and involve industry partners who work on (and are concerned with) these issues within the major organizations that work to scale up promising interventions.

If a submission is not accessible, we will refer the authors to SIGACCESS guidance on accessible PDFs for revision before posting it on the website. We will support this process to the extent possible, as authors new to ACM may be unfamiliar process, or not have access to the tools, for preparing accessible PDFs.

4. WORKSHOP FORMAT

We propose a synchronous, virtual event approximately one month in advance of the one-day, in person workshop at CHI 2023. While this choice splits conversations, this allows us to ensure high-quality participation in each format without being dependent on Internet connectivity or high-quality audiovisual conferencing equipment at the venue.

The virtual event will last for up to three hours and focus on the following: (1) overviews of HCI and IS with a goal of ensuring a baseline understanding among all participants, (2) large group discussion of points of intersection between HCI and IS, and (3) reflecting on the presentations and discussions and organizing themes to be shared during the in-person workshop at the CHI conference.

Those who can attend CHI in Hamburg will then participate at a one-day in-person workshop. This in-person workshop will maximize opportunities for collaborative idea generation, creativity, and community-building. We anticipate a maximum of 30 participants. Supplies needed will be a projector, self-stick flip charts, post-it notes, and markers, all of which the organizers can provide.

4.1. Asynchronous Engagement

All workshop information, including recruiting and submission information, the information contained in this proposal, and accepted submissions, will be available on a public website before, during, and after the workshop as both an organizing tool and record of the day’s progress.

Additionally, in advance of the virtual workshop, we will create a Slack workspace for participants in both workshops and any co-authors on position papers who are interested in the topic but may not be able to attend. This workspace will be private, to facilitate discussion that builds toward a shared understanding across both the virtual and in-person events, and the time between and after. During the in-person event, we will rotate shared note taking responsibilities and make informal notes available to remote participants.

In addition to encouraging all authors to follow best practices for accessible publications and presentations (which we will share with authors), we will distribute a survey about accessibility needs for the events in advance and work with participants and CHI organizers to address those needs. For this, we will need guidance from the accessibility and workshop chairs about what kinds of support (e.g., in-person captioners?) will be organized through the conference and which are the responsibilities of organizers or participants.

4.2. Workshop Goals & Structure

The primary goals of the workshop will be to map commonalities, unique contributions, and untapped opportunities for collaboration between HCI and IS with the goal of moving innovations into real-world service settings. Through this, we will construct a research and practice agenda that leverages the strengths of each field and their synergistic contributions.

This agenda will support development of two summary articles, one written for a HCI audience (e.g., a summary article for Interactions) and one written for an implementation audience (e.g., a summary article in Implementation Research and Practice).

A secondary goal of this workshop is to facilitate new collaborations at the intersection of HCI and IS. Specifically, we will solicit small and manageable commitments from attendees surrounding what they will do in the next six months to promote these collaborations. Examples might include setting up at least one check-in meeting with another attendee.

To accomplish these goals, the first portion of each workshop will provide brief overviews of HCI and IS with an emphasis on their overlap and unique contributions. The second will primarily be a working meeting with a focus on the interests and position statements of the participants, building on the conversations from the first session. Finally, time at the end will ensure not only larger group reflection, but concrete next steps to achieve the desired articles and set up future collaboration opportunities and communication mechanisms.

4.2.1. Virtual workshop agenda.

We list workshop time rather than a specific hour, given the virtual nature of this portion.

• 0:00–0:15: Brief introductions and summary of goals, as well as plans for including results of the virtual workshop in the in-person schedule.

• 0:15–0:45: Overview of HCI and IS, including highlights we identify in position papers.

• 0:45–1:00: Encourage participants to cluster what they see as key overlaps or tensions using a Miro board, which each cluster representing a breakout topic.

• 1:00–1:10: Break. Organizers refine groups as necessary.

• 1:10–2:00: Discussion within breakout groups about key in-sights from each field as well as topics where more work is needed.

• 2:00–2:25: Rotate. Form new groups with one or more members of each breakout group to share out discussion from their groups.

• 2:30–2:50: Plenary discussion and sharing.

• 2:50–3:00: Next steps. Participants make tiny commitments for the next six months, organizers note how discussion at this event will be carried forward into the agenda for the in-person workshop and plans for after.

4.2.2. In-person workshop agenda.

• 9:00 – 9:30: Introduction and Grounding: Brief introductions of the organizers and goals of the workshop; quick participant introductions.

• 9:30 – 10:15: Overviews of HCI and IS, including highlights and provocations, we identify in position papers, with a goal of ensuring a baseline understanding among all participants.

• 10:15 – 10:45: Large group discussion of points of intersection between HCI and IS.

• 10:45 – 11:15: Coffee break and informal discussion.

• 11:15 – 12:00: “Speed Networking:” In rotating pairs, participants discuss collectively relevant topics that they would like to explore further and establish shared interests; goal is for all participants to talk to each other.

• 12:00–12:45: Large Group Agenda-Setting: Reflecting on the presentations and speed networking discussions, the larger group synthesizes the general topics of conversation into organizing themes for the rest of the workshop.

• 12:45 – 1:45: Lunch: Seating arrangements to maximize new connections. Small group of organizers to spend part of this time setting up topic groups (n = 4–5) for the afternoon session.

• 1:45 – 3:30: Topic Breakout Groups: Breakout groups will be formed using the Open Space method (Owen, 2008). Participants propose discussion topics by posting them on a bulletin board and groups are formed around these topics. Open Space is guided by five principles: 1) whoever comes is the right person; 2) whenever it starts is the right time; 3) wherever it happens is the right place; 4) whatever happens is the only thing that could have happened; 5) when it is over, it is over.

• 3:30 – 4:30: Large Group Discussion/Reflection: Report-outs from small groups lead into discussion of major priorities moving forward and next steps.

• 4:30 – 5:00: Concrete Next Steps: Participants break into two groups based on their own interest in how to move forward, each facilitated by a relevant group of organizers. One group will focus on putting together outlines for the summary articles, while the other can focus on setting up new projects and collaborations.

• 6:00: Optional group dinner.

Throughout this schedule, organizers will engage in active facilitation practices. This includes ensuring a range of possible ways to contribute to discussions, establishing norms of conduct at the beginning of the workshop, and encouraging connection between new potential collaborators throughout the workshop.

5. POST-WORKSHOP PLANS

Following the workshop, paper-writing teams (each led by one of the workshop facilitators) will meet monthly to advance their manuscripts. To reduce the risk of siloing between the two groups, a subset of the facilitators will participate as contributing authors to both of the manuscripts. We will assess interest in one or more special issue (e.g., a special issue on methods for combining IS and HCI), though this will be a secondary goal, and may also inform the development of future, more focused workshops at the intersection of IS and HCI, to be hosted at future HCI, IS, or health informatics conferences.

Accessible, plain-language summaries of discussions, identified HCI and IS commonalities/unique contributions, and specific opportunities for collaboration will be posted on the workshop website and/or circulated as Medium posts.

We will work with our institutions’ offices of media relations and communications to facilitate broad awareness and understanding of workshop results via various media channels (e.g., newsletter, departmental/university websites) and we will engage social media to further disseminate findings to scientific and public communities (e.g., via Twitter).

6. CALL FOR PARTICIPATION

Human computer interaction (HCI) and implementation science (IS) each have been applied to improve the delivery and adoption of innovative health interventions. This workshop will explore similarities and differences between fields, with a goal of articulating a research agenda at their intersection.

We invite position papers on based on one of two prompts, designed to evoke opportunities for collaboration between HCI and IS to address health service impact:

• Share a case study that encountered substantial barriers to scale up or barriers to scale out (i.e., translating to another setting) for an innovation, including how these barriers were successfully (or unsuccessfully) addressed.

• Share a critical reflection on how one’s current work or research interests relate to the intersection of HCI and IS and what you think this will contribute to the discussion.

Position papers should be written in language approachable to researchers and practitioners working across fields and follow guidelines for accessible PDFs. No particular format is required., Maximum length is 1,000 words.

Selected participants will be a diverse group including faculty, graduate students, practitioners, researchers from multiple relevant fields, and those who have done substantial work in the field alongside those interested in becoming involved but who have not yet made a significant contribution.

For each selected position paper, at least one author must participate in 1. a remote workshop conducted in advance (date to be determined) and/or 2. an in-person workshop day at CHI in Hamburg and must register for at least one day of the conference.

CCS CONCEPTS.

• Human-centered computing; • Applied computing →Life and medical sciences;