HCI Contributions in Mental Health: A Modular Framework to Guide Psychosocial Intervention Design

Abstract

Many people prefer psychosocial interventions for mental health care or other concerns, but these interventions are often complex and unavailable in settings where people seek care. Intervention designers use technology to improve user experience or reach of interventions, and HCI researchers have made many contributions toward this goal. Both HCI and mental health researchers must navigate tensions between innovating on and adhering to the theories of change that guide intervention design. In this paper, we propose a framework that describes design briefs and evaluation approaches for HCI contributions at the scopes of capabilities, components, intervention systems, and intervention implementations. We show how theories of change (from mental health) can be translated into design briefs (in HCI), and that these translations can bridge and coordinate efforts across fields. It is our hope that this framework can support researchers in motivating, planning, conducting, and communicating work that advances psychosocial intervention design.

1. INTRODUCTION

Many people seeking mental health care prefer psychosocial interventions [7, 50, 53, 81, 101, 102, 127], such as psychotherapy, counselling, and case management, over pharmacological treatments [81]. These interventions are often complex, in that that they may involve many components, target a range of behaviours across a range of stakeholders, rely on expertise and skills from people delivering and receiving the intervention (and this expertise may vary), may be designed to work across a range of care settings, and may have varying levels of flexibility to adapt the intervention [45, 47, 59, 111]. Intervention designers often turn to technology to support parts or all of complex psychosocial interventions in mental health. Common aims include using technology as an opportunity to increase an intervention’s scale or reduce its cost, to reach people in more settings, to provide increased interactivity, and to increase the number of touch points with the intervention [52, 66, 68, 90, 95, 106, 112].

HCI researchers and practitioners have made extensive contributions to the design of psychosocial interventions in mental health – cf., [67, 106, 112, 122] for relevant reviews. These contributions are as varied as they are important: research has focused on developing tools for learning about mental health [42, 89], providing remote screening and measurement tools [23, 89, 94], designing tools to facilitate remote therapy [43, 97, 105, 109], providing increased support for existing therapies between sessions [52], and even developing new forms of mental health interventions inspired or made possible by new technology-mediated experiences [113]. From the design perspective, these contributions can operate at many different levels, such as developing new technical capabilities, proposing design patterns that can replace or extend parts of existing psychotherapies, enabling new therapeutic approaches, or by providing a deep understanding of people’s lived experiences with existing mental health services and current (digitally supported) tools to inform future design.

Prior literature has however also highlighted tensions and interdisciplinary challenges arising from the differences in the traditional approaches to intervention design and associated research practices within HCI and (mental) health fields [2, 67, 106, 112]. Some of the often cited friction points refer to the relative value of participants’ lived expertise versus clinical knowledge in intervention design; difference in emphasis placed on the creativity and innovation during the (ideally iterative) design process versus establishing efficacy of (ideally fixed and theory-derived) interventions on target populations; and the associated misalignments of focus on formative vs summative methodologies as core research agendas. Such disparities are often further entrenched through funding mechanisms in many countries [2], such as the differing expectations of the National Institutes of Health vs the National Science Foundation in the US, or the Engineering and Physical Sciences Research Council vs the Medical Research Council in the UK.

In this paper, we hope to contribute to the emerging literature that argues for the need of bridging these divides, so far often under the banner of incorporating user-centred design into the mental health intervention development (cf., [41, 56, 68, 85, 99]).

The core challenge across such bridging work is the need to balance (i) the interest in (socio)technical innovation often driven by so far unmet user needs that could be addressed by digital technologies; (ii) with the requirement of building interventions that are ‘psychologically active’– i.e., that the experiences enabled by the intervention successfully shift the user’s mental health state (ideally in a positive direction).

In other words, a successful design of a technology-enabled psychosocial intervention must:

1. Incorporate sufficiently ‘psychologically-active’ elements that lead to the mental health impacts (which should rely on established and validated ‘theories-of-change’ from psychology [19, 32])

2. Address users’ needs and lived challenges through design of innovative digital systems (where co-production and user-centred design methodologies are often an excellent fit)

It is this duality of interdisciplinary requirements that brings a dilemma: On one hand, too strong an emphasis on innovating or adapting intervention systems to provide an engaging user experience can run the risk of inadvertently removing psychosocially active interventions, especially when those components involve potentially uncomfortable experiences [67]. This leads to calls for such research to also clinically evaluate the resulting prototypes—such as randomised control trials in clinical populations (e.g., [106]). However, such methods are pragmatically difficult to achieve within typical HCI research and funding lifecycles (cf., also [2, 16, 60]). On the other hand, efforts that strictly implement existing psycho-active components can be insufficiently responsive to user needs and thus fail to gain adoption, or may miss out on innovation opportunities enabled by new technologies [99, 106, 112]. For example, the design of existing psycho-active components is often limited by the capabilities that were available at the time the theories were developed (often decades ago). In effect, strict reliance on existing intervention approaches without innovation of the underlying theories of change may constrain the kinds of interventions and intervention elements that designers imagine [104].

Framework overview:

To help address these issues, we propose a framework (Figure 1) that provides a common language and a conceptual model that could help unify these two perspectives in ways that are generative and open to technology-driven innovation, directly include the ‘psychological-active’ elements into the design process, and offer an interface to communicate our findings to our colleagues in the mental health space (and vice-versa).

Figure 1:

Our proposed framework argues that psycho-social interventions can be seen prescribing ’sets of experiences’ that are expected to lead to psychological effects for the participants …and thus can be translated into design briefs. This figure provides an overview of the four key types of design briefs—at the capability, component, intervention system, and intervention implementation level—that correspond to different functions within an intervention.

Our argument proceeds in two steps:

In the first step, we propose that a useful model for HCI designers is to see psychosocial interventions in mental health as prescribing a particular set of experience trajectories (cf., [11]) that are expected to lead to the psychological effects for the participant(s). In particular, we show how this framing can support designers and researchers in fluently translating between interventions’ theories of change (i.e., descriptions of the psycho-active elements and how these bring about the change in mental health) and design briefs (as an established way of specifying interaction design aims in HCI)¹.

Such re-framing of intervention design into experience design enables us to bridge the tension between psychological efficacy and design innovation – both now are positioned as complementary requirements / boundaries on the experience to be supported. In our experience, such framing is then understandable for clinicians and HCI researchers alike. In particular, we highlight how theories of change in mental health already (i) serve as a way to both orient intervention design targets and as a metric to drive evaluation later in the process; (ii) encapsulate the requirements and boundaries on the ‘types of experiences’ that intervention users need to experience for the mental health intervention to be successful; and (iii) are inherently ‘modular’ in the sense that these requirements are functional and—to a large extent—agnostic to exact implementation.

This brings us to the second step: we argue that approaching theories of change as modular design briefs gives technologists and designers flexibility to imagine new technical and interface capabilities that can implement parts of the functionality prescribed by theory of change; as well as propose new theory of change opportunities based on new technology. This opens up the possibility to contribute to the design of complex psychosocial interventions on multiple levels, without the need to always to do so only as part of a full intervention system and associated efficacy evaluation.

To illustrate this reasoning, we propose a vocabulary of four conceptual types of design briefs, corresponding to different functional ‘levels’ within a theory of change where design contributions can happen, and the associated evaluation methodologies required. Specifically, these include:

• capabilities (as the basic interaction design elements of a user’s interaction, whether these are scaffolded by technology or people — i.e., building blocks for interactions that could lead to psychological effects, close to ‘traditional’ HCI research);

• components (incorporating one or more capabilities to deliver a particular psychologically active experience trajectory — i.e., one step in a theory of change);

• intervention systems (combining multiple components into an intervention — i.e., implementing a full theory of change); and

• intervention implementations (as a particular adaptation of an intervention system tuned to a particular deployment context and addressing specific implementation barriers, often through introduction of new socio-technical components).

Across the framework, the goal is to articulate the different ways in which HCI can do influential innovation work and evaluate its success on foundational levels (such as new capabilities or components) without the requirements to immediately embed these into higher levels (systems or implementations). This can help sidestep the methodological difficulties with evaluation models that are not well aligned with traditional design practice in HCI (such as large scale, multi-site randomised controlled trials) but that are crucial for evaluating the psychological impact of intervention systems / implementations, and the corresponding uptake of designs in mental health community (cf., [60]). We also help articulate the types of contributions that HCI can deliver at each level. This includes how one might argue—and think about—the implications that work on one level can have for the higher levels (e.g., new capability enabling a new theory of change component) and lower levels (e.g., by a user study at a system level specifying a so-far unmet need within a particular component).

In summary, we hope this framework can streamline interdisciplinary collaborations, orient newcomers (or even long time researchers) to work at the intersection of HCI and complex psychosocial interventions, enable HCI researchers to do innovative work (while focusing evaluation energy where it is most useful), and create clarity for researchers, reviewers, and readers about what a given piece of research is—and is not—aiming to do. Further, by highlighting interactions between different contribution types, the framework has the potential to help researchers and practitioners envision innovation-to-implementation pathways, supporting the emerging cross-disciplinary field to build more effectively on each others’ work, and enhance the impact of HCI research on public health.

2. RELATED WORK

This paper was inspired by our own conversations, over a period of years, in which we sought to situate our work at the intersection of HCI and mental health relative to each other’s work, HCI, and the efforts of our clinical partners. Across these conversations, we found a need for a vocabulary that could describe the scope of each of our projects and contributions and link them to the underlying theories of change that describe complex psychosocial interventions.

As background before describing the framework, we describe complex psychosocial interventions and the challenges to their uptake and sustainment. We also highlight some of HCI’s contributions thus far to the design of complex psychosocial interventions, with a focus on mental health, as the framework of contributions should encompass more than our own work. We also briefly review forms of intermediate knowledge and contribution types in HCI that helped us develop goals for how our framework could facilitate the work of HCI researchers.

2.1. Complex psychosocial interventions for mental health

Psychosocial interventions (e.g., psychotherapy, counselling, and case management) involve interpersonal and/or informational activities, strategies, and techniques with a goal of improving functioning or wellbeing and reducing symptoms [47]. Many people, especially those in lower income, geriatric, marginalised, or rural populations, prefer this mode of treatment for mental health care [7, 50, 53, 81, 101, 102, 127] and substance abuse [34]. However, such modes of treatment are often unavailable in the non-specialty settings where many people seek care, due to delivery models that require one-to-one, in-person treatment from trained mental health professionals who are in high demand [57].

Importance of theories of change:

Like most interventions in the health domain, psychosocial interventions should be under-pinned by theories of change (also called program theories). For example, Moore and colleagues [86] refer this concept simply as the “understanding of the causal assumptions about how the intervention [should] work”, and the latest UK Medical Research Council guidance on developing complex interventions[111] defines it as a “description of how an intervention is expected to lead to its effects and under what conditions. It articulates the key components of the intervention and how they interact, the mechanisms of the intervention, the features of the context that are expected to influence those mechanisms, and how those mechanisms might influence the context. […] Implementation questions should be anticipated in a programme theory at all stages of intervention design”.

In other words, the theory of change describes the ensemble (and often sequence) of experiences that an intervention seeks to create, the expectations about how these experiences are supposed to result in the desired psychological change (i.e., what psychological mechanisms the intervention relies on), and the assumptions under which such mechanisms should be expected to operate (e.g., what are the socio-cultural contexts that would scope the effectiveness of the proposed approach).

As we discuss later, implementation science approaches are particularly concerned with the limits within the established theories of change (that is, for whom and under which contexts the interventions can ‘work’), as well as how such barriers can be identified and addressed. Each theory of change brings a wealth of assumptions about what needs to be true about the target populations. This is clear for constructs such as age (a cognitive restructuring intervention for 25-year-olds might not work for 5-year-olds). However, it is crucial to understand that similarly strong impacts on how likely a theory of change is to work for any given audience can come from socio-cultural aspects such as education or cultural background (cf., the ‘features of context that are expected to influence psychological mechanisms’ from the definition above). See Figure 2 for a specific example of how a (simplified) theory of change can look like for an established mental health intervention (behavioural activation).

Figure 2:

An simplified description of an example theory of change, in the context of a behavioural activation intervention, which is an approach often used to treat depression (cf., [77, 79, 80]).

Role of theories of change in design:

Theories of change have been used extensively to guide psychosocial intervention design and evaluation [18, 26, 32, 39]. A crucial aspect is that while theories of change describe an overall intervention, they are not prescriptive in how any given element must be achieved. This supports researchers and intervention designers in knowing not just what they might design for, but what they also might change: for example, they might identify an element of a theory of change that can be replaced with other elements that achieve the same function, add new elements and functions that enhance existin intervention functions, or notice elements whose functions are potentially unnecessary and could be removed; cf., emerging work on design of ‘modular interventions’ in psychology [73, 121]. In this way, theories of change are somewhat akin to Benford et al.’s experience trajectories [12] or journey maps [51], but with a greater focus on the psychological function of each element and less on the overall experience.

Broader design context of psychosocial interventions:

One reason that psychosocial interventions may be unavailable, despite people preferring them, is that the theories of change that describe them are often complex and have complicated interactions with the settings in which people would access them. The interventions may include several components, target multiple behaviours, rely on expertise and skills from people delivering and receiving the intervention, and may have varying levels of flexibility to adapt the intervention [111]. They may or may not integrate with various care settings [45, 59], which affects the resources available to support their adoption and sustainment [47].

As a result of their complexity, existing psychosocial interventions are often difficult to deliver at scale, can require extensive training and supervision [4, 40, 107], come with high user burden and cognitive load for the clients [71, 74, 88], and do not align well with existing individual or organisational socio-cultural practices [29, 88]. There are clear calls to increase the reach, scalability, affordability of interventions, to reduce the training needs on who can deliver interventions, and to make interventions more flexible and context sensitive to needs of those receiving and delivering the programmes [57].

To address challenges in the delivery of complex interventions, the health research community has sought to draw on technological innovations [10, 30, 41, 55, 57, 78, 117]. Health researchers have sought to use technology to help manage the complexity of existing interventions, making them usable in more settings and by more people. Technology’s capabilities for personalization, asynchronous engagement, and just-in-time support have been suggested as ways of increasing engagement and tailoring complex interventions to client and recipient needs and preferences [6, 10, 33, 48]. Flexibility in connecting via technology may also help address stigma associated with seeking mental health support [10, 21, 124]. Further, depending on their design, cost, and infrastructure support, use of technologies to deliver complex interventions has the potential to reduce or magnify disparities [20, 24, 119].

2.2. HCI research to advance complex, psychosocial interventions

Both the HCI and mental health fields see technology and human-centred design as having the potential to address challenges with adopting and sustaining complex psychosocial interventions in practice [41, 56, 72, 87, 92, 128].

HCI contributions to psychosocial mental health interventions to date are varied – cf., [106, 112] for a recent review. These include work spanning development of new technical capabilities that could be incorporated into interventions, such as digital interfaces for logging mood [94] or ability to infer mental health status based on sensed data or activity traces [23]. They also include fully realised systems designed to support mental health interventions [105, 110], some of which have been implemented in real-world delivery settings [103] — although, as Sanches et al. [106] note, these are often not fully validated. Intervention systems can be designed to replace existing in-person interventions with digitally mediated variations [66, 90, 95], to augment existing in-person interventions with supports in between sessions [52], or to imagine new interventions that could only exist in technology-mediated forms (e.g., [113]).

HCI has also been instrumental in advanced methods for understanding and designing complex interventions, especially in terms of introducing the focus on user-centred design and co-production methods into intervention design (e.g., [5, 8, 41, 63, 85, 99, 108, 129]. This work includes recommendations about specific methods, such as principles for co-design of mental health interventions with children and young people [56] or for participatory design [87]. Usability evaluation of complex interventions has also received attention, such as through the adaptation of usability scales [74], adaptation of the cognitive walkthrough method [69], and development of heuristics to guide design and implementation of complex interventions [88]. Further, methods like asynchronous remote communities [75] have supported building longitudinal understanding of participant needs and experiences with interventions [13] as well as provided a platform for prototyping new experiences [52], particularly among stigmatised or hard to reach populations [76, 96].

These developments also highlight the importance of contributions that provide empirical understanding of people’s lived experiences with existing interventions (rather than developing new systems directly). Such work can inspire and guide the design of interventions and their elements, as well as visions of future states. For example, research on people’s existing technology use for peer support highlighted the need for research on how to design for 1) matching peers on similarities beyond diagnosis; (2) enhancing accessibility; and (3) proactively mitigating risk through training and intervention [91]. At a larger scale, researchers have examined trends and practices in digital mental health and argued for approaches that decolonialise digital mental health, by moving away from rigid classifications and centring aspects of people’s lived experience including power relationships and the surveillance aspect of many digital mental health systems [93]. Discursive design has also made important contributions to considering future intervention designs. For example, Fit4Life describes a near-future system that creates a dystopian lived experience for its users, providing a vital vision of a future our field should seek to avoid [100]. These visions support consideration of how an intervention system that may be effective at its immediate goals of improving health outcomes, but, once implemented in a broader context, may face adoption issues, or worse, become adopted and cause harms.

Researchers have also called for methods that bridge design and medical approaches to research and development [8]. These approaches need to address tensions between innovations that may support a more engaging user experience with fidelity to theories of change that describe the intervention [68], and they should support designers in considering redesign of intervention artefacts alongside implementation strategies that might change organisations and people to better do the intervention [70, 99]. They also should help intervention designers navigate questions of what kinds of expertise are necessary to address a given set of research or design aims [2]. Approaches intended to do so include theory-based development of behaviour change interventions [15, 128] and design processes that combine human centered design and implementation science [56, 67, 83]. Intervention researchers have urged greater use of such approaches in complex psychosocial interventions, even when they are not digital [5, 63, 68]. These models outline extensions of the intervention design processes to incorporate organisational considerations that affect adoption and fidelity to the theory of change. We observe, however, that they focus at the scope of designing and evaluating an overall intervention system or implementation of an intervention system. Consequently, the guidance they offer may be ill-suited for HCI researchers seeking to contribute to the design of complex psychosocial interventions through the development of novel capabilities or components. A goal of this paper, then, is to describe a framework for navigating tensions between different approaches in HCI and the design of complex psychosocial interventions at a range of scopes.

2.3. Contributions and intermediate knowledge in mental health and HCI

Similarly to other design domains focusing on complex socio-technical systems, research on mental health interventions has been fraught with philosophy of science questions for both mental health and HCI communities. The key challenge is that while we can only design, develop, and test specific ‘point solutions’ of interaction designs or mental health interventions for a given population, our aims are—often—to learn about more general, transferable, or abstract approaches to addressing similar problems (for other populations, slightly different systems, etc).

As neither mental health nor HCI are prone to immutable, all-encompassing ‘laws’, the difficulty has been in identifying approaches to describing such intermediate knowledge in ways that retain the key insights, while enabling sufficient degrees-of-freedom to account for any context-to-context differences and adaptations [49]. Intermediate knowledge can be generative (annotated portfolios, methods and tools, and design patterns) or evaluative (heuristics, experiential qualities, and criticism), and both mental health and HCI research have discussed various needs for and types of intermediate knowledge in their fields. In what follows, we point to the discussions on these topics within mental health (e.g., form vs function, or identification of ‘intervention principles’), and HCI design more broadly (e.g., around design patterns, heuristics).

Rather than highlight a definitive form of intermediate knowledge, this section highlights the difficulty of the issues, strengths of some approaches, and dissatisfaction with the current state of intermediate knowledge across HCI and mental health. It also shows ways in which these concerns directly impact design within the mental health space: for example, any theory of change is in fact an attempt at describing a (more or less) general set of approaches that are likely to help some people, in some situations, if implemented well. In this respect, the tension between general patterns and point solutions is at the heart of designing digital interventions in this space.

2.3.1. Intermediate knowledge in mental health.

One of the ways of addressing these questions in mental health and implementation science is the distinction between ‘form’ and ‘function’ [45, 46, 54, 59]. In lay terms, the key challenge is how one describes the active components underpinning the theory of change for a given intervention — that is, how can we specify the types of experiences that clients should go through with sufficient detail (to know what is supposed to be implemented) but also enough freedom for adaptation (as it is clear that exactly the same interaction will not work for all)?

Traditionally, this has been done by describing the ‘form’ of the intervention: for example, providing a specific, standardised module (e.g., what a therapist should say and what exercises are to be done), with the expectation that implementation of such a module with fidelity is what would be expected to deliver the effects. In design, this leads to what we would call a point solution, such as standardising an interaction design construct (such as a ‘leaderboard’) by a specific implementation of a solution (such as app implementing a very specific version of leaderboards).

The alternative is to describe the ‘function’ of what the given module is aiming to achieve and the constraints on potential implementations, and allow the implementation of that function to vary depending on the context. We note that these ideas on the importance—and difficulty—of outlining such intermediate knowledge emerge in psychology repeatedly, potentially under different terms such as kernels [35], practice elements [22], intervention principles [84], principles of change [37], or describing interventions as requiring a “theoretical deconstruction into components and then an agreement about permissible and prohibited variation in the delivery of those components” [111]. For designers, such functional descriptions of target experiences for the users can serve as generative (e.g., “how might we achieve this?”) as well as evaluative (“does our intervention / component achieve this function?”) design targets.

2.3.2. Intermediate knowledge in HCI.

We see analogous discussions in HCI around the questions of categorising and articulating the design knowledge generated through empirical, technical, or theoretical research, and there have been many efforts to define intermediate knowledge that balances providing sufficiently prescriptive guidance while leaving researchers and designers degrees of freedom in which to innovate.

One of the best-known forms of intermediate knowledge in HCI is design patterns. Architect Christopher Alexander introduced design patterns as a general, repeatable approach to addressing a particular kind of problem. Such problems could occur at a range of scopes–the book was subtitled towns, buildings, and construction [3]. The idea of design patterns carried over into software engineering [98] and interface design [118]. However, as HCI design patterns gained prevalence and instantiation into stencil libraries and other pattern libraries, Höök and Löwgren [49] critiqued how the associated ease of reuse may encourage researchers and designers to reuse existing patterns rather than imagine new interactions, what Höök and Löwgren describe as “not a fruitful starting point for a knowledge-oriented academic discourse.”

Höök and Löwgren introduced strong concepts as an alternative form of generative intermediate knowledge. A strong concept is design knowledge that is “generative and carries a core design idea, cutting across particular use situations and even application domains; concerned with interactive behaviour, not static appearance; is a design element and a part of an artefact and, at the same time, speaks of a use practice and behaviour over time.” They describe social navigation and seamfulness as two examples; in a paper reflecting on efforts to translate the trajectories conceptual framework, Velt, Benford, and Reeves note that trajectories could also be seen as a strong concept [120].

In their work on theory-guided design of social computing systems, Kraut and Resnick also respond to limitations they perceive of design patterns with design claims [62]. Design claims differ from design patterns in that they use theory to make casual claims about the effects of using a specific design choice. Design claims also include conditions necessary for the claim to be true; this might include attributes of the person using the system, the associated task, or the context of use. Kraut and Resnick also point out that the use of theory in design claims allows for researchers to make claims about the potential effect of an interface even before the existence of evidence that the particular interface works. Consequently, use of theory to make design claims can inspire new interfaces, and so design claims also respond to HCI’s need for generative intermediate knowledge.

Other conversations in HCI have emphasised the types of contributions different kinds of knowledge can make (e.g., [126]). One example comes from systems research: Fogarty provides a framework in the workshop paper “Code and Contribution in Interactive Systems Research” [38]. This framework describes how HCI systems contributions can operate at the level of techniques and functionality. Systems contributions typically must address both technique and functionality even though many only introduce novelty at one level. A researcher may combine novel and known functionalities to create a novel sytem. However, they might also combine entirely known techniques to contribute a novel functionality, or they might contribute a novel technique that supports better achieving a known functionality. Beyond directly building systems, researchers may also articulate a research sketch or vision with some functionalities unimplemented, but described in ways that motivate the present or future systems contributions. Overall, such framework can support systems researchers in considering what kinds of contributions they might make with their work, and establishing relationships between novel and known techniques and functionality can help to motivate research – for both researchers and readers seeking to understand a work’s contributions.

2.3.3. Design briefs as a bridge between mental health and HCI intermediate knowledge.

In this paper, we draw on design briefs as a form of intermediate knowledge. We see a design brief as encapsulating a set of constraints and expectations on the experience trajectories for those using the system, as well as providing a specification of contextual or user-specific constraints and needs.

This approach to design briefs emphasises the required psychological or interaction-design functionality, analogously to the ‘function’ part of the form-function debate in the previous section: various design solutions may achieve each design brief and so a brief is prescriptive about what ‘must occur’ but not about ‘how’. As a result, like design claims or strong concepts, well-crafted design briefs can inspire researchers and designers to innovate on ways to achieve that functionality, while codifying a set of knowledge about what psychological or interaction design experiences is necessary (e.g., given a combination of specific theory of change and knowledge about unmet user needs from empirical work). Additionally, the description of what ‘must occur’ can also be used as an evaluation target to assess novel systems at each scale.

We argue that this approach to design briefs can then serve as the middle ground between the psychological efficacy and interaction design innovation: if each part of a theory of change is seen as prescribing a particular experience trajectory for those impacted by the intervention, the constraints on experience can be captured in a design brief, alongside any more traditional HCI descriptions of user needs and interaction design goals.

The rest of the paper is dedicated to showing how the experience trajectory constraints coming from psychology (relevant elements from the theory of change), technology development (possible system capabilities), and HCI knowledge (lived experience, socio-cultural context, user needs) can come together to combine into a design brief.

3. FRAMEWORK – ‘VOCABULARY’ OF DESIGN BRIEF LEVELS

Our goal is to provide a language that can help combine these disparate sets of requirements across psychology and design, in ways that are close to how we—as HCI designers—are already used to thinking about interactive systems design.

As alluded to in the introduction, the proposed framework is a result of a year-long conversation between the authors, drawing on our own experiences in research at the intersection of HCI and mental health, including how we each position our own research and coordinate efforts among our HCI and mental health collaborators. This included asynchronous conversations as well as intense synchronous collaborations. As we developed the framework and corresponding vocabulary, we increasingly used it in our own, separate projects, and we then revised it based on questions and feedback from collaborators. Once we had a complete set of vocabulary, we began sharing it with others who work in this area, through talks and drafts, and used feedback to further refine the framework. The text below is the resulting synthesis of these iterations.

Design briefs as a common language.

The insight emerging from the related work is the similarity of the role of theories of change in mental health to the role that design briefs play in interaction design: both aim to provide a set of constraints and requirements on the experience trajectories that should—or could—be generated for those impacted by the system (whether digital or human-led). In turn, this then enable us to draw out a joint focus in HCI and mental health on designing ‘the right experiences’ for people, which can transcend the interaction vs. intervention design boundaries and provide a shared mental model guiding where and how HCI contributions can happen in psychosocial intervention design.

For example, Figure 3 provides an illustration of this process with a particular element of the theory of change underpinning behaviour activation interventions (which was outlined on page 5). In particular, the theory of change can often be seen as bringing the ‘idealised’ experience trajectories (“people should do X”), HCI might bring the socio-technical lens and deep understanding of users’ needs (“people find it really hard to do A”; “…already do B”; “…would like to do C”), and the state of technical capabilites provides hard constraints on what is (newly) possible.

Figure 3:

A conceptual diagram of how the constraints and expectations on experience trajectories—coming from psychological theories of change, HCI understanding of lived contexts, and the available technical capabilities—can combine in a single design brief.

Such design briefs are expected to evolve over time (e.g., as the field develops new technology capabilities and/or better understanding of any socio-technical barriers). Similarly, each design brief brings a wealth socio-cultural and pragmatic assumptions regarding ‘for whom’ such experiences might ‘work’ – these will likely come from each of the three components: limits on the psychological mechanisms underpinning theory-of-change (cf., section 2.1), socio-technical and pragmatic concerns from HCI, and limitations inherent in existing technologies.

Four types of design briefs and their properties.

In the rest of this section, we propose a conceptual hierarchy of four such types of design briefs—describing possible capabilities, components, intervention systems, and intervention implementations of a mental health support—that come with different approaches to design, deployment, and evaluation.

We note that the differences between design brief levels should not be seen as a linear progression within any given project, but rather as an analogy of different magnification levels on a microscope: each zooming in to different functional constituents of what makes an intervention tick (e.g., the ‘chemical components’, ‘cells’, ‘full organism’, and ‘living environment’ of the intervention). As such, the purpose of this conceptual hierarchy of design briefs is to enable and allow researchers and practitioners to identify and scope their work in ways that connect it to meaningful (future) application, but also provide clear ‘interface’ and boundaries on how the developed capability / component / system could be embedded into a broader mental health intervention deployments.

In what follows, we describe each of these levels individually, with specific focus on the conceptual grounding of what should be included in a corresponding design brief, example descriptions for three mock projects (distress rockets, leaderboards, and goal setting), and the associated expectations for evaluations. Section 4 then discusses how we can use this vocabulary to articulate (HCI / mental health) contributions, and support innovation-to-implementation pathways across projects.

Figure 4 on page 9 gives an overview of the key points of this section; readers are also encouraged to revisit the Figure 1 from page 2 that provides a simplified visual overview of how these types related to each other, and their overall function in the intervention design.

Figure 4:

An overview of the key properties of the four types of design briefs, with focus on describing the role they play in the intervention development, and the key evaluation criteria associated with each type – a ’crib’ sheet summarising the core messages from the rest of this section.

3.1. Capabilities

…a basic building block of future interventions, a ‘technology piece doing X’ or a ‘human providing Y’

Capability design briefs.

Design briefs for a capability describe expected properties of interaction between the user and the system, but do not focus on the impact of any such interaction on the user’s mental state / health. In this sense, capability design briefs are closest to what a traditional systems’ contribution in HCI would be: answering questions such as “can we make the system / system+user do X?” In other words, capabilities are focused on enabling a particular interaction, and they do not necessarily specify how such interaction would be incorporated into a theory of change intervention. An example would be a design brief requiring us to “make it possible to launch projectiles that ‘look pretty’” … but we do not need to specify if such capability (“fireworks!”) is going to be embedded into a fireworks display at a child charity event or as a tool for football hooliganism.

This often means that capabilities are akin to blueprints or templates of interaction design that can be filled in with specific content, and thus re-usable across multiple therapeutic systems. These can be technological (e.g., ability to show a user’s current heart rate or access to an interactive worksheet), user-driven (e.g., capability of getting answer to a question within 30s), or even human-scaffolded (capability to receive feedback aligned with a specific theory from a trained therapist). The brief should thus provide boundaries and requirements of the types of interactions that should be enabled by this technical, socio-technical, or human-driven capability. From the perspective of HCI, is it most likely that innovation will come from envisioning and developing new technical / socio-technical capabilities, which might (or might not) be inspired by the capabilities currently delivered through human support; by identifying user needs that cannot be supported at this moment; or perhaps by interactions that are known to be very helpful but impossible-to-scale if these were to be provided by humans.

Evaluation requirements.

Given that the design briefs for capabilities focus on immediate interaction outcomes, well-known usability & interaction design methodologies are likely to be sufficient. In other words, all that we are attempting to prove—as designers of the technology—is that the new capability is able to support a pre-specified set of behaviours / interactions, including that it is usable and accessible. Capabilities that are focused on assessment or measurement should be evaluated for bias. As such, any number of methods commonly used across CHI communities (from lab studies, proof-of-concept examples, technical benchmarking etc) might be suitable, depending on the design brief focus.

We note that the linkage to mental health interventions is not necessarily embedded within the design brief, but could be ‘just’ articulated through outlining how such new capability would be of benefit to components / intervention systems in future— Section 4.1.1 discusses this in more depth.

Example design brief directions.

Distress rockets project:

To imagine a particular capability, say that someone believes that the ability to create illumination that can be seen for a certain distance at night would be valuable for summoning help. They specify a certain brightness, a duration for which it must be seen, core usability (e.g., that someone can initiate the signal) requirements, and perhaps some safety requirements for the device itself as they imagine how it might need to be transported and used. Based on this brief, a firework vendor creates a new firework flare—or may even find a product in their existing line—verifies that these fulfil the brightness, duration, and core usability requirements, and considers it a success. The rocket may have value for a variety of uses beyond the design brief, and that may be seen as a benefit but carry a risk for misuse.

Leaderboards project:

To shift to an example more prominently featured in HCI, consider leaderboards, a design element featured in Kraut & Resnick’s work [62]. Social comparison theory describes how people will seek out and process information about others in appraising their own behaviours and goals: this is sufficient knowledge to motivate a design brief that requests the creation of a feature that provides a list of people, ranked according to some defined function, with appropriate metadata. Evaluation at the capability level needs to ensure that the data and rankings correct (e.g., through proof of algorithmic correctness, or benchmarking tests) and that people correctly interpret the information (e.g., through simple visualisation experiments). It, however, need not assess the effects of receiving such information on how users’ appraise or change their behaviour.

Specifying and recalling goals project:

The theory of change for many complex psychosocial interventions, including behavioural activation, involves one or more iterations of people setting goals, pursuing them between sessions, and then appraising their progress in a future session [2]. This motivates the need for a capability that enables users to record specific, measurable, actionable, relatable, and timely (SMART) goals. Further requirements for such a capability could include the need to enable recall of such goals during the week for the client, and collaboratively in the next session. Again, this leads to a design brief for a capability, which itself be comprised of various known technical capabilities (collaborative text entry, structured input, secure data storage and access). Depending on the design brief details, an evaluation might be a simple lab study that focuses on usability of the interface and reliability of the technology, and does not need to evaluate the quality of the goals set, or whether the goals settings had any impact on user’s longer term behaviours.

3.2. Components

… a basic building block for mental health intervention, has a clear ‘psychological function’, i.e., the resulting users’ experience is affecting a specific part of the intervention ‘theory of change’ that eventually leads to impact on mental health.

Component design briefs.

The key difference between the components and capabilities is in the role that these are supposed to play in a given theory of change: while capabilities provide simple interaction design building blocks that can be used in a number of contexts; an intervention component should have a clearly defined psychological impact on those who are supposed to use it — and will likely serve as enabling a particular step within a broader theory-of-change. As a result, components will be likely composed from an ensemble of capabilities, some of which could be novel and exciting, and some of which might be well-established from an interaction design perspective but very exciting from an intervention design perspective as these are now being put to novel use (e.g, “let’s use fireworks to stop a street fight!”).

Design briefs for intervention components should thus focus on the psychological function that the component is supposed to have on the user. The brief should be clear about the parts of the user experience that are expected to be affected by this component: these could be on interaction level (e.g., user engagement with digital prompts will show higher reflective comments / self-awareness of mood), on socio-technical constructs (e.g., increased number of parent-child discussions of emotions per day), or on psychological constructs (e.g., survey based measure of change in beliefs about malleability of mood).

Design briefs should also include any pre-conditions which are necessary for this function to work (or likely to prevent it from working). In other words, the components will be clearly scoped in terms of the assumptions about the personal, pragmatic, or intervention contexts in which the component is supposed to be used – for example, a component brief might assume that the target participants are motivated to engage, have access to particular technologies or knowledge, or have successfully gone through a number of previous steps on a defined theory of change. In other words, while a component design brief may inherit some constraints from the overall intervention (what it must do, what kinds of capabilities are viable within the overall intervention, and the abilities and needs of the people who will interact with it), it is not focused on the overall theory of change or the other pieces that would be necessary to make it work in a full intervention system or implementation system.

Evaluation requirements.

The evaluation of components should focus on the psychological change that they are supposed to enact. However, the scope of the design brief should enable the researchers to focus on well-defined ‘proximal’ outcomes (cf., [60])]: i.e., changes that are expected to shift quickly and are closely linked to (ideally even a single) interaction that the component is aiming to support. For example, the researchers might be required to show that our component provides a (temporary!) boost in mood, enables the user to identify ‘well-crafted goal’, or helps users perceive a difficult conversation as more cooperative (than if they did not have access to the new component). In other words, there is no need to directly prove any lasting impact on users’ mental health (e.g., changes in depression or anxiety, which would require weeks-long large scale RCT), as long as one can argue how the functionality described in the design brief (and tested through proximal outcomes) would fit into established (or even hypothesised) theories of change. As a result, researchers should be able to mostly remain within the timescales and size of usual user-study evaluations within common HCI practice.

We note that even on this level, usability, accessibility, and equity of the component should continue to receive attention. Just because capabilities have been validated on their own does not mean that the ensemble of capabilities in a component remain usable, accessible, and equitable.

Example design brief directions.

Distress rockets project:

A search and rescue agency can propose a design brief for a distress signal component, which should enable a user to indicate that they encountered a problem at night, and be seen by people even if they are not in direct line of sight (such as in forested areas). A developer might realise that the rocket capability developed previously might be helpful, perhaps with a small carrying case (quality-of-life capability); and a pictorial training manual for the user (psychoeducation capability). The system evaluation now shifts to whether users are able to launch the rocket at a test site (e.g., at night), and verifying that it does get viewers’ attention and that they can correctly perceive the bearing to it and estimate range. With these functions achieved, they consider the evaluation a success.

Leaderboards project:

Designers of an online community consider using leaderboards to promote contributions, but first want to test the effects of leaderboards on two proximal measures: people’s goals and their self-appraisals of contributions. Guided by social comparison theory–which predicts that such comparisons may not have an effect unless the behaviour is relevant to the person, and that the effects of comparisons vary based on factors such as one’s standing in the comparison and whether someone has the self-efficacy to believe they can close a performance gap–they carefully select study participants and manipulate presentation of results to assess effects across a range of conditions described by theory. They find that in many situations, leaderboards cause people to set higher goals and appraise their contributions a bit lower, but that when leaderboards show someone as far below the top contributors, it either had no effect or was actually demotivating, and so the community designers take this into consideration for when and to whom they plan to show leaderboards.

Specifying and recalling goals project:

At this level, the focus shifts to a design brief requiring that people are provided support to set “good” goals, where what is ‘good’ may be derived from existing psychological theory on goals (e.g., SMART goals [17]) and/or from research describing what goals must do to support one or more interventions’ overall theories of change (e.g., a behavioural activation intervention [77, 80]). The recording goals capability from the previous section might be combined with expectations of other capabilities (such as access to a clinician who is trained to facilitate the process of setting high quality goals) and assumptions about the types of clients for whom the component should work (e.g., well motivated, brings expertise in their priorities, and have a clear understanding of what they have tried and what resources are available to them).

Researchers may evaluate the resulting component in a lab setting and focus on just the goal setting activity, or, to increase external validity, they might embed their goal setting component in an existing therapy that makes use of goal setting and evaluate it with clients and clinicians. Evaluation might include review of transcripts of the process by which goals were set and/or the resulting goals (as in past work on developing exercise plans [1]). Note, again, that the researchers can claim success in evaluation even without testing for any changes in clients’ distal outcomes (e.g., depression scores), as long as their component enables the therapist and a client to set ‘good’ enough goals (as defined by the design brief). As a result, it might be possible to test the value of the newly proposed component within a handful of sessions (if pragmatics are a concern), rather than requiring longer / more complicated study designs.

3.3. Intervention system

…an ensemble of intervention components which, together, should lead to psychological change on targeted distal mental health outcomes (e.g., anxiety / depression / self-harm …)

Intervention system design briefs.

Briefs at the scope of intervention systems focus on describing a full theory of change that should work in real-world use, even if that is in a best-case-scenario or many of the details around how to make the intervention work in practice (e.g., training, technology support, policies that make the service reimbursable, or even some technical components, such as user account creation) have yet to be worked out.

The design brief may include some description of for whom this should work (e.g., socio-cultural assumptions that are crucial for the theory of change and intervention system to work), and assumptions the team is willing to accept for the initial efficacy trials (e.g., level of training available for clinical partners, access to technology for clients).

Evaluation requirements.

In contrast to the components, the focus is on the ‘full’ functioning of the proposed intervention. Outcomes of interest can include feasibility, acceptability, appropriateness [123], engagement, and appropriation as well as assessment of proximal outcomes of individual (sets of) components and/or the actual health outcomes. The design brief may also include some experiential expectations (e.g., target usability or user burden) as well as a description of the range of socio-cultural indicators might mediate the hoped-for effects.

In effect, while focus on proximal outcomes is still an excellent practice (cf., [60] for similar discussion in the context of behaviour change), the intervention value should eventually be shown through impact on the main distal outcomes at least for some well-defined test population and context. As a result, it is nearly certain that the team would require methodologies emerging from mental health space to show how the system measurably affects users’ mental health indicators over time, similar to any other clinical intervention. These might include randomised controlled trials [16] or emerging methods such as optimisation studies [61], but they also could focus on approaches common in traditional HCI work, such as open trials (e.g., [64, 125]) which can help understand the appropriation and perceived effects of the system in-situ in the initial phases. We note that these studies might, or might not, be directly led by HCI researchers – see [15, 27] for a discussion of the evaluation opportunities in these contexts and the notion of ‘clinical hand-overs’.

Researchers may assess intervention fidelity (i.e., “is the intervention being delivered as designed?”), primarily as a way to assess validity of the evaluation of health outcomes. In other words, it is valuable to know whether if improvements in the health outcomes are not seen, is it because the intervention system does not produce the desired effects, or because the intervention is not actually being delivered or engaged with as designed? This, combined with the assessment of proximal measures, can be especially valuable for understanding the root cause of any negative results in the evaluation of the intervention system. As before, usability, accessibility, and equity of the ensemble of components may be evaluated, as well as measures such as user burden [116].

Example design brief directions.

Distress rockets project:

At this level, the rockets are now integrated into a system that support their safe transportation and quick, safe deployment. The team plans to test them in a specific scenario: can someone, on seeing a street fight break out after a football match, deploy them, does the security team see them, and can they respond with aid in time? They create a controlled environment in which they can conduct this test with a small number of different users and find that the use of the rockets is feasible, users and the security team rate it as appropriate, and users find it acceptable. They scale up their testing and add a condition in which they compare it to the timeliness of the response to the current way of requesting aid: ringing a large bell in a town square. This also succeeds and leads to responses more reliably, and they declare their intervention system, based on distress rockets, effective.

Leaderboards project:

For leaderboards, the focus shifts to evaluating their effects, which could be achieved in a field experiment, such as past work on the effects of leaderboards on contributions to online communities [44], or in a lab setting, such as a study that evaluated effects on when people stopped playing online games (and at what score) [25]. In each of these studies, the research questions were still centred on the leaderboard but the evaluation carried all the way through its effects on the overall system outcomes. An alternative approach might have built up a more complex theory of change that would lead to increased contributions to the online community through a variety of components and evaluated both quantitative outcomes (compared to the existing system) as well as people’s user experiences with the set of components as assembled into the system (e.g., perhaps each component is effective on its own, but the ensemble leads to an exhausting and burdensome experience of the system always asking for more).

An intervention that includes a goal setting component:

At this stage, researchers may integrate their previously developed goal setting component into an overall intervention system – this might be a new intervention (enabled by the existence of a new component), or it may replace the previous component in an existing intervention. In the case of behavioural activation (Figure 2), a technology-augmented goal setting component may be introduced, replacing a goal setting component in which the clinician and client previously used a paper worksheet to coordinate their goal setting efforts. At this level, the focus is no longer on goal setting capabilities or components, but on intervention systems that incorporate goal setting components alongside others and their overall feasibility, appropriateness, acceptability, and efficacy.

The system might be evaluated in a field deployment focused on the lived experience with the overall intervention system, an RCT in a highly resourced and incentivized setting, or an optimisation study that seeks to still tweak which components are needed, when, and how much. In the case of technology-augmented goal setting component being added to behavioural activation, researchers might examine long-term outcomes in comparison to behavioural activation delivered with the previous approach (an RCT). Additional attention may be paid to measuring and comparing clients’ goal setting skill development over time with each approach, especially if researchers are concerned that the technological components may be lending too much support, reducing opportunities to learn this important behavioural activation skill. Finally, a subset of clients and clinicians might be interviewed to understand how they experience this component within the larger intervention context.

3.4. Intervention implementation

… a fully fledged intervention system which has been fine-tuned to a particular context and ideally includes a set of socio technical components that address barriers to implementation in that context. The questions are no longer could this work, but rather ‘does’ it work in this context, and how can it be improved?

Intervention implementation design briefs.

Implementation design briefs will take into account factors that affect adoption and sustainment in particular settings. The key focus here is on how a given intervention system fails or could be amplified in a given real-world deployment scenario. In particular, even intervention systems that have been shown to be usable and effective in one setting may face barriers to adoption and ongoing use in a range of service settings–or even the same setting, once the supports associated with a research project are withdrawn.

At the intervention implementation level, the core challenge is often to leave the design brief for the intervention system (intended mechanisms of action, desired outcomes, and higher order constraints) intact while seeking to identify and develop components that can mitigate barriers to adoption and sustainment of the system. This may include implementation strategies integrated into the core intervention [58] and adjunctive interventions designed to increase people’s ability to engage in the primary intervention [114]. Researchers at the intersection of HCI and implementation science have called for increased attention to this perspective in the design process², envisioning intervention strategies that might better prepare or provide ongoing support for individuals and organisations as they adopt and use novel intervention systems [67].

The resulting design briefs will be akin to much CSCW research, where researchers have identified socio-technical components that can address fit of a given tool with a deployment context – the CSCW and HCI communities already have a long history of examining similar issues of adoption and sustainment of technical systems in other contexts, often with a goal of informing designs that better align with existing work practices, where work is located, and existing infrastructures [36]. The implementation science field has also identified a range of determinants for successful adoption and sustainment of an intervention [14, 31], adaptations made to interventions and implementation strategies to facilitate adoption and sustainment [82, 115], and causal pathways that describe mechanisms of change for implementation outcomes [65].

While HCI researchers should consider implementation concerns—at least sometimes—we caution against letting these concerns to overly constrain innovation in capabilities, components, and intervention systems. HCI research can motivate changes to implementation constraints by showing that an approach can be feasible and effective. For example, once telehealth or self-paced complex interventions have been shown to be effective and cost-effective, regulators have an incentive to make it easier to allow them and payers have more motivation to figure out how to reimburse for them. In this way, the evidence for a new intervention system can motivate new or changed design briefs for an implementation. On the other hand, researchers can also take the constraints in a well-defined and motivated design brief for an intervention implementation as, well, constraints, to drive innovation at other levels. For example, articulating the facilitators or barriers to adoption or sustainment in a given setting may inspire researchers to modify capabilities (e.g., new technologies that use available infrastructure), components (e.g., to be culturally responsive), or the intervention system (e.g., to be delivered within the short timeframe of a primary care visit followed by asynchronous work, possibly removing some mechanisms of the original intervention).

Evaluation requirements.

The research team will likely work with implementation science researchers and be embedded in a particular deployment context. If the aim is to determine the psychological effectiveness of the newly improved intervention, the methodological concerns from the previous level remain, and might be further amplified – for example by the need to employ factorial design or cluster-randomised experimentation. However, HCI research might also focus on other aspects of the implementation practice. For example, it might examine the lived experience with the implemented intervention, relying on the CSCW in-situ research methods as per above, with the resulting evaluations describing identified barriers and facilitators as well changes made to the intervention as people and organisations adopted and used it (in implementation science, these are known as reactive adaptations [115]). Additionally, scales such as user burden and implementation measures (e.g., acceptability of intervention, intervention appropriateness, and feasibility of intervention) continue to be valuable, and it is often the case that in non-idealised setting and trials, additional barriers will become salient. When organisations (e.g., health systems) and clinicians are involved, longitudinal assessment of intervention implementations can be especially important. While one might expect delivery of the intervention to improve with time (e.g., as clinicians become more skilled, as the organisation adapts to address remaining barriers to success or to accommodate new workflows). However, it is often the case that an implementation is delivered most reliably shortly after initial adoption and training, and then drifts, potentially reducing efficacy [9].

Example design brief directions.

Distress rockets project:

The distress rockets, having been shown to be effective in a carefully controlled study, are deployed. However, on the night of their first use, the home team has won. When a riotous crowd gathers, and someone fires a distress rocket, this is interpreted as a celebration. No help arrives and the football crowd only becomes more energised. Researchers develop a public awareness campaign that the unique colour and shape of this firework indicates distress.

Leaderboards project:

The online community adopts leaderboards, and contributions soar. While the effect eventually starts to wane, moderators are able to introduce other strategies to focus community members’ attention on the leaderboard at key times (e.g., a horror movie leaderboard for Halloween), reinvigorating contributions temporarily and meeting the community’s goals.

An intervention that includes a goal setting component:

Having found promising results in an RCT of technology-augmented behavioural activation (incorporating the new goal setting component as well as several other new components), the researchers then seek to scale this treatment out to more settings. After development of a clinician training plan, the intervention system is successful when deployed in clinics similar to that in which it was developed and initially evaluated. However, in clinics that have different resources and serve different patient populations, researchers observe that clients often arrive at sessions late and encounter many barriers (e.g., transportation to the session, housing security) that use much of the allocated time. In sessions that are supposed to focus on goal setting, goal setting is often crowded out by these other needs, and clients may leave without having fully developed a SMART goal or even selected a problem to work on. When clients do get to goal setting, they report that the goal setting component guides them toward goals that seem irrelevant to their priorities or resources, indicating that the component was not designed and evaluated with a sufficient diversity of participants. Finally, the interface for the technology-enhanced components is often hard to use on the devices available in these clinics and not all participants can access the digital action plans intended to support applying skills between sessions.

Taken together, these issues result in worse outcomes than the previous paper-based behavioural activation intervention. The clinic responds by (1) increasing the scheduling blocks for each session and making health navigators available to support clients with their barriers, (2) working with additional clients and clinicians to expand and adapt the examples suggested by the system, (3) re-designing the interface to work better on lower resolution screens, and (4) creating an option to print action plans and supporting materials at the end of each session, for clients to take home.

4. FRAMEWORK – THE ‘SO WHAT?’

The previous section has introduced a language which we hope can help both HCI and mental health researchers identify and articulate the range of different types of contributions across the innovation-to-implementation dimension in mental health. This section pivots to using this vocabulary to articulate how we expect it being used and useful within HCI research. In other words, how could this modular set of concepts help us think about how we—as designers—can envision, design, build, test, and talk about our new systems in the context of mental health interventions?

In particular, we argue for benefits in three interrelated ways, by: (i) focusing on design briefs as a bridge between mental health requirements and interaction design; (ii) articulating the scope of necessary evaluation … and thus also of what can be ignored at each level; and (iii) providing a conceptual structure to enable a more practical innovation-to-implementation pathway and collaboration between HCI and mental health sciences.

4.1. Design briefs as the bridge between mental health and interaction design – two contribution types

The focus on design briefs—as descriptions of experiences necessary for a complex psychosocial intervention ‘to work’—provide a bridge between interaction design choices and intervention requirements. As an implication, this enables a clear differentiation of two broad kinds of contributions—one focused on developing new systems (in response to a well known brief); and the other on identifying new design briefs as design challenges that should be addressed—together with a shared language to freely move from one to another, and from HCI to mental health domains.

4.1.1. Responding to a design brief.

…where the contribution involves developing (socio-)technical elements or systems and requires the designer to be able to provide some argumentation as to how these newly developed elements correspond to the functionality outlined in the brief.

For example, this might comprise:

1. developing a new capability (e.g., developing or adapting sensors to allow an ‘interaction A’);

2. creating a new app feature that delivers a specific component (e.g., providing in-the-moment support for emotion regulation effectively without a therapist engagement);

3. deploying a full intervention system in a test context (e.g., adapting a web-based application to deliver an online CBT intervention with a well-trained medical staff); or

4. working with implementation science researchers to identify a specific implementation barrier and then build on user-centred design methodologies to adapt existing components—or build new ones—to resolve it within the target deployment context (e.g., by transitioning a county hospital to digitally mediated therapy during COVID by developing a new remote training module for patients who struggle using the system).

In each of these cases, the contribution is about developing, adapting or testing (a set of) digital elements, and understanding the extent to which these can deliver on the presumed theory of change (as embedded in the level-specific design briefs). It is likely that most of the traditional HCI research—especially if pursued without clinical collaborators—would most comfortably lead on aspects (1) and (2); but we are also seeing an increased involvement in (3) and (4).

Interconnections between levels:

We argue, however, that contributions on each level must at least provide a ‘vision’ of how the newly developed element fits into the higher levels and how the function that it provides could be plausibly beneficial (although not necessarily, e.g., cost-effective at this time) for a well-defined intervention system.

1. For example, one could develop a new capability which is in direct response to a challenge identified with a crucial component of an existing intervention system—e.g., supporting clinicians’ feedback on cognitive reappraisal (component) as part of established online CBT (intervention system)—at which point the arguments are simple, such as a pointer to the description of the identified challenge.

2. Alternatively, one could develop a new component that provides a previously unknown capability (e.g., in-situ support for anger management through peer-support), which would then have to be argued for in more detail (e.g., drawing on existing psychological literature outlining the difficulty in applying anger management techniques as one of key challenges in existing therapies).

We note that in both cases, the researchers do not necessarily need to show how their new component can be embedded in any given therapy (or test that it improves its effects), as long as their conceptual motivation of the design brief functionality is sound, and they are able to show effects on more proximal—e.g., interaction level—outcomes as defined in the brief. However, the vision serves as a forcing function to motivate the potential usefulness of the developed capability/component to deliver psychological change, as well as articulate and consider downstream socio-technical and socio-cultural challenges even in ‘early’ stages. The descriptions in the previous sections have already hinted at this ability for scoped evaluation; and section 4.2 will return to this aspect in more detail.

4.1.2. Creating a new design brief.

…where the contribution involves articulation of a new design brief (such as through a careful analysis of in-situ appropriation of an existing mental health intervention system), and providing some evidence as to why the new functionality—as identified by the design brief—is needed or beneficial for a mental health intervention.

This type of contribution outlines how new design problems can be put forward—often by identifying challenges or opportunities in existing intervention elements. The aim is to provide a clearly-enough defined design brief (i.e., description of the necessary psychological function that the element is supposed to play), which is also well enough motivated so that others can take it up as a design challenge to provide a solution to the newly identified problem.

The resulting design briefs can roughly follow the same general structure as described below, with the type / form of specifics different depending on the level in the framework:

• [necessary] Requirements on ‘psychological function’ for the given element:

  → what needs to be true about user’s experience for it to be a psychologically ‘active’, together with argumentation about the psychological effects are likely (e.g., by referring to psychological literature, empirical data, or other believable argumentation); as well as the known limitations of the underlying theory change in terms of socio-cultural backgrounds and other key participants’ characteristics? As shown in Figure 2 this may include both requirements and optional elements (e.g., the option of including social support components for reinforcement in behavioural activation). We note that while capabilities may not directly deliver a psychological function on their own (cf., Section 3.1), it is important for researchers arguing for a capability’s utility in mental health to relate their design brief to known (or envisioned) components in which the capability could support an important psychological goal.

• [optional] Requirements & resources brought in by the specific study context:

  → what are the inherited requirements / resources that we have to rely on within the specific design context that the designers have chosen?

  The idea here is that these requirements are not necessary part of the design problem, or inherent constraint on the type of experience trajectories we need to support, but might instead be a result of specific partnerships, population lived experiences, research decisions, and similar. We note that this does not make them any less important for a successful design solution, but they are not ‘core’ part of the psychological problem we are trying to solve (i.e., they are not directly a requirement for the intermediate knowledge we are trying to generate).

• [optional] Predetermined choices about lower-level solutions:

  → what are choices we have already made about the types of solutions we will design / re-use in this project (e.g., the component has to rely on voice-assistants capabilities; or we have to embedded into therapy system X).

  Again, the idea is that these are not necessary requirements, but might be important considerations given the expertise of the research team, because the community is driven by a specific set of research questions (e.g., aim to investigate the limits of a specific mobile sensor capability), or due to pragmatic concerns emerging from target populations and implementation concerns (e.g., all our doctors use this type of technology already).

Identifying and well-defining a crucial, so-far-unsolved interaction design problem (associated with one of the design brief levels) is a crucial contribution in the context of the framework. We suggest that there are three common approaches through which such briefs will be most likely generated.

Empirical accounts:

The first relies on empirical user data – e.g., by seeing how existing systems (whether designed by the researchers or not) work in the real world, likely on the levels of implementation, system, or component. This might include identification of things we thought should work but do not (‘bad’ design), noticing previously unknown challenges with use (newly identified needs by observing barriers), or articulating opportunities to extend the existing brief through new elements (e.g., by co-production with target populations). Many empirical accounts in HCI contribute to partial design briefs. For example, “Design Opportunities for Mental Health Peer Support Technologies” presents an analysis of interviews with 18 people about how they currently use technology to access peer support for mental health as well as their visions of what kind of support they would like [91]. The analysis implies experiential aspects against which implementation systems might be evaluated, and it also implies partial design briefs for various components that could use peer support to deliver part of an intervention. The paper has informed more than 100 subsequent publications across mental health and care work. However, seen through the lens presented in this work, we also see opportunities where the work might have helped even more readers connect the results in their own work, such as by explicitly arguing for the development of certain components and catalysing research into the capabilities that could help realise them.

Literature reviews:

The second draws on literature based argumentation – e.g., by identifying a challenge in existing implementations / systems / components and suggest a new functionality that would address these (e.g., through illustrating how similar design problem have been addressed elsewhere). For example, Slovak and Antle et. al. [112] identify future opportunities for digitally supported emotion regulation interventions. They combined a review of learning sciences with psychological research on emotion regulation to propose a conceptual framework that distinguishes four core approaches to supporting emotion regulation training (didactic vs experiential; on-the-spot vs offline) and outlines the theory-driven intervention targets that HCI design could focus on. They then use this framework to map out existing HCI research, identify gaps (e.g., fragmentation along technological lines and lack of research focus on key psychological constructs such as cognitive reappraisal) and outline a set of opportunities for future work. In the language of this paper, their review provides set of design goals that guide how the research community can compare existing design briefs, and develop new ones.

Critique:

Finally, a crucial way of creating new design briefs can be through thoughtful critical questioning of existing approaches, whether these existing approaches are explicitly identified in literature and in current practice or are implied as an extrapolation of ongoing work (cf., [100]). For example, emerging literature around decolonising mental health [93] highlights problems in existing digital mental health technologies (e.g., power differentials, disregard for lived experiences, and inattention to structural contributors to mental health problems) and argues for different approaches that could lead to more equitable care. Work that highlights parts of accepted design briefs that should be seen as problematic and re-thought need not provide solutions, or even fully developed alternative design briefs, to make a valued contribution to the field.

4.2. Evaluation approaches for different design brief types

Second, the framework and vocabulary provides scope and argumentation for evaluating intervention elements within the ‘minimal’ requirements necessary for whichever level we are focused on (e.g., developing a new capability or component), without having to be immediately concerned with higher levels (e.g., whether the developed prototype was to be robust enough to be immediately deployed in a real-world setting).

In other words, each level provides us with an ‘excuse’ to not worry about some of the aspects that would be hard to evaluate (e.g., whether or not our new interaction element prototype is deployable at scale), while providing us with a language of why the tested element could still work if included in higher levels (thus both helping motivate our work and significance of our contribution, while hopefully reducing the HCI innovation-to-implementation gap). While we have alluded to this in previous examples, this section aims to provide an overview of how contributions on different levels will change in terms of their interaction design and evaluation targets, together with the likely types of HCI work required.

1. Innovating on capabilities (e.g., UIST / CHI / interaction design driven work):

   1. such as identification of new capabilities (e.g., on-going tracking) that enable previously impossible psychological functions (such as real-time feedback) that then lead to new intervention components and then intervention systems (e.g., JITAIs)
   2. capability that enables re-implementing delivery of existing psychological functions in better ways (e.g., automated reminder message on a phone rather than a call from an assistant).
2. Innovating components (e.g., CSCW / user-centred design work), either as:

   1. providing new / better psychological function implementation, e.g., by building on newly available capabilities;
   2. re-designing existing capabilities to better fit in with user’s needs (around delivering a previously supported psychological function).
3. Innovating intervention systems, regardless of whether that is through:

   1. combining and slightly adapting existing components in ‘better’ ways (such as to achieve better context-fit, or reducing UX barriers etc);
   2. exchanging prior components for better ones (such as enabling ‘therapist feedback’ daily rather than bi-weekly through automation);
   3. innovating on the theory of change by adding new components (e.g., by adding a new problem-solving-through-peer-support component to an online CBT).
4. Innovating on the intervention implementations, such as:

   1. developing collaboration and coordination tools that support distributing the work of an intervention over various roles in a health system, making it more sustainable;
   2. new training techniques that increase the number of people who can deliver an intervention well;
   3. building tools that support designers in adapting intervention elements to be more culturally responsive.

Within each level, the general structure of the evaluation remains the same: showing how the interactive elements have managed to support the functionality (i.e., set of experiences) as defined by the design brief. What is different, however, are the likely types of evaluation methods that will be sufficient to make claims about the match between the design, and the corresponding design brief; as discussed in the ‘evaluation requirements’ in sections 3.1-3.4.

Underlying tensions:

The two primary points we would like to again emphasise here are:

1. The key difference in evaluation scope between the focus on capabilities vs intervention components—i.e., one focusing on simple interaction design characteristics; the other on their psychological impact—but which seem to have been to date often conflated in HCI.

2. The change in required methodologies when moving from evaluating components to the focus on evaluating intervention systems, which bring much higher burden of evidence and more extensive and rigorous evaluation methods (like randomised control trials). Similarly to Klasnja et al. [60], we invite designers to be well aware of the benefits and costs associated with this choice, and be clear about which level of the design brief the project aims to address.

We note that such underlying tension between qualitative and quantitative evaluation methods has commonly crept up in our interdisciplinary projects. Our perception is that there is a need for quantitative evidence (following usual clinical/psychological methodologies) if one wants to be able to make claims about efficacy (i.e., whether or not the designed system ‘works’ in terms of reliably enabling psychological change). However, this does not diminish the importance of non-quantitative work (usually framed as process evaluation in psychology). This is crucial to develop an understanding of appropriation (i.e., how is the system actually used), identify existing socio-technical or psychological challenges and gaps in provision (i.e., developing new briefs for future work) and would also support mechanistic studies (e.g., developing an empirical understanding of ‘why’ the system might work for some but not others). In other words, while qualitative methods are crucial throughout the four levels—and are necessary to help identify or check for any socio-technical challenges—the parallel importance of quantitative evaluation grows on the system and implementation levels, as the targeted outcomes become more difficult to shift within individual interactions with the system. We refer the interested reader to an excellent discussion of this topic in related domains [28, 60].

4.3. Innovation to implementation pathway

The last role that we envision this framework to play is one connecting the two fields of HCI and complex psychosocial interventions (especially for mental health), and supporting a shortening of the existing challenges with moving innovation to real-world implementation.

Whether pursuing our own projects through to implementation or seeking to inspire others’ implementations, we believe that most HCI researchers working on complex interventions or their underlying components and capabilities would hope for their research to make an even greater positive impact in the world. We hope—and believe—that our shared vocabulary can support this by providing a conceptual structure that: (i) supports teams of HCI and health intervention researchers in planning and coordinating their work, (ii) supports HCI researchers in understanding and responding to needs identified in health research (e.g., by creating new capabilities or components in response to design briefs articulated in mental health); and (iii) facilitates the uptake and evaluation of HCI innovations in existing mental health interventions and settings (e.g., by aligning novel socio-technical components with psychological functions described by theories of change).

5. CONCLUSIONS

HCI has made and will continue to make vital contributions to the design of complex psychosocial interventions, especially in mental health. By describing a framework that characterises four distinct scopes of HCI contributions to complex psychosocial interventions, and links those scopes to the theories of change that describe how interventions work, we hope to support HCI researchers in planning, conducting, and communicating our research and to bridge between interaction design and mental health goals, streamlining collaboration between fields.

Through use of design briefs that focus on theory-of-change functionalities—i.e., experiences that are psychologically ‘active’—and evaluation foci at each scope, we also hope that this framework can reinforce for HCI researchers what is important and must be achieved in addressing any given design challenge, while also highlighting degrees of freedom in which researchers and designers might innovate. This, in turn, supports researchers to incorporate interaction design and clinical expertise in research, and helps navigate tensions between (i) building technical instantiations of evidence-based interventions that are known to work; and (ii) innovating new capabilities, components, or even entirely new interventions and implementations.

The inherent modularity of theories of change—and the scopes we describe in this framework—also facilitates contributions to the domain of complex psychosocial interventions at various levels, while motivating one’s contribution though connections to other levels. Finally we emphasise that contributions at each of these scopes can be done by (i) building or testing an new interactive artefact (i.e., creating something in response to a design brief); or by (ii) articulating a need to design for a new or different set of user experiences, together with an explanation of how these would lead to psychological change (i.e., creating a new design brief and providing links to plausible theories of change).

CCS CONCEPTS.

• Human-centered computing → HCI theory, concepts and models; • Applied computing → Consumer health.