User-Centered Design for Psychosocial Intervention Development and Implementation

Abstract

The current paper articulates how common difficulties encountered when attempting to implement or scale-up evidence-based treatments are exacerbated by fundamental design problems, which may be addressed by a set of principles and methods drawn from the contemporary field of user-centered design. User-centered design is an approach to product development that grounds the process in information collected about the individuals and settings where products will ultimately be used. To demonstrate the utility of this perspective, we present four design concepts and methods: (a) clear identification of end users and their needs, (b) prototyping/rapid iteration, (c) simplifying existing intervention parameters/procedures, and (d) exploiting natural constraints. We conclude with a brief design-focused research agenda for the developers and implementers of evidence-based treatments.

Much attention has been paid to the “research-practice gap” in mental healthcare, wherein evidence-based treatments (EBT) – typically established through decades of development and rigorous empirical testing – are not routinely employed in service delivery (Kazdin, 2008; McHugh & Barlow, 2010). Recently, the field of implementation science has emerged, explicitly tasked with improving the use of well-researched interventions in everyday service settings (Eccles & Mittman, 2006), and some have argued that new ways of connecting science and service may be necessary to close the research-practice gap and truly raise quality of care (e.g., Kazdin & Rabbitt, 2013). In line with this call for new approaches, we articulate in this paper how many of the contemporary difficulties encountered during EBT implementation are exacerbated by fundamental design problems – embedded in both EBT themselves and typical EBT implementation processes – and which may be effectively addressed by a set of principles and methods drawn from the field of user-centered design.

Gaps in EBT Design, Implementation, and Effectiveness

EBT are defined as interventions that have produced therapeutic change in controlled trials, while evidence-based practice refers to integration of research knowledge with clinical expertise and patient characteristics, culture, and preferences (American Psychological Association, 2006; Kazdin, 2008). Although there is recognition that mental health service quality should extend beyond EBT to the broader concept of evidence-based practice, and a number of recent research examples that suggest the field may be moving slowly toward complementary approaches (e.g., Garland et al., 2014; Schoenwald et al., 2008; Weisz & Chorpita, 2011), manualized EBT protocols remain the primary medium through which research evidence is packaged and disseminated for use (Garland, Hawley, Brookman-Frazee, & Hurlburt, 2008). Despite their prevalence, numerous concerns about EBT appear to contribute to their low level of use by community practitioners (Chambless & Ollendick, 2001; Kazdin, 2008). While some of these concerns represent important questions surrounding the methods through which EBT are tested (e.g., research sample generalizability, relevance of psychiatric symptom outcome measures), many are exacerbated by the design or structure of EBT, user responses to those designs, and the ways that elements of EBT design interact with implementation processes. Although the implementation and widespread reach of EBT in service systems are known to be influenced by a range of factors operating across multiple system levels (Aarons, Hurlburt, & Horwitz, 2011) – and intervention characteristics are commonly included in leading implementation frameworks (e.g., Damschroder et al., 2009; Rogers, 2003) – specific characteristics of the programs implemented are typically given less attention than the individuals, systems, and processes involved. Further, despite acknowledgment that intervention characteristics are important, existing frameworks provide almost no guidance surrounding specific methods for ensuring that EBT successfully meet user needs.

Key design issues that continue to impact EBT implementability include flexibility, complexity, and effectiveness, as well as the frequently one-directional relationship between program development and implementation. First, there is ongoing debate surrounding the extent to which EBT are able to effectively balance structure and flexibility when introduced to service providers working in community contexts (Chambless & Ollendick, 2001; Hill & Owens, 2013; Lieb, Mayfield, Miller, & Pennucci, 2004). Flexibility (e.g., via allowable adaptations) is inherently appealing when working to deliver individualized or locally-relevant services (Lyon, Lau, McCauley, Vander Stoep, & Chorpita, 2014), but is accompanied by increased uncertainty (Chorpita & Daleiden, 2014) and may lead to lower clinician performance (e.g., Jewell, Handwerk, Almquist, & Lucas, 2004). Second, most psychosocial interventions are complex, and carry numerous decision points during the course of their use (Chorpita, Bernstein, & Daleiden, 2008). High complexity can interfere with the extent to which EBT are readily accessible to providers or organizations interested in adopting them to improve their practice. Further, trends suggest increasing complexity as EBT developers pursue applications to wider populations, but this added complexity often carries little additional benefit (e.g., Chaffin et al., 2004). As a partial function of their complexity, EBT are exceedingly difficult to train and learn. Indeed, perhaps the best-established implementation truism is that even intensive “train and hope” approaches are unlikely to result in meaningful practitioner behavior change without ongoing consultation or coaching (Beidas & Kendall, 2010; Fixsen et al., 2005; Lyon, Stirman, Kerns, & Bruns, 2011). As a result of the effort required, high-quality EBT training is expensive and elusive for many practitioners.

Third, although widely-cited papers originally documented the effectiveness of EBT relative to usual care (e.g., Weisz, Doss, & Hawley, 2006), some recent research has raised questions about their actual superiority (e.g., Spielmans, Gatlin, & McFall, 2010; Weisz et al., 2012; Weisz et al., 2013). Findings such as these threaten to undermine the legitimacy of EBT and the basic arguments on which their implementation is based. As discussed below, this is particularly problematic considering that EBT are typically disseminated as static tools to be used only as directed by developers, rather than products to be improved over time (Chambers, Glasgow, & Stange, 2013). Although it is often a foregone conclusion that the transfer of an EBT to a community context will be accompanied by an inevitable “voltage drop” in which its effectiveness falters relative to the original efficacy trials, there is growing acknowledgement that ongoing adjustment and evaluation of EBT to ensure that a program is response to the local context provides an opportunity to increase intervention effectiveness over time (Aarons et al., 2012; Chambers et al., 2013). However, there are currently few clear principles or procedures available to guide this work effectively.

Finally, all of the concerns described above are perpetuated by existing divisions between intervention design (e.g., development and testing) and intervention implementation processes (e.g., training, scale-up, sustainment). Although interventions are not always developed following a uniform or consistent process, a university-based research team most traditionally develops and tests an intervention extensively, articulating its key functions and structure, before it is disseminated more broadly as a fully-formed, static intervention protocol. Chorpita and Daleiden (2014) recently distinguished this dominant emphasis on “design time” control (i.e., determining features in advance) from “run time” control, which they defined as configuration based on how a product interacts with its environment. This distinction is akin to the common breakdown of roles and responsibilities within the construction industry, in which architects design new buildings and contractors subsequently execute that design. Although there are advantages to this arrangement (e.g., development of a high degree of specialization), it can also produce blueprint designs that are ultimately too expensive, impractical, or even impossible to construct within real world constraints. Seen this way, EBT suffer from a disconnect between the design context and implementation context, which leads to (a) design decisions that are not necessarily applicable to the constraints of the contexts in which they will ultimately be used and (b) implementation processes that devote excessive resources to adhering to design time specifications that may be inappropriate (e.g., strict emphases on intervention integrity). Despite recognition that EBT implementation involves substantial problem solving and frequent compromise (Aarons et al., 2011; Aarons et al., 2014), emerging implementation strategies (Powell et al., 2015) have typically been used to implement relatively static EBT in a unidirectional fashion, emphasizing design time control. Because successful implementation occurs as a function of both the intervention itself and the destination context (Rogers, 2003), the failure of most implementation processes to incorporate ongoing, iterative intervention development increases the risk that local users will find new technologies to be low on key implementation outcomes such as acceptability, feasibility, and appropriateness (Proctor et al., 2011) and may limit effectiveness (Chambers et al., 2013).

A reconceptualization of how intervention technologies are initially developed, prepared for dissemination, and then ultimately implemented may help to capitalize on opportunities to increase intervention appropriateness and effectiveness over time. Just like other consumer products, EBT can be intentionally designed using methods that ensure the needs of target users are incorporated into the development process. Such processes are relevant to both the initial development of new interventions as well as the revision of existing interventions. We next discuss how user-centered design can be applied to satisfy user needs and improve EBT.

User-Centered Design to Improve EBT Development, Packaging, and Implementation Usability and User-Centered Design

The need to create products that intended audiences find compelling and easy to use transcends any single discipline or industry. Over the past two decades, a field of user-centered design (UCD) has developed, largely rooted in human-computer interaction, industrial design, and cognitive psychology. UCD is an approach to product development that grounds the process in information about the people who will ultimately use the product (Courage & Baxter, 2005; Norman & Draper, 1986). Although UCD borrows concepts from other disciplines (e.g., participatory research), it bundles them uniquely in a comprehensive set of principles and procedures intended to make products more accessible and appealing and to improve their effectiveness over time. Colloquially, design is often distinguished from engineering: while engineering may build functional, yet inelegant, solutions to problems that meet technical specifications, design emphasizes parsimony, ease of use, aesthetics, fitness to purpose, and results in products that meet requirements in compelling ways. Because EBT have historically focused primarily on the identification of robust, often complicated, solutions for highly specified problems (i.e., diagnosable clinical disorders), it could be said that they have overemphasized engineering to the detriment of design.

Usability – the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use (International Standards Organization, 1998) – is a principal outcome of a user-centered approach to design. Usability has been conceptualized as a combination of constructs including (a) learnability, (b) efficiency, (c) memorability, (d) error frequency/severity, and (e) satisfaction (Nielsen, 1994). Maguire (2001) has identified that usable systems result in increased productivity, a reduction of errors, require fewer resources for training and support, are more acceptable to users, and enjoy enhanced reputations within the user market. In contrast, problematic system design can easily undermine otherwise appealing and effective products (Littlejohns, Wyatt, & Garvican, 2003; Karsh, 2004). Other authors have included additional criteria for usable products such as functional minimalism (i.e., too many features, functions, or components will reduce usability), low cognitive load (i.e., minimize the amount of thinking required to complete a task), clear feedback to users (e.g., about product status, success or failure), and exploitation of the natural constraints present in an environment (i.e., designing products in ways that incorporate unalterable characteristics of the intended context of use) (Norman, 1988; Tognazzini, 2014).

Applied to psychosocial interventions, this list of design principles for ensuring usability suggests that well designed EBT should (a) provide clinicians, service recipients, and other users opportunities to rapidly build understanding of or facility in their use (learnability); (b) minimize the time, effort and cost of using the EBT to resolve identified problems (efficiency); (c) remember and successfully apply important elements without many added supports (memorability); (d) prevent or allow rapid recovery from errors or misapplications (error avoidance/reduction); (e) be viewed as acceptable and valuable compared to other products available within the larger mental health marketplace (satisfaction/acceptability/reputation); (f) maintain simplicity (low cognitive load); and (g) be designed – first and foremost – to fit their context of use (exploit natural constraints). Table 1 displays these design goals.

Table 1.

Design goals for evidence-based treatments (EBT) in mental and behavioral health

Principle	Description
Learnability	Well-designed EBT should provide users opportunities to rapidly build understanding of, or facility in, their use.
Efficiency	Minimize the time, effort, and cost of using the EBT to resolve identified problems.
Memorability	Users can remember and successfully apply important elements of the EBT protocol without many added supports.
Error reduction	Prevent or allow rapid recovery from errors or misapplications of EBT content.
Satisfaction / Reputation	Be viewed as acceptable and valuable, especially compared to alternative products available within the larger mental health marketplace.
Low cognitive load	Simplify task structure or the number of steps required in order to minimize the amount of thinking required to complete a task.
Exploit natural constraints	Successful designs should incorporate or explicitly address the static properties of an intended destination context that limit the ways a product can be used.

UCD principles can be applied not only to the creation and improvement of software and physical products, but also to design of effective social phenomena, such as service design (Goldstein, Johnston, Duffy, & Rao, 2002; Zomerdijk & Voss, 2010) or instructional design (Gagne, Wagner, Golas, Keller, & Russell, 2004; Van Merriënboer, Kirschner, & Kester, 2003). In mental and behavioral health, some limited work has applied design principles to topics such as the instructional design of clinician training programs (Weingardt, 2004). Authors have also begun to explore the relevance of these ideas to mental and behavioral health interventions (most notably Chorpita & Daleiden, 2013), although not within an explicit UCD framework. Wu et al. (2014) have discussed the relevance of the related discipline of engineering to implementation and mental health services, and others have advocated for usability testing and iterative design in the context of health information technologies to support clinical decision-making (e.g., Bickman, Kelley, & Anthay, 2012; Lyon et al., in press-b). Nevertheless, a UCD approach has not yet been applied to the development or implementation of EBT themselves. To illustrate the utility of UCD, we briefly present a selection of concepts and methods below through which (a) initial psychosocial intervention design and (b) redesign of existing interventions (frequently in the context of implementation activities) can be brought into better alignment with the needs of the end users. These include: careful identification of intervention end users and their needs, prototyping and rapid iteration, simplifying existing intervention parameters and procedures, and exploiting natural constraints. For each, we present a definition, example techniques from UCD, and potential applications to the design or redesign of psychosocial interventions in mental health.

Identifying Users and User Needs

Definition

The UCD field places strong emphasis on explicitly identifying primary, secondary, and sometimes tertiary users in order to ensure that new products effectively meet their needs (Cooper, Reimann, & Cronin, 2007; Grudin & Pruitt, 2002). Primary users are the target group for a product whose needs are prioritized in the design or redesign process. Redesign of an existing innovation may sometimes be prompted by the identification of a new set of primary users. Secondary users are those who are likely to be generally satisfied with the design elements identified on the basis of the primary user(s), but who may have additional needs that can be accommodated without compromising a product’s ability to meet the primary user(s) needs. Negative users are those the product is explicitly not intended to serve, and whose input should not be considered as design decisions are made (Cooper et al., 2007).

Techniques

Product developers tend to underestimate user diversity in their design processes, but careful identification of representative user needs can correct this bias and enhance product quality (Kujala & Kauppinen, 2004). In the absence of this information, developers are likely to base designs on people similar to themselves (Cooper, 1999; Kujala & Mäntylä, 2000). Use of diverse user groups is important when designing products for organizations, which inevitably contain individuals representing different user types (Kujala & Kauppinen, 2004). In the domain of computer technologies, the increasing ubiquity of digital products has prompted suggestions that designers move beyond generic user models toward more nuanced understandings of their needs and desires (e.g., Dillan & Watson, 1996).

One parsimonious model for user identification is the lead user approach, wherein the experiences of particularly advanced users are collected to uncover system problems and solutions (which lead users often identify on their own) (von Hippel, 1989). Although this method has been found to improve the efficiency of the product design process (Olson & Bakke, 2001), some lead user needs may be too advanced to be relevant to less experienced users (Kujala & Kauppinen, 2002). Hackos and Redish (1998) proposed a process for incorporating a broader variety of users into the design process that includes: (a) brainstorming a preliminary list of users, (b) articulating user characteristics, (c) describing and prioritizing main user groups, (d) selecting typical and representative users from those groups, and (e) gathering information from users to inform the redesign of the user group descriptions. Some evidence exists to suggest the utility of this process in producing more usable systems (Kujala & Kauppinen, 2004).

Applications

Similar to ineffective digital technology development, EBT development processes tend to emphasize the needs and perspectives of intervention developers over those of well-defined user groups. Indeed, substantial disconnects have been identified between developers, who are typically doctoral-level researchers or trainees working in academic settings, and public-sector mental health therapists, who are likely to be among the end users of the protocol (Weisz et al., 2006). Because Masters-level therapists provide the bulk of mental health services in community settings (Hyde, 2013), this group is an important set of primary users. Nevertheless, available evidence suggests that EBT are not particularly well aligned with the needs of this group (Addis, Wade, & Hatgis, 1999). As a result, many of these intended EBT users do not view EBT as necessary or relevant to their work (Nakamura, Higa-McMillan, Okamura, & Shimabukuro, 2011) or, if they do, are struggling to use them routinely and successfully (Becker, Smith, & Jensen-Doss, 2013). Other studies have indicated that therapists sometimes question the relevance or effectiveness of EBT for their specific populations or struggle to deliver them when presented with engagement difficulties, crises, or comorbidities (Chandler, Peters, Field, & Juliano-bult, 2004; Kazdin, 2008; Whaley & Davis, 2007). Chorpita and colleagues (2014) have pointed out that these situations often require therapists to go “off protocol,” changing their intervention plans in unexpected ways that are inconsistent with the intended treatment. Deviations such as these represent the EBT equivalent of a software “workarounds,” in which temporary fixes are used to bypass identified system usability problems. Although workarounds can often be effective problem-solving strategies in the short term, they are likely to decrease usability and product functioning if unaddressed. Well-designed systems reduce the need for such workarounds. A UCD perspective may also allow EBT to be designed to address complex or changing client problems and some existing interventions already reflect these principles. Examples of interventions designed to address these issues come from contemporary work on “transdiagnostic” and “modularized” approaches to intervention (Barlow, Allen, & Choate, 2004; Chorpita, Daleiden, & Weisz, 2005b; Roy-Byrne et al., 2010; Weisz et al., 2012) as well as more principle-driven (rather than traditionally manualized) interventions (Koerner, 2013). Both of these types of EBT are intended to allow for more flexibility in addressing client problems identified across different stages of treatment.

Drawing from the UCD literature, a clearer conceptualization of users and user needs should involve more explicit articulation of user types and incorporation of user perspectives across intervention development phases. At a minimum, involving lead or expert users (e.g., clinicians with expertise using multiple EBT in community settings) in early, formative information gathering to drive development processes represents a relatively cost effective method of cataloging protocol problems and potentially innovative solutions, although this may result in limited scalability if it does not result in a product that is responsive to less experienced users. Developers may also leverage primary users’ feedback to produce variations of EBT products for adoption by users at different levels of expertise, as is common in digital technologies (Kujala & Kauppinen, 2004). In this case, different classes of primary users may include line staff in juvenile justice mental health cottages, nurses and social workers in primary care settings, or Ph.D. level psychologists in specialty clinics; all of whom may desire or need different configurations of an EBT (e.g., versions that provide greater theoretical rationale or information about basic techniques, such as psychoeducation). Furthermore, findings indicating that incomplete penetration and sustainment are the norm (Stirman et al., 2012) suggest that it may be equally important for intervention developers to explicitly articulate the characteristics of negative users who are not intended targets for an intervention (e.g., those with low attitudes toward the use of research evidence in practice).

In addition, EBT design processes may be most effective if they incorporate a developmental perspective and attend to how individual service provider and service system needs change over time (Chorpita & Daleiden, 2014). In software design, competitive products must remain sufficiently adaptable to account for changing user expectations, previously unidentified bugs, evolving hardware, or a shifting marketplace. For these reasons, software updates are pushed out to users with astonishing regularity (e.g., every 6 weeks; Khomh, Dhaliwal, Zou, & Adams, 2012). Well-designed EBT should be similarly responsive to changes in staff needs. These changes may include increases in individual staff expertise due to experience (e.g., new therapists becoming increasingly comfortable in clinical interactions) or specific development of specific new competencies, but may also include decreases in a system’s collective expertise due to staff turnover. In either situation, mechanisms that allow service systems to update components of EBT to meet these changing needs are vital. Possibilities include developing tools to “push out” regular intervention updates to practitioners along with just-in-time training (Dimeff, Paves, Skutch, & Woodcock, 2010).

Beyond clinicians, consumers of mental health services are a frequently ignored, but essential, primary EBT user type (Sanders & Kirby, 2014). When considered, consumer perspectives are most commonly captured in surveys of mental healthcare satisfaction (e.g., Solberg, Larsson, & Jozefiak, 2014), but their needs extend well beyond this construct. From a design perspective, satisfaction is only one of many indicators of a usable and effective product, and one that may be particularly subject to bias – and therefore less informative – when a user has no experience with a comparable product to which to compare a new innovation. Although service recipients generally report favorable experiences receiving EBT (Hodgetts & Wright, 2007), research has documented that EBT consumers may also experience a number of barriers to successfully engaging in, and benefitting from, those interventions. These include questions about their accessibility, perceived flexibility and cultural relevance, among others (Becker, Spririto, & Vanmali, in press; Hodgetts & Wright, 2007). UCD methods that evaluate (e.g., via contextual inquiry or other participatory approaches; Holtzblatt, Wendell, & Wood, 2004) and address these types of concerns directly have the potential to increase the “patient-centeredness” of interventions by incorporating consumer preferences and ensuring responsivity to consumer needs (Methodology Committee of the Patient-Centered Outcomes Research Institute, 2012). Indeed, one promising approach to better include consumer needs is through direct user participation in the initial development, redesign, or evaluation of interventions themselves (Boote, Baird, & Beecroft, 2010). For instance, Holmqvist and Walsh (2014) found that although web- and telehealth-based CBT for insomnia were equally effective, service recipients favored web-based delivery. In that scenario, use of a web-based platform is likely to increase the user- and patient-centeredness of the intervention. Again, due to their built-in flexibility, newer transdiagnostic or modular approaches to service delivery (described above) may be more likely to be responsive in these ways.

Secondary EBT users may include system administrators, who make decisions about innovation adoption; paraprofessionals, who may use versions of EBT within traditional or re-conceptualized service roles; as well as other key stakeholders. Although a large body of implementation-focused research has documented the impact of organizational processes to successful implementation – including leadership buy-in and administrative support (Beidas & Kendall, 2010) – outside of a few noteworthy exceptions (e.g., Schoenwald et al., 2008), little research has focused on ways to design EBT to be responsive to the needs of these decision makers while keeping intervention effectiveness intact or to systematically plan evaluations of the modifications needed to better fit setting constraints. Additionally, given perpetual workforce shortages and rising populations of individuals in need of mental health services (U.S. DHHS, 2013; Kakuma et al., 2011), it may be the case that, in some situations, traditional mental health therapists are not the optimal “intervention pilots” or front-line service deliverers. Reconsideration of primary EBT users and service delivery models also opens the door to task-shifting strategies, which have quickly become popular in global health and, increasingly, domestically (Patel, 2009). Task shifting involves workforce reorganization and redistribution of some tasks that have traditionally been completed by highly trained service providers (e.g., clinical psychologists, psychiatrists) to other types of professionals. For example, in the Improving Access to Psychological Therapies project, cognitive-behavioral approaches were explicitly packaged as low intensity versus high intensity to efficiently expand workforce capability and ensure that people with less severe problems could receive care from psychological wellbeing practitioners with less intensive training (Clark et al., 2009; Glover, Webb, & Evison, 2010). Revising an existing EBT to be applicable to a new class of service providers (e.g., paraprofessionals) may be an appropriate impetus for system redesign. Redefined roles of this type may also help to make room for the development and use of simplified “disruptive innovations” within the EBT domain, which typically make products available to new sets of consumers or users who were not previously considered part of the target market (Rotheram-Borus, Swendeman, & Chorpita 2012; Hwang & Christensen, 2008). Disruptive innovations are defined as “cheaper, simpler, more convenient products or services that start by meeting the needs of less demanding customers” (Christensen, Bohmer, & Kenagy, 2000, p.2). Applied to healthcare, disruptive innovations often have the effect of increasing service capacity by either (a) reducing the need for clinician involvement by delivering services directly to consumers (e.g., client-facing apps that provide first-line interventions) or (b) shifting service delivery tasks to other types of professionals or paraprofessionals. The latter may include behavioral health service delivery in primary care or tertiary care settings that more typically emphasize physical health or the use of new service delivery innovations (e.g., via avatar-based interventions) to promote acceptability and accessibility. In both cases, increasing capacity may also have the effect of identifying and addressing problems in less severe populations of service users.

Prototyping and Rapid Iteration

Definition

The concept of prototyping is ubiquitous within the field of UCD and related disciplines. Rapid prototyping is a process of making ideas tangible in order to quickly test and make improvements based on feedback (Wilson & Rosenburg, 1988). The mantra, “Fail early and often,” conveys the spirit of rapid prototyping. Using this technique early in the design process allows for the inexpensive exploration of novel ideas and solutions prior to more costly production. It is especially helpful to mock up important interactions that will be crucial to success or workflow ease. Prototyping, and rapid iterations based on evaluation of each successive prototype, is an excellent example of the overarching design process of collecting data and feedback at all stages of a product development cycle (Maguire, 2001).

Techniques

Prototyping is iterative and involves the sequence of developing a prototype, reviewing that prototype with users, and then refining it based on their feedback. At its essence, prototyping involves the creation of a “low-fidelity” version of a product that contains key functions of interest in order to test a concept, facilitate rapid evaluation and feedback, or answer a specific question (e.g., deciding between two design alternatives). Later, fully functional “high-fidelity” prototypes may be created that are more similar to the final product and typically offer real interactive content (McGuire, 2001).

Prototyping is frequently visual and sketching may be an especially helpful tool to use in early rapid prototyping. With sketching, a design team represents a product in a quick and disposable manner, with just enough detail to learn how someone will use a product rather than evaluating that product or becoming distracted by how it looks. For example, to design a website or smart phone application, designers often begin with paper wireframes (quick drawings of user screen views) and schematic drawings of the content and interactions on and between pages. Testing with a paper prototype means the designer can rapidly redraw an interaction to better fit the flow to user needs and preferences. Robust evidence supports the use of prototyping to improve a product’s match with users and overall “ease of use” (Gordon & Bieman, 1997).

Applications

Applied to EBT development, rapid prototyping may hold opportunities to accelerate the glacial pace through which research innovations have historically been developed, tested, and translated into typical practice (Balas & Boren, 2000). Prototyping may be used either to facilitate more rapid and responsive initial EBT development processes or to redesign existing EBT in the context of larger-scale implementation (e.g., successive, systematic testing of different intervention components or configurations). Prototyping may be distinguished from typical pilot testing by its speed, repetition (many small tests are required), and the fact that it is largely exploratory (i.e., focused on challenging core assumptions and altering an intervention in meaningful ways) versus confirmatory (i.e., intended to establish feasibility).

The concept of microtrials (Howe, Beach, & Brody, 2010), represents a feasible approach to engaging in rapid, small-scale prototyping to develop and evaluate components of psychosocial interventions. Microtrials are short tests of the effects of circumscribed environmental or behavioral manipulations on proximal outcomes or mechanisms of change. Microtrials share some similarities with clinical analogue trials, but rather than using analogue conditions (e.g., vignettes [Wright, Weinman, & Marteau, 2003]; participants that approximate clinical populations [Barlow, Agras, Leitenberg, & Weincze, 1970]) to represent the phenomena of interest, microtrial studies are more likely to reflect real-world conditions (e.g., real service recipients and clinical interactions). Recently, microtrials have been identified as a feasible method for testing individual parenting techniques (e.g., praise) to determine their discrete merit and ultimately drive more individualized, tailored service delivery (Leijten, Dishion, Thomaes, Raaijmakers, & Matthys, 2015). Early in an intervention development process, microtrials may have the potential to support the collection of “proof of concept” evidence for specific, previously untested techniques using a within-subjects, case study research design. These techniques can often be delivered across a single session, after which developers evaluate (a) the extent to which service recipients found the technique acceptable and appropriate and (b) any changes in proximal outcomes. Such an approach provides opportunities for multiple, simultaneous small-scale tests of variations within a case study research design or related framework.

Applied to the redesign of established EBT, rapid prototyping allows changes to be made to existing content on an ongoing basis in the context of small- or even large-scale implementation. For example, systematic adaptation and iteration of evidence-based progress monitoring, suicide risk detection/intervention, or group delivery of previously individualized interventions can be integrated with strong existing procedures for quality improvement to better fit the needs and constraints faced within a large health maintenance organization (Steinfeld et al., 2014). This stands in contrast to the dominant scientific model in which each incremental change to an existing EBT protocol must be subjected to a new randomized trial for testing (Chorpita, Daleiden, & Weisz, 2005a). In this way, rapid prototyping may systematize the processes through which practitioners tend to apply EBT in the real world, which commonly involve a series of non-systematic adaptations (Stirman, Miller, Toder, & Calloway, 2013). Rapid prototyping may be distinguished from other models of EBT development and deployment (e.g., Rounsaville, Carrol, & Onken, 2001; Weisz, Jensen, & McLeod, 2005) in the extent to which it functions as a continuous, nonlinear quality improvement process and provides opportunities to quickly evaluate the viability of new concepts or variations. Because it opens the door to a greater number of variants and decreases the costs associated with testing new alternatives, rapid prototyping may be more likely to result in new EBT protocol innovations than current models.

Design Simplification

Definition

With the goals of increasing learnability and decreasing the cognitive load required as users interact with a product, design simplification has long been a hallmark of UCD (Norman, 1988). Simplification is an overarching principle with specific applications to multiple design activities, such as the processes of scoping product functions and features (i.e., avoid unnecessary options) or determining the ways products present information to users (i.e., effective information visualization – see below). Simplicity is particularly important in the context of UCD because the common practice of collecting user input across phases of development has the potential to identify a nearly endless set of enhancements, which introduces considerable opportunities for “scope creep,” or the creation of a product or product features that add complexity and extend well beyond original specifications. Vigilant application of the principle of simplification works to contain this complexity while enhancing user experiences and facilitating the scalability of products (Rogers, 2003; Yamey, 2011).

Techniques

In the context of website usability, Nielsen and Loranger (2006) note that while it is “easy to build a bulky design by adding layer upon layer of navigation and features; it's much more difficult to create simple, graceful designs.” Early in the emergence of the field of UCD, Norman (1988) articulated a series of principles for transforming difficult tasks into simple ones. Among them, he suggested simplifying the structure of tasks (e.g., by leveraging technology and reducing the load on attention, short-term memory, and long-term memory) through designs that show alternative courses of action, make visible information that would otherwise be invisible, and help users to readily evaluate the implications or outcomes of their actions. The goal of simplification can either be achieved by (a) keeping primary tasks unchanged, but incorporating new supportive infrastructure or external memory devices to supplement human perceptual abilities (e.g., dashboard instruments that communicate the state of object in question [such as an automobile]), or (b) reducing the complexity of a task itself (e.g., introducing Velcro to replace shoelaces, or digital watches to replace analog). Frequently-completed tasks are often most ripe for simplification, given the potential of even small changes to save substantial time and effort over each occurrence of a behavior.

A particularly important set of simplification tools relates to information visualization techniques, designed to communicate information simply and effectively. Although information visualization is frequently discussed surrounding the display of quantitative data (Tufte, 2001; Ware, 2013), clear presentation of visual information is essential to the functioning of any product to allow users to understand the state of a system, what actions are possible, and how they should be completed (Norman, 1988).

Applications

When scaling up behavioral health interventions, Aarons and Chaffin (2013, para. 5) observed that, “within an existing network, the less change required, the more implementation may occur.” Considering the complexity of contemporary EBT, deliberate simplification may enhance the potential of effective programs to spread within service systems. Efforts to simplify EBT interventions and implementation processes may take several forms, including methods of simplifying EBT themselves (i.e., reducing the complexity of the task itself) or improving EBT manuals and related materials (i.e., keeping the task unchanged, but incorporating new or different supports). These simplification activities may occur as new interventions are developed, or in service of reducing the complexity of existing protocols via EBT redesign.

Increasingly, mental health services researchers have begun to argue for the dissemination and implementation of key intervention competencies, principles, or practices rather than full EBT packages (e.g., Beidas, Koerner, Weingardt, & Kendall, 2011; Rotheram-Borus et al., 2012), an approach that may represent one compelling pathway to EBT simplification. For instance, increasing evidence has emerged that routinely monitoring psychotherapy client outcomes and providing data-driven feedback to therapists can reduce premature dropout and improve outcomes irrespective of the particular intervention approach used by the therapist (Bickman, Kelley, Breda, de Andrade, & Reimer, 2011; Lambert et al., 2003). Considered from this perspective, routine outcome monitoring may represent a potential minimum intervention necessary for change when working to enhance the quality of community-based services (Scott & Lewis, 2014).

Furthermore, recent advances in methods for distilling effective interventions to their “common elements” and processes have emerged (e.g., Chorpita et al., 2005a; Embry & Biglan, 2008), which parse the research literature at a finer level of detail than complete treatment packages. Recognizing that the identification of these specific elements of practice – which may be recombined in thousands of novel configurations – has the potential to increase the complexity of mental health intervention, Chorpita and colleagues have also developed a set of decision-making frameworks and tools that help to simplify these process (Chorpita et al., 2008; Daleiden & Chorpita, 2005). Although evidence for the effectiveness of interventions informed by these models is just emerging (e.g., modular interventions Weisz et al., 2012), such approaches have facilitated the dismantling of complicated, existing intervention protocols to identify components that may, theoretically, be rearranged to produce simplified interventions. Lyon and colleagues (2014a; in press) described the initial stages of such an effort, in which a small subset of elements from existing, evidence-based interventions were selected to construct a brief, simplified intervention for use by school-based mental health clinicians. Focusing on intervention-setting appropriateness (i.e., “fit”), the authors identified multiple constraints in the education sector, such as short windows for service delivery (approximately three to four sessions), which drove their simplified design.

Despite the potential for streamlined intervention content to improve intervention design, even traditionally-structured EBT can benefit from redesign improvements that keep the intervention largely unchanged, but incorporate more useful supports to facilitate accessibility, usability, and scale-up. A particularly concrete application of the simplification principle may involve the redesign of EBT manuals using effective information visualization techniques. Driven by findings that users generally experience exhaustive documentation as burdensome, frustrating, and unhelpful (and, as a result, do not read manuals), the computer industry has moved to abandon comprehensive manuals in favor of quick, effective reference guides and other forms of minimalist documentation (Salvo, Zoetewey, & Agena, 2007; Smart, Madrigal, & Seawright, 1996). A similar problem exists within mental health, where concerns about traditionally-structured, established EBT manuals abound (e.g., Addis & Krasnow, 2000; Borntrager, Chorpita, Higa-McMillan, & Weisz, 2009). Interestingly, some of the best examples of the innovative redesign of psychosocial intervention manuals and materials come from global health, where simplification of intervention content has been essential to allow for the translation of existing content into a form that is usable by paraprofessionals or local service recipients. Rahman (2007) described the simplification of cognitive-behavioral techniques for delivery to mothers demonstrating perinatal depression in rural Pakistan. The intervention was simplified using local imagery (e.g., depictions of culturally-relevant mothers and children) to represent content that was normally written and to facilitate administration by existing “Lady Health Workers” who lacked prior mental health experience. Other international researchers have had similar success (e.g., simplifying handouts for service recipients with low rates of literacy; Kaysen et al., 2013), but this type of manual simplification has rarely been applied domestically. Many of the design techniques described in previous sections (e.g., the construction of hassle maps) have the potential to facilitate this process and more research is needed surrounding the ways to best streamline different components of EBT protocols via a redesign process. Although EBT are situated within human interactions and reciprocal social processes and some degree of complexity is unavoidable, emerging evidence indicates that it is possible to reduce complexity without deleterious consequences (e.g., Rahman, Malik, Sikander, Roberts, & Creed, 2008).

Exploit Natural Constraints

Definition

Within the context of design, environmental constraints represent properties of an intended destination setting that limit the ways a product will be designed or used. Product design depends largely on this type of constraint, which may include limitations on or requirements for a product’s form, function, budget, operating conditions, or time to completion, among others (Moggridge, 2007). Constraints of this type are unavoidable when working in the real world and must be considered during the design or redesign process if a new product is to function well in a destination context. For any given product, natural constraints restrict possibilities for action, but simultaneously make other pathways (those consistent with the constraints) more accessible.

Techniques

Addressing natural constraints in a setting involves both their identification and their incorporation into the product development cycle. Some constraints are ubiquitous and, while attention to them is essential to produce an effective design solution, nearly all solutions will attend to them implicitly (e.g., gravity). The most important natural constraints to articulate are therefore not necessarily those that are universal, but those which are likely to impact the usability and usefulness of a product. Discussion of natural constraints often involves the consideration of affordances, or properties of an environment (or objects in the environment) that pull for particular behaviors and, in doing so, exclude others. A door knob, therefore, “affords” turning while a lakefront walking path “affords” walking, running, and standing to enjoy the view. Importantly, affordances rely heavily on human perception and have no existence independent of a joint consideration of the individual and environment (Zaff, 1995). Although affordances and constraints are traditionally conceptualized (within industrial design) as physical attributes of an environment, Norman (2004) has discussed cultural constraints, or learned conventions shared by a group, as a type of affordance that influences user expectations and perceptions and, in doing so, impacts design decisions.

Applications

In the context of EBT, natural constraints are frequently conceptualized as barriers to implementation, represented as antagonistic to the internal validity of the EBT and as factors that must be overcome if program adoption and sustainment are to occur. Constraints may include practitioner caseload size, the regularity or duration with which clients attend sessions, practitioner time for training, and organizational or individual priorities (Lyon et al., 2014c; Southam-Gerow, Rodriguez, Chorpita, & Daleiden, 2012). When engaging in the redesign of an existing EBT, the identified core components of the intervention (Damschroder et al., 2009), if known, may be considered one particularly important set of design constraints.

Although discussion of ways to maximize intervention-setting fit is common in the field of EBT implementation, there has been little research focused explicitly on procedures for realizing this goal (Aarons et al., 2012; Lyon et al., 2014c). Countless studies have cataloged a litany of barriers (often post hoc) that interfere with successful program installation (e.g., Hasson, Andersson, & Bejerhom, 2011; Langley, Nadeem, Kataoka, Stein, & Jaycox, 2010; Lewis & Simons, 2011). In response, there is now growing consensus that the time has come to move implementation science beyond the identification of barriers toward the development of explicit strategies that facilitate EBT use and client improvement (Proctor, Powell, Baumann, Hamilton, & Santens, 2012). A re-conceptualization of EBT implementation barriers as design constraints both acknowledges the intimate relationship between those barriers and characteristics of the intervention model itself and simultaneously places responsibility for attending to those constraints on program designers and redesigners. Intervention redesign processes intended to address contextual constraints may be informed by work such as the Interagency Collaborative Team model articulated by Aarons, Hurlburt, and colleagues (Aarons et al., 2014; Hurlburt et al., 2014). In this model, interagency seed teams consisting of stakeholders with varied and complementary expertise (e.g., EBT experts and individuals with knowledge of the local context) support the implementation and redesign of EBT in complex systems. Emerging evidence suggests that the ICT process can facilitate the alignment of the structure of evidence-based practices with local contextual variations (Hurlburt et al., 2014).

A UCD perspective conceptualizes program adaptations as less a nuisance to be minimized during the primary implementation phase of a project, and more an essential feature of intervention design to be considered from the first moments of its conceptualization. Explicit co-design processes – in which EBT developers specify major core components and an overarching structure, but allow service providers and recipients opportunities to determine more specific aspects of the intervention in real time – have been proposed as a method for the development of complex, contextually-appropriate practices in fields such as education (Penuel, Roschelle, & Shechtman, 2007) and mental health (Chorpita, Bernstein, & Daleiden, 2011). Such approaches have considerable potential to account for natural constraints because local providers are given license to address them as they arise. Frazier and colleagues (2012), engaged in an elaborate process to redesign an existing EBT – the Summer Treatment Program (STP) for disruptive behavior problems (Pelham et al., 1997) – for use in a low-income, urban after-school environment that paid explicit attention to a natural constraints. Their process involved a full year of discussion; resource mapping and needs assessment; participant observation; modeling, practice, and feedback; and ongoing problem solving to identify constraints and establish a final, flexible set of intervention principles and specific tools that emphasized the goals of the after-school environment (activity engagement and instruction, behavior management, and academic enrichment). Notably, the overarching goal of this project was not the implementation of the STP, but the identification of a set of practices that could best support the existing mission of the destination context (i.e., a design solution that highlighted environmental constraints). This orientation reflects a central tenet of UCD in that it places primary importance on the goals and needs of end users.

Discussion and Future Directions

This paper has focused on the ways that principles and processes from the field of UCD can be leveraged to move EBT development and implementation into a new era of contextual appropriateness, scalability, and effectiveness. “Design thinking” holds promise in the extent to which it can drive a reconceptualization of EBT users, support more rapid innovation and testing of novel approaches, simplify standard EBT designs, and incorporate – rather than struggle to overcome – barriers and other natural constraints for new and existing interventions. From a UCD perspective, all human-made products are designed and “the alternative to good design is bad design, not no design at all” (Martin, 1990, p.12). Failure to explicitly consider a product’s design is a guaranteed pathway to problematic design.

As noted earlier, the existing division between design time and run time control is akin to the common separation of architect and contractor responsibilities, where one professional completes their tasks before handing off a product to the next. Although this model has value, adopting a hybrid model for professionals interested in intervention development and implementation processes might prove to be better aligned with a UCD perspective (along with its built-in expectations for iteration, redesign, and continuous improvement) and, ultimately, more effective. Interestingly the traditional distinction between architect and contractor has also begun to break down in favor of emerging design-build models that integrate both components (Reed Construction Data, 2013). Design-build is a project delivery system used in the construction industry in which a single entity is responsible both for planning and executing a new building. In this system, design and construction phases are overlapping rather than sequential, which saves time and allows for greater responsiveness to client needs or difficulties encountered during implementation of the design schematic. A comparable shift in the mental health field may be useful, in which developers begin to consider themselves designer-builders and take on the responsibility of contextually-appropriate application of their EBT blueprints. Unfortunately, relative to the consolidation of the design and build components of the construction industry, there are fewer clear incentives for adopting some of the UCD approaches we have articulated. Nevertheless, given the current emphasis on accountability in healthcare and recent findings questioning the size of EBT effects (Driessen, Hollon, Bockting, Cuijpers, & Turner, 2015; Weisz et al., 2013), interest in design-oriented initiatives that can improve effects through locally-relevant adaptations (Chambers et al., 2013) may become increasingly appealing.

A Design-Based Research Agenda

The current paper has begun to apply a UCD perspective to the development, redesign, and implementation of psychosocial mental health interventions, but remains largely theoretical. Research is needed to begin to test the applicability of these principles and evaluate the results of engaging in user-centered processes to improve key outcomes at the levels of the system (e.g., appropriateness, efficiency, penetration/reach), service provider (e.g., adoption, acceptability), and client (e.g., acceptability, improved functioning). Even as increasing research has focused on developing and testing implementation strategies to improve EBT uptake and sustainment (Powell et al., 2015), studies that have examined aspects of EBT design (e.g., simplicity, usability) and their connections to identified implementation outcomes (Proctor et al., 2011) are almost nonexistent. Pursuit of this research agenda could involve the construction of comprehensive usability assessment protocols to evaluate existing EBT, test their applicability to different user types, identify avenues for redesign, and drive simplification. This might include developing test scenarios to assess how easily practitioners can use different components of the intervention (e.g., engaging in prolonged exposure in an anxiety protocol) following a training. This approach is similar to emerging methods for assessing practitioner skill acquisition following training (e.g., behavioral rehearsals; Beidas, Cross, & Dorsey, 2014) except that the intent would be to evaluate and revise the intervention or training protocol rather than assessing the practitioner’s competence and determining the need for additional supports. A more economical option may be to adapt one of the many existing usability self-report measures. The 10-item System Usability Scale (SUS; Brooke, 1996), for instance, is generally considered to be among the most sensitive, robust, and widely used scales of its type (Sauro, 2011; Tullis & Stetson, 2004). The SUS yields a total score ranging from 0–100, with scores >70 indicating an acceptable level of usability when applied to digital technologies. New norms could be established across a range of EBT protocols, thus providing a benchmark against which adaptations of existing interventions or new protocols could be compared. Once established, a measure of EBT usability may also be used to explore the statistical relationship between usability and the growing number of measures evaluating implementation constructs (Lewis et al., 2015).

It may also be possible to glean design-relevant information from available data drawn from clinical trials or routine service delivery. For instance, EBT adherence data – frequently collected in the context of clinical trials, but often not feasible in typical service settings – could be leveraged to explore the nuances of usability problems. Within a UCD framework, adherence data may be conceptualized as an indicator of EBT task completion; a common metric in usability testing. Adherence checklist elements that are frequently omitted or delivered incorrectly may reflect EBT usability issues and suggest the need for redesign of those intervention elements or their related training procedures. In routine service delivery, settings that engage in routine outcome monitoring (e.g., using a measurement feedback system; Bickman et al., 2012) could facilitate a UCD approach by allowing for simultaneous “A/B” trials of different intervention configurations to determine what combination or sequencing of intervention components (e.g., preceding exposure procedures for anxiety with a brief motivational intervention intervention) is most effective at a session and case level.

Finally, although we presented only a subset of UCD concepts in the current paper, additional direction may be gleaned from the worlds of effective design and technology development, especially as it relates to large-scale rollout of new projects. For instance, the mental health field may want to take notice of the increasing need for interoperable systems within health information technology, where there is now widespread acknowledgment that nearly ubiquitous digital tools need to be able to work together and share information in a way that enhances convenience and value for the user (Fontaine, Ross, Zink, & Schilling, 2010). As EBT are increasingly scaled up across large service systems, alignment of multiple interventions within organizations is quickly becoming a priority (Chorpita et al., 2011). Designing and testing EBT within interoperability in mind (e.g., mechanisms for information sharing, elimination of redundancies) may be one way to facilitate this task.

Conclusion

It is our perspective that a UCD approach represents a promising collection of methods and a way forward for researchers and practitioners interested in supporting the dissemination and implementation of psychosocial interventions. EBT researchers who adopt a design-informed approach will be well advised to embrace the mentality of rapid trial and error and nonlinear progress that can be observed in the design and technology communities. Calls to “fail early and often” are useful reminders to take chances, try new approaches, hedge bets by exploring multiple innovations simultaneously, and reconceptualize success as the exploration of new possibilities rather than the confirmation and solidification of the status quo. A similar perspective has emerged within the National Institute of Mental Health in their “Fast-Fail” drug trials (http://www.nimh.nih.gov/research-priorities/research-initiatives/fast-fast-fail-trials.shtml), but no specific funding mechanisms currently exist to support comparable, rapid work in behavioral health.

The transition to more rapid approaches may be uncomfortable for some researchers accustomed to the traditionally slow scientific slog down a single investigative pathway, but is likely to pay important dividends related to scientific discovery and the widespread use of well-designed, contextually appropriate, and empirically-based interventions. Just as the first computers were complicated machines, accessible to and understood by only expert users, so too EBT protocols have historically only been available to highly trained (and often highly motivated) mental health providers. It is our hope that redesigning EBT protocols and implementation processes can make them as accessible and ubiquitous as computing has become for large segments of the general population.