Human-centered design methods to achieve preparation phase goals in the multiphase optimization strategy framework

Abstract

Background

The public health impact of behavioral and biobehavioral interventions to prevent and treat mental health and substance use problems hinges on developing methods to strategically maximize their effectiveness, affordability, scalability, and efficiency.

Methods

The multiphase optimization strategy (MOST) is an innovative, principled framework that guides the development of multicomponent interventions. Each phase of MOST (Preparation, Optimization, Evaluation) has explicit goals and a range of appropriate research methods to achieve them. Methods for attaining Optimization and Evaluation phase goals are well-developed. However, methods used in the Preparation phase are often highly researcher-specific, and concrete ways to achieve Preparation phase goals are a priority area for further development.

Results

We propose that the discover, design, build, and test (DDBT) framework provides a theory-driven and methods-rich roadmap for achieving the goals of the Preparation phase of MOST, including specifying the conceptual model, identifying and testing candidate intervention components, and defining the optimization objective. The DDBT framework capitalizes on strategies from the field of human-centered design and implementation science to drive its data collection methods.

Conclusions

MOST and DDBT share many conceptual features, including an explicit focus on implementation determinants, being iterative and flexible, and designing interventions for the greatest public health impact. The proposed synthesized DDBT/MOST approach integrates DDBT into the Preparation phase of MOST thereby providing a framework for rigorous and efficient intervention development research to bolster the success of intervention optimization.

Plain Language Summary

• 1. What is already known about the topic? Optimizing behavioral interventions to balance effectiveness with affordability, scalability, and efficiency requires a significant investment in intervention development.

• 2. What does this paper add? This paper provides a structured approach to integrating human-centered design principles into the Preparation phase of the multiphase optimization strategy (MOST).

• 3. What are the implications for practice, research, or policy? The proposed synthesized model provides a framework for rigorous and efficient intervention development research in the Preparation phase of MOST that will ensure the success of intervention optimization and contribute to improving public health impact of mental health and substance use interventions.

Introduction

An optimization approach to building and adapting behavioral interventions prioritizes continual and quantifiable progress toward solving individual and societal problems, such as addressing the unmet need for mental health and substance use services, promoting critical health behaviors, and preventing the development of mental and physical health problems related to adverse childhood events. Intervention optimization is a process that seeks to identify an intervention that yields the best-expected outcome, given the real-world conditions imposed by the setting in which the intervention will be implemented (Collins, 2018). To increase the potential for public health impact, interventions should be designed for the contexts in which they will be used, prioritizing implementability as much as effectiveness. Often, the intervention design itself is overlooked as an implementation determinant, with more attention being paid to strategies to improve intervention adoption and sustainment (Lewis et al., 2021). Given the widely documented, nearly two-decade gap in time that it takes evidence-based interventions to develop in scientific laboratories and make their way to community settings (Estabrooks et al., 2018), some intervention developers are now adopting an implementation-forward approach. This approach means they are identifying implementation determinants at the outset and considering them throughout the intervention development process. Though adopting an implementation-forward orientation is a laudable goal, the challenge is finding a way to develop interventions for implementation contexts systematically and effectively.

Intervention optimization is an expedient way to develop interventions for successful implementation. The multiphase optimization strategy (MOST) is an innovative, principled framework that guides the development and optimization of multicomponent interventions such that they are effective, affordable, scalable, and efficient. (Collins, 2018). MOST outlines three phases that comprise the development of interventions: (a) Preparation, (b) Optimization, and (c) Evaluation. Each phase has a specific purpose and activities, but phase goals may be achieved using various research methods dictated by the research question and type of intervention being developed. In the Preparation phase, researchers derive/revise the conceptual model for the intervention, develop a set of candidate components, define the optimization objective of the intervention package (i.e., operational definition of the best-expected outcome under key constraints of the implementation setting), and conduct pilot testing of the candidate components. In the Optimization and Evaluation phase, researchers identify the experimental design that best matches the research question and can yield the most valuable scientific information with the least resources expended.

Research methods used to complete the Preparation phase activities are less clearly defined as these activities are commonly researcher/project-specific and based on theory and empirical evidence. However, the information gathered in the Preparation phase is critically important as it lays the foundation for the entire optimization process. Intervention optimization using the MOST framework requires a systematic approach overall, and the Preparation phase of MOST would benefit from more systematic yet flexible approaches to accomplish Preparation phase goals. Concrete ways to achieve Preparation phase activities, such as defining the optimization objective, which often aligns with implementation determinants informed by stakeholders, are a priority development area. In a recent commentary detailing key directions for MOST research in the next decade, Collins and colleagues noted that within the MOST framework, “approaches for obtaining information and managing input from stakeholders have not yet been developed” (Collins et al., 2021, p. 2003; see Windsor et al., 2021 for a notable exception).

We propose that the discover, design, build, and test (DDBT) framework, which centers on an array of human-centered design (HCD) strategies (e.g., cognitive walkthroughs, usability testing, direct observation; (Lyon, Brewer, et al., 2020) may serve as a systematic guide to accomplish the goals of the MOST Preparation phase including effectively designing novel or adapted intervention components in ways that improve their implementability from the outset. Although DDBT may be applicable across multiple phases of MOST, integrating DDBT into the Preparation phase of MOST provides a theory-driven and methods-rich roadmap for preparing a new or adapted intervention for optimization. In DDBT, HCD strategies can be used to achieve specific Preparation phase objectives, such as refining the conceptual model, identifying and designing candidate intervention components, specifying the optimization objective, and conducting pilot studies.

We begin with a brief overview of the two frameworks on which our argument is built—MOST and DDBT. We then discuss the overlap in core principles shared by MOST and DDBT approaches to intervention development. Next, we discuss practical applications of this synthesized approach by describing how the HCD strategies leveraged in DDBT can be used to address each Preparation phase goal. We present an applied example of a project using this synthesized approach to design a digital intervention for children coping with post-separation/divorce interparental conflict that is effective, affordable, scalable, and efficient in the context of the family court. The intention is to provide scaffolding for researchers interested in using DDBT to accomplish the goals of the MOST Preparation phase.

The MOST Framework

MOST is an engineering-inspired framework for developing behavioral or biobehavioral interventions that strategically balances intervention effectiveness with affordability, scalability, and efficiency (Collins, 2018). The guiding tenet of MOST is that interventions should be optimized to maximize efficacy and/or effectiveness within the constraints of a well-defined optimization objective, and only after an intervention has been optimized should it be evaluated in an RCT (Collins, 2018). Thus, “optimization” in MOST refers to a process of systematically determining (a) the most effective combination of intervention components that (b) the specified real-world implementation determinants as defined by the optimization objective. Intervention components are defined as any part of the intervention that can be separated for experimental study (Collins, 2018). A candidate component may include the intervention content (e.g., cognitive restructuring, behavioral activation), engagement strategies (e.g., coaching calls, text reminders), delivery of content structures (e.g., intervention algorithms, measurement-based care), or delivery fidelity approaches (e.g., clinician checklist, training, and supervision protocol). MOST studies often also include a “constant component” that is not under experimental control but is included in all conditions based on clinical or theoretical reasons (e.g., psychoeducation, established minimum best practice). When investigators build interventions based on knowledge of the relative contribution of each intervention component, the result is optimized interventions that are not only effective but also, over time, will be incrementally more efficient, affordable, and scalable (i.e., able to be implemented at scale without extensive adaptation). Notably, MOST aims to continually optimize interventions across time and settings, making them more effective and efficient with each iteration.

MOST is distinct from the classical treatment package approach to intervention development. The classical treatment package approach examines behavioral interventions as a package (i.e., the effect of all components collectively). It seeks to establish the efficacy of an intervention package through an RCT prior to making necessary adaptations to the intervention for it to be implemented in the intended treatment setting. Consequently, successful implementation often requires more than one resource-intensive RCT to establish preliminary efficacy, assess the effectiveness of the treatment package in the intended setting, and examine implementation strategies and outcomes (Aarons et al., 2017; Curran et al., 2012; Onken et al., 2014). Following this approach often takes more than 15 years to yield an effective and implementable intervention (Karlin & Cross, 2014).

In contrast, MOST prioritizes implementation objectives from the outset. It analyzes component effects to identify an optimized intervention comprising only components that meet the prespecified optimization objective before evaluating an intervention package in a resource-intensive RCT. See Collins (2018) and Guastaferro and Collins (2019) for a comprehensive introduction to MOST. Using MOST, intervention developers move iteratively through three phases (Preparation, Optimization, and Evaluation) to optimize an intervention package. In the Preparation phase, researchers (a) identify or develop a conceptual model to guide intervention component development; (b) identify, create, and/or refine candidate intervention components; (c) conduct pilot studies, as needed, to assess and improve the feasibility and acceptability of each intervention component before transitioning to the Optimization phase; and (d) identify the optimization objective for the intervention package (e.g., maximum cost per patient vs. highest effectiveness).

Then, in the Optimization phase, researchers conduct an optimization trial to identify components of the optimized intervention based on the estimated effects of those components (independently and in combination). Optimization trials are rigorous, randomized experiments that use various experimental designs (e.g., factorial trial, sequential multiple assignment randomized trial, micro-randomized trial) to collect the empirical information needed to optimize an intervention for efficacy and implementation. For example, a factorial experimental design allows researchers to examine the main and interaction effects for all intervention components considered for inclusion in the optimized intervention package. Although optimization trials typically focus on behavioral markers of change (e.g., weight loss, minutes of exercise), component effects could be conceptualized as any outcome that is of interest to the investigator, including implementation determinants (e.g., usability, engagement). Implementation determinants could be considered candidate intervention components in the Optimization phase if, for example, the purpose of the optimization trial is to assess implementability effects of design alternatives. As such, the Preparation and Optimization phases guide the systematic (and often iterative) development and refinement of an optimized intervention package to be evaluated in the final Evaluation phase (Freedland, 2020). At the end of the Optimization phase, the researcher has identified the optimized intervention package, considering the optimization objective.

Lastly, in the Evaluation phase, the optimized intervention package is evaluated against a suitable comparison, often but not always in a two-arm RCT. MOST includes an RCT experimental design for intervention evaluation like the classical treatment package approach. However, using MOST, the optimized intervention package subjected to experimentation in the RCT has been through two initial phases (Preparation and Optimization) focused on iteratively developing an affordable, effective, and scalable intervention package. The majority of intervention scientists will be familiar with many of the activities outlined in the Preparation phase. The advantage of MOST is to provide a framework to achieve these activities in a systematic and intentional way. As a result, researchers using MOST are able to incorporate real-world implementation constraints from the outset of intervention design rather than struggle to overcome these constraints later. In addition to the design of the intervention itself, the MOST framework applies to optimizing implementation strategies to promote the adoption and sustained use of interventions (Broder-Fingert et al., 2019). Ultimately, MOST researchers aim to illuminate ways to continually improve the effectiveness, affordability, scalability, and efficiency of the intervention.

MOST is a comprehensive and iterative implementation-forward framework for optimizing multicomponent interventions based on empirical information about component effects and real-world constraints (Guastaferro & Collins, 2021). Researchers across several public health priorities have used MOST to successfully optimize multicomponent interventions including, but not limited to, smoking cessation (Piper et al., 2016, 2018; Schlam et al., 2016), weight loss (Spring et al., 2020), and prevention of sexually transmitted infections in college students (Tanner et al., 2021; Wyrick et al., 2020).

The DDBT Model

The DDBT model was introduced as a framework to study and improve the usability of intervention components as a key determinant of intervention implementation (Lyon, 2019). Usability is the degree to which the individuals for whom an intervention is intended can accomplish specified goals easily, effectively, and efficiently in a satisfactory manner and in the specific context where it will be implemented (Lyon & Koerner, 2016). DDBT is a three-phase model that combines elements of HCD and implementation science to inform the systematic development of intervention components. The overall goal of DDBT is to maximize usability and implementation outcomes—and ultimately service recipient outcomes—of novel or redesigned interventions. The first phase, Discover, involves learning about the contextual constraints of the intervention (i.e., the setting in which it will be implemented), the needs and preferences of individuals/users, and optimal design elements of the intervention components (e.g., digital vs. in-person). In this first phase, stakeholders play a fundamental role. Stakeholder engagement is critical in DDBT, and HCD more broadly, as user preferences are a key intervention-level implementation determinant. In other industries, consumer preferences and interest in a new product drives its advancement through the development process. In other words, if a product is not of interest to potential consumers, it will not be developed. This mindset is not common for behavioral interventions, which likely contributes to challenges with intervention implementation, especially adoption and engagement. Stakeholders are broadly defined as any individual who will potentially impact the implementation of the intervention. Stakeholders may be primary users, such as clinicians or other providers who will deliver the intervention, and/or patients, clients, participants, or families who will directly use the intervention, or secondary users, such as administrators who make decisions about which interventions to adopt. Their input determines the primary issues or user needs to be addressed by the intervention, identification of anticipated multilevel implementation determinants, and how current interventions are working in the targeted setting.

The second phase of DDBT, Design and Build, involves iteratively developing and refining intervention components (e.g., via rapid prototyping) so that they are designed to meet the needs of key stakeholders—especially those who will directly use, or benefit from, the intervention (i.e., primary users)—and align with system constraints. DDBT intentionally does not prescribe specific design processes but researchers using this model use a variety of techniques (see Dopp et al., 2019 for a glossary of techniques). Typically, the evaluation includes multiple approaches to data collection and information gathering, such as usability testing and observations. Intervention theory can, and we argue, should be considered as an input when determining which intervention components should be tested, revised, and ultimately retained. Although it is not required by DDBT, many researchers using this model have considered intervention components using a tasks and packaging breakdown model (see Table 2 in Lyon, Brewer, et al., 2020). See Bearss et al., (2022) for an applied example.

Table 2.

Example Application: DDBT Design and Build Activities for the MOST Preparation Phase.

Identify set of candidate intervention components. The research team reviewed the stress and coping literature to identify coping strategies that would be appropriate intervention content components. Reappraisal, distraction, and relaxation coping strategies are related to fewer mental health problems among children (Compas et al., 2017), making intervention components based on these strategies key candidates for inclusion in an optimized coping intervention. Subsequently, the candidate intervention components are skills training in reappraisal, distraction, and relaxation. Design intervention components as digital modules. Leveraging data from the Discover phase, the research team developed four digital modules—a psychoeducation module to serve as a constant component and three skill-building components. The constant component provides basic education about divorce and brain science-based explanations of the connections among feelings, thoughts, and actions. Content of the constant component draws from the work of other research teams designing interventions for youth that maximize engagement (Schleider et al., 2020). Each candidate component (i.e., reappraisal, distraction, and relaxation) was designed to teach one coping strategy that was expected, based on the conceptual model, to increase the use of the coping behavior and, in turn, decrease children's maladaptive responses to IPC events (Rhoades, 2008; Sandler et al., 1994). The research team designed two initial module prototypes. They conducted a brief design survey with N = 10 children ages 9–12 to help select and name characters for the intervention. Assess preliminary feasibility of the digital modules. To ensure that the digital modules met the design criterion of 30 min or less specified by qualitative data, the researchers conducted seat time testing sessions with adults (n = 3) and children (n = 4). The researchers cut content to meet the desired seat time based on these data. Assess usability, appropriateness, acceptability, and perceived usefulness of the digital modules. The researchers conducted user testing sessions of the digital module prototypes with stakeholders (family court professionals, parents, and children; N = 18). User testing sessions included a think-aloud protocol, direct observation of task completion, and mixed methods feedback data collection. Qualitative assessment included questions regarding participants’ overall impressions of the module, experience of anything confusing, unusual, or difficult in the module; parts of the module participants liked the most and the least, and ideas for improvement. Participants were also asked for their opinion on whether children would like the module, whether parents would approve of their children participating in the module, and whether family law professionals would refer families to the module. Quantitative assessment included anticipated (before) and experienced (after) ease of use ratings, completion of the 10-item System Usability Scale (Brooke, 1996), the 4-item AIM, and the 4-item IAM (Weiner et al., 2017), and two items on perceived usefulness (i.e., “The module gave good ideas” and “Kids whose parents get divorced or split up should do this module”).

Note. IPC = interparental conflict; AIM = Acceptability of Intervention Measure; IAM = Intervention Appropriateness Measure; DDBT = discover, design, build, and test; MOST = multiphase optimization strategy.

The third and final phase of DDBT, Test, involves small-scale pilot testing of the newly designed intervention components to ensure they are acceptable to end-users as well as feasible and appropriate for the setting before being formally tested for efficacy or effectiveness (Leon et al., 2011; Lyon et al., 2019). Many researchers adopt a mixed methods approach to this phase, using qualitative methods and quantitative ratings (e.g., Weiner et al., 2017) to assess feasibility, contextual appropriateness, and acceptability. Like MOST, DDBT is applicable to the development or redesign of both client-facing interventions and implementation strategies (Lyon et al., 2019).

Across all three phases of the DDBT framework, HCD strategies are used to achieve the key objectives. A wide range of HCD strategies can bolster intervention implementation. Most prioritize basing the design of intervention components on the preferences and needs of the individuals who will ultimately use the intervention and the setting in which it will be implemented (Dopp et al., 2019). In intervention development, HCD strategies are used to gather data on the wishes, preferences, beliefs, values, and ideas of key stakeholders and develop design solutions. There are at least 30 distinct HCD strategies relevant to intervention development/redesign and implementation (see Dopp et al., 2019 for full review), including design-oriented focus groups and surveys, co-creation sessions, heuristic evaluations, and observational usability tests. Often, intervention developers adopt a multi-method approach to make use of several HCD strategies. For example, following a focus group or qualitative survey to learn about potential interest in and preferences for the intervention, an intervention developer may conduct a co-creation session during which the intervention developer and potential user(s) design an intervention component prototype side-by-side. The intervention developer may decide to conduct user testing sessions with another group of potential users to directly observe them interacting with the prototype, and then conduct an experience sampling study in which potential users engage with the prototype for a period of time during the course of their daily activities. Key themes in HCD research include early and repeated involvement of stakeholders in all phases of the intervention development process and a mixed-method, iterative research design.

Alignment of Principles Underlying MOST and DDBT

We suggest that combining the MOST and DDBT frameworks enhances the intervention optimization process. Importantly, the two frameworks share many core principles, illustrating their shared vision of an implementation-forward approach to intervention development. Beyond that, the two frameworks offer complementary perspectives and research methods that provide a theory-driven and methods-rich roadmap for preparing a new or adapted intervention for optimization.

First, MOST and DDBT both take a systematic, phased, and iterative approach to intervention development. They share the assumption that interventions should be developed based on factors that define successful implementation in real-world settings and ultimately contribute to public health impact. Notably, MOST and DDBT are “problem-agnostic,” suitable for any complex (i.e., multicomponent) psychosocial, behavioral, or public health intervention or implementation strategy. This principle applies to all behavioral intervention development processes broadly defined, including the establishment of novel interventions as well as redesigning interventions for new contexts or populations, regardless of the mode of delivery (i.e., digital, in-person, hybrid, individual, group) or recipient of the intervention (e.g., direct to consumers, training models for clinicians, organization-level implementation strategies). Importantly, both frameworks explicitly acknowledge that intervention researchers should be responsive to changes in systems and constraints over time (e.g., during application in new contexts) and understand that interventions may require redesign to maintain fit.

Second, although intervention effectiveness is a critical element of MOST and DDBT, both contend that intervention developers should balance effectiveness with intervention-level implementation determinants such as affordability, scalability, and efficiency. Prioritization of these determinants ultimately depends on the constraints of the intended setting in which the intervention will be implemented and the users or beneficiaries of the intervention. Thus, the goals of DDBT and MOST are complementary and inextricably linked. For example, an intervention must be highly usable (DDBT goal) to be maximally effective (MOST goal), successfully implementable at scale (DDBT and MOST goal), and efficient (MOST goal).

Third, and possibly most important, DDBT and MOST emphasize the critical role of engaging stakeholders, intervention gatekeepers, and intervention users or beneficiaries in the intervention development process. Input from these sources directly informs the intervention's implementability and needs to be prioritized early in the development process (i.e., in the Preparation phase of MOST and throughout the phases of DDBT). DDBT provides explicit methods and a structure for gathering this information as part of the MOST Preparation phase, and MOST provides the structure to rigorously evaluate the candidate intervention components that emerge from the DDBT process.

Using HCD Strategies of DDBT to Accomplish The Preparation Phase Goals of MOST

The DDBT framework can be used to accomplish the goals of the MOST Preparation phase: determining the key optimization objective for the intended setting, creating theory- and evidence-based intervention components and pilot testing to assess and improve components’ usability, feasibility, acceptability, and appropriateness (all key implementation determinants; Lyon & Bruns, 2019; Weiner et al., 2017). The DDBT framework primarily employs HCD methods to design or refine interventions and/or implementation strategies with the overarching goal of improving usability and other key intervention-level implementation determinants.

Figure 1 outlines the MOST framework and describes how the DDBT framework can be embedded within the Preparation phase of MOST to meet specified goals. As we outline each step, we describe an ongoing example project that applied the synthesized DDBT/MOST approach to develop a digital intervention for children aged 9–12 years who experience high levels of interparental conflict in the context of parental separation/divorce.

Figure 1.

Application of DDBT to accomplish the preparation phase of MOST. Note. Adapted with permission from Collins (2018). DDBT = discover, design, build, and test; MOST = multiphase optimization strategy.

The first phase of the DDBT framework, Discover, addresses two MOST Preparation phase activities: identifying or developing the conceptual model and the optimization objective of the intervention (Table 1). The conceptual model is based on theory and extant empirical evidence; it clarifies hypothesized mechanisms and associated intervention targets to inform component selection. In the case of limited theory and/or empirical literature, the conceptual model can be informed by other sources including, but not limited to, secondary data analyses, clinical experience, unpublished literature, and conference presentations (Collins, 2018). The Discover phase uses methods designed to contextualize the conceptual model by deepening understanding of the implementation context. Engaging in mixed-method research activities, intervention developers prioritize structured contextual observation of primary users and other stakeholders (i.e., providers and consumers/patients) who are relevant to natural processes in the intended implementation setting. For example, structured contextual observation could include shadowing providers to capture workflows in the intended setting and/or watching intervention session recordings while providers explain their thought processes and the challenges they faced (Lyon, 2019).

Table 1.

Example Application: DDBT Discover Activities for the MOST Preparation Phase.

Develop conceptual model. The researchers developed a conceptual model using a combination of empirical literature and clinical experience. A review of the literature indicated that decades of children who experience high-IPC parental separation/divorce are at risk for mental health problems and that this risk pathway is explained by their cognitive, emotional, and behavioral reactions to IPC events (Davidson et al., 2014; Long et al., 1988; Rhoades, 2008). Children who are exposed to high levels of IPC often endorse maladaptive cognitive appraisals (e.g., self-blame, safety concerns, fear of abandonment), intense emotions (e.g., fear, anger, sadness), and coping behaviors (e.g., avoidance, involvement) (Cummings et al., 2006; Grych, 1998; O’Hara et al., 2021). Empirical literature matches the team's clinical experience working with children who have reported these experiences and demonstrated difficulty engaging in adaptive coping behaviors to help themselves cope with the distress of exposure to IPC. Identify key stakeholders. The research team identified key stakeholders as follows: (a) family court professionals who either have clinical experience with high-IPC separating/divorcing families or administrative duties that involve selecting interventions for these families, (b) separating/divorcing parents who have experienced high IPC with their child's other parent, and (c) children ages 9–12 of separating/divorcing parents who have been exposed to high IPC. Assess intervention needs and preferences. Guided by the CFIR (Damschroder et al., 2009), the researchers conducted qualitative interviews with key stakeholders (N = 47) to assess intervention needs and preferences. Interview questions assessed user needs and resources, general recommendations for developing an intervention that is acceptable to both children and parents, and desired intervention characteristics (e.g., What should the intervention look like? What topics should the intervention cover? What are common IPC scenarios that can be used as relatable examples?). Identify optimization objective. Guided by CFIR (Damschroder et al., 2009), the research team identified key factors related to developing the most effective intervention within the constraints of the setting in which it would be implemented from the qualitative interviews with stakeholders. Interview questions assessed issues related to cost, duration, facilitators and barriers to implementation, the context of possible implementation settings, and factors that would be prioritized by: (1) the family court in their decision to adopt the intervention, (2) by parents in their decision to allow their child to participate in the intervention, and (3) by children in their decision to engage with the intervention. Based on these data, the optimization objective was defined as “all active components,” meaning that the intervention will comprise all candidate intervention components demonstrating a meaningful effect.

Note. IPC = interparental conflict; CFIR = consolidated framework for implementation research.

Additionally, the DDBT Discover phase emphasizes HCD methods to engage primary user stakeholders to ensure interventions meet their needs. These methods are essential for identifying optimization objectives in the MOST Preparation phase. An exemplary Discover phase method is a systematic user-identification process which begins with creating an overly inclusive list of potential users, clarifying the most relevant subset of user characteristics, and then describing, prioritizing, and selecting main primary user groups (Lyon, Koerner, et al., 2020). Primary users of interventions most often include providers and intervention recipients whose needs and constraints must be directly addressed in the intervention design. When an existing intervention is being redesigned, the Discover phase typically involves direct user testing (e.g., cognitive walkthroughs, task-based usability testing) of the original intervention components to identify modification targets. User testing is an efficient approach to understanding how an intervention component operates with users as studies are typically conducted with samples as small as five participants (Turner et al., 2006) and lower cost and “lower fidelity” prototypes—such as paper versions of redesign solutions for a digital intervention—to make the most efficient use of available resources (Lyon, Brewer, et al., 2020; Lyon, Koerner, et al., 2020). Another illustrative Discover phase method identifies stakeholder needs via semistructured interviews, focus groups, and survey studies that target the aspects of an intervention that might be misaligned with a particular context. There are related examples in the literature of MOST researchers using mixed methods approaches to identify or clarify optimization objectives for interventions addressing smoking cessation and substance use prevention (Bernstein et al., 2018; Whitesell et al., 2019). These stakeholder engagement and data collection methods can enable efficient and effective work with primary end-users to identify preliminary optimization objectives for the intended treatment setting.

The second phase of the DDBT framework, Design and Build, addresses two MOST Preparation phase activities: identify a set of candidate intervention components and conduct pilot studies (Table 2). The Design and Build methods allow researchers to iteratively develop, evaluate, and refine prototypes of intervention components to meet the needs of the identified stakeholders (Lyon et al., 2019) and then prioritize intervention components for testing (see Lyon, Coifman, et al., 2021; Lyon, Pullmann, et al., 2021, note that when an intervention/strategy is being redesigned, this prioritization typically happens in the Discover phase). Specifically, researchers build prototypes of new intervention components based on findings and insights from the Discover phase and rapidly evaluate and iterate these prototypes with small samples in user testing sessions to improve component usability and other implementation determinants. In the Design and Build phases, stakeholders engage in similar user testing methods as the Discover phase (e.g., think-aloud protocol, structured observation [Beidas et al., 2014; Lyon et al., 2019]), and the primary goal is to understand if the identified usability problems and implementation constraints have been adequately addressed. In addition to the focus on usability, researchers may use HCD methods to iteratively improve components’ feasibility, acceptability, and appropriateness.

The third phase of the DDBT framework, the Test phase, addresses the final MOST Preparation phase activities of identifying a set of candidate intervention components to be empirically examined in the Optimization phase and conducting intervention component pilot studies (Table 3). The Test phase focuses explicitly on ensuring components are usable, feasible, acceptable, and appropriate, in the intended treatment setting. Test phase methods include small-scale pilot testing of the newly designed/refined intervention components using various methods, such as a one- or two-arm pre-post trial or a pilot randomized factorial trial and are often conceptualized as pilot hybrid effectiveness-implementation trials (Curran et al., 2012). Primary outcomes in the Test phase include usability (Lyon, Pullmann, et al., 2021) and other implementation determinant measures completed by primary users (e.g., providers, consumers, patients) such as the Acceptability of Intervention Measure (AIM), Intervention Appropriateness Measure (IAM), and Feasibility of Intervention Measure (FIM, Weiner et al., 2017). Intervention recipients sometimes complete outcome measures to determine the feasibility of assessment and preliminarily evaluate intervention impact. Qualitative interviews may be conducted to explain and elaborate on quantitative data. Additional implementation feasibility data may also be collected, such as the number of training hours needed for providers to achieve adequate intervention fidelity, costs to implement the intervention, including provider training time, delivery time, and costs of ongoing supervision to ensure intervention fidelity over time.

Table 3.

Example Application: DDBT Test Activities for the MOST Preparation Phase.

Pilot testing. In preparation for a randomized optimization trial to assess the main and interactive effects of intervention candidate intervention components, the researchers will pilot study procedures with N = 6 children, including the randomization protocol, assessment measures, intervention components, and daily diary data collection app. Research assistants will track issues encountered with the randomization protocol and the assessment measures. Mixed methods feedback data of the intervention components and daily diary app will be collected. ◾ Qualitative assessment includes questions regarding the child's overall impressions of the intervention components and experience of anything confusing, unusual, or difficult in the intervention components. ◾ Quantitative assessment includes completion of the 10-item System Usability Scale (Brooke, 1996), the 4-item AIM and the 4-item IAM (Weiner et al., 2017), and two items on perceived usefulness (i.e., “The module gave good ideas” and “Kids whose parents get divorced or split up should do this module”).

Note. DDBT = discover, design, build, and test; MOST = multiphase optimization strategy.

Summary, Limitations and Future Directions

In this article, we reviewed MOST and DDBT, two innovative, implementation-forward frameworks for developing, improving, or redesigning behavioral interventions. Given their many shared conceptual features and to address the identified need for concrete ways to achieve MOST Preparation phase goals, we proposed a synthesized model (DDBT/MOST) that utilizes the DDBT framework as a theory-driven and methods-rich roadmap for achieving MOST Preparation phase goals. The Preparation phase provides the critical foundation for the subsequent phases of MOST and cannot be overlooked or undervalued. A recent systematic review by Landoll et al., (2022) proposed a checklist for Preparation phase activities to standardize reporting of this phase of MOST to enhance rigor and reproducibility. Though the checklist ensures Preparation phase activities are reported accurately, the authors note there is room for methodological guidance about accomplishing the goals of this phase. Specifically, concrete methods for carrying out Preparation phase activities systematically will equip intervention researchers with critical tools to conduct rigorous and efficient Preparation phase research, reaping benefits that will flow to all subsequent phases of intervention development, including optimization and evaluation trials. The DDBT framework provides one approach that is informed by HCD to achieve the Preparation phase goals of MOST.

It is important to note that DDBT is not the only framework for systematically achieving the Preparation phase activities and that many other approaches similarly include a phased approach to intervention development (see O’Cathain et al., 2019 for a taxonomy of intervention development approaches). DDBT integrates elements from many of the key categories highlighted by O'Cathain et al., (2019), including intervention development that is target population-centered, implementation-based, and stepped or phase-based. However, beyond its natural fit with the fundamental principles of MOST, DDBT offers several unique advantages. First, DDBT provides a useful (but not prescribed) set of methods for intervention evaluation and redesign. Second, DDBT explicitly centers stakeholder feedback throughout the intervention development process. Third, DDBT makes use of HCD strategies (i.e., user testing, observation) to systematically identify usability issues and drive redesign solutions. Nevertheless, DDBT is one of many intervention development approaches, and diverse approaches will be useful for different contexts, needs, and objectives.

DDBT also has important limitations. DDBT is user-centered but deliberately does not always involve full sharing of power with users/participants. This limitation stems from the HCD roots of DDBT and its contrast with a fully participatory process based on the rich traditions of Community-based Participatory Research (Minkler & Wallerstein, 2011). DDBT is also primarily focused on optimizing interventions for usability/implementability and less focused on optimization for effectiveness. In the Test Phase, determining preliminary efficacy is important but secondary to determining usability, feasibility, acceptability, appropriateness. This is one reason that DDBT is an excellent framework to achieve the Preparation phase goals of MOST, prior to the Optimization phase of MOST. Finally, DDBT is an overarching framework to guide redesign processes, but it is not a step-by-step guide for how to redesign intervention components. A prescriptive “how to” guide would have benefits like making the redesign process very concrete and ensuring standardization, but it would also have drawbacks like less flexibility and assuming a one-size-fits-all redesign process.

The proposed synthesized DDBT/MOST approach to developing interventions should be evaluated for practical application and studied more broadly. For example, we see the great additional potential for incorporating DDBT/HCD methods in the Optimization and Evaluation phases of MOST. In line with the MOST guiding principle of continual improvement (Collins, 2018) and as highlighted by Huffman et al., (2020), ongoing feedback from stakeholders and end-users is critical throughout intervention development phases. For example, there may be valuable uses of DDBT/HCD in the MOST Optimization phase to evaluate outcomes other than effectiveness, such as component usability. In such cases, Optimization phase activities could include factorial designs in which competing intervention component design options are evaluated using in vivo usability assessment techniques in service settings (Lyon, Brewer, et al., 2020; Lyon, Koerner, et al., 2020).

The proposed synthesized DDBT/MOST approach integrates DDBT into the Preparation phase of MOST thereby providing a framework for rigorous and efficient research that centers the user voice in intervention development. This systematic approach will enhance the rigor and reproducibility of Preparation phase activities of MOST, bolster the success of intervention optimization, and contribute to advancements in intervention and implementation science more broadly.