Using Intervention Mapping to developmentally adapt an online HIV risk reduction program for adolescent men who have sex with men

Abstract

Adolescent men who have sex with men (AMSM) experience a dramatic health disparity in HIV, accounting for over 80% of new diagnoses among youth. Current evidence-based HIV prevention programs, however, focus primarily on adults and heterosexual youth, thereby missing the unique experiences and socio-environmental contexts of AMSM aged 13–18. To address these gaps, we used the Intervention Mapping (IM) protocol to developmentally adapt an existing evidence-based online HIV risk reduction program (i.e., Keep it Up!/KIU!), originally designed for young adult MSM aged 18–29, into a new intervention called SMART Squad.

Using a hybrid of IM creation and adaptation tasks, we specified three behavioral outcomes and identified corresponding performance objectives for SMART Squad based on the original goals of KIU!. We constructed matrices of change objectives using determinants from the Information–Motivation–Behavioral Skills model, modifying them for the younger population with additional theoretical and empirical evidence and expert review. SMART Squad activities were operationalized from theory-based behavior change methods matched to the change objectives and guided by program themes, components, and scope imported from KIU!.

The final SMART Squad intervention comprises 6 episodes/modules delivered in 2 sessions plus 2 booster episodes occurring 1 and 3 months after the main program. It is currently being evaluated nationally as part of a stepped-care package of 3 programs, in which the receipt and sequencing of interventions is tailored to individual AMSM development and needs. Despite substantial changes to KIU!, IM was a useful method for retaining the hypothesized essential elements of the eHealth HIV risk reduction program. Challenges and recommendations for future researchers and practitioners are discussed.

Introduction

Young men who have sex with men (YMSM) in the US represent approximately 2% of youth (Kann et al. 2018) but account for over 80% of HIV diagnoses in individuals aged 13–24 (Centers for Disease Control and Prevention [CDC], 2018). Moreover, almost half of youth living with HIV are undiagnosed, the highest among any age group (CDC 2018). Condom use and HIV testing rates among adolescent MSM (AMSM, defined here as 13–18 years old) remain low (Kann et al. 2018; Mustanski et al., 2020), suggesting AMSM as critical targets for primary prevention. Fostering health-promoting and risk-reduction skills, habits, and behaviors early in their sexuality development could help temper the rate of HIV infection during young adulthood.

Despite this disproportionate burden, current evidence-based HIV prevention programs almost exclusively focus on adults and heterosexual youth. None of the current 65 programs identified by the CDC as effective HIV risk reduction interventions are designated for AMSM (CDC, 2019a). Interventions targeting MSM focus on adults over the age of 18, who may be further along in their general psychosocial and sexual orientation identity development relative to early adolescents (Savin-Williams 2011; Mustanski et al. 2014b; Boislard et al. 2016). Young adult (18–29 years old) and adult MSM are also expected to have more complex skills around sexual decision-making; accessing sexual health information, products, and services; and analyzing external influences on behavior (CDC, 2019b) and are legally allowed to enter certain venues prohibited to minors (e.g., bars, clubs). Conversely, programs targeting early adolescents are designed primarily for heterosexual youth, which may overlook topics essential to prevention among MSM, such as LGBTQ stigma, anal sex, rectal STI testing, and pre-exposure prophylaxis (PrEP) (Mustanski et al. 2011; Nelson et al. 2019). Because issues affecting sexual health decisions among AMSM are unique (DuBois et al. 2015; Mustanski et al. 2017a; Boislard et al. 2016), interventions must be specifically designed to ensure the appropriate content resonates.

Responding to this gap, we developed SMART, a suite of three online HIV prevention interventions for AMSM ages 13–18 (Ventuneac et al. 2019) that is currently being evaluated in both English and Spanish throughout the US, Puerto Rico, Guam, and American Samoa. SMART comprises a stepped-care strategy, reflecting the public health prevention model of low-cost interventions for a population, selective interventions for groups at heightened risk, and intensive interventions for individuals indicated as having the highest susceptibility (National Research Council and Institute of Medicine 2009). Each unit in SMART is developmentally adapted from an intervention designed for YMSM: SMART Sex Ed is a universal LGBTQ-inclusive sex education program adapted from Queer Sex Ed, originally designed for 16–20-year-olds (Mustanski et al. 2015). SMART Squad is an HIV risk reduction program adapted from Keep it Up! (KIU!) (Mustanski et al. 2018), a CDC “best-evidence” intervention for 18–29-year-olds (CDC, 2019a). Finally, SMART Sessions is a videoconference motivational interviewing protocol for AMSM at highest sexual risk and based on the CDC-best-evidence Young Men’s Health Project for 18–29-year-olds (Parsons et al. 2014). By adapting promising/effective interventions from similar populations, we hoped to reduce the time and resources required to develop intervention content and increase the odds that the adapted interventions would be effective with AMSM. Because of the large scope, we chose to revise each intervention separately. KIU! was the most complex to developmentally adapt, given extensive multimedia content and the age difference, so we selected a systematic framework for planning health promotion programs called Intervention Mapping (IM) (Bartholomew Eldredge et al. 2016) to guide the process. This paper describes our use of IM to adapt KIU! into SMART Squad.

Keep It Up!

KIU! is a web-based intervention that has been shown to reduce condomless anal sex and STI incidence among racially diverse YMSM aged 18–29 (Mustanski et al. 2018). Accessed via a desktop, laptop, or tablet computer, the intervention comprises seven modules, presented in three sessions, plus booster sessions at 3- and 6-months post intervention. Each module focuses on some context relevant to the lives of YMSM (e.g., hooking up online, navigating a bar, dating) and uses diverse multimedia (e.g., videos, animation, games) to target information, motivation, and behavioral skills (the IMB model; Fisher et al. 1994) related to HIV risk reduction. Excluding boosters, KIU! takes approximately one hour to complete; however, there are 24-hour breaks between sessions. The development and evaluation of KIU! have been described elsewhere (Mustanski et al. 2017b; Mustanski et al. 2013; Mustanski et al. 2018).

Intervention Mapping

Adapting an intervention may inadvertently affect its effectiveness (Botvin 2004), so modifications should be made cautiously, systematically, and with empirical and/or theoretical support. IM is a six-step protocol that, comparable with other adaptation frameworks (e.g., ADAPT-ITT, Wingood and DiClemente 2008) (Escoffery et al. 2019), organizes the program planning process around prescribed tasks (Bartholomew Eldredge et al. 2016). More uniquely, IM emphasizes the methodical specification of the mechanisms of behavior change. Through a series of logic models and matrices (examples presented below), IM explicitly links intervention components to behavior change methods that target theoretical determinants of behavior and modify them to achieve desired behavioral outcomes, which in turn improve downstream health and quality of life outcomes. These mechanisms serve as the blueprints from which an intervention is constructed.

IM has been widely used to develop new and adapt existing behavioral interventions, including eHealth programs (Miranda and Cote 2017; Boekhout et al. 2017). Here, we aim to demonstrate its utility for developmentally adapting the complex, multimedia KIU! and discuss challenges and lessons learned. We selected IM because we thought that comprehensively delineating KIU!’s theoretical mechanisms of action would be the best way to preserve them in light of substantial necessary changes to not only intervention content but also the technology. Additionally, given the dearth of intervention specification in the literature (Hoffmann et al. 2013), we aim to transparently characterize SMART Squad’s mechanisms/blueprints for future researchers and program implementers to examine.

Methods

Figure 1 depicts the six steps of IM. The steps for adapting an existing intervention mirror those for developing a new one, but the specific tasks differ. The tasks for adaptation focus on replicating and maintaining fidelity to the original content as much as possible.

Figure 1.

Intervention Mapping (IM) steps with tasks for program creation and program adaptation. Adapted from Bartholomew Eldredge et al. (2016) and Rodriguez et al. (2018). Straight, solid-line arrows indicate the typical process pathways dictated by the IM protocol. Curved, dotted-line arrows and bolded tasks indicate the hybridized process used to adapt Keep It Up! into SMART Squad. The loops in Steps 3 and 4 represent our borrowing from some adaptation tasks (e.g., adapting pieces of content) while remaining in the program creation track.

Establishing a program-planning group with comprehensive expertise is critical to any intervention development process. Given the size and scope of the SMART trial, investigators and staff were organized into several working groups (WGs) shared across the interventions: The Intervention Adaptation and Delivery WG comprised five PhD-level researchers (including authors DL, DM, KM, and BM [lead]) with expertise in HIV, sexual minorities, adolescent health, health communications, and eHealth intervention development. For SMART Squad, it led the developmental adaptation of KIU! activities and partnered with a video production company to create filmed content. The Technology WG, led by authors KM and RS, oversaw the engineering of non-video components and the web-based intervention platform with a staff of software developers, a graphic designer, and a quality control specialist. Third, the Linguistic and Cultural Adaptation WG consisted of researchers and staff at the University of Puerto Rico, who further adapted all content for Spanish-speaking participants; that process is outside the scope of this paper but is the focus of a forthcoming manuscript.

We enlisted advisory groups to provide consultation throughout the adaptation process. Two online Youth Advisory Councils (YACs) of racially diverse (50% minority) AMSM aged 14–18 from 24 states (N=46) gave near-real-time feedback on the design of individual elements, including language, messaging, graphics, format, and interactivity (Li et al. in press). Intervention content was iteratively developed and presented back to the YACs for critique. A Community Collaboration Board, comprising organizations that serve racially/ethnically diverse MSM, reviewed an early prototype of the intervention to ensure that content would be acceptable and culturally relevant to their clients and to discuss future implementation considerations (Ventuneac et al. 2019). Finally, a Content Advisory Team of seven local AMSM aged 14–18 beta-tested the intervention content and delivery system using a concurrent think-aloud protocol (Peute et al. 2015) to identify remaining usability issues. All activities were conducted with approval from the Institutional Review Board at Northwestern University, and we obtained a waiver of parental permission for all AMSM participants.

Results by IM Step

Step 1: Understanding the problem

The aim of Step 1 is to understand the health problems and/or behaviors affecting the target population. Because of the Intervention Adaptation and Delivery WG’s extensive history of research on HIV among YMSM and AMSM (e.g., Mustanski et al. 2014a; Mustanski et al. 2011), including the development of KIU!, we did not conduct a full needs assessment. We focused on identifying critical differences in sexual health education needs between the age groups (Nelson et al. 2019) by leveraging findings from the YACs and our research on AMSM. For example, although there is substantial variation in sexual developmental trajectories (Savin-Williams 2011), AMSM ages 13–18 are generally less sexually experienced (or have not initiated sex), have fewer sexual partners, and are less out about their sexuality than YMSM ages 18–29 (Mustanski et al. 2011). AMSM also generally have less access to certain environments (e.g., bars) and preventive resources (e.g., condoms, HIV testing, PrEP) for reasons including age-specific barriers, transportation issues, and/or lack of awareness. We also examined usability and acceptability feedback from past KIU! participants (Madkins et al. 2019) to inform updates to intervention functionality, with mobile compatibility being one of the most requested features.

Step 2: Linking outcomes to intervention objectives

In Step 2, program planners state their new/adapted program’s outcomes and objectives. We selected KIU! for adaptation because its outcomes, content, and form fit our target population and goals. However, as we began planning adaptations to make in the next step, two factors prevented us from replicating most of KIU! and just updating selected components. First, after comparing KIU!’s learning objectives to AMSM-specific factors identified in Step 1, we determined that key differences between AMSM and YMSM in terms of psychosocial and identity development, as well as access to health-promoting and risk environments and resources, warranted substantial changes. For example, KIU! is delivered in conjunction with an HIV test, but SMART Squad would not be because of low rates of testing in AMSM (Mustanski et al., 2020). KIU! includes a module focused on bars, a setting inaccessible to SMART participants.

Second, we needed to account for the changing sociotechnical landscape. KIU! was designed for desktop computers, but young people increasingly access the internet through mobile devices (Lenhart 2015), particularly for sexual health information (Mitchell et al. 2014). Redesigning the intervention platform for mobile compatibility necessitated functionality changes for some of the intervention’s interactive elements. User expectations of technology have also evolved, so style, graphics, videos, and activities needed to be refreshed to remain acceptable to AMSM. As such, we continued with development using a hybrid of creation and adaptation tasks specified by IM, treating SMART Squad as a novel intervention while attempting to keep within form, functionality, and content parameters set by KIU! (see Figure 1).

The primary behavioral outcomes of KIU! are to reduce condomless anal sex, use condoms consistently, decrease condom use errors and failures, reduce alcohol and drug use before sex, and get tested regularly for HIV. For developmental reasons, we decided to focus on correct and consistent condom use and HIV testing. We also expanded a secondary focus in KIU! related to healthy relationships to include psychosocial development more broadly because AMSM are more likely to be just starting to disclose their sexual identity, meet other AMSM, and initiate sexual activity (Mustanski et al. 2014b). Thus, the behavioral outcomes for SMART Squad are as follows: SMART Squad participants will (a) correctly use a condom 100% of the time if/when they engage in anal sex, unless they are in a long-term (at least a year), exclusive, trusting relationship with a partner who has tested HIV-negative; (b) get tested for HIV/STIs regularly (every 3 months) when sexually active; and (c) develop healthy identities and relationships with media, friends, and romantic/sexual partners.

Following IM creation tasks, for each behavioral outcome, we identified a set of performance objectives, or stepwise, incremental actions that are prerequisite to achieving the overarching behavioral goal. We crossed performance objectives with behavioral determinants from the IMB model (Fisher et al. 1994), on which KIU! was based, to construct behavior change matrices (see Table 1). In each performance-objective-by-determinant cell, we specified relevant mechanistic targets for the intervention, also called change objectives. Thorough review by study staff ensured all KIU! learning objectives were reflected in either the new SMART Squad performance and/or change objectives. A partial matrix for the condom use behavioral outcome is shown in Table 1; complete matrices can be found in Appendix A (available online).

Table 1.

Partial matrix of change objectives for the condom use behavioral outcome in SMART Squad

Performance Objectives╲Determinants	Information	Motivation				Behavioral Skills
	Knowledge	Attitudes	Vulnerability	Norms	Perceived Barriers	Skills	Self-Efficacy	Intentions
Behavioral Outcome 1. SMART participants will correctly use a condom 100% of the time if/when they engage in anal sex (unless they are in a long-term, exclusive, trusting relationship with an HIV-negative partner).
PO 1.1. Make an informed decision to have anal sex.	K1.1.1. List reasons why guys have sex.	A1.1.1. Believe there are positive benefits of abstaining from sex, including preventing HIV/STIs, and engaging in sex. A1.1.2. Believe they have agency in deciding whether or not to have sex.		N1.1.1. Believe some guys choose to abstain from sex.	B1.1.1. Recognize there is a lot of pressure to have sex.	S1.1.1. Be able to determine whether or not having anal sex is right for them.	SE1.1.1. Feel confident making a decision around anal sex.	I1.1.1. Intend to engage in anal sex only when ready.
PO 1.2. Decide to use a condom during anal sex.	K1.2.1. Identify how HIV/STIs are transmitted. K1.2.2. List consequences of HIV/STIs.	A1.2.1. Believe condoms are effective at preventing HIV/STIs.	V1.2.1. Believe they are susceptible to HIV/STIs.	N1.2.1. Believe other guys use condoms during anal sex.				I1.2.1. Intend to use a condom during anal sex.
PO 1.3. Obtain a condom.	K1.3.1. Identify places to obtain condoms.	A1.3.1. Believe they should have a condom even if not currently sexually active.	V1.3.1. Believe contracting HIV/STIs is bad.	N1.3.1. Believe other guys obtain condoms.	B1.3.1. Perceive it is easy to get a condom, including for free.	S1.3.1. Be able to obtain a condom.	SE1.3.1. Feel confident obtaining a condom.	I1.3.1. Intend to obtain a condom.
PO 1.4. Properly store a condom.	K1.4.1. List rules for storing condoms.	A1.4.1. Believe proper storage maintains condom effectiveness.	V1.4.1. Believe they are more susceptible to HIV/STIs if they do not properly store a condom.	N1.4.1. Believe other guys properly store use condoms.	B1.4.1. Perceive it is easy to store a condom discretely.	S1.4.1. Be able to store a condom properly and discretely.	SE1.4.1. Feel confident storing a condom properly and discretely.	I1.4.1. Intend to store a condom properly and discretely.
PO 1.5. Have a condom available when engaging in sex.	K1.5.1. List strategies for ensuring condom availability. K1.5.2. Identify backup plan if condom is not available.	A1.5.1. Believe it is important to be prepared with a condom just in case they have sex.	V1.5.1. Believe they will be less likely to use a condom if they don't have one readily available.	N1.5.1. Believe other guys keep condoms available just in case.	B1.5.1. Perceive it is easy to increase condom availability prior to engaging in sex.	S1.5.1. Be able to have condoms available in multiple locations. S1.5.2. Be able to procure a condom quickly if one is not available.	SE1.5.1. Feel confident in having condoms available.	I1.5.1. Intend to have condoms available.

Step 3: Designing the program plan

The aim of Step 3 in developing a new intervention is to match the change objectives to theory- and evidence-based behavior change methods (e.g., modeling, cues to action), which are then operationalized into practical applications/activities for the specific intervention context (e.g., vignette, reminders). A partial activities map for condom use is shown in Table 2 (see Appendix B, available online, for complete maps). When designing practical applications for SMART Squad, we tried to duplicate or match program themes, components, and scope from KIU! wherever possible. For example, both KIU! and SMART Squad end with participants selecting three prevention or risk-reduction goals for themselves, followed by the intervention’s helping them think through how to overcome barriers to achieving those goals. KIU! instructs YMSM to think about their sexual, health, and emotional needs; in SMART Squad, we modified the concept by adding life needs (e.g., doing well in school), changing “sexual” to “physical” needs, and coining the mnemonic “everyone needs HELP (health, emotional, life, and physical).”

Table 2.

Partial activities map for the condom use behavioral outcome in SMART Squad

Determinant/change objective: SMART Squad participants will…	Behavior change method(s)	Practical application
Performance Objective 1.3. Obtain a condom.
K1.3.1. Identify places to obtain condoms.	Consciousness raising, active learning	Condom finder decision-support tool lists common places to obtain condoms based on various input parameters (e.g., cost, urgency, transportation).
A.1.3.1. Believe they should have a condom even if not currently sexually active.	Modeling, shifting perspective, cultural similarity, entertainment education	Soap opera shows AMSM talk about having condoms even if not sexually active.
V.1.3.1. Believe contracting HIV/STIs is bad.	Modeling, shifting perspective, cultural similarity, entertainment education, anticipated regret	Soap opera shows HIV-negative AMSM discussing and facing negative consequences of STIs and HIV-positive AMSM talking about physical, emotional, and social consequences of HIV.
N1.3.1. Believe other guys obtain condoms.	Modeling, cultural similarity, entertainment education	Soap opera shows AMSM obtain condoms.
B1.3.1. Perceive it is easy to get a condom, including for free.	Modeling, cultural similarity, entertainment education, active learning	Soap opera shows AMSM obtain condoms. Condom finder decision-support tool lists common places to obtain condoms based on various input parameters (e.g., cost, urgency, transportation).
S1.3.1. Be able to obtain a condom.	Active learning, goal setting, planning coping responses	Condom finder decision-support tool lists common places to obtain condoms based on various input parameters (e.g., cost, urgency, transportation). Tailored goal setting includes a goal about obtaining condoms and helps brainstorm potential barriers and strategies around those barriers.
SE1.3.1. Feel confident obtaining a condom.	Active learning, goal setting, planning coping responses	Condom finder decision-support tool lists common places to obtain condoms based on various input parameters (e.g., cost, urgency, transportation). Tailored goal setting includes a goal about obtaining condoms and helps brainstorm potential barriers and strategies around those barriers.
I1.3.1. Intend to obtain a condom.	Tailoring, goal setting	Tailored goal setting includes a goal about obtaining condoms and helps brainstorm potential barriers and strategies around those barriers.

In many instances, substantial adaptation was needed for the content and/or technology. We utilized Mohr et al. (2015)’s principles for optimizing eHealth interventions to make decisions around updating content: First, we identified the behavioral change strategies (reflected in the matrices and activities maps) and instantiation components (the technology-enabled user experience) in KIU! that needed to be preserved. Second, the three WGs suggested content and technology changes based on their expertise in AMSM HIV prevention and web development, as well as feedback from YACs and previous KIU! users. Third, the leads of the WGs, including the primary developer of KIU!, assessed proposed updates using questions outlined by Mohr et al. (e.g., does the change interfere with the principle being tested, what is the consequence of not making the change). Bug fixes and usability enhancements were implemented quickly, whereas larger content or feature updates were compared to the matrices and often presented to the YACs for feedback. Adaptations were implemented if they were aligned with the change objectives and received positive reception from the YACs.

Major changes occurred in cases where original KIU! activities were not developmentally appropriate for AMSM, like the aforementioned bar module. We designed new applications that used similar theoretical change methods but reflected updated information or context. As many AMSM are meeting sexual and romantic partners online through geosocial networking applications (Macapagal et al. 2018), we created a simulated hookup app in lieu of KIU!’s virtual bar activity in order to teach AMSM skills around assessing and mitigating situations that may lead to higher HIV risk. In another example, the YACs indicated that the CDC’s HIV/STI testing locator (https://gettested.cdc.gov/) was too complicated (“required [multiple] steps”, “amount of text [was] unnecessary and overwhelming”), so we built a simpler interface for the same data.

KIU! employs various forms of video-based storytelling, including an episodic soap opera, to change YMSM’s sexual motivations (from the IMB model) through methods such as dramatic relief and entertainment education (Bartholomew Eldredge et al. 2016) that target beliefs about partner communication around condoms. Deciding this format could effectively engage and target other motivation-based determinants among AMSM, we expanded the soap opera to encompass the entire SMART Squad intervention and subsume change objectives previously covered by other types of videos (e.g., condom demonstration). Non-video activities that were independent from the soap opera in KIU! were in SMART Squad integrated into the storyline to create a single interactive narrative. The modules were thereafter renamed episodes.

YMSM in the KIU! trials said they wanted the ability to connect with other participants. Because this aligned with our third behavioral outcome of developing healthy relationships, we created a moderated forum/discussion board for SMART Squad participants to interact on. We added other features such as progress bars, text narration, and video subtitles to increase usability, also based on user suggestions, as well as the Technology WG’s expertise in web development best practices. As previously noted, such navigational instantiation components help maintain the usability of technology-based interventions but are not hypothesized to alter the theoretical mechanisms of the intervention (Mohr et al. 2015; Li et al. 2019).

Finally, practical and technological parameters influenced our redesign. Whereas KIU! is a standalone intervention, SMART Squad is delivered only to participants who have completed SMART Sex Ed. To differentiate it from and build upon the information-based SMART Sex Ed, we made a concerted effort to increase interactivity and reduce text in SMART Squad. The platform also needed to reflect our adapting SMART Squad for Spanish-speaking participants and thus included features to allow us to switch out content and add videos subtitles. Similarly, knowing that HIV prevention science among AMSM is still evolving, we purposefully presented information likely to change (e.g., PrEP) in simpler formats (e.g., text, images; cf., video) to facilitate future editing. Compared to KIU!, the SMART platform required far greater accessibility across operating systems, devices, browsers, and screen sizes/resolutions; increased automation; and better technical support. Furthermore, there were additional data privacy and security considerations because SMART participants are minors; for example, we developed a mechanism that logs participants out of the intervention after 15 minutes, chosen based on consultation with Technology WG data safety experts, the YACs, and youth ethics researchers.

Step 4: Producing the program

In Step 4, intervention developers produce the new/adapted materials. For each activity and element, we created design documents that contained target change objectives, written descriptions of the activity’s form and functionality, draft messages, and low-fidelity visual mockups. The design documents were refined through continual feedback from the YACs and discussions among the WGs. After finalizing the soap opera script, we worked with our video production partner to film and produce the videos. Simultaneously, the Technology WG developed non-video activities; the overall intervention platform; content management, participant tracking, and data systems; and integration with the scripted videos. Study staff and the Content Advisory Team beta-tested the program for technical and content errors, and after intervention components were finalized in English, the design documents and final products were given to the Linguistic and Cultural Adaptation WG to adapt for Spanish-speaking AMSM.

Table 3 describes the final SMART Squad intervention. The soap opera centers on four racially and geographically diverse AMSM from around the US and Puerto Rico who log onto the SMART Squad platform and meet “face-to-face” in a virtual world representing the program. As the characters navigate different challenges related to dating and sex offline, they access the virtual world to get advice and support from each other and additional pedagogical agents, such as an HIV-positive character and the “Sexpert.” SMART Squad users have opportunities to interact with the content through the characters, who periodically address the camera and ask for the participants’ input, which then leads into an active learning component. The story figuratively depicts the experiences that users have in the intervention.

Table 3.

Description of SMART Squad and comparison to KIU! source content

Episode	Soap opera main concepts	Active learning components	KIU! source content by format and modulea
1	Introduction to the SMART Squad virtual world and soap opera characters Health, emotional, life, physical (HELP) needs Tips for finding a boyfriend	Interactive lessons Balancing HELP needs Simulated hookup app* Summary quiz* Reflection How did a past decision affect your HELP needs?	Character introduction – soap opera (3) Sexual, emotional, health needs – text (6), soap opera (B2) Tips for finding a boyfriend – YMSM interviews (B1) Looking for a relationship – soap opera (B2) Meeting/balancing your needs – activity (6) Simulated club game – animated activity (4) Identifying hookup risks – animated activity (2)
2	Social and emotional consequences of HIV (stigma) Having to have difficult conversations in a relationship	Interactive lessons Differentiating aggressive, passive aggressive, and assertive communication* Summary quiz* Reflection When would be the right time to disclose an HIV-positive status to a potential dating partner?* How would living with HIV affect your HELP needs?*	Social and emotional consequences of HIV – YMSM interview and soap opera (6) Making assumptions about responsibility to disclose HIV-positive status – soap opera (6) Good vs. bad communication – scripted video (6)
3	Resisting peer pressure to use drugs* Cheating Choosing when to have sex* Talking to a healthcare provider about sexual health HIV and STI testing (including oral and rectal) Condom use norms (including discontinuation) Behavioral and biomedical prevention strategies	Interactive lessons Resisting (drug) pressure statements* Choose-your-own-adventure incremental steps to cheating Fact sheet about HIV risk factors* Fact sheet about biomedical prevention Summary quiz* Reflection Why might a person accidentally step outside their relationship? Decision support HIV and STI testing locator tool*	Reducing drugs and alcohol before sex – animated activities (2, 4), soap opera (6, B1) Cheating – animated activity (6), reflection (B2) Making assumptions about monogamy – soap opera (3,  4) Visiting a testing clinic – soap opera (B1) Condom use norms – text (B1) Discontinuing condoms in a relationship – soap opera (B2) Behavioral and biomedical prevention strategies – text, scripted video (B1) HIV facts – text (3, 6) Fact sheet about biomedical prevention – text (B1) CDC testing locator – link (B1)
8-hour break			Two 24-hour breaks
4	Societal pressures on attitudes, behaviors, identity, and gender expression* Condom demonstration SCORE before you score (safety tips for meeting up with someone online)* Reinitiating condoms in a relationship Alcohol use contributing to a high-risk hookup	Interactive lessons Evaluating your environment (for risk factors) Summary quiz* Reflection How do you celebrate your sexual orientation and/or gender expression?*	Community, family, relationships on health – YMSM interviews (1) Condom demonstration – activity (4), demo video (B1) Increasing hookup safety – text (2) Reinitiating condoms in a relationship – text (B2) Reducing drugs and alcohol before sex – animated activities (2, 4), soap opera (6, B1) Identifying hookup risks – animated activity (2)
5	What to do after a condom break (HIV testing, rectal STI testing, PEP vs. PrEP) Negotiating condoms before sex	Interactive lessons Identifying risk factors that led to a bad hookup Resisting pressure (to have condomless sex) statements* Summary quiz* Decision support PrEP/PEP clinic locator tool*	What to do after a condom break – soap opera (B1) Negotiating condoms before sex – animated activities (2,  5), soap opera (4), scripted video (6) Identifying hookup risks – animated activity (2) Good vs. bad communication about condoms – scripted video (6) PrEP/PEP information – text (B1)
6	Intrinsic and extrinsic pressures to have sex Sexting* Agency in sexual decision making Social barriers to obtaining condoms* Summary of condom, PrEP, and testing key takeaways	Interactive lessons Summary quiz* Decision support Where to find condoms* Tailored goal setting and strategizing around barriers	How mood affects sex – animated activity (2) Obtaining condoms – text (B1) Negotiating sex when there is differential power – animated activity, text (5) Goal setting and strategizing around barriers – activity (7)
1 month after main intervention			3 months after main intervention
7	Review of HIV/STI testing, hookup apps, online safety, planning ahead for sex, agency in sexual decision making Couples testing and PrEP in a relationships	Interactive lessons Conditions for discontinuing condoms in a relationship checklist Decision support Goal setting progress and strategizing around barriers	Review of HIV/STI testing and condoms – soap opera (B1) Behavioral and biomedical prevention strategies – text, scripted video (B1) Couples testing – soap opera (B2) Discontinuing condoms in a relationship – soap opera (B2) Goal setting progress and strategizing around barriers – activity (B1)
3 months after main intervention			6 months after main intervention
8	Testing, PrEP, and condom discontinuation in a long-term exclusive relationship Epilogue of soap opera character storylines	Decision support Goal setting progress and strategizing around barriers	Testing, PrEP, and condom discontinuation in a long-term exclusive relationship – soap opera (B2) Goal setting progress and strategizing around barriers – activity (B1)
Through-out	N/A	National-level resources page* Learning map for easy navigation* Quick access to key tools (e.g., various locators)* Ability to favorite activities and videos* Forum with customized avatars and “Ask the Sexpert” board* Access to previously viewed SMART Sex Ed content	Links to previous modules – links (B1, B2)

Note:

^*indicates a component added or substantially adapted for developmental reasons and/or to improve relevance, usability, engagement, and/or active learning.

^aKIU! module topics: 1 = healthy and whole person; 2 = hooking up online; 3/interspersed = assumptions; 4 = bars/clubs; 5 = dating (an older partner); 6 = serious relationships; 7 = goal setting; Booster 1 (B1) = repeat testing; Booster 2 (B2) = long-term relationships

The active learning components broadly comprise three types. The most varied in terms of form and functionality, interactive lessons convey information and rehearse skills through game-like activities. To learn verbal strategies around negotiating condom use, for example, SMART Squad participants assist one of the soap opera characters in responding to his partner, who is pressuring the character to have sex without a condom; participants choose responses to pressure statements and receive feedback on their choices. The second type, reflections, asks participants to apply intervention concepts, such as identifying health, emotional, life, and physical needs, to their own lives and then type their responses into the platform. The reflections are presented back to the participants prior to the goal setting activity in order to prime them to select goals that are most relevant to them. The third type of activity is decision support. These tools aid participants in finding nearby resources (e.g., PrEP providers, places to obtain condoms) or making decisions about their lives (e.g., selecting sexual health goals and barriers).

Compared to KIU!, SMART Squad’s six main episodes are delivered in two sessions with an 8-hour break in between, whereas KIU!’s seven modules were delivered in three sessions with 24-hours breaks. We made this change given feedback that the breaks posed a barrier to completion. The two booster episodes occur respectively at 1 and 3 months instead of 3 and 6 months after the main content to fit within the larger SMART suite. Lastly, the national delivery model for the SMART trial prevented local tailoring of some intervention content as was done in KIU! (i.e., recorded interviews with YMSM in each KIU! trial city). Despite these adaptations, the underlying theoretical mechanisms of behavior change remain nearly identical. Appendix C (available online) presents a comparison of SMART Squad to KIU! based on IM fit categories.

Steps 5 & 6: Developing implementation and evaluation plans

SMART Squad is being evaluated along with SMART Sex Ed and SMART Sessions in an ongoing pragmatic efficacy trial to assess their effects on sexual risk behaviors among racially diverse AMSM. The interventions are hosted and administered by the research institutions, which directly recruit and enroll AMSM into the program. SMART uses a sequential multiple assignment randomized trial design (Murphy 2005) to test the stepped-care structure of the suite.

Discussion

In the context of a persistent public health epidemic (i.e., HIV) with no existing evidence-based interventions for a key target population (i.e., AMSM), adapting a program shown to be effective among a similar group (i.e., YMSM) can increase the efficiency of development and maximize the potential for the adapted program to achieve the desired outcomes. Our aim in this paper was to illustrate the utility and process of IM for developmentally adapting a complex eHealth intervention in a rigorous way. SMART Squad underwent substantial redesign because of differences in population and delivery context, as well as broader changes in technology and HIV prevention. However, we retained the hypothesized essential elements of KIU! and systematically delineated how the adapted intervention is supposed to change sexual health behaviors among AMSM. The process was not without challenges, which we discuss here along with recommendations for future researchers.

First, before beginning the adaptation process, we debated whether to adapt the three SMART suite interventions as separate but cumulative units or as a single unit with internal risk-based tailoring. Because each original intervention used different behavioral change methods, the types and amount of adaptation needed for each intervention was different, and there would be large temporal and experiential differences among participants due to the stepped-care structure, we decided against combining the adaptations. Future program planners developing stepped-care packages could take the alternative approach to ensure greater cohesion among units.

Second, IM traditionally instructs those adapting programs to make minimal changes to the evidence-based intervention, correcting only for mismatches between the program and the new target community while keeping all else the same (Highfield et al. 2015). This attempt to “freeze” interventions is an unrealistic standard for eHealth interventions when technology and user expectations evolve quickly. Updating the KIU! platform for a present-day, younger population involved not only changes in content presentation but extensive redesign of functionality (e.g., accessibility, automation, adaptability) and form (e.g., sleeker design, inconspicuousness) to avoid obsolescence (Mohr et al. 2013). Thus, we decided to incorporate IM creation tasks in our adaptation process. KIU! was not originally designed with IM, but we reverse-engineered the IM matrices of behavior change in order to retain KIU!’s mechanisms and methods in SMART Squad. In the absence of empirical data from a dismantling study, this approach allowed us to theoretically identify the essential elements of KIU!. We then applied Mohr et al. (2015)’s principles to selecting proposed adaptations. Future program planners could consider this method when needing to extensively adapt technology-based programs.

Third, many hi-tech features we designed conceptually in Step 3 (e.g., animated role-playing games) had to be scaled back during production in Step 4 (e.g., choose-your-own-adventure comic) due to pragmatic limitations, such as web browser parameters, the need to function across different devices, potential participant burden (e.g., data usage, loading time), and budgetary constraints. Developers must sometimes temper expectations around what is feasible and consider whether certain instantiation components can be simplified without affecting underlying mechanisms (Mohr et al. 2015). Conversely, periodic upgrades may be necessary to keep pace with technological advancements and stay acceptable and engaging to the target population (Chambers et al. 2013); however, adapters should ensure that new features do not alter instantiation components that may be critical to behavior change. For example, we considered the act of typing in one’s responses an important aspect of the reflection activities in KIU!. Despite optimizing SMART Squad for video content on smartphones, we nevertheless felt it important to incorporate pop-up prompts into which participants write their reflections.

Fourth, SMART Squad features multiple types of media that are highly interconnected but had widely varying development times. The soap opera videos took approximately 12 months from scripting to final edits; individual interactive components ranged from weeks to months. It should be no surprise that creating high-quality digital media and technology takes time. The challenge for eHealth intervention developers is to anticipate the differences during Step 3 because certain media (e.g., videos) become difficult and costly to change once they go into production. This also underscores the importance of selecting more easily updateable formats (e.g., text) for information likely to change over time (Li et al. 2019). The activities map in Step 2 served as an organization tool and checklist for the change objectives included in SMART Squad. We isolated all change objectives operationalized as videos to focus on creating that content first. We then targeted the activity with the next longest development time, identified by the Technology WG. Strategically prioritizing the development sequence and having clear processes for audit and review against the change objectives and for functionality testing helped ensure that we covered all necessary content and stayed within our development timeline and that all the disparately created components came together before beta-testing and launch.

Fifth, our selection of technological features has implications for implementation. Rather than use a third-party platform to house SMART Squad, as was done for KIU!, we built a custom platform and content management system to allow for more flexibility in designing active learning components and to set up for future Spanish-language content and scientific updates, such as the approval of PrEP for adolescents (BusinessWire 2018). However, the bespoke nature of the software limits dissemination without substantial technological support, and technological complexity is often proportional to development and upkeep costs (Li et al. 2019). Alternative implementation approaches to the direct-to-consumer strategy used in the SMART trial may be viable, as was discussed with our Community Collaboration Board (Ventuneac et al. 2019), but will require backend updates. Furthermore, as media become outdated, new content will need to be created. A major benefit of IM, though, is that future iterations of SMART Squad can use the intervention blueprint here (i.e., change matrices, activities map) to guide adaptation decisions.

Future program planners should consider the following when interpreting our work: Designing new interventions and adapting interventions are traditionally separate, parallel processes in IM, but the nature of eHealth led us to use an amalgam of both. Although this may be a deviation from protocol, had we used only IM adaptation tasks, SMART Squad would have more closely replicated KIU! at the expense of contemporary and developmental relevancy. IM is also a very detailed and intensive process that requires time and expertise not all teams may have. One researcher (author DL) had graduate-level training in and prior research experience using IM and could therefore quickly and efficiently execute the construction of change matrices and their translation into practical applications in Steps 2 and 3, respectively. Developers less familiar with IM would likely need to spend some time learning those parts of the protocol before being able to apply them. Furthermore, we benefited from having on-site developers who could provide rapid feedback throughout the production phase. Future planners should build-in additional time for this iterative process during Step 4, as there will inevitably be negotiations between conceptual and practical design.

Despite these limitations, SMART Squad is a novel HIV risk reduction program that addresses a critical need among AMSM. We believe that using the IM protocol to identify and retain the underlying behavior change mechanisms and methods from the original CDC best-evidence intervention helped ensure the best chance for effectiveness in the new population. Given continually increasing investment in eHealth across various health domains (Bennett and Glasgow 2009), more rigorous and comprehensive documentation of program adaptation is needed. IM is a useful tool for researchers and practitioners to do so, and we hope this report will serve as an exemplar to facilitate others’ use of this method in the future.