Technology-Based Interventions to Reduce Sexually Transmitted Infections and Unintended Pregnancy Among Youth

Abstract

Purpose

Technology-based interventions to promote sexual health have proliferated in recent years, yet their efficacy among youth has not been meta-analyzed. This study synthesizes the literature on technology-based sexual health interventions among youth.

Methods

Studies were included if they: 1) sampled youth ages 13-24; 2) utilized technology-based platforms; 3) measured condom use or abstinence as outcomes; 4) evaluated program effects with experimental or quasi-experimental designs; and 5) were published in English.

Results

16 studies with 11,525 youth were synthesized. There was a significant weighted mean effect of technology-based interventions on condom use (d=.23, 95% CI [0.12, 0.34], p<.001) and abstinence (d=.21, 95% CI [0.02, 0.40], p=.027). Effects did not differ by age, gender, country, intervention dose, interactivity, or program tailoring. However, effects were stronger when assessed with short-term (1-5 months) compared to longer-term (greater than 6 months) follow-ups. Compared to control programs, technology-based interventions were also more effective in increasing sexual health knowledge (d=.40, p<.001) and safer sex norms (d=.15, p=.022) and attitudes (d=.12, p=.016).

Conclusions

After 15 years of research on youth-focused technology-based interventions, this meta-analysis demonstrates their promise to improve safer sex behavior and cognitions. Future work should adapt interventions to extend their protective effects over time.

Introduction

The risk of unplanned pregnancy and sexually transmitted infections (STIs), including HIV, is high among youth. Youth ages 13-24 comprise nearly half of the 20 million new STI cases and more than 20% of new HIV diagnoses each year in the United States [1, 2]. Worldwide, adolescents and young adults account for 45% of all new HIV infections [3]. Rates of unintended pregnancy are also elevated among youth, with girls ages 15-19 having higher rates of unintended pregnancy than girls and women in any other age group [4, 5]. Further, complications from childbirth are the second leading cause of death among 15-19 year old girls worldwide [5].

Efforts to provide youth with sexuality education to reduce HIV/STIs and unintended pregnancy have been underway for decades.[6, 7]. Only recently, however, has a movement emerged toward utilizing technology-based platforms for sexual health program delivery [8–10]. Technology-based interventions have been alternatively referred to as eHealth, mHealth, digital media, or new media interventions, and typically utilize computers, smart phones, text messaging, and/or other web-based platforms. As digital tools have become increasingly accessible and sophisticated in their functionalities, many researchers have heralded the promise of technology-based interventions to improve sexual health [8–10]. Compared to traditional interventions, new media approaches may allow for broad reach at relatively low costs, improved fidelity during intervention delivery, greater privacy and comfort for teens learning about sensitive topics, and increased capacity for individually tailoring prevention messages [11, 12]. Additionally, such tools may provide fruitful means for engaging today’s digitally native youth, among whom the ubiquitous use of technology plays a central role in key developmental tasks [13, 14].

The promise of new media tools has prompted a recent proliferation of technology-based interventions targeting sexual health among youth. While recent systematic reviews have highlighted potential impacts and limitations of these interventions [15–17], little is known regarding their combined efficacy. Determining whether and through what mechanisms technology-based interventions promote youth sexual health is critical, as such knowledge can inform future interventions and justify the allocation of resources to their development.

To date, there have been only three published meta-analyses across adolescent and adult populations that provide insights into the efficacy of technology-based interventions for sexual health. The first focused on interventions to reduce HIV infection and included 12 randomized controlled trials published between 2002-2008 with participants of any age [18]. This study found a small but significant overall increase in condom use among participants who completed a technology-based intervention compared to those who were randomized to a control group, with an effect size that was comparable to similar in-person interventions (d across studies = .26). These effects were stronger when interventions included individualized tailoring (i.e., materials matched to the needs of specific participants) and had a greater number of sessions. The second meta-analysis focused exclusively on programs employing computer-based administration, finding that such interventions were effective in improving sexual health knowledge, sexual self-efficacy, safer sex intentions, and safer sexual behavior among adolescents and adults [19]. The third meta-analysis identified significant effects of new media interventions for increasing condom use and STI testing within non-clinical populations [20]. Importantly, this meta-analysis found that interventions produced the largest effect sizes when they targeted female adolescents [20].

Despite these preliminary findings, at least three critical gaps inhibit our ability to draw conclusions about the overall efficacy of technology-based sexual health programs for youth. First, there are currently no meta-analyses focused exclusively on youth, even though youth are at heightened risk for unplanned pregnancies and STIs [3, 4, 21, 22] and are also some of the most frequent users of new media technologies [13, 23]. Second, no meta-analyses have included abstinence as a behavioral outcome. Abstinence can be a developmentally-appropriate objective of comprehensive sexuality education—particularly in programs targeting early adolescent populations. Third, prior meta-analyses and systematic reviews of technology-based sexual health programs have only included studies published through 2014 [15, 16, 18–20, 24]. Given the rapidly changing landscape of technology and the many new interventions introduced each year, an updated review of the literature is warranted.

Thus, the goal of this meta-analysis is to synthesize the growing literature on technology-based sexual health interventions among youth ages 13-24 and to determine their overall efficacy on two key behavioral outcomes: condom use and abstinence. Additionally, we will examine a number of secondary outcomes identified as important components of safer sexual decision-making within health behavior theories [25] that have guided many intervention efforts in this area. These outcomes include safer sex attitudes, norms, self-efficacy, behavioral intentions, and sexual health knowledge. Finally, we will examine whether characteristics related to the intervention recipients (age, gender, country) or the intervention design (delivery method, use of tailoring, program interactivity, follow-up duration) moderate the effectiveness of technology-based interventions on sexual health outcomes.

Methods

Search Strategy

We conducted a comprehensive search of Medline, PsycINFO, and Communication Source databases to extract relevant studies published through May 2017. We used the following combination of key words, with asterisks used as “wild cards” to find any variations: (adolescen* or teen* or youth or middle school or high school) AND (sexual health or safe* sex or sex* education or sexually transmitted disease or sexually transmitted infection or STD or STI or HIV or AIDS or pregnancy or reproductive health or condom* or contracept* or protected sex or unprotected sex) AND (intervention or program or education or trial) AND (technology or internet or web-base* or computer-base* or online or social media or social network* or SNS or eHealth or mHealth or electronic health or mobile health or texting or text messag* or digital media or new media or gaming or SMS or mobile phone or cell phone or phone app* or Facebook or Twitter or Instagram or instant message or web 2.0 or media 2.0). Additional studies of potential relevance were located by examining prior reviews and meta-analyses [15–20, 24]. We also examined the reference lists of all included articles to search for additional studies. This search produced an initial 1,932 scientific articles.

Selection Criteria

Studies were included if they met the following criteria: 1) focused on youth between the ages of 13-24 (i.e., mean sample age 13-24 and no participant older than 29); 2) utilized technology as the primary mode of delivering an HIV/STI or pregnancy prevention intervention (studies that utilized technology but also included extensive in-person components were excluded [26, 27]); 3) included a behavioral outcome measure of either condom use/unprotected sex (referred to as “condom use” in this paper; the effect sizes for unprotected sex were recoded so that the direction of effect always indicated greater protection) or abstinence/delayed intercourse (referred to as “abstinence”); 4) evaluated program effects with an experimental or quasi-experimental design; 5) were published in English; and 6) provided sufficient statistics to calculate effect sizes. These selection criteria resulted in a final sample of 16 articles (see Supplemental Figure 1). In total, we calculated 16 independent effect sizes for condom use, 8 for abstinence, 6 for safer sex attitudes, 5 for norms, 5 for self-efficacy, 5 for behavioral intentions, and 9 for sexual health knowledge.

Data Extraction

Two of the authors (K.K. and J.L.S.) independently coded the primary studies. The following data were abstracted: 1) demographic and sample characteristics; 2) intervention characteristics (e.g., type of technology used, number of sessions, program interactivity); 3) methodological characteristics (e.g., length of follow-up, type of comparison group); and 4) type of outcome measurement(s). The mean percentage agreement across all coding categories was 88%. Discrepancies between coders were resolved through discussion with the first author (L.W.). Additionally, three of the authors (L.W., K.K., and J.L.S.) rated the 16 studies for quality and risk of bias using the Cochrane Collaboration’s tool for assessing risk of bias (see Supplemental Table 1).

Calculation of Effect Sizes

The standardized mean difference, d, was used as the indicator of effect size (i.e., the treatment group and control group means divided by the pooled standard deviation). According to Cohen [28], effect size d can be interpreted as small (.20), medium (.50), or large (.80). When ds were reported in an article, they were directly extracted. If ds were not reported, other statistics that could be converted to ds (e.g., summary statistics, odds ratios) were calculated using Comprehensive Meta-Analysis V2.0 [29]. When no statistics in the study could be converted to a d, study authors were contacted and appropriate data were requested. When more than one follow-up time point was reported, effect sizes were calculated based on data from the longest follow-up for which data were available. To ensure the consistency and interpretability of effect sizes, higher values always indicate the technology-based intervention group performed better than the control group.

We used random effects meta-analytic procedures for the primary analyses across all independent effect sizes; this procedure allowed for the possibility of differing variances across studies [30]. The Q statistic and I² were used to examine whether significant heterogeneity existed among effect sizes. Effect sizes for hypothesized moderators were calculated along with their 95% confidence intervals, and those effect sizes were compared using the Q_b statistic. For these analyses, mixed effects models were utilized to allow for the possibility of differing variances across subgroups. These models employ random effects assumptions, while stratifying the effect sizes by fixed factors such as age and study dose [30]. Analyses were conducted using Comprehensive Meta-Analysis V2.0 [29].

Results

Study Characteristics

Table 1 provides a summary of the studies included in this meta-analysis, including sample characteristics and moderator variables. A total of 11,525 participants (mean age=18.42) were enrolled across 16 technology-based interventions for youth. The majority of studies were conducted in the United States (k=11). Many studies used combined samples of boys and girls (k=9); however, a few studies analyzed data from boys (k=3) and girls (k=4) independently.

Table 1.

Study Characteristics

Authors	Sample and Study Description	Age M (range)	Dose	Interactive	Tailoring	Follow-Up (months)	Outcome(s)
Antonishak et al [31]	2,284 young women from U.S.; race/ethnicity: 56% White, 17% Black, 19% Hispanic; recruited online; intervention via website and email	22.3 (18–29)	High	Yes	No	12	Condom Use
Bannink et al [32]	1,702 adolescents from the Netherlands; 46% female; recruited in secondary schools; intervention via website and email	15.9 (15-16)	Low	No	Yes	4	Condom Use
Bull et al [67]	1,578 young adults from U.S.; 65% female; race/ethnicity: 30% White, 35% Black, 14% Hispanic; recruited through snowball sampling; intervention via social media	20.0 (16-25)	High	Yes	No	6	Condom Use; Self-efficacy; Intentions
Downs et al [49]	300 sexually active teenage girls from U.S.; race/ethnicity: 15% White, 75% Black, 10% mixed race; sexual orientation: 100% heterosexual; recruited in health center; intervention via computer	nr (14-18)	Mod	Yes	No	6	Condom Use; Abstinence; Knowledge
Howard et al [39]	254 low-income teenage girls from U.S.; race/ethnicity: 99% Black; recruited in family planning clinics; intervention via website and computer	nr (“teen”)	Low	No	No	3-6	Condom Use; Knowledge
Kiene and Barta [50]	157 college students from U.S.; 71% female; race/ethnicity: 81% White, 5% Black, 6% Hispanic; sexual orientation: 98% heterosexual; recruited from psychology student participant pool; intervention via computer	18.9 (nr)	Low	Yes	Yes	1	Condom Use; Attitudes; Norms; Intentions; Knowledge
Lightfoot et al [64]	133 adolescents from U.S.; 45% female; race/ethnicity: 49% Black, 47% Hispanic; recruited from alternative high schools; intervention via computer	16 (14-18)	Mod	Yes	Yes	3	Condom Use; Abstinence
Lim et al [34]	994 young adults from Australia; 58% female; recruited from music festival; intervention via texting and email	19 (16-29)	High	No	No	12	Condom Use; Knowledge
Lou et al [33]	1337 adolescents from China; 45% female; recruited from high schools and universities; intervention via website and email	nr (15-19)	High	Yes	Yes	2.5	Abstinence; Attitudes; Norms; Knowledge
Mevissen et al [68]	171 young adults from the Netherlands; 61% female; sexual orientation: 100% heterosexual; intervention via website	20.8 (18-25)	Low	Yes	Yes	3	Condom Use; Attitudes; Norms
Mustanski et al [48]	102 sexually active young adult men from U.S.; race/ethnicity: 25% White, 13% Black, 46% Hispanic; sexual orientation: 100% men who have sex with men; recruited from HIV testing center; intervention via website and video game	21.3 (18-24)	Mod	Yes	Yes	3	Condom Use; Attitudes; Norms; Self-Efficacy; Intentions; Knowledge
Norton et al [46]	198 sexually-active college students from U.S.; 70% women; race/ethnicity: 85% white; sexual orientation: 100% heterosexual; recruited from psychology student participant pool; intervention via computer DVD	18.6 (nr)	Low	No	No	2	Condom Use
Peskin et al [41]	1571 adolescents from U.S.; 59% female; race/ethnicity: 17% Black, 74% Hispanic; recruited from urban middle schools in Texas; intervention via computer	14.3 (nr)	High	Yes	Yes	12	Condom Use; Abstinence; Attitudes; Self-Efficacy; Intentions; Knowledge
Roberto et al [47]	326 10th grade adolescents from U.S.; 56% female; race/ethnicity: 97% White; recruited from rural high schools; intervention via computer	15.6 (nr)	High	Yes	Yes	2.5	Condom Use; Abstinence; Attitudes; Norms; Self-Efficacy; Knowledge
Suffoletto et al [45]	52 sexually-active young adult women from U.S.; race/ethnicity: 65% Black, 6% Hispanic; sexual orientation: 100% heterosexual; recruited from emergency rooms; intervention via text messages	21.4 (18-25)	High	Yes	Yes	3	Condom Use; Abstinence
Ybarra et al [69]	366 sexually-active adolescents from Uganda; 16% female; recruited from high schools; intervention via website	16.1(13-19)	Mod	Yes	Yes	6	Condom Use; Abstinence

Note. Dose = High (7 or more sessions), Moderate (3-6 sessions), Low (1-2 sessions). nr = not reported. Sexual orientation is included in the sample description whenever it was reported in the text; many studies did not report the sexual orientation of their participants.

The delivery of program content varied widely across different digital media technologies. Specifically, 5 programs were delivered exclusively via computer programs; 2 were delivered exclusively through internet websites; 1 was delivered exclusively through texting; 1 was delivered exclusively through social media; and nearly half of programs (7 of 16) were delivered with more than one method, such as an Internet website with email follow-up [31–33] or a combination of text messaging and email delivery ([34]). It was common for interventions to be interactive (i.e., accept input from the user as the program progressed; k=12) and to utilize tailoring (i.e., include program components that were matched specifically to the user based on user characteristics; k=10). The dose of intervention varied widely across studies, with k=5 programs including 1-2 sessions and k=8 programs including 7 or more sessions.

Primary Outcomes

Condom Use

While individual effect sizes ranged from d=−.07 to .67, the weighted mean effect size for condom use across these studies was d=0.23 (95% CI [0.12, 0.34]; p<0.001), indicating that technology-based interventions have a small but significant protective effect on condom use behavior among youth (see Figure 1). To examine whether publication bias may have inflated the effect size of the interventions on condom use, a fail-safe N was calculated. This fail-safe N was 92, suggesting 92 non-significant studies would need to exist in order to reduce the effect size to a trivial level (p>.05).

Figure 1.

Forest Plots for Primary Outcomes

Statistical testing indicated marginal heterogeneity among studies with regard to the condom use outcome (Q=24.27, p=.061, I²=38.19). Thus, we examined the potential impact of several moderating variables. As shown in Table 2, there was only one marginally significant difference based on the duration of follow-up assessment, with shorter follow-ups (less than 6 months) producing stronger effects (d=0.32, p<.001) than programs with follow-ups of longer duration (6 months or more; d=0.14, p=.093). No significant differences were found by participant age, gender, study dose, program interactivity, program tailoring, or whether the study was conducted in the U.S.

Table 2.

Weighted Mean Effect Sizes for Condom Use by Categorical Moderator Variables

					Between Groups
Variable	k	d	95% CI	p	QB	p
Gender					2.95	ns
Female only	4	.36	[0.19, 0.53]	<.001
Male only	3	.20	[−0.14, 0.53]	.246
Mixed gender	9	.17	[0.03, 0.31]	.016
Age					0.33	ns
Sample mean age ≤ 18	9	.20	[0.08, 0.31]	.001
Sample mean age > 18	7	.27	[0.03, 0.51]	.025
Intervention Dose					0.18	ns
Low (1-2 sessions)	5	.34	[0.15, 0.52]	<.001
Moderate (3-6 sessions)	4	.22	[0.03, 0.41]	.025
High (7+ sessions)	7	.17	[−0.01, 0.35]	.070
Interactivity					0.00	ns
Interactive	11	.23	[0.09, 0.38]	.002
Static	5	.23	[0.06, 0.41]	.002
Tailoring					0.07	ns
Used tailoring	9	.24	[0.09, 0.39]	.002
Did not use tailoring	7	.21	[0.04, 0.39]	.018
Country					0.02	ns
U.S.	11	.22	[0.09, 0.35]	.001
Non-U.S.	5	.24	[−0.02, 0.50]	.069
Follow-Up					2.73	.099
Less than 6 months	9	.32	[0.18, 0.47]	<.001
6 months or more	7	.14	[−0.02, 0.31]	.093

Note. N = sample size; k = number of studies; d = weighted mean effect size; CI = confidence interval; ns = not significant. Mixed effects models are presented for moderator analyses.

Abstinence

Individual study effect sizes for abstinence/delayed sex ranged from d=.01 to .65, with an overall weighted mean effect size for abstinence across studies of d=.21 (95% CI [0.02-0.40]; p=.027). This indicates that technology-based interventions have significant protective effects on abstinence over time (see Figure 1). To examine whether publication bias may have inflated the effect size of the interventions on abstinence, a fail-safe N was calculated. This fail-safe N was 18, suggesting 18 non-significant studies would need to exist in order to reduce the effect size to a trivial level (p>.05).

Statistical testing indicated significant heterogeneity among studies with regard to the abstinence outcome (Q=16.93, p=.018, I²=58.65); thus, we examined the potential impact of several moderating variables. As shown in Table 3, the only significant moderator was the duration of follow-up. Similar to the effects on condom use, intervention effects on abstinence were significantly stronger for short-term follow-ups (i.e., less than 6 months; d=.45, p<.001) than those with long-term follow-ups (d=.07, p=.321). No significant differences were found by gender, age, intervention dose, tailoring, or country of study. Of note, all 8 studies that included abstinence as a primary outcome were interactive interventions; thus, we could not examine the effects of interactive versus static programs for abstinence.

Table 3.

Weighted Mean Effect Sizes for Abstinence by Categorical Moderator Variables

					Between Groups
Variable	k	d	95% CI	p	QB	p
Gender					2.86	ns
Female only	2	.11	[−0.11, 0.32]	.318
Male only	2	.01	[−0.27, 0.29]	.925
Mixed gender	4	.36	[0.05, 0.67]	.021
Age					0.26	ns
Sample mean age ≤ 18	7	.23	[0.02, 0.43]	.030
Sample mean age > 18	1	.05	[−0.61, 0.70]	.888
Intervention Dose					0.06	ns
Low (1-2 sessions)	0	–	–	–
Moderate (3-6 sessions)	3	.18	[−0.01, 0.38]	.061
High (7+ sessions)	5	.23	[−0.08, 0.54]	.150
Interactivity					–	–
Interactive	8	.21	[0.02, 0.40]	.027
Static	0	–	–	–
Tailoring					0.01	ns
Used tailoring	7	.21	[−0.02, 0.44]	.069
Did not use tailoring	1	.23	[−0.05, 0.50]	.104
Country					0.10	ns
U.S.	6	.22	[−0.01, 0.46]	.064
Non-U.S.	2	.16	[−0.16, 0.48]	.333
Follow-up					7.00	.008
Less than 6 months	4	.45	[0.21, 0.69]	<.001
More than 6 months	4	.07	[−0.07, 0.22]	.321

Note. N = sample size; k = number of studies; r = weighted mean effect size; CI = confidence interval; ns = not significant. Mixed effects models are presented for moderator analyses.

Secondary Outcomes

We examined 5 secondary outcomes, including safer sex attitudes, social norms for safer sexual activity, self-efficacy, behavioral intentions to practice safer sex, and sexual health knowledge. As shown in Table 4, compared to control programs, technology-based interventions were efficacious in increasing sexual health knowledge (d=.40, p<.001), social norms for safer sexual activity (d=.15, p=.022), and safer sex attitudes (d=.12, p=.016) among youth. Compared to controls, there were no significant differences in the effect of technology-based interventions on safer sex intentions or perceived self-efficacy to engage in safer sexual behavior.

Table 4.

Effect Sizes for Safer Sex Related Supplemental Outcomes

Variable	k	d	95% CI	p
Safer Sex Attitudes	6	.12	[−0.02, 0.23]	.016
Safer Sex Norms	5	.15	[0.02, 0.29]	.022
Safer Sex Self-Efficacy	5	.07	[−0.05, 0.19]	.249
Safer Sex Intentions	5	.06	[−0.02, 0.14]	.158
Sexual Health Knowledge	9	.40	[0.25, 0.55]	<.001

Note. Random effects models are presented. k = number of studies; d = weighted mean effect size; CI = confidence interval.

Data Quality

The majority of study designs were rigorous (14 RCTs) and there was limited evidence of attrition bias, with retention rates across studies generally exceeding 80%. However, some uncertainty was noted in whether studies had utilized best practices for concealing the allocation of participants into study conditions. For 8 studies, it was also unclear if selective reporting of outcomes could have been an issue since these studies had not been preregistered. However, overall data quality across studies was deemed to be high, and most studies were found to have a low risk of bias (see Supplemental Table 1).

Discussion

Results of the current meta-analysis, which synthesize nearly 15 years of research on the development and evaluation of youth-focused technology-based interventions, highlight the great promise of these approaches to improve safer sex behaviors among youth. Pooling data from 16 studies with over 11,000 adolescents, this meta-analysis found a significant positive effect of technology-based sexual health interventions for improving two key sexual behaviors among youth: increasing consistent condom use (effect size d=.23) and delaying sexual activity (effect size d=.21). Across studies, technology-based programs also led to increases in sexual health knowledge, safer sex attitudes, and positive norms surrounding safer sexual activity. Importantly, these effect sizes are comparable or even exceed the effects of in-person interventions, with generally small to moderate intervention effects noted for in-person programs (effect size d=.13 for condom use and d=.11 for abstinence in a meta-analysis by Johnson et al [6]).

Technology-based interventions offer many possible benefits over traditional, face-to-face interventions [11, 35, 36]. Most notably, these programs can be administered with high fidelity without extensive facilitator training. This may result in cost-effective programs that are capable of reaching a greater number of youth than are in-person programs. These approaches can also increase youths’ openness to learning by providing a safe, controlled, and familiar environment to receive sexual health knowledge and skills. This may be especially salient for sexual minority youth who, in traditional face-to-face interventions (e.g., school-based), may receive sexual education that is stigmatizing, inaccurate, or irrelevant to their specific needs [37]. Further, technology-based programs offer ample opportunities for customizability, interactivity, and individual tailoring [11, 35, 36]. Technology-based interventions also have the potential to reach adolescents in resource-limited areas as worldwide access to mobile phones and internet technologies is rapidly increasing among young people, even in low- and middle-income countries and rural areas [23, 38].

Results suggest that the positive effects of technology-based interventions on sexual health may be robust across a number of different factors. Several potential moderating factors did not impact the efficacy of technology-based programs for youth, including the age and gender of participants and the intervention interactivity, tailoring, and dose. For example, while intervention dose varied from a single 15-minute session [39] to a full 12 months of content delivery [31, 34], intervention dose did not influence effect sizes for condom use or abstinence across studies. This finding differs from a previous systematic review of in-person sexual health programs, which found that programs with higher duration and intensity produced the greatest treatment effects [40]. However, our findings are consistent with a more recent analysis of new media interventions [20], which found that the program duration did not influence the size of effects on condom use. Indeed, the effects of technology-based program dose on youth sexual health outcomes may be complex. On the one hand, the lack of association between intervention dose and outcomes could simply reflect the challenges of differentiating between the dose intended and the dose received within some technology-based programs [41]. On the other hand, this finding may reflect the growing potency of technology-based interventions. As technology has become increasingly available, interactive, and customizable in recent years, these interventions have clearly shown effectiveness, even at small doses. Future work could investigate whether such intervention impacts vary by how actively youth engage with technology on an everyday basis.

One crucial insight gleaned from this meta-analysis is that the effects of current technology-based programs tend to decrease over time. Specifically, for both condom use and abstinence, stronger effects were found in the short-term (i.e., 1-5 month follow-up) compared to studies that evaluated intervention effects over 6 months or more. Adolescent sexual behavior is complex, and it is perhaps not surprising that program effects appear to diminish over time. The lack of long-term sustainability is a problem that has also plagued many programs delivered to youth in-person [42]. The knowledge and skills adolescents learn in specific programs may fail to adequately prepare them for long-term changes in their dating and sexual relationships and their evolving sexual interests and desires. Given the initial promise of technology-based programs, an exciting extension of this work would be to evaluate longer-term, adaptive interventions utilizing Sequential Multiple Assignment Randomized Trial (SMART) designs [43]. These designs will allow investigators to examine the impact of added program components (e.g., booster sessions) over time to target the specific youth for whom treatment effects are diminishing. In fact, SMART designs may be particularly amenable to technology-based interventions, given unprecedented opportunities to automate assignment to intervention options (i.e., tailoring) based on participants’ responses at critical decision points [44].

Limitations and Future Directions for Intervention Efforts

A number of important limitations in the studies included in this meta-analysis are worthy of future research attention. First, it is worth noting that there was substantial variation across studies in the way that the outcome measures were defined. For example, for both condom use and abstinence, some investigators focused exclusively on acts of vaginal sex [45–47] or anal sex [48], whereas others included acts of vaginal, anal, and oral sex in their definitions [41] or referred only to “sex” or “intercourse” without a definition [32, 39]. Additionally, the timeline for measuring these behaviors differed. For example, some studies measured condom use at last sex [45, 47], whereas others measured the frequency of condom use over a specified period of time [49, 50]. Likewise, for the outcome measure of abstinence/delayed sex, some studies focused on whether youth had ever initiated sexual activity [41, 47] whereas others focused on abstaining from sexual activity over a specified period of time [49]. Similar variation was observed in the definitions of the secondary outcomes under investigation. It is clear that the sexual health intervention literature lacks a “gold standard” for measuring sexual risk behavior and related attitudes and cognitions. This lack of consistency in measurement has been observed in other meta-analyses examining sexual risk behavior [6, 18, 51–53]. The measurement variation across studies could be obscuring our ability to detect the most precise estimates of intervention success, including which behaviors are most effectively addressed by technology-based interventions.

Second, only one of the included studies measured a biological outcome (i.e., Chlamydia infection [49]) and no programs assessed intervention effects beyond one year. These issues raise important questions about the long-term clinical impacts of these programs; however, they are not unique to technology-based programs. In fact, a recent systematic review of all school-based (i.e., in-person) sexual health programs among youth in the U.S. [54] found only seven interventions that measured HIV/STI incidence or HIV/STI testing as outcomes. Further, among the 98 in-person interventions reviewed by Johnson et al. [6], the average outcome assessment occurred at 13 weeks. Future efforts to evaluate technology-based programs would be enhanced by data on HIV/STI testing and/or diagnosis among program recipients, as well as long-term follow-up to measure the longitudinal impacts of these programs.

Another potentially fruitful avenue for efforts to improve adolescent sexual health outcomes will be to identify the specific, replicable behavioral change techniques that are utilized in technology-based interventions to enhance behavior change, as is increasingly being done for in-person interventions [55]. It also will be critical for researchers to identify common design elements that are specific to technology-based interventions going forward (for examples, see [36, 56]). As technology rapidly evolves, recognizing common design features across programs (e.g., tailoring, online social support and communication, automatic feedback) will be essential for comparing interventions delivered across a range of technological platforms. In this meta-analysis, we tested several possible technological design features that could serve as moderators of intervention effectiveness, such as whether the program was interactive and whether personal tailoring was utilized. However, there were too few studies to examine further design elements that may increase program impact, such as self-monitoring or personalized reminders [36]. These remain interesting avenues for future work as new technology-based programs continue to emerge [57, 58]

As previously discussed, technology-based interventions may be a beneficial avenue for delivering non-stigmatizing, accurate, and relevant sexual health education to sexual minority youth. However, with the exception of one intervention targeting young men who have sex with men [48], no study included in this meta-analysis specifically reported tailoring intervention content to be inclusive of this population. Sexual minority youth experience disproportionate rates of sexual risk [59], and face a number of stressors that may impact their overall sexual health. Researchers should more explicitly consider the unique needs of sexual minority youth in the development and evaluation of sexual health programming, in addition to providing needed services at various levels of societal and individual intervention [60]. Technology-based approaches may be particularly valuable within this population, as sexual minority youth are more likely than heterosexual youth to use the internet to look for sexual health information [61].

Future studies also should consider the potential moderating role of other important demographic factors, including race/ethnicity, gender, and socio-economic status, given that these group comparisons were not possible in the current study. Sexual health disparities among racial and ethnic minorities have been well documented, with rates of reported STIs higher among some groups (e.g., Black and Hispanic individuals), compared to White individuals [62]. Although a “digital divide” remains in terms of access and use of technology in the United States [63], recent statistics suggest that rates of smartphone access are comparable among Hispanic and White adolescents, and that Black youth are more likely to have smartphones compared to other racial/ethnic groups [13]. Even among low income youth (i.e., household income less than $30,000), 91% of adolescents report that they access the Internet via a mobile device [13]. Thus, sexual health interventions delivered via technology may hold particular promise for reaching these at-risk groups.

Finally, there is a need for greater attention to the comparative effects of content delivered in-person versus through technology. Only one study in our meta-analysis compared a technology-based intervention to an in-person control [64]. However, the structure and quality of technology-based interventions may differ from in-person treatments in important ways, including the possibility for anonymous communication (e.g., discussion boards), the provision of unique forms of social support and skills practice (e.g., using avatars or simulated person-to-person communication), and possibilities for different types of self-monitoring [36]. Additionally, with advancements in technology and incorporation of tailored, interactive features (e.g., “ask the expert”), technology-based interventions may offer the opportunity for personalized, rich communication and support from intervention personnel that once was limited to in-person interventions [36, 65]. However, interventions delivered in-person may increase the perceived credibility of information sources and provide a sense of accountability or connection [65]. Thus, although this meta-analysis suggests that technology-based interventions are effective compared to no-treatment or attention-matched controls, their relative efficacy in relation to traditional, in-person sexual health interventions remains a question for future work.

Summary and Implications

After nearly 15 years of research on youth-focused technology-based interventions, this meta-analysis demonstrates the great promise of these approaches to improve the safer sex behavior and cognitions of youth. Pooling data from 16 studies with over 11,000 adolescents and young adults, we found that technology-based interventions were more effective than control programs at improving condom use, abstinence, sexual health knowledge, safer-sex attitudes, and safer-sex norms. However, effects were stronger in studies with shorter-term follow-ups compared to follow-ups of 6 months or more. Future work should focus on intervention adaptations and supplements that may extend their protective effects over time. Future work should also move beyond program development and into the realm of implementation science to ensure these programs are broadly disseminated for maximum impact [66].