Text Messaging, Teen Outreach Program, and Sexual Health Behavior: A Cluster Randomized Trial

Abstract

Objectives. To consider whether Youth All Engaged! (a text message intervention) intensified the effects of the adolescent pregnancy prevention Teen Outreach Program (control) for youths.

Methods. In this trial performed in Denver, Colorado, from 2011 to 2014, we randomized 8 Boys & Girls Clubs each of 4 years into 32 clubs per year combinations to ensure each club would serve as a treatment site for 2 years and a control site for 2 years. Control intervention consisted of the Teen Outreach Program only. We enrolled 852 youths (aged 14–18 years), and 632 were retained at follow-up, with analytic samples ranging from 50 to 624 across outcomes. We examined program costs, and whether the intervention increased condom and contraceptive use, access to care, and pregnancy prevention.

Results. Control program costs were $1184 per participant, and intervention costs were an additional $126 per participant (+10.6%). There were no statistically significant differences in primary outcomes for the full sample. Hispanic participants in the intervention condition had fewer pregnancies at follow-up (1.79%) than did those in the control group (6.72%; P = .02).

Conclusions. Youth All Engaged is feasible, low cost, and could have potential benefits for Hispanic youths.

Childbearing during adolescence is associated with reduced economic opportunities, and children born to adolescents more often face poverty and higher risk for outcomes associated with poverty (e.g., child abuse and neglect). Preventing adolescent births addresses these issues and pays dividends to taxpayers.¹ Despite declining adolescent birth rates between 2004 and 2013, additional intervention is warranted to address adolescent pregnancy, particularly for racial and ethnic minority groups. In 2013, 3807 female adolescents aged 15 to 19 years, gave birth in Colorado, with disparities found among White, non-Hispanic adolescents (14 births per 1000), Black adolescents (26 births per 1000), and Hispanic adolescents (42 births per 1000).² In Denver, 75% of all babies born to adolescents are to Hispanic female adolescents.³

The US Department of Health and Human Services identified 36 evidence-based programs for preventing adolescent births.⁴ One is the Teen Outreach Program (TOP), which is focused on youth development, including sexuality education, with a minimum of 25 hour-long classroom sessions delivered by an adult, plus 20 community service learning hours.⁵ Even evidence-based programs face challenges in effectively serving diverse youths⁶ who may not relate to the content of the intervention and drop out.^7–9 Programs that rely exclusively on face-to-face delivery fail to capitalize on youth communication preferences, including cell phones and social media.¹⁰ Although there are standalone programs that have successfully used text messaging to support healthy behavior,^11,12 many do not capitalize on the evidence that demonstrates increased effects when content is theory driven.¹³ We submit that exclusively virtual programs may not connect to youths as successfully as those that rely on real-world supports¹⁴; these programs may not reach high-risk youths,¹⁵ and their program effects decay quickly¹⁶ or are not consistently measured past 3 or 6 months.^17,18 Youths are the largest consumers of cell phone minutes and text messaging,^19,20 and they integrate technology into their daily lives through chats, games, and social media exchanges with their real-world peers and family. This understanding illustrates opportunities for reaching youths, but we must consider that it also demonstrates youths have competing demands for attention online,²¹ making it arguably critical to attend to designing dynamic, engaging, and interactive content in social media interventions that facilitate attention to and absorption of material to maximize effect. The “Integrated Theory of mHealth”²² was created specifically for health promotion via mobile and social media. It integrates traditional social and behavioral science constructs used in health promotion interventions with best practices in health communication for message design and state-of-the-art evidence on how to elicit responses in the social media environment by offering a useful framework for the design of the Youth All Engaged! (YAE!), a never-before evaluated curriculum we describe here.

We created the hybrid TOP + text message program (YAE!, formerly known as TOP411) and explored opportunities to reach high-risk youths outside the classroom, through the Denver Metro Boys & Girls Clubs, which serve primarily minority youths and those living in poverty.²³

The rationale to address adolescent pregnancy among minority youths assisted by text message technology was explored through 4 research aims:

1. What is the feasibility and cost of delivering YAE! and TOP in Boys & Girls Clubs?

2. What is the impact of YAE! delivered with TOP relative to TOP alone on the average percentage of sex acts protected by condoms or contraception over the past 3 months assessed at program completion?

3. What is the impact of YAE! delivered with TOP relative to TOP alone on access to contraceptive or sexually transmitted infection clinical services over the past 9 months assessed at program completion?

4. What is the impact of YAE! delivered with TOP relative to TOP alone on ever being pregnant or causing a pregnancy assessed at program completion?

Because of adolescent pregnancy disparities for Hispanics in Colorado, we explored research questions 2 to 4 comparing Hispanic with non-Hispanic participants in an ad hoc analysis, with ethnicity as a moderator.

METHODS

The following is a description of the study design, recruitment and enrollment of participants, measures employed, and analyses implemented.

Design and Recruitment

We implemented a cluster-randomized trial over 4 years in 8 Boys & Girls Clubs. Cluster randomization prevented contamination from sharing text messages among members at a single site, which we felt was likely if youths were randomized to YAE! At the study outset, we created 32 unique randomization units, that is, each of the 8 clubs participating each year for 4 study years (8 clubs × 4 years = 32), and randomized all 32 clubs per year combinations to ensure each club would be an intervention site in 2 years and a control site in 2 years. Within each year, 4 clubs were assigned to the intervention condition, stratified by club (8 strata), and year (4 strata) to balance demographic characteristics between clubs.

There were 9250 14- to 18-year-old members of participating Boys & Girls Clubs between September 2011 and September 2014 who were naïve to TOP eligibility for study enrollment. Of these, 3643 were active members, having attended 1 of the clubs in the 3-month period before program delivery. All 3643 were approached by club staff and invited to participate; 854 agreed. Reasons given for nonparticipation were lack of interest in TOP and competing commitments for other club and after school activities. Analyses that compared gender, age, and race/ethnicity of youths who did not participate with those who were enrolled showed no statistically significant differences. Our CONSORT diagram is shown in Figure A (available as a supplement to the online version of this article at http://www.ajph.org).

Enrollment and Condition

Enrollment occurred annually preceding inaugural TOP or TOP/YAE! sessions. We applied for and received a waiver of parental consent for participation in the study. Eligible youths viewed a video-based informed consent on the day of but before enrollment in the study. Of 854 participants who viewed the video-based informed consent on a computer, 852 consented to participate, responding in the affirmative to specific questions that documented comprehension and agreement to participate on a self-administered online survey after viewing the video. They then self-administered baseline study assessments via Research Electronic Data Capture, a secure online survey system, and were then made aware of their study assignment. Participants completed follow-up surveys identical to baseline assessments immediately following the final TOP or TOP/YAE! session, and again 12 months after completing TOP or TOP/YAE! In the first 2 years of data collection, we offered youths $5 to complete baseline surveys and $10 for each follow-up survey. To increase survey response rates, we offered $5 for baseline and $15 for each follow-up in years 3 and 4.

We offered all participants 25 weekly TOP sessions over 9 months and 20 hours of community service learning, receiving a stipend of $2.50 per program session in TOP, regardless of enrollment arm.

In addition to TOP, youths in the intervention received YAE!, the aforementioned text message intervention, which was carefully designed to reinforce sexual and reproductive health content delivered through TOP throughout the 25-week program. YAE! is among only a handful of social media interventions that attends carefully to compelling and engaging text message design informed by the Integrated Theory of mHealth,²² which allows adaptation to accompany any in-person, evidence-based youth sexual health intervention. YAE! was developed using an iterative youth-informed formative evaluation.²⁴ Intervention participants received between 5 and 7 messages weekly, of which 40% were bidirectional (i.e., requesting a response), which allowed us to document that youths were reading messages and to assess engagement. Messages were standardized and automated, sent through the Patient Relationship Management System²⁵; youths without phones received messages through Text Free, an online system, viewing messages on computers at the Boys & Girls Club. Examples of messages can be found in Table 1.

TABLE 1—

Examples of Text Messages Sent to Participants: Youth All Engaged! Program; Denver, CO; 2011–2014

Type	Content	Desired Theoretical Outcome
Question	If ur friend got pregnant, what would u tell her to do?	Social support
Quiz	Where do people in Colorado volunteer most? 1 = hospitals, 2 = education/schools, 3 = religious places, 4 = other/unsure.	Civic engagement
Club reminder	Ur TOP Club will meet at < club name > on < date > at < time > . Will u go? Reply 1 = yes, 2 = no, 3 = not sure.	Cue to action
Myth-fact (tailored for gender)	Female: If a guy wants to have sex, it’s his responsibility to get the condoms. 1 = myth, 2 = fact, 3 = unsure. Male: If a girl wants to have sex, it’s her responsibility to get birth control. 1 = myth, 2 = fact, 3 = unsure.	Norms
Myth-fact	“Less than 50% U.S teens are having sex.” Reply 1 = myth, 2 = fact, 3 = unsure.	Norms
Myth-fact	It’s a FACT! The Centers for Disease Control reported that only 43% of US teens are having sex. Not every1 is doing it.
Poll	Will an unplanned pregnancy prevent you from reaching your goals? Text 1 = yes, 2 = no, 3 = unsure.	Improved future orientation (social capital theory)
Fun fact	1 out of 3 teens say that it’s hard to talk about condoms. Think safe sex is important? Learn how to talk about it!	Positive norms re: healthy communication
Quote	“Communication is key.” “Be polite...you don't start taking each other for granted ever, you know.”—Ice Loves Coco	Role modeling for healthy communication

Measures

To assess aim 1 (feasibility and cost), data on platform and administrative costs for YAE! and TOP were estimated using 2013 US dollars, including personnel costs to run TOP sessions, incentives for participation, supplies, indirect costs, transportation costs, training and fidelity observation costs, and fees paid to implement the curriculum. We did not track youth message plans; formative evaluation demonstrated most youths with cell phones had unlimited text messaging plans, indicating no cost to them for receiving texts. Total study costs were summed across the cost components and divided by the total study participants enrolled at baseline to estimate the average program administration costs per participant in TOP and YAE! We assessed feasibility of program delivery by documenting the number of TOP sessions attended. We documented intervention fidelity by tracking the number of messages sent and the number of responses received to bidirectional messages per participant.

To assess aim 2 (average percentage of protected sex acts), participants reported if they ever had sex and the number of times they had sex in the past 3 months. Those with sexual experience reported the number of encounters in 3 months protected by condoms or contraception. The average percentage of protected sex acts was calculated as the number of times sex was protected by condoms and contraception divided by the number of times a person had sex in the previous 3 months. Scores were distributed continuously, ranging from 0% (never protected) to 100% (fully protected). Participants reporting 0 sexual encounters in the last 3 months were considered abstainers and were coded as 100% protected. We also coded them as having missing data for this outcome and ran analyses using both approaches. Participants reported whether they had accessed contraceptive or sexually transmitted infection services in the past 9 months (aim 3) and whether they had ever been pregnant or caused a pregnancy (aim 4), coded dichotomously (yes/no).

We anticipated covariates based on theory and planned to include those in models based on empirical assessment when any variable predicted a given outcome at a P value of less than .05, making covariate lists outcome dependent. Covariates included demographic characteristics: age, gender, and ethnicity (do you consider yourself to be Hispanic or Latino? Yes/No, hereafter referred to as Hispanic), followed by self-reported race, in which participants identified with 1 or more racial groups, including White, Black, Asian, American Indian, Native Hawaiian, or other. We conducted a subanalysis with all participants comparing outcomes of Hispanic to non-Hispanic participants. We asked participants to indicate (yes/no) if they failed a course in the past year, were living with 1 or both parents, if any parent was born outside the United States, if they had ever been coerced or coerced another into sex, and had ever been jailed; we also asked them to indicate parental education level.

The Clinical Trials Registry number is NCT01535651. We experienced no adverse events during the study.

Data Analysis Plan

Our first aim was descriptive and included frequency data, means and SDs to assess costs, and feasibility of program delivery. To assess outcomes for aims 2 to 4, our planned sample of 800 participants, with 200 each year for 4 years, assumed a 30% program attrition to generate 620 participants at program completion, to document postintervention outcomes (power 0.80; α < =0.05) similar to other interventions for adolescents,^16,26 with assumptions that 45% of adolescents would be sexually experienced and 73% sexually active in the past 3 months as documented in nationally representative samples.^27,28 Estimates took into consideration published data of sexual behavior intraclass correlation coefficients, which ranged from 0.001 to 0.02 arising from within-group similarities in Boys & Girls Clubs.²⁹ We followed with an intent-to-treat approach, in which participants were analyzed based on assigned condition, regardless of level of engagement with the program.

Analyses used a multilevel regression framework, using SAS Proc Mixed (SAS Institute, Cary, North Carolina) for continuous and SAS Proc Glimmix for categorical outcomes, accounting for nesting within club/year combinations.

We examined equivalence between conditions for baseline demographic characteristics, and we included variables found to differ at baseline in addition to those named previously as covariates in analyses. Analyses for aims 2 and 3 were analysis of covariance models that predicted each outcome from intervention condition, covarying baseline values for outcome, club, year, any variables found to differ between conditions at baseline, and potential covariates that predicted outcomes at a P value of less than .05. The critical P value for primary outcome analyses was a P value of less than .0125, which corrected for the 4 primary outcomes using a Bonferroni–Hochner calculation.³⁰ Secondary ad hoc subgroup analyses include Hispanic ethnicity in the analysis of covariance models to explore the interaction term between intervention condition and ethnicity, controlling for main effects of ethnicity and condition.

We examined if those lost to follow-up were different than those retained, and whether this differed by condition on 22 baseline variables (e.g., the same demographic, behavioral, and sexual risk variables listed in Table 2). For each of the 22 variables, we tested the main effect of retention status (retained vs not retained) and the interaction term between retention status and condition. Of all tests we conducted, there were 3 main effects of retention status. At baseline, those retained were more likely to report sexual acts protected by contraception (P = .006) and living with both parents (P = .004), and were less likely to report having had intercourse (P = .02). There was a single significant interaction term in which the effect of retention status depended on the condition for age at baseline (P = .04); intervention participants lost to follow-up were older than those retained (P = .006).

TABLE 2—

Demographic and Behavioral Characteristics for Youths at Baseline: Youth All Engaged! and Teen Outreach Program; Denver, CO; 2011–2014

	Youth All Engaged!		Teen Outreach Program
Baseline Measure	Analytic Cell No.	% or Mean ±SD	Analytic Cell No.	% or Mean ±SD	P of Difference
Age or grade level	316	14.90 ±1.02	315	14.98 ±1.14	.51
Gender: female	314	51.59	313	51.12	.75
Race/ethnicity
White	316	9.18	315	7.94	.50
Black	316	20.25	315	18.73	.73
Hispanic	316	42.41	315	43.81	.68
Multiple/other	316	28.16	315	29.52	.71
Failed a course	295	31.86	292	32.88	.98
Living with both parents	316	51.58	315	54.29	.74
Mother’s education	240	2.48 ±1.14	240	2.36 ±1.12	.24
Father’s education	197	2.22 ±1.12	212	2.07 ±1.11	.32
Parents not born in United States	296	37.84	289	40.14	.67
Ever had sex	309	22.98	303	27.72	.54
Ever coerced into sexual activity	294	10.88	305	10.82	.83
Ever coerced someone else into sexual activity	302	3.31	304	2.63	.74
Ever jailed or arrested	306	11.11	302	10.93	.68
Has cell phone	316	71.20	315	68.89	.29
Condoms in past 3 moa
Sexually active	40	68.7 ±42.5	42	68.4 ±45.0	.89
Includes abstainers	306	95.9 ±18.5	300	95.6 ±19.9	.78
Contraception in past 3 mob
Sexually active	39	84.6 ±31.7	40	79.2 ±38.4	.20
Includes abstainers	304	98.0 ±12.3	298	97.2 ±15.6	.73
Access to contraceptive or STD services	316	4.11	315	4.44	.92
Ever pregnant/caused pregnancy	313	0.32	311	3.86	.03

Note. STD = sexually transmitted disease.

Source. Teen Outreach Program and Teen Outreach Program411, 2012–2015 Baseline surveys administered before program enrollment. All analyses conducted in multilevel modeling framework to account for participants nested within clubs/years. Presented P values are adjusted for dummy coded club and year variables.

^aAverage percentage of sex acts protected by condoms in past 3 months.

^bAverage percentage of sex acts protected by contraception in past 3 months.

We also found variations in missing data for each outcome based on participants’ sexual history and comfort with answering specific questions. For example, the sample size for average percentage of protected acts assessed among sexually active participants was lower due to a lower level of recent sexual activity among sexually experienced participants than assumptions for sample size estimations. At baseline, 233 of 852 participants reported ever having had sexual intercourse, and 142 reported sex in the previous 3 months. When possible, we used logical imputation for missing values, but participants choosing “do not want to answer” were coded as missing. At follow-up, 293 of 590 participants reported ever having had intercourse (49.6%, which was similar to the 47% reported in Youth Behavioral Risk Surveillance System),²⁸ but only 114 reported sex in the previous 3 months (38.90% of those sexually experienced); this diverged from nationally representative surveys that showed more than 70% of sexually experienced youths had been sexually active in the past 3 months.²⁷ The maximum sample size for the average percentage of protected acts in the past 3 months among sexually experienced participants was 114, reducing the effective number and power to detect significant effects for these outcomes. Participants reported differentials in sex in the previous 3 months at baseline compared with follow-up, with a small number of participants with sexual outcome data at both time points (e.g., 66 reported sex in the previous 3 months for baseline and follow-up), thus affecting the analytic sample size for these outcomes because baseline values were used as covariates that affected the power to detect effects among sexually active youths. Thus, we included Cohen’s d or odds ratio effect sizes for continuous and binary outcomes.

We conducted sensitivity analyses in 2 ways to address the issue of missing data, including using full information maximum likelihood to estimate effects using all available data and multiple imputation to create a complete data set for covariates or baseline values to address missing data on relevant covariates. For sensitivity analyses, study conclusions did not change, and we did not report on these analyses further (although the results are available from authors). Tables 3 and 4 report the observed analytic number, not the number based on imputed values.

TABLE 3—

Postintervention Estimated Effects on Behavioral Outcomes by Condition: Youth All Engaged! and Teen Outreach Program; Denver, CO; 2011–2014

	Youth All Engaged!		Teen Outreach Program
Outcome Measure	Analytic Cell No.	% or Mean ±SD	Analytic Cell No.	% or Mean ±SD	Effect Size,a d (95% CI) or OR (95% CI)
Condoms in past 3 mo—sexually activeb	27	80.7 ±34.2	28	70.2 ±39.0	0.30 (0.20, 0.39)
Condoms in past 3 mo—includes abstainersc	217	94.1 ±22.1	200	92.7 ±24.1	0.06 (0.04, 0.08)
Contraception in past 3 mo—sexually actived	24	92.8 ±22.4	26	80.4 ±40.0	0.39 (0.30, 0.48)
Contraception in past 3 mo—includes abstainerse	248	97.5 ±14.8	253	95.9 ±19.5	0.09 (0.07, 0.11)
Access to contraceptive or STD servicesf	313	5.75	311	8.04	0.75 (0.35, 1.62)
Ever pregnant or caused pregnancyg	256	3.13	255	3.92	0.73 (0.17, 3.09)

^aMeasures of effect size are presented as Cohen’s d (95% confidence intervals [CI]) for continuously measured outcomes and odds ratios (OR; 95% CI) for binary outcomes.

^bCovariates were baseline average percentage of sex acts protected by condoms in past 3 months—sexually active, baseline pregnancy history, club, y, and ever coerced into sexual activity.

^cCovariates were baseline average percentage of sex acts protected by condoms in past 3 months—include abstainers, baseline pregnancy history, club, y, age, ever coerced into sexual activity, and ever failed a course.

^dCovariates were baseline average percentage of sex acts protected by contraception in past 3 months—sexually active, baseline pregnancy history, club, y, and ever coerced into sexual activity.

^eCovariates were baseline average percentage of sex acts protected by contraception in past 3 months—include abstainers, baseline pregnancy history, club, y, ever coerced into sexual activity, and ever failed a course.

^fCovariates were baseline access to contraceptive or sexually transmitted disease (STD) services, baseline pregnancy history, y, age, and gender.

^gCovariates were baseline pregnancy history, club, y, age, ever coerced into sexual activity, coerced someone else into sexual activity, and ever failed a course.

TABLE 4—

Secondary Analyses of Postintervention Estimated Effects by Hispanic vs Non-Hispanic Participants: Youth All Engaged! and Teen Outreach Program; Denver, CO; 2011–2014

	Youth All Engaged!		Teen Outreach Program
Outcome measure	Analytic Cell No.	% or Mean ±SD	Analytic Cell No.	% or Mean ±SD	Condition by Ethnicity, B (SE); Pa
Condoms in past 3 mo—sexually activeb					0.50 (0.22); .06
Hispanic	11	75.3 ±40.1	12	49.4 ±46.6
Non-Hispanic	16	84.5 ±30.2	16	85.8 ±23.2
Condoms in past 3 mo—includes abstainersc					0.4 (0.04); .36
Hispanic	99	94.2 ±22.5	95	91.5 ±26.8
Non-Hispanic	118	94.1 ±21.9	105	93.9 ±21.5
Contraception in past 3 mo—sexually actived					0.04 (0.19); .84
Hispanic	11	93.5 ±15.3	12	75.0 ±45.2
Non-Hispanic	13	92.3 ±27.7	14	85.0 ±36.1
Contraception in past 3 mo—includes abstainerse					0.04 (0.03); .20
Hispanic	109	98.4 ±10.8	114	95.5 ±20.6
Non-Hispanic	139	96.8 ±17.2	139	96.3 ±18.7
Access to contraceptive or STD servicesf					−0.41 (0.70); .57
Hispanic	132	5.30	137	9.49
Non-Hispanic	181	6.08	174	6.90
Ever pregnant/cause pregnancyg					−4.38 (1.72); .02
Hispanic	112	1.79	119	6.72
Non-Hispanic	144	4.17	136	1.47

^aAll analyses conducted in multilevel modeling framework to account for participants nested within clubs/year. Presented P values are adjusted for baseline scores, baseline pregnancy history, and dummy coded club and year variables (except for access to services in which adjusting for club created convergence problems). Additional covariates that were specific to each outcome are noted. We controlled for Hispanic ethnicity and treatment condition when assessing the interaction between Hispanic ethnicity and treatment.

^bCovariates were baseline average percentage of sex acts protected by condoms in past 3 months —sexually active, baseline pregnancy history, club, year, and ever coerced into sexual activity.

^cCovariates were baseline average percentage of sex acts protected by condoms in past 3 months —include abstainers, baseline pregnancy history, club, year, age, ever coerced into sexual activity, and ever failed a course.

^dCovariates were baseline average percentage of sex acts protected by contraception in past 3 months —sexually active, baseline pregnancy history, club, year, and ever coerced into sexual activity.

^eCovariates were baseline average percentage of sex acts protected by contraception in past 3 months—include abstainers, baseline pregnancy history, club, year, ever coerced into sexual activity, and ever failed a course.

^fCovariates were baseline access to contraceptive or sexually transmitted disease (STD) services, baseline pregnancy history, year, age, and gender.

^gCovariates were baseline pregnancy history, club, year, age, ever coerced into sexual activity, coerced someone else into sexual activity, and ever failed a course.

RESULTS

Of 852 participants who completed a baseline assessment, 436 were assigned to TOP/YAE! and 416 to TOP alone. We retained 317 participants in TOP/YAE! and 315 in TOP alone. A CONSORT diagram showing recruitment and study assignment is available in the Appendix (available as a supplement to the online version of this article at http://www.ajph.org).

Data on demographic and behavioral characteristics, including sexual behaviors as an equivalence of the sample enrolled at baseline retained at follow-up (Table 2) document a significantly higher proportion of those in the TOP alone compared with those in TOP/YAE! who had experienced a pregnancy at baseline (P = .03). Pregnancy history was included as a covariate in outcome analyses.

Aim 1: Cost and Feasibility of Program Delivery

Average per participant program cost for TOP administered to 416 participants was $1184 (95% confidence interval [CI] = $954, $1424). Average additional per participant program cost for 436 YAE! participants was $1310 (95% CI = $1083, $1549) for an additional $126 per participant in YAE! versus TOP alone or a 10.6% cost increase (95% CI = $101, $153).

TOP/YAE! participants attended a mean of 10.9 ±8.8 sessions, with a mean of 10.8 ±13.9 community service learning hours, and TOP alone participants attended a mean of 13.0 ±8.9 sessions and 12.5 ±12.9 community service learning hours. Participants in TOP/YAE! received 75 text messages, of which 40% were bidirectional. A detailed analysis of 2 years of text message delivery³¹ documented that 80% of participants responded to at least 1 bidirectional text message, and responded an average of 13 times to bidirectional messages. Participants aged 16 years and older, female adolescents, and Hispanic participants responded significantly more frequently to bidirectional messages.

Aims 2 to 4: TOP Only vs TOP/YAE! Impacts on Outcomes

Table 3 shows follow-up unadjusted means and SDs for the continuous and dichotomous outcomes for TOP only versus TOP/YAE!, with results from the impact analyses in terms of effect sizes and associated P values. Postintervention effects showed no differences between TOP only versus TOP/YAE! for any outcomes.

Table 4 shows ad hoc follow-up unadjusted means or percentage values for Hispanics in the TOP alone and Hispanics in TOP/YAE! for all outcomes. We documented no effect of condition dependent on Hispanic ethnicity for condom and contraceptive use or access to services. However, we observed the percentage of pregnancies reported among Hispanics in TOP alone was 6.72% compared with 1.79% in TOP/YAE! (P = .02). Although we confirmed baseline equivalence in all but 1 demographic and outcome variables, stratified by Hispanic versus non-Hispanic, before conducting subgroup analyses (Table A, available as a supplement to the online version of this article at http://www.ajph.org), the P value could not be calculated to compare baseline pregnancy rates among Hispanics in the TOP alone to those in TOP/YAE! because there were no pregnancies among TOP/YAE! participants at baseline, raising a possibility of nonequivalence at baseline.

DISCUSSION

This rigorously designed cluster-randomized trial allows confidence in multiple findings. We maintained a degree of fidelity consistent with published literature on health-related text message programs³² in delivering automated text messages that reinforced TOP content and showed only a 10.6% increase in cost of adding YAE! to TOP. This supported our first aim that showed it was feasible to enhance TOP with a text message supplement, and this enhancement could be delivered with fidelity and at low cost.

There were no statistically significant differences between TOP/YAE! and TOP alone on any primary outcomes. Subanalyses that explored differences for Hispanics on average percentage of sex acts protected by condoms or contraception in the past 3 months or access to contraceptive or STI services in the past 9 months were not supported.

There was a significant difference in pregnancy outcomes in our analyses for Hispanic participants in TOP/YAE! compared with TOP alone. We could not confirm baseline equivalence on this outcome for Hispanic participants; there was a small sample size, and we conducted a large number of moderator analyses. However, our analysis provides evidence that use of YAE! with in-person curricula might have potential to benefit Hispanics in preventing pregnancy, which warrants future research using larger sample sizes of Hispanics.

We are cautious to recommend a TOP/YAE! replication with 25 sessions over a 9-month period. Although we documented no differences between those invited and not participating and those who did participate, with less than one-quarter of youths invited participating, we could not generalize results to all Boys & Girls Club members. Competing programming options in Boys & Girls Club activities and program length might have been a deterrent to study participation. For those enrolled, mean attendance data (11.9 sessions of 25 required) suggested that expectations for youths to attend 25 weekly sessions over 9 months was unrealistic.

Our effect sizes adding YAE! text messages were small. However, we considered the impact of small positive effects of exposure to technology on the change at the population level, particularly because technology allows opportunities to reinforce and extend reach over time to large numbers of individuals at low added cost per person. If TOP or other programs can incorporate the use of text—particularly bidirectional texting, as it has been shown to generate better engagement and better effects³³—they may be able to improve impact. However, our experience with TOP participants completing only half the intended number of sessions suggests programs cannot be lengthy, and that exploring shorter hybrid programs might be more successful in generating population effects.

The next steps appropriate for research should explore whether and how technology can deliver content to supplement face-to-face sessions or replace some when they may be less optimal due to length, staffing, or cost. Hybrid programming combining the “best” of face-to-face interventions with convenient, easy to access technology based content is logical and mimics 21st century youth technology use.

A limitation of our research was the small number of youths who were sexually active in the past 3 months, which might increase the analytic bias because younger participants and those with lower risk had higher retention. It might be that TOP or TOP/YAE! could be more effective for higher-risk adolescents; however, they were more often lost to follow-up, so we could not ascertain this. We advocate approaches that reduce biases associated with retention of youths over time.