Impact of an Intervention Designed to Reduce Sexual Health Risk Behaviors of African American Adolescents: Results of a Randomized Controlled Trial

Abstract

Objectives. To replicate an evidence-based HIV risk reduction program and assess its impact on 2 behavioral outcomes—inconsistency of condom use and frequency of sex—6 months after the program.

Methods. The study was an individual-level randomized controlled trial in which we randomly assigned 850 youths (aged 14–18 years) to 1 of 2 conditions. The treatment (Becoming a Responsible Teen) is a group-level sociocognitive and skills training sexual education course; the control is a general health intervention that includes the same initial informational component as the treatment. Participants were recruited over 3 summers (2012–2014) from a summer employment program in New Orleans, Louisiana, that serves primarily African American adolescents.

Results. Six months after program exposure, we found no statistically significant difference between treatment and control group members’ self-reported inconsistency of condom use or frequency of sex (P > .05).

Conclusions. Although previous evidence has indicated that this particular program can be effective, this study’s findings indicate that it was not effective in this setting with this specific population. Results should provide an incentive to learn why the intervention works in some cases and what conditions are necessary for causal impacts.

The deleterious health, social, and economic outcomes associated with unintended adolescent pregnancy and sexually transmitted infections have made adolescent sexual health a particular concern for health professionals and policymakers over the past 20 years.^1–5 One response has been the development of programs that purport to reduce adolescent sexual risk by helping young people to modify behaviors that lead to these outcomes. Another has been the evaluation of these programs. Since 2009, the US Department of Health and Human Services (HHS) has sponsored an independent review of the evidence for these interventions. This review has identified more than 35 programs that have some evidence of positive impacts on sexual behaviors or related outcomes.⁶ However, the review also identified the absence of studies that “test how these programs perform when implemented on a broader scale, in different settings, or with different populations”^7(p1) as a primary weakness in the research.^7,8 In light of this, HHS incorporated this approach into Office of Adolescent Health (OAH) Teen Pregnancy Prevention Program funding as a means to bolster the current evidence base.⁹

We report findings from a study of an evidence-based adolescent pregnancy prevention intervention, Becoming a Responsible Teen (BART), conducted from 2012 to 2014 in Orleans Parish, Louisiana. Pregnancy and sexually transmitted infection rates for African American youths in New Orleans, which are well above the national average, demonstrate a high level of baseline risk for this population. More details of the public health setting for our target population are included in Appendix A (available as a supplement to the online version of this article at http://www.ajph.org). As with other programs built on social psychological theory that have been developed to reduce sexual risk behaviors, the intervention is premised on the idea that imparting knowledge alone is insufficient to reduce risk behaviors. In addition to providing youths with information, the intervention aims to build motivation to reduce these risk behaviors through the development of skills, efficacies, attitudes, and intentions.^10–14 Citing a randomized controlled trial published in 1995 (involving nearly 250 African American adolescents recruited from health clinics in a midsized southern city), the HHS review indicates that the intervention shows evidence of reducing unprotected anal and oral sex and increasing condom-protected intercourse.^9,14 The objective of this study was to add to the generalizable evidence of the intervention’s effectiveness by rigorously evaluating its impact on pertinent but not identical outcomes in a new setting: a summer work program in Orleans Parish that serves a similar population of primarily African American adolescents.

In an earlier published study, we found positive postprogram effects on many of the theoretical antecedents to behavior change that the program aims to influence, including knowledge, self-efficacy, attitudes, and intentions.¹⁵ We did not, however, find any impact on perceived risk. The purpose of this study was to investigate the impact of the intervention on 2 behavioral outcomes that are relevant to improving sexual health by reducing HIV and other sexually transmitted infections and unintended pregnancy—inconsistency of condom use and frequency of sex 6 months after the end of treatment. These measures were identified before the collection of any outcome data, as required by the Consolidated Standards of Reporting Trials (CONSORT). They were selected because they were behavioral outcomes specifically addressed by the intervention, deemed relevant to the HHS Teenage Pregnancy Prevention Evidence Review, and identified as primary outcomes of interest by the grantee who implemented the intervention.

METHODS

This study was a randomized controlled trial in which eligible individual participants were randomly assigned to the BART intervention (treatment) or the Healthy Living intervention (control). BART is based in theory that suggests that knowledge alone is insufficient to reduce risk behaviors; instead, behavior change must be motivated by theoretical sociocognitive antecedents of behavior. As such, both conditions included the same initial informational session that provides facts deemed necessary to prevent HIV, other sexually transmitted infections, and unintended pregnancy. After the initial information session, the control condition did not include the topic of sexual health behavior. Subsequent treatment sessions, however, did include information and activities intended to motivate sexual risk reduction. Because the control included the same HIV information component in the initial information session, this study tested the effects of the treatment sessions, which include training intended to address the theoretical determinants of behavior change (skills building and attitude and belief modification).

Interventions

BART.

The intervention includes 4 core components—information, skills training, opportunities to practice skills, and social support—that are meant to increase participants’ motivation to reduce risk by building knowledge and awareness of risk, clarifying participants’ values related to sexual behaviors, developing and enhancing participants’ risk-reduction skills, building attitudes supportive of condom use, and fostering intentions to reduce high-risk behaviors, including unprotected oral, anal, and vaginal sex; frequency of sex; and number of sexual partners. By addressing these motivational antecedents of sexual behavior, the program ultimately aims to increase safer sex behaviors, thereby preventing the transmission of HIV and other sexually transmitted infections and preventing adolescent pregnancy among African American adolescent youths. Program content is designed to be delivered in 8 weekly sessions. Sessions are expected to take between 90 and 120 minutes, depending on the session. Table 1 provides a detailed description of intended program content by session.

TABLE 1—

Intended Program Content, by Session: Becoming a Responsible Teen; Orleans Parish, LA; 2012–2014

Session	Overview
1. Understanding HIV and AIDSa	Provides information on what HIV is, how it is transmitted, risk and protective behaviors, and HIV prevalence among the target population; it also dispels common HIV myths.
2. Making Sexual Decisions and Understanding Your Values	Reviews information on HIV transmission, risks, stereotypes, and prevalence; it also includes activities intended to personalize risk and to help participants identify support systems.
3. Developing and Using Condom Skills	Presents facts about condoms, examines attitudes toward condoms and common barriers to their use, and provides demonstration of how to use condoms.
4. Learning Assertive Communication Skills	Presents ways to negotiate safer sex, identifies common communication problems and possible solutions, and demonstrates different communication styles.
5. Practicing Assertive Communication Skills	Presents tips for assertive communication, explores ways to say no; demonstrates and allows participants to practice assertive communication through role-play.
6. Personalizing the Risks	Presents personal accounts of HIV through in-person presentations or videos.
7. Spreading the Word	Participants link assertive communication skills to their lives and identify ways to get out of risky situations; demonstrates and allows participants to practice sharing what they have learned.
8. Taking BART With You	Reviews HIV facts; participants discuss how their behaviors or attitudes have changed and experiences in sharing what they learned.

Note. BART = Becoming a Responsible Teen.

^aThis same initial session is offered in the Healthy Living intervention.

Healthy Living.

Participants assigned to the control condition were offered Healthy Living, an 8-session general health and nutrition course designed for the study that aims to influence participants’ health behaviors with informational components on nutrition, healthy eating, body image, and exercise, as well as some basic HIV prevention facts. The control program administered the same information-only first session about HIV as the intervention; the remaining sessions were focused strictly on health and nutrition and did not contain any sexuality education components or incorporate core elements of the intervention.

Program Delivery and Setting

In collaboration with partners, the Louisiana Public Health Institute implemented programming over 3 consecutive summers (2012–2014) as an educational component of a summer employment program that is funded by the city government and serves primarily African American adolescents. The treatment and control interventions were offered as an optional health education program for youth workers. Before randomization within each site, youths were assigned to sites by city government staff. Criteria for assignment included age, what type of programming the applicant indicated he or she wanted to receive, and availability. The Louisiana Public Health Institute hired and trained health educators to administer the program; health educators were expected to teach both interventions.

Across all 3 cohorts, programming was offered at 18 employment sites and was facilitated by 41 health educators. In the first year of the study, both interventions were implemented for the most part over 8 weeks with 1 session per week. In the latter part of the first year, and throughout the second and third years, the duration was modified slightly, but not the dosage of the program. Because the summer job program did not make an 8-week-long intervention practicable, both programs were implemented over the course of 5 or 6 weeks. Both interventions were delivered in gender-specific groups of between 5 and 15 persons by 2 health educators, 1 male and 1 female. Additional information on program implementation and delivery can be found in the final impact report, available through the OAH Web site.¹⁶

Sample Recruitment

To be eligible to participate in the study, individuals had to (1) be aged between 14 and 18 years, (2) be assigned to a summer employment site implementing the study, (3) have not previously participated in other OAH-funded youth pregnancy prevention initiatives operating in Louisiana, and (4) provide participant assent/consent and, if younger than 18 years old, parental consent to participate in the study. The recruitment process did not differ between the treatment and control groups. Each year, evaluation staff recruited participants and screened for eligibility using consent packages that were provided to potential study participants and their parents (adolescents who had been accepted into the summer employment program and could be placed at sites administering the interventions). Packages contained a cover letter explaining the study, separate parent program and evaluation consent forms, and participant evaluation assent forms. Assent forms included eligibility questions related to the potential participants’ gender, age, and any previous participation in other adolescent pregnancy prevention programs operating in the city. Additional information can be found in the final impact report available through the OAH Web site.¹⁶

Randomization

Various employment program restrictions (participants were assigned to different sites and shifts) and intervention fidelity requirements (which required gender-specific classes containing 5–15 participants) necessitated a blocked randomization design. Individuals were randomly assigned to the treatment or control condition in 3 cohorts (year of enrollment) and at each job site, according to gender and work shift. That is, each summer, youths at each employment site were individually randomized into gender-specific treatment and control groups according to whether they worked in the morning or afternoon. Full details of the data collection timeline can be found in Table A (available as a supplement to the online version of this article at http://www.ajph.org).

All individuals in attendance on the first day of work who met eligibility criteria were individually assigned by the evaluators into the treatment or control condition using the random allocation (RALLOC) command in Stata version 12 (StataCorp LP, College Station, TX). Adolescents who were not present the first day but who were otherwise eligible and attended work within the first 2 weeks were randomized (using a coin toss) into the study on a rolling basis, provided there was space in the class. For both allocation methods, the probability of being assigned to the treatment or the control condition was 50%.

In all, 1230 youths were assigned to study sites over the course of the evaluation. Of these, 850 individuals were eligible, provided consent or assent, and were then randomized into the treatment (n = 427) or control (n = 423) condition across 86 gender-specific classes (43 classes for each condition). See CONSORT diagram (Figure 1) for details.

FIGURE 1—

Consolidated Standards of Reporting Trials (CONSORT) diagram for adolescents: Becoming a Responsible Teen; Orleans Parish, LA; 2012–2014.

Measures

To assess impacts on self-reported contraceptive use and sexual activity for the impact study, we collected baseline, outcome, and covariate data via a self-administered questionnaire at baseline (before the first program session attended) and 6 months after the program. Data collection procedures did not differ across the conditions. Incentives were provided to all study participants for completing questionnaires, regardless of whether they received any programming.

We operationalized inconsistency of condom use as the percentage of times in the past 3 months a participant did not use a condom while engaging in any type of sex. Persons who indicated that they were not sexually active and those who were sexually active but used a condom every time they had sex were considered to have engaged in the risk behavior 0% of the time. We obtained data from the following 2 items on the questionnaire administered at baseline and 6 months after the end of programming: “In total, how many times have you had any type of sex in the past 3 months?” and “Now, think about the number of times that you had any type of sex in the past 3 months. How many of those times did you use condoms?” Respondents were instructed that any type of sex referred to oral, anal, or vaginal sex, and not masturbation. Frequency of sexual activity was operationalized as the self-reported number of times in the past 3 months a person engaged in any type of sex. Persons who indicated that they were not sexually active were considered to have had sex zero times. Additional information can be found in the supplemental materials and the final impact report.¹⁶

Data Analysis

We examined whether the intervention affected participants’ reported inconsistency of condom use and frequency of sex within an intent-to-treat framework. To answer the research questions, we regressed outcomes on treatment status, blocking variables (indicators for cohort, site, shift, and gender) and individual-level covariates including the baseline measure of the outcome variables (inconsistency of condom use and frequency of sex at baseline). Because assignment was random, a simple difference of means on the outcome variables should provide an unbiased estimate of program impact; however, we statistically adjusted for covariates to increase the precision of our estimates and to account for blocking procedures. We mitigated the loss of cases due to item nonresponse with dummy variable adjustment for missing pretest and covariate data and multiple stochastic imputation for missing outcome data.

We conducted 6 sensitivity analyses (specified before analysis) that tested whether our findings were sensitive to our analytic decisions to

1. control for covariates,

2. impute or otherwise adjust missing data,

3. exclude unreliable data, and

4. include outliers in analysis and to test our decisions related to

5. condensing programming and

6. data collection procedures.

Results confirmed benchmark findings. Additional information can be found in the supplemental materials and the final impact report.¹⁶

RESULTS

The full set of 850 adolescents who were randomized into the study constitutes the full sample. Treatment participants were those who were randomly assigned to receive the treatment intervention (n = 427); control participants were those who did not receive the intervention (n = 423). The analytic sample is the subset of this sample for whom we had sufficient data (completed the baseline and 6-month follow-up questionnaires and did not have unreliable responses at either administration). The analytic sample consisted of 688 adolescents (treatment, n = 342; control, n = 346) and represents 81% of those initially randomized. See the CONSORT diagram (Figure 1) for details.

Baseline Equivalence

Baseline data indicate that, of the 688 participants included in the analytic sample, 362 (53%) were female; 576 (84%) identified as Black, and 23 identified as Hispanic (3%). On average, participants were aged 15 years at baseline and self-reported engaging in sex 1 time in the previous 3 months.

Baseline equivalence of the analytic sample is convincing; regression-adjusted differences, presented in Table 2, between the treatment and control group were small and statistically insignificant (P > .05 in all cases). We assessed equivalence of the treatment and control groups on preintervention measures of our primary and secondary outcomes (consistency of condom use and frequency of sex) and key covariates. We used a 2-step procedure to establish balance; we first generated model-based estimates of the differences between groups and then examined the statistical significance of the differences. Separate models were run for each of the baseline variables. We used ordinary least squares regression models to estimate differences in continuous baseline measures and constructed linear probability models to estimate differences in dichotomous baseline measures. Additional information can be found in the final impact report.¹⁶

TABLE 2—

Participant Characteristics and Sexual Behaviors at Baseline for Treatment and Control Samples: Becoming a Responsible Teen; Orleans Parish, LA; 2012–2014

Baseline Characteristic or Behavior	BART (n = 342), Mean ±SD or % (SD)	Control (n = 346), Mean or % (SD)	Adjusted Differencea	P
Age, y	14.96 ±0.79	14.96 ±0.85	0.03	.44
Sex (female)	52.63 (0.50)	52.60 (0.50)	0.26	.94
Race (African American)	84.80 (0.36)	82.66 (0.38)	1.35	.63
Hispanic	2.92 (0.17)	3.76 (0.19)	−0.64	.66
Parental educationb	2.48 ±0.96	2.48 ±0.90	0.00	.99
Family structure (lives with both parents)	25.73 (0.44)	20.23 (0.40)	5.28	.10
Inconsistency of condom usec	10.40 (25.57)	12.79 (27.14)	−2.26	.34
Frequency of sexual activityd	0.98 ±2.77	1.17 ±3.42	−0.10	.67

Note. BART = Becoming a Responsible Teen; P = significance of the baseline equivalence statistic, or the P value of the difference between the treatment and control groups. n = 688 participants in the analytic sample.

^aRegression adjusted difference between treatment (BART) and control groups.

^bParental education refers to the mean level of parents’ education reported by participants (scores range from 1 = less than high school to 5 = graduate degree).

^cInconsistency of condom use refers to the proportion of times in the past 3 months a person reported having any type of sex without using a condom.

^dFrequency of sexual activity refers to the number of times in the past 3 months a person reported having any type of sex.

Inconsistency of Condom Use

Regression findings presented in Table 3 suggest that the intervention had no significant effect on participants’ inconsistency of condom use at the 6-month follow-up. The treatment effect of 2% indicates that 6 months after the intervention, on average, members of the treatment group reported using condoms 2 percentage points less often when they had sex over the past 3 months than did members of the control group, controlling for other covariates in the model. This difference is not statistically significant. Sensitivity analyses (details of which can be found in the final impact report) corroborated these results. In each of the sensitivity tests, the adjusted mean difference in participants’ inconsistency of condom use reported by treatment and comparison groups remained insignificant.

TABLE 3—

Estimated Intervention Impact on Sexual Behaviors at the 6-Month Follow-Up: Becoming a Responsible Teen; Orleans Parish, LA; 2012–2014

Outcome	No.	BART, Mean	Control, Mean	Adjusted Differencea (SE)	P	g
Inconsistency of condom useb	688	9.05	6.74	2.31 (1.80)	.20	.10
Frequency of sexc	688	1.40	1.73	−0.17 (0.37)	.65	.03

Note. BART = Becoming a Responsible Teen; g = standardized effect size (Hedges g) of treatment indicator averaged across the 10 imputations. Included in the regression model as covariates are age, race, ethnicity, parental education, family structure, and the following blocking covariates: cohort, site, work shift, and gender.

^aThe regression adjusted mean difference between treatment (BART) and control groups.

^bInconsistency of condom use refers to the percentage of times in the past 3 months a person reported having any type of sex without using a condom.

^cFrequency of sexual activity refers to the number of times in the past 3 months a person reported having any type of sex.

Frequency of Sexual Activity

Results also indicated that the intervention had no impact on participants’ frequency of sex at the 6-month follow-up. The coefficient for the treatment indicator indicates that, compared with control participants, treatment participants, on average, reported having had sex 0.17 fewer times in the past 3 months. Again, this difference is not statistically significant, and sensitivity analyses confirmed the benchmark results.

DISCUSSION

Findings indicate that the intervention did not have a significant impact on the sexual behaviors of youths who were offered the intervention. Six months after the end of the program, treatment and control group members reported statistically insignificant differences in inconsistency of condom use and frequency of sex. Contrary to our expectations, the addition of motivational components as part of the BART program produced no discernible differences in participant-reported risk behaviors 6 months after the end of the program when contrasted with an information-only control program. These results, consequently, do not support the earlier research evidence cited by the HHS-sponsored evidence review that found that the program is effective in promoting safe sex behaviors, including reduced instances of unprotected sex, increased instances of condom use, and abstinence up to 12 months after exposure.^8,14

One hypothesis as to why the intervention did not achieve the anticipated behavioral impacts in this setting and with this population is that, although it appears to have been implemented with reasonable fidelity (details of the implementation study can be found in the final impact report), the program itself may not sufficiently motivate participants to reduce high-risk sexual behaviors.¹⁶ The theory of change posits that, to achieve desirable behavior modification, not only must a sexual health program impart knowledge and skills, but it must also motivate participants to use those skills through attitudinal development, by increasing self-efficacies to successfully use those skills, and by raising their awareness of their personal vulnerability to risk.^17,18 Findings from a previously published study conducted under the auspices of this grant showed that at the conclusion of programming, the intervention had positive and significant impacts on many of these latent constructs; however, the program did not have an effect on participants’ perceived vulnerability.¹⁵ This is important because the theoretical framework for the intervention identifies perceptions of vulnerability as a main motivator of behavior change, and several of the program’s components focus specifically on perceptions of risk.

Another potential explanation for the apparently discrepant results could be that the participants in both studies were different enough in some material way. Participants in this and the previous study, although largely similar, did report some differences in background characteristics and past sexual behaviors that suggest that youths in the previous study were more sexually active than were the ones in our study (details of the samples used in this and the previous study can be found in the supplementary materials at http://www.ajph.org and elsewhere).^14,16 However, secondary analyses, reported in the final impact report, do not support this interpretation.¹⁶

Yet another consideration is that outcome measures used in this study quantify different specific risk behaviors than those cited as evidentiary by the review. To investigate whether these findings are unique to our primary outcome measures, we examined program impact on the following additional secondary outcomes reported 6 months after the program: onset of sexual activity (initiation of any type of sex), recent sexual activity (any type of sex reported in past 3 mo), frequency of unprotected oral sex in the past 3 months, frequency of unprotected anal sex in the past 3 months, frequency of unprotected vaginal sex in the past 3 months, and number of lifetime sexual partners. The findings were not unique. Our analyses revealed no statistically significant difference between treatment and control groups in any of these outcomes at 6-month follow-up. Additional information on secondary analyses can be found in the final impact report.¹⁶

Plausible explanations lie beyond the scope of our data. For instance, the sample for the original causal study was drawn from a health clinic; adolescents were recruited into the study because they were receiving health care at the clinic. It is possible that because youths were receiving the intervention in this setting, they were more receptive to the intervention’s health promotion messages than were youths in our study. Finally, these studies were conducted 20 years apart. It is conceivable that any historical change in adolescents’ social, normative, educational, and informational environments now as compared with then could help explain differences in findings.

Regardless as to the reason, results of this randomized controlled trial demonstrate that the offer to receive the treatment relative to a control condition that contains only the informational components of the program had no statistically significant impact on participants’ self-reported condom use and frequency of sex 6 months after the end of the intervention. To the extent that they are of equal quality, study results such as this that fail to reject the null hypothesis and find that the intervention has failed to effect the behavioral change hypothesized should be of equal evidentiary value as those that find otherwise. The goal of OAH-funded adolescent pregnancy prevention evaluations is to build our empirical understanding of what works in youth risk reduction. Though there is evidence that this particular program can be effective in decreasing risk behaviors, findings from this study indicate that the program was not effective in this setting with this specific population. These results should provide more opportunity or incentive to learn why the intervention works in some cases and not in others and what conditions are necessary for the desired causal impacts.