Predictors of responsiveness among early adolescents to a school-based risk reduction intervention over 3 years

Abstract

This study assesses potential predictive factors for unresponsiveness to the “Focus on Youth in the Caribbean (FOYC)” intervention using longitudinal data from 1360 Bahamian sixth-grade youth. Results from hierarchical logistic regression analyses indicate that the intervention had a greater impact on knowledge, skills, self-efficacy, and condom use intention among low and medium initial scorers. High initial scores in knowledge, skills, self-efficacy, and intention were predictive of relative unresponsiveness to the intervention. Advanced age and male sex were predictive of unresponsiveness to the intervention for HIV/AIDS knowledge. Female gender was predictive of unresponsiveness to the intervention for self-efficacy. High academic self-evaluation was predictive of unresponsiveness to the intervention for condom use intention. The greatest intervention impact was observed at the six-month post-intervention follow-up; these intervention-related gains were sustained over the subsequent follow-up periods. Youth with higher risk attributes (lower knowledge, skills and self-efficacy) were more likely to respond to a risk reduction intervention.

Introduction

Early adolescents are at a pivotal time in their lives regarding personal decisions that impact their future health. In this challenging stage of child development, social, environmental, and personal factors interact with each other to form the foundation from which most health-related decisions develop [1]. In an all too common scenario, many adolescents confront life-changing difficulties after adopting risky behaviors (e.g., early sexual initiation and substance use) [2]. Today’s youth are disproportionately affected by HIV/AIDS. According to the Joint United Nations Programme on HIV/AIDS, an estimated five million young people between the ages of 15 and 24 are living with HIV, about 40% of new HIV infections are among young people, and more than half of all STIs other than HIV occur among young people in the same age group [3]. Because of their high vulnerability to HIV infection, their great potential to change their behaviors and the impact of infection in this age group on the global HIV epidemic, young people have been an important target population for HIV-prevention efforts

Over the past decade, several HIV prevention interventions have demonstrated effectiveness in reducing risk behaviors among young people [4–6]; these interventions have been designated “best-evidence” HIV prevention interventions by the Centers for Disease Control and Prevention (CDC) [7]. One of these programs, Focus on Youth (FOY), is a school-based HIV prevention program that provides adolescents who are at the pre-risk phase (e.g., the majority of them are not sexually active) with the skills and knowledge they need to protect themselves from HIV and other STDs [8]. Long-term intervention evaluations have found that FOY and several FOY-based adapted programs, including Focus on Youth in the Caribbean (FOYC) were effective in increasing youth’s HIV/AIDS knowledge, perceptions of their ability to use condoms and their intention to use condoms, and in reducing unprotected sex among sexually active youth [9–12].

Appropriately, an intention-to-treat analysis was widely used to evaluate the efficacy of the above-mentioned intervention programs including FOY [7]. However, young people are diverse. Specific intervention programs may be more or less effective for subgroups of youth with certain characteristics. Information about these variations in intervention responsiveness would inform intervention deployment and development decisions. Dissemination efforts targeting subgroups of youth with characteristics associated with responsiveness to a specific intervention could enhance the program’s impact. Identifying the characteristics of youth who are unresponsive to the interventions could be helpful in modifying programs to improve their effectiveness among less responsive populations.

A limited number of studies have assessed the differential impact of HIV prevention interventions according to the personal characteristics of participants. Evaluations of an adolescent sexual health intervention in Tanzania showed that the intervention had a greater impact on knowledge and sexual attitudes among males and improvements in knowledge were greater in unmarried compared to married young people [13,14]. A parent-based HIV prevention program for youth found that parents of preadolescent females discussed a larger number of sexual topics [15]. Some studies have found that programs targeting younger school children or youth who were virgins at program initiation have had greater success in influencing sexual behaviors compared with those targeting older school youth or youth who were already sexually active [16]. A differential intervention effect on condom use has also been reported among youth who live with HIV; HIV-positive youth with relatively low risk of disease transmission or rapidly growing risk are more responsive to program intervention than youth at high risk [17].

Factors other than gender, age and prior sexual experience are likely to influence intervention responsiveness. HIV/AIDS knowledge and condom use skills are considered pre-requisite for effective behavior change [14] and thus it is likely that these attributes may influence an individual’s responsiveness to a risk-reduction intervention. The purpose of this secondary analysis was to examine the extent to which the initial levels of knowledge, skills, perceptions and intentions predicted changes in the longitudinal trends of mean scores for knowledge, skills, perceptions and intentions and to identify individual level predictive factors for unresponsiveness to the intervention. The specific questions addressed in this study were: (1) Did trajectory patterns of HIV/AIDS knowledge, condom use skills, self-efficacy, and condom use intention differ by pre-intervention levels of knowledge, skills, perceptions, and intention that are categorized into low, medium, and high scorers?; and, (2) What are the characteristics of youth who made no improvement in their HIV/AIDS knowledge, condom use skills, self-efficacy, and condom use intention over the 36 months post-intervention period?

Method

Study site

Despite an effective response to the HIV epidemic, The Bahamas, with an estimated population of 353,658 over 700 islands, has an estimated adult HIV prevalence rate of 2.2% [18]. AIDS is the leading cause of death among adults 15 to 44 years of age in The Bahamas. Heterosexual activity is the predominant mode of transmission. The island of New Providence was selected as the study site because it houses 65% of the population and 86% of the HIV infected Bahmians reside on this island [19].

Intervention Description

Focus on Youth in the Caribbean (FOYC) was adapted by the Bahamian Ministry of Health and our research team from the intervention programs FOY. FOY is a Best Evidence Program and is included in the Diffusion of Behavioral Intervention portfolio of the CDC [7]. A detailed description of the development of FOYC can be found in our previous publications [20, 21]. Briefly, FOYC is a school-based HIV prevention program which consists of 10 sessions, each approximately 75 minutes in length. FOYC is based on Protection Motivation Theory [20] and emphasizes healthy decision-making, goal setting, communication, negotiation, consensual relationships, and information regarding both abstinence and safer sex. The youth control condition, the Wondrous Wetlands (WW), also consists of 10 sessions, each of which presents information in an interactive and hands-on fashion regarding the protection of the environment, including The Bahamas’ wetlands.

A 1-hour booster session was given to youth (both FOYC and WW) after completion of the 12-month assessment. For FOYC youth, the booster focused on a review of HIV facts and application of a decision-making model introduced previously to the youth in the FOYC curriculum, whereas for WW youth, it focused on steps to preserve water, a Bahamian natural resource.

Participants and intervention assignment

The participants in the study included 1,360 sixth-grade students who were recruited from 15 of the 26 public elementary schools from New Providence. The 15 schools were randomly assigned using a random numbers table to the intervention group (10 schools received FOYC) and to the control group (five schools received WW). All of the children attending these schools received the assigned curriculum; approximately two-thirds of eligible youth from each of the 15 schools voluntarily participated in the assessment of the intervention, resulting in 863 sixth-grade participants from the 10 FOYC schools and 497 from the five control schools. The data in the present analysis were obtained from six surveys of the youth (a baseline survey and five follow-up surveys conducted at six, 12, 18, 24 and 36 months post-intervention). The follow-up rate was 95% at six months, 89% at 12 months, 85% at 18 months, 86% at 24 months, and 82% at 36 months post-intervention. The mean age of youth at baseline was 10.4 years (range 10 to14 years) and 53% were females. Ninety-nine percent of youth were of African descent.

Data Collection Procedures

Data for the assessment of FOYC were collected using the Bahamian Youth Health Risk Behavioral Inventory, a paper-and-pencil questionnaire administrated in the classroom setting. The instrument was adapted from the Youth Health Risk Behavioral Inventory [22] through extensive ethnographic research and pilot testing. The questionnaire was read out-loud by project staff while the students marked their responses on the questionnaires; approximately 45 minutes were required for completion. Students were informed that participation was voluntary and their answers were confidential. Teachers were asked to leave the classrooms during the survey. Each student was given a voucher worth $5 Bahamian after completing the survey. Parental consent and youth assent were obtained before participation in the trial. The research protocol and questionnaires were approved by the institutional review boards at Wayne State University and Princess Margaret Hospital in Nassau.

Measures

HIV/AIDS Knowledge

A 20-item scale including true and false statements was used to assess level of knowledge regarding HIV transmission and prevention (e.g., “if you touch someone with AIDS you can get AIDS”). The internal consistency estimate (alpha) of these items was 0.60. Correct responses were scored 1 and incorrect 0, resulting in a summary score of 1 to 20 for each participant.

Condom use skills

Condom use skills were assessed using the Condom-use Skills Checklist [23]. The validated scale includes 15 true and false statements describing the detailed steps of condom use from opening a condom pack for use to disposal after use (e.g., “unroll condom before placing on the penis”). The internal consistency estimate of these items was 0.42. Correct responses were scored 1 and incorrect 0, resulting in a summary score of 1 to15 for each participant.

Condom use self-efficacy

A six-item scale was used to assess condom use self-efficacy (e.g., “I could put a condom on correctly”). Agreement was measured through a five-point Likert scale (1=strongly disagree to 5=strongly agree). The internal consistency estimate of the scale was 0.88. A composite score was calculated as a mean score across the six items (range 1 to5).

Condom use intention

Intention to use condoms was measured using the question, “if you were to have sex in the next six months, how likely is it that you or your partner would use a condom?” Youth rated the likelihood on a five-point Likert scale ranging from 1 (very unlikely) through 5 (very likely).

Academic self-evaluation

Academic self-evaluation was measured using the question “compared to other students in your class, what kind of student are you?” Five response options include: one of the best, above the middle, in the middle, below the middle, and near the bottom. The last three categories were combined into single “middle or below” category due to the low frequency of responses in these categories.

Unresponsiveness to FOYC intervention

A participant whose score of HIV/AIDS knowledge, condom use skills, self-efficacy or condom use intention never increased from the baseline survey through the 36-month follow-up survey or whose score increased but then relapsed by the 36-month follow-up survey is defined as a FOYC non-responder in each of these four outcomes categories (knowledge, skills, self-efficacy and intentions).

Analysis

Descriptive statistics (mean and standard deviation) of the four outcome variables (HIV/AIDS knowledge, condom use skills, condom use self-efficacy and condom use intentions) were calculated for both intervention and control groups at baseline, and six, 12, 18, 24, and 36-month follow-up. Within group and between group differences of these variables were tested using repeated measures ANOVA and student’s t test.

In the stratified analyses, at baseline the four outcome variables were further categorized into low (below the 25th percentile), medium (25th to 75th percentile), and high (above the 75th percentile). Correlation analysis was conducted to examine the overlapping of the initial score category in these outcome variables using Cochran-Mantel-Haenszel statistics.

Given the hierarchical nature of our data (students clustered within classes in 15 schools) and possible intra-class correlation by class and school, multilevel logistic regression analysis was conducted using the generalized linear mixed model (GLIMMIX) procedure to identify individual- and school-level predictive factors for unresponsiveness to the “FOYC” intervention. Dependent variables included lack of improvement in the four outcome variables during the 36 month follow-up period. Independent variables included age, sex, student academic self-evaluation, initial score in each of the four outcome variables, and intervention group assignment. Regression coefficients were calculated for each predictor variable. To adjust for the clustering effects of classroom and/or school, the intra-class correlation coefficient were calculated for all outcome variables.

All statistical analyses were performed using the SAS 9.2 statistical software package (SAS Institute Inc., Cary, NC, USA). A significance level of 0.05 was adopted in bivariate comparisons and multivariate analyses.

Results

1) Longitudinal trends of mean scores for HIV/AIDS knowledge, condom use skills and condom use self-efficacy

Figure 1 displays the longitudinal trends of mean scores for HIV/AIDS knowledge among Bahamian youth by intervention and control groups and initial levels of HIV/AIDS knowledge during the 36 months post-intervention. Overall, HIV/AIDS knowledge increased significantly among youth in both the intervention and control groups, although the increase in knowledge was significantly greater in the intervention group compared to the control group [4.2 vs. 3.2, t=4.12, df (1211), p<.0001]. A large increase in knowledge in the intervention group occurred at the six-month follow-up; this increase resulted in an upward displacement of the curve for the intervention group [2.0 vs. 0.6, t=5.97, df (1165), p<.0001]. After the six-month follow-up, the trajectories were similar for the two groups, but the upward displacement was retained for the intervention group throughout the follow-up period.

Figure 1.

Change in HIV/AIDS knowledge through 36 months intervention period

Figure 1 also illustrates that when the control and intervention participants were further stratified into three categories (low, medium, and high scorers) based on initial HIV/AIDS knowledge scores, the low and medium initial scorers who received the intervention (compared to those who did not receive the intervention) demonstrated the greatest improvement. The initial high scorers in the intervention group also showed improvement in their knowledge in comparison to the controls [1.6 vs. 0.4, t= 3.78, df (343), p=.0002]. Among the intervention youth, large increases occurred at the six-month follow-up for both low and medium initial scorers while high initial scorers remained at their initial level through 12 months of follow-up and then began to increase.

Figure 2 demonstrates changes in the mean score of condom use skills over time by intervention and control groups and initial levels of condom use skill. Overall, condom use skills increased significantly in the intervention group from a mean score of 7.7 at baseline to 9.0 at the 36-month follow-up, while it remained almost constant in the control group [1.2 vs. −0.1, t= 6.35, df (797), p<.0001]. When stratified by initial level, the trajectories were quite different for each category (low, medium, high initial scorers). Among both intervention and control youth, the largest improvements were observed among low initial scorers; however, the increase was significantly greater in the intervention group compared to the control group [3.6 vs. 2.1, t=4.36, df (197), p<.0001]. Intervention group participants with medium initial scores showed a significant increase compared to their medium scorers counterparts in the control group [1.4 vs. 0, t=5.20, df (327), p<.0001]. Mean scores of condom use skills decreased among participants with high initial scores in both the intervention and control groups, with a significantly greater decrease in the control group [0.7 vs. 1.9, t=4.25, df (269), p<.0001].

Figure 2.

Change in condom use skills through 36 months intervention period

Longitudinal trends of means scores for condom use self-efficacy are presented graphically for intervention and control groups in Figure 3. There was no significant difference in mean self-efficacy scores between the two groups at baseline. A greater increase was observed for the intervention group compared to the control group at the six-month follow-up [0.8 vs. 0.4, t= 4.66, df (1160), p<.0001]. Similar to the pattern seen in Figure 1 regarding the time trend for knowledge, after the six-month follow-up, the trajectory patterns were similar for the two groups, but the intervention group retained the upward displacement of its trajectory experienced in the first six months throughout the follow-up period. When stratified by initial self-efficacy scores, low and medium initial scorers in both the intervention and control groups showed the largest improvements, with significantly greater increases in the intervention group [2.5 vs. 2.1, t=3.25, df (350), p=.0013; 1.5 vs. 1.1, t=4.48, df (570), p<.0001]. Self-efficacy for high initial scorers in the intervention group remained almost constant throughout the intervention period, while it decreased for the control group [0.2 vs. −0.1, t=1.98, df (290), p=.0485].

Figure 3.

Change in condom use self-efficacy through 36 months intervention period

Figure 4 demonstrates the longitudinal trends of means scores for condom use intention for the intervention and control groups. Overall, condom use intention increased significantly among youth in both the intervention and control groups except that the mean intention score for the control group had a sharp decrease at the 18-month follow-up. The score increase in intention was not significantly different between the two groups. When stratified by initial intention score, low and medium initial scorers in both the intervention and control groups showed the largest improvements, with significantly greater increases in the intervention group [3.4 vs. 3.0, t= 2.38, df (406), p=.018; 1.4 vs. 0.8, t=3.16, df (289), p=.0018]. Mean scores of condom use intention decreased among participants with high initial scores in both the intervention and control groups.

Figure 4.

Change in intention to use condom through 36 months intervention period

2) Relationships among HIV/AIDS knowledge, condom use skills, condom use self-efficacy and condom use intention

Correlation analysis indicated that HIV/AIDS knowledge is strongly correlated with self-efficacy [χ²=25.38, df (1), p<.0001] and condom use skills [χ²=6.07, df (1), p<.0138]. Approximately 60% and 40% respectively of youth who had low scores in HIV/AIDS knowledge (≤10 points) also had low scores in condom use skills (≤6 points) and self-efficacy (≤1.5 points). In addition, condom use skills are strongly correlated with self-efficacy [χ²=11.77, df (1), p<.0006]. Forty-four percent of youth who had medium scores in condom use skills also had medium scores in self-efficacy. Further, self-efficacy is strongly correlated with condom use intention [χ²=83.47, df (1), p<.0001]. About 56% of youth who had low scores in self-efficacy also had low scores in condom use intention (≤2.0 points).

3) Predictive factors for unresponsiveness to FOYC intervention

Results of the hierarchical logistic regression analysis are presented in Table 3. The estimated coefficients for individual-level and school-level variables are listed under Models 1 through 4. The results of models 1–4 indicate that high initial scores in the four outcome measures were predictive of relative unresponsiveness to the intervention. The intervention was significantly related to improvement in the four outcome measures. Advanced age was associated with lack of improvement in HIV/AIDS knowledge. Female gender was predictive of improvement in HIV/AIDS knowledge but lack of improvement in self-efficacy. High academic self-evaluation was associated with lack of improvement in condom use intention.

Discussion

This paper presents results from a detailed intervention impact evaluation for overall and predefined population subgroups: low, medium, and high initial levels of HIV/AIDS knowledge, condom use skills, self-efficacy perceptions, and condom use intentions. There was great variation of intervention impact according to the subgroups examined. The intervention had a greater impact on each of the four outcomes among low and medium initial scorers, although the intervention also had a significant impact on HIV/AIDS knowledge and condom use skills among initial high scorers. The intervention had little impact on condom use self-efficacy and condom use intention among high initial scorers. Large increases in all four outcomes among low and medium initial scorers occurred in the immediate post-intervention period; subsequently, the intervention upward displacement of the trajectory was retained throughout the 36 months follow-up period.

There was some evidence that intervention impact on HIV/AIDS knowledge and condom use self-efficacy differed according to age group and gender. Intervention impact on knowledge decreased with age of the participant at the time of baseline survey. Older youth were less likely to improve their HIV/AIDS knowledge. A possible explanation is that many of the older youth (51.3% of those who are 11–13 years old) are grade six repeaters; as such, it is possible that they may be less interested or motivated by the grade six curriculum in general. Female youth were more likely to demonstrate improvement in HIV/AIDS knowledge but less likely to improve condom use self-efficacy compared to males. Surprisingly, youth with a relatively high academic self-evaluation were less likely to improve condom use intention. This may result from a lack of salience because their intentions to have sex were relatively low at baseline (i.e., 69% of youth with high academic self-evaluation described themselves as “very unlikely” to have sex in the next six months compared to 60% of youth with low academic self-evaluation).

The intervention was only effective on condom use intention among low and medium initial scorers. The lack of impact on condom use intention among initial high scorers may result from a “ceiling effect” for intention at baseline [20]. The sharp decreases in mean scores of condom use skills and intention among initial high scorers in both the intervention and control groups at the six-month post-intervention follow-up may reflect the effects of regression-to-the-mean when repeated measurements are made on the same subject [24]. The higher response rates among initial low-scorers have parallels in other fields. For example, Olds et al found that low income mothers under stress were more likely to respond to an early home intervention parenting program than were mothers without fewer domestic sources of stress [25, 26].

There are several potential limitations in this study. First, categorization of baseline knowledge, skills, self-efficacy, and intention (low, medium, and high) to define subgroups was arbitrary, even though the cutoffs we used 25th to 75th percentiles are frequently employed [27]. Further, we attempted to minimize potential classification bias by including overall longitudinal trajectories for both the intervention and control groups in comparisons to trajectories of subgroups. Second, subgroup analysis was not powered to see a difference in primary behavior outcomes (e.g., sexual behavior and condom use) as large portions of the participants were not sexually active. Third, condom use intention was assessed using a single item. More sensitive scales or multiple items could have resulted in greater reliability. Finally, these analyses were not included in our original hypotheses and thus should be regarded as hypothesis generating. Future studies are needed to further validate these research findings.

This study has confirmed that among youth overall, the FOYC intervention has a long-term impact on HIV/AIDS knowledge, condom use skills, condom use self-efficacy, and condom use intention, but the intervention effects differed by subgroups (age, sex, initial level of knowledge, skills, self-efficacy, and intention). Future research efforts should go beyond group analysis and focus on more complex analytical models which include potentially important subgroups. Gaining a better understanding of differing responses to HIV prevention interventions will allow us to modify these interventions to strengthen effectiveness. The finding that the intervention has the strongest impact on youth possibly at greater risk at baseline has important public health implications and is consistent with findings in other fields.

Table 1.

Results of hierarchical logistic regression analyses of predictive factors for unresponsiveness to FOYC HIV prevention intervention

Variables	Estimated models
	HIV/AIDS knowledge (1)			Condom use skills (2)			Self-efficacy (3)			Intention to use condom (4)
	β	SE	t	β	SE	t	β	SE	t	β	SE	t
Fixed effect
Intercept	−9.094	1.572	−5.79***	−4.957	1.540	−3.22**	−8.220	1.614	−5.09***	−4.056	1.466	−2.77*
Baseline knowledge/skills/self-efficacy/intention	0.325	0.035	9.37***	0.706	0.059	11.91***	1.213	0.086	14.03***	1.115	0.063	17.83***
Intervention (control=ref)	−1.034	0.232	−4.46***	−1.270	0.197	−6.44***	−0.812	0.193	−4.20***	−0.531	0.232	−2.29*
Age	0.389	0.123	3.17**	0.034	0.127	0.27	0.227	0.123	1.85	−0.070	0.120	−0.58
Gender
Female	−0.454	0.173	−2.62*	−0.104	0.170	−0.62	0.664	0.183	3.64***	−0.140	0.168	−0.83
Male (ref)
Academic self-evaluation
One of the best	−0.075	0.550	−0.14	−0.064	0.529	−0.12	1.108	0.811	1.37	1.773	0.700	2.53*
Above the middle	−0.194	0.552	−0.35	−0.073	0.530	−0.14	1.019	0.810	1.26	1.475	0.699	2.11*
Middle or below (ref)
Random effect
School†	0.034	0.069	0.31	0.008	0.041	0.05	-	-		0.078	0.072	2.54*
Class (nested within school)†	0.179	0.116	4.20*	-	-		0.063	0.099	0.49	-	-
Intra-class correlation	0.052			0.045			0.041			0.043

^*P<0.05;

^**P<0.01;

^***P<0.001.

^†Chi-Square.