HealthMpowerment.org: Feasibility and Acceptability of Delivering an Internet Intervention to Young Black Men Who have Sex with Men

Abstract

Young Black men who have sex with men (BMSM) are disproportionately affected by HIV/AIDS in the United States and continue to experience rapidly increasing HIV incidence. We designed a tailored, theory-based interactive HIV/STI prevention website for young BMSM, called HealthMpowerment.org (HMP) and conducted a small pilot trial comparing HMP to currently available HIV/STI websites. We present findings demonstrating feasibility and acceptability of delivering the intervention to the target population of young BMSM. Retention rates were 90% and 78% at one and three month follow-ups, respectively. Evaluation immediately after the intervention’s completion revealed that participants who used the HMP website reported high levels of user satisfaction and interest and low levels of website difficulty and frustration. At the end of the intervention, there was a trend in increased behavioral intentions to use condoms and engage in preparatory condom use behaviors in the intervention group compared to the control group (p=.10). We observed a reduction in mean scores on the CES-D scale among those in the intervention group that was not seen in the control group at the one-month follow-up, though this was not statistically significant. Feedback from exit interviews with study participants suggested that HMP is relevant to the prevention needs of young BMSM. Overall, the findings support the acceptability and feasibility of delivering this prevention program to a group that has few interventions despite bearing a significant burden of the epidemic. Future trials, combining internet and mobile phone technologies, are planned to test HMP among larger and more diverse populations of young BMSM.

Introduction

Young Black men who have sex with men (BMSM) are disproportionately affected by HIV/AIDS in the United States (US) and continue to experience a rapidly increasing HIV incidence (Hall et al., 2008; Prejean et al., 2011). Nevertheless, few interventions have focused specifically on the prevention needs of BMSM and none have been developed specifically for young BMSM (Johnson et al., 2009). Moreover, while reported rates of engaging in risky sexual behaviors among young BMSM is high, behaviors alone do not explain racial disparities in HIV infection rates – as BMSM do not report more risk behaviors than White MSM (CDC, 2002; Millett, Flores, Peterson, & Bakeman, 2007). The National HIV/AIDS Strategy specifically acknowledges the need to target prevention and education efforts towards high-risk groups, particularly BMSM (National HIV/AIDS Strategy for the United States, July 2010).

The internet offers the opportunity to reach BMSM who engage in risk but may be difficult to access by traditional means such as group or individual level venue based interventions (Elford, Bolding, Davis, Sherr, & Hart, 2004). Several studies confirm that both White MSM and MSM of color are receptive to internet-based data collection and interventions, such as chat room discussions, individual outreach activities, educational services and message board forums (Bolding, Davis, Sherr, Hart, & Elford, 2004; Bull, McFarlane, & King, 2001; Fernandez et al., 2004; Klausner, Levine, & Kent, 2004; Rhodes, 2004; Turner, 2001). BMSM routinely use the internet for both sexual and non-sexual communications partly because of stigma and privacy issues (Wilson et al., 2009).

Studies demonstrate equivalent outcomes from therapist and computer-delivered interventions for a range of health problems, including HIV (Feil, Noell, Lichtenstein, Boles, & McKay, 2003; Lenert et al., 2003; Napolitano et al., 2003; Newman, Kenardy, Herman, & Taylor, 1997; Sciamanna et al., 2002; Selmi, Klein, Greist, Sorrell, & Erdman, 1990; Tate, Jackvony, & Wing, 2003; Tate, Wing, & Winett, 2001; Woodruff, Edwards, Conway, & Elliott, 2001). Interventions that used tailoring (customizing the information to meet the unique needs and interests of a specific individual) (Kreuter, Farrell, Olevitch, & Brennan, 2000) and included more sessions were more efficacious (Noar, Black, & Pierce, 2009). The primary goal of the current study was to assess the feasibility and acceptability of delivering a novel, theory-based, tailored internet intervention developed for young BMSM (Hightow-Weidman et al., 2011).

Methods

Description of the intervention

The intervention, healthMpowerment.org (HMP), described in detail previously, (Hightow-Weidman, et al., 2011) was based on the IOM’s Integrated model of behavior theory that incorporates several theories of health behavior (Ajzen, 1985; Ajzen & Fishbein, 2004; Bandura, 1977, 2004; Fishbein & Ajzen, 2005; Rosenstock, Strecher, & Becker, 1994). We conducted formative work with the target population to develop and test the site prior to this study. Preliminary satisfaction, content acceptability, and usability findings supported the use of the internet to deliver risk reduction messages to young BMSM (Hightow-Weidman, et al., 2011). Key interactive features of the site include live chats with an HIV expert, interactive quizzes of varying levels of difficulty, personalized health and “hook-up/sex” journals, and decision support tools for assessing and modifying risk behaviors (See Figure 1 for selected screen shots).

Figure 1.

Selected Screen Shots from HealthMpowerment.org

Participants

Young BMSM were recruited from local flyers (n=8), HIV clinic providers (n=15), community outreach (n=13), internet web sites (n=6), and through word of mouth/friends (n=23) from September 2009 to January 2010. Men, 18–30 years old who had sex with another man in the last 12 months, and identified as Black/African American were eligible. Sixty-five men were screened for participation (6 were ineligible due to age, 1 did not have internet access and 8 did not attend the enrollment visit). Participants received $30 per survey for completing three surveys, $30 for completing weekly online diaries, and an additional $30 if they completed the entire study. The University of North Carolina at Chapel Hill Institutional Review Board approved this project.

Study Procedures

Participants were randomly assigned to either 1) the intervention website (HMP) or 2) a control group which was provided with a list of five websites that provide general HIV/STI information. The control group websites were selected to reflect currently available websites that young BMSM may learn about in their community, through friends, or via HIV or STI clinic providers. Subjects were asked to log onto the assigned website, spend at least 30 minutes on the site weekly for four weeks, and complete a weekly log to document the time they spent visiting the assigned website. Follow-up assessments were performed at one and three months. Exit interviews were conducted in-person at one-month follow-up to determine overall satisfaction with the site(s) and garner suggestions for modifications.

Measures

Demographics

Each participant completed a baseline questionnaire to obtain demographic information such as age, sexual orientation, education, and annual income. The questionnaire also assessed internet usage patterns including amount of time spent online, typical access platforms (laptop, mobile phone) and main reasons for use (e.g. to access health information, find social and sexual partners).

Sexual Behaviors

At baseline and each follow-up participants reported, by serostatus, their number of male and female sex partners in the preceding 30 days, the number of times they engaged in various sexual activities with their partner(s), by serostatus, and the number of times they engaged in each activity with and without a condom.

Intervention satisfaction

Participants’ overall reaction to the website was measured with four items on a five-point Likert-scale. Measures included participants’ ratings of the website(s) on scales of terrible/wonderful, difficult/easy, frustrating/satisfying and dull/stimulating. Twenty additional questions on a five-point Likert scale measured overall satisfaction with the site(s) and included items such as: “Overall, I am satisfied with how easy it is to use this website” and “I think I would use this website frequently” (alpha=0.94). Questions were adapted from the Computer System Usability Questionnaire and the Questionnaire for User Interface Satisfaction (Chin, Diehl, & Norman, 1988; Lewis, 1995).

Intentions

Behavioral intention items included 8 questions that assessed participants’ intentions to use condoms and engage in preparatory condom use behaviors (such as buying and carrying condoms) as well as to discuss HIV status with partners (alpha=0.83). A sample question is: “In the next 3 months how likely are you to tell your partner that you need to use a condom?” Questions were designed specifically for this study.

Attitudes

Attitudes and extent of motivation for engaging in safer sexual behaviors in the next three months were assessed with four items rated on a four-point Likert-type scale (alpha=0.68). A sample question is “In the next three months, how motivated or unmotivated do you feel about having safer sex?” Items were adapted from the information, motivation and behavioral skills model of AIDS risk behavior change (Fisher & Fisher, 1992; Fisher, Fisher, Bryan, & Misovich, 2002).

Condom use self-efficacy

Self-efficacy was assessed with 27 questions that asked participants how confident they felt using condoms and rated on a five-point Likert-type scale. A sample question is: “I feel confident in my ability to discuss condom usage with any partner I might have” (alpha=0.82). These questions were adapted from the validated condom use self-efficacy scale (alpha=0.85) (Brafford & Beck, 1991).

HIV/AIDS Knowledge

We used the HIV/AIDS Knowledge Questionnaire to assess knowledge needed for HIV prevention (Carey, Morrison-Beedy, & Johnson, 1997). All of the 18 questions are answered, “true”, “false” or “do not know.” A single summary score is obtained by summing the number of items correctly answered (“don’t know” responses are scored as incorrect) (alpha=0.88).

Depression

Depressive symptoms were assessed with the Center for Epidemiologic Studies Depression Scale (CES-D), a 20-item validated survey of clinically significant distress as a marker for clinical depression (alpha=0.89) (Radloff, 1977). Respondents indicated the frequency of each symptom over the past week on a four-point Likert-type scale. A CES-D score of ≥16 suggests a clinically significant amount of depressive symptoms.

Statistical Analysis

Initial comparisons between the intervention and control groups were done using simple bivariate analysis, using the chi-square test statistic for categorical variables and the t-statistic for continuous variables. We also compared website satisfaction between the intervention and control groups at one month using the t-statistic. Analysis of change in study variables across all three data collection points was done using hierarchal linear modeling (HLM) where the study variable of interest was modeled as a function of both time and intervention status. The a priori critical alpha level was set at alpha=.10 due to the preliminary nature of the analysis and relatively small sample size. Analyses were conducted using SAS software, version 9.2 (SAS Institute Inc., Cary, NC).

Results

Participant Characteristics

We enrolled 50 young (age 18–30), BMSM and randomized them to either HMP (n=25) or the control websites (n=25). Participants completed their one-month follow-up evaluations in person; retention was 90% (22 in the intervention group and 23 in the control group). Table 1 outlines baseline demographic data and measures of internet use. The mean age of participants was 23.7 years, 48% were currently enrolled in school, and 60% earned less than $21,000 in the previous year. Sixty-three percent identified as gay and 31% as bisexual. There were significantly more HIV-infected participants enrolled in the control group than the intervention arm (p=.02). Three-month follow-up evaluations were conducted online; 39 (78%) participants completed surveys.

Table 1.

Baseline Characteristics of Young Black Men who have Sex with Men in HealthMpowerment Feasibility Study

Variable	Intervention N=25	Control N=25	p-value
Age
Median (range)	25 (19–30)	23 (19–30)	.24
Sexual Identity			.11
Gay	18 (72%)	13 (54%)
Bisexual	5 (20%)	10 (42%)
Other	2 (8%)	0
Income last year			.38
<10,999	12 (48%)	6 (24%)
11,000–20,999	4 (16%)	8 (32%)
21,000–30,999	4 (16%)	3 (12%)
≥31,000	4 (16%)	4 (16%)
Refuse	1 (4%)	4 (16%)
Education status			.82
Some high school	2 (8%)	3 (12%)
High school or GED	5 (20%)	4 (16%)
Some college	12 (48%)	10 (40%)
College	3 (12%)	5 (20%)
More than college	3 (12%)	3 (12%)
Currently enrolled in school	14 (56%)	10 (40%)	.26
HIV-infected	6 (27%)	15 (63%)	.02
Number of male partners by serostatus: mean [SD]
HIV+	0.36 [0.49]	0.88 [2.03]	.25
HIV−	2.59 [2.38]	1.13 [1.70]	.02
Unknown	1.14 [1.42]	2.08 [2.59]	.13
Alcohol use			.49
None	4 (16%)	3 (12%)
Once a month or less	8 (32%)	5 (20%)
>Once a month but < once a week	2 (8%)	5 (20%)
One or more times per week	11 (44%)	11 (44%)
Every day	0	1 (4%)
Drug use (not alcohol)	13 (52%)	12 (48%)	>.99
Internet Usage
How often go online			.22
≥Once a day	21 (84%)	22 (88%)
Weekly	4 (16%)	3 (12%)
Hours/day spend online
Median (range)	5 (0–13)	4 (0–20)	.30
Device used most to access the Internet			.18
Laptop	9 (36%)	14 (56%)
Desktop	11 (44%)	5 (20%)
Mobile phone	5 (20%)	6 (24%)
Where access the Internet			.24
Home	15 (60%)	16 (64%)
Work/School	6 (24%)	2 (8%)
Other	4 (16%)	7 (28%)
Used the Internet to find sex partners	15 (60%)	19 (76%)	.23

Intervention Feasibility and Satisfaction

Evaluation immediately after the intervention’s completion revealed that participants who used the HMP website reported high levels of user satisfaction and interest and low levels of website difficulty and frustration (Table 2). There was no difference between the intervention and control groups between reported number of log-ins or time spent using the site(s). Participants logged on to HMP most often on Mondays and Thursdays; they most frequently visited the site between 4pm and 9pm. Less than 0.4% of user log-ins resulted in a server related error. The measured average user visit duration for intervention participants was 19 minutes. On average, participants reported spending 30 minutes to an hour on the website(s) every week, which did not significantly differ by intervention condition (Table 2).

Table 2.

Satisfaction and Reported Use of HealthMpowerment.org and Control Websites

Variable	Intervention Mean Scores [SD] N (%)	Control Mean Scores [SD] N (%)	p-value
Overall reaction to the website
Terrible----wonderful	4.45 [0.67]	3.91 [0.85]	.02
Difficult----easy	4.27 [1.03]	3.91 [1.08]	.26
Frustrating----satisfying	4.23 [1.11]	4.22 [0.95]	.97
Dull-----stimulating	4.18 [1.05]	3.83 [1.30]	.33
Average Number of Logins per user	6.16 [4.69] range 1–19	4.60 [3.06] range 1–13	.17
Completed all 4 journal entries	16 (64%)	13 (52%)	.26
Reported Use of Site by Participants
Login times per week			.17
1–2	15 (68%)	12 (52%)
3–4	7 (32%)	8 (35%)
>5	0	3 (13%)
Average time spent on site/week			.20
<30 min	8 (36%)	3 (13%)
30 min-1 hr	7 (32%)	11 (48%)
> 1 hr	7 (32%)	9 (39%)
Overall Website Satisfaction	4.11 [0.61]	3.98 [0.67]	.46

Secondary Efficacy Outcomes

At the end of the intervention, trend in greater behavioral intentions to use condoms and engage in preparatory condom use behaviors in the intervention group (p=.10) was observed. There was a reduction in mean scores on the CES-D scale among those in the intervention group that was not seen in the control group at the one-month follow-up, though this was not statistically significant (Table 3). There were no differences between the intervention and control groups in HIV-related knowledge, levels of condom use self-efficacy, or attitudes toward engaging in safer sex at both one and three months follow-up

Table 3.

Changes in Secondary Outcomes among Participants at One and Three-Months Post-Intervention

	Baseline		1 month			3 months
Variable	Control	Intervention	Control	Intervention	p value	Control	Intervention	p value
Condom percent*	69.0%	56.5%	75.7%	78.0%	.89	81.5%	64.7%	.40
Behaviorl intentions	3.35	3.45	3.49	3.55	.61	3.40	3.63	.10
Attitudes	3.27	3.38	3.55	3.61	.66	3.41	3.62	.23
Self-efficacy	4.57	4.44	4.45	4.48	.24	4.49	4.49	.23
Knowledge	79.1%	84.0%	86.2%	83.6%	.55	88.3%	82.55%	.34
CES-D	13.29	15.26	13.06	11.26	.50	16.00	13.21	.42

^*Condom percent includes only subjects who reported anal sex in last three-months (34/50 at baseline and 17/39 at three-months), and excludes those reporting positive-concordant sex. Higher scores indicate more positive outcomes except for CES-D.

Sexual behaviors

Self-reported condom use significantly increased (p=.05) among all participants from baseline (average=63.13%) to one-month follow-up (average=76.68%); this increase was not related to the intervention condition (Table 3). A decrease in the number of male sex partners in the last three months over time for all participants was observed, with an average of 3.49 at baseline and 2.23 at the three-month follow-up (p=.03; Table 4). Whether or not a participant was in the intervention or control had no effect on either of these changes.

Table 4.

Number of Reported Sex Partners in the Last Three-Months Among All Participants at Baseline, One and Three-Months

Number of sex partners	Baseline	1 month follow-up	3 month follow-up
Zero	7 (14.0%)	13 (28.9%)	13 (33.3%)
One to three	22 (44.0%)	24 (53.3%)	19 (48.7%)
Four or more	21 (42.0%)	8 (17.8%)	7 (18.0%)

Retention

Forty-five participants completed the one-month follow-up (22 from the intervention and 22 from the control). Thirty-nine completed the three-month follow-up (21 from the intervention and 18 from the control). Between baseline and the three-month follow-up, 16% of the intervention group and 28% of the control group were lost to follow-up; this difference was not statistically significant. However, while only 13% of HIV-negative participants were lost to follow-up, 33% of HIV-positive participants did not complete the three-month assessment (p=.10).

Exit Interviews

Users expressed a high level of approval for HMP and provided constructive advice for refining and improving the intervention. Overall, users felt the site was useful, relevant and filled a particular void in the prevention needs of young BMSM. Young men appreciated the website’s attention to overall health and wellness and not just HIV/STI prevention.

While participants randomized to the control websites did not report differences from HMP participants in quantitative measures of website satisfaction, important distinctions emerged during their exit interviews. In general, control participants were mixed in their reviews regarding their overall impressions of both the tone and content of the control websites. A substantial number of youth noted that the control websites seemed to provide information directed at a white, heterosexual population and/or conveyed the information in an overly moralistic fashion.

One feature that all participants identified as missing was a community networking function that would allow users to post responses, leave comments, chat, contribute to a group blog or provide “testimonials”. Additionally, many requested more interactive pieces as well as the ability to access the website using a mobile device.

Discussion

The primary goal of the current study was to assess the feasibility and acceptability of delivering a novel, theory-based and tailored internet intervention to young BMSM. HMP is the first online intervention developed by and for young BMSM. Though prior research has focused on both HIV infected and uninfected youth (Lightfoot, Comulada, & Stover, 2007; Lightfoot, Rotheram-Borus, & Tevendale, 2007) and BMSM (Coury-Doniger, Wilton, Herbst, & Painter; Wilton et al., 2009) few interventions that are culturally sensitive to BMSM’s HIV prevention and treatment needs have been developed. In the current study, we were able to enroll 50 BMSM and retain 90% at one-month follow-up and 78% at three months– one of the highest retention rates seen in online prevention studies. Prior HIV internet studies have had only limited success in enrolling and retaining minority populations (Bowen, Horvath, & Williams, 2007; Bull, Lloyd, Rietmeijer, & McFarlane, 2004; Bull, Vallejos, Levine, & Ortiz, 2008; Carpenter, Stoner, Mikko, Dhanak, & Parsons, 2009). We believe we achieved this level of retention by enrolling subjects in-person thus building initial rapport, and by offering a retention bonus for those who completed all surveys (Bull, et al., 2004; Bull, et al., 2008). The financial costs, both in monies spent and personnel time devoted needs to be considered when designing retention strategies for future online interventions.

Overall users found the intervention acceptable and expressed high levels of satisfaction. Exit interviews suggested that participants had limited knowledge about the availability of websites providing information on HIV and other STIs specifically for young BMSM. There was overwhelming support for further development and dissemination of HMP. Comments from participants in the control arm provided us with additional important feedback regarding continued refinement of HMP. While many people, including adolescents, Blacks and those with HIV access and trust information they obtain online, many websites are not credible, leaving these young men vulnerable to half-truths and misinformation (Benotsch, Kalichman, & Weinhardt, 2004; Borzekowski & Rickert, 2001; Kalichman et al., 2003). Since 80% of the US population regularly use the internet, including 56% of all Blacks, 61% with an income less than $30,000 and 38% of those with no high school diploma, the creation of a factually-based website can fill a significant void in prevention needs for young BMSM by providing up to date, accurate and relevant information.

We did see a difference between intervention and control participants, with the former expressing a trend toward greater intentions to use condoms and to engage in preparatory condom use behaviors. Because the intervention was theory-based, other variables such as knowledge, attitudes, and self-efficacy were assessed, though these variables either did not change or did so equally between the intervention and control groups. While intentions may not always coincide with actual behaviors, we interpret the change in behavioral intention as a necessary, even if not entirely sufficient, precursor to changes in behavior. In the context of this relatively short feasibility study it is a reasonable indirect measure of behavior change (Albarracin, Johnson, Fishbein, & Muellerleile, 2001; Bandura, 2001; Fishbein, 1980). In general, HIV sexual behavior risks showed positive changes over the course of the project regardless of participation in either the intervention or control group. The beneficial effects seen in the control condition could have resulted from the provision of HIV/STI information through the control websites or could have been due to study participation itself. This suggests that even though we saw greater changes in behavioral intention among the intervention group, the change may not have operated through the theoretically predicted avenues of knowledge, attitudes, or self-efficacy or it could be that no change in these constructs were identified because both groups were exposed to similar HIV prevention information and experienced similar changes. Future studies with more participants should explicitly test the pathway through which any behavioral changes are produced in this and other online interventions.

Despite our randomization procedures, we were surprised to have enrolled more HIV-infected subjects in the control group. Due to our low sample size we were not able to account for this in our analysis (Cohen, Cohen, West, & Aiken, 2003). HIV status was also assessed through self-report, raising the possibility that individuals who self-reported as HIV-negative did not disclose HIV infection because of fear, stigma, or a lack of knowledge about their current HIV status. Furthermore, if we assume that the effect of the intervention varied by HIV status this raises the possibility that we could have either failed to detect effects where they truly existed, or might have attributed effects to the intervention that were in reality a function of HIV infection. Moreover, many of the men in our sample had not had any recent episodes of anal sex or were HIV-infected men engaging in seroconcordant sex, thus limiting our ability to detect any meaningful changes in condom use over the study period or to explain the decrease in condom use behaviors at three-months in the intervention group. Future studies should account for differential effects due to HIV status both through statistical controls and a stratified randomized design to ensure equal representation by HIV status in both study conditions.

While our three-month retention rate is comparatively high (78%), we did find some differences both by intervention group and participant HIV-status. Sixteen percent of the intervention group dropped out, as opposed to 28% of the control group (p=.31), and 13% of HIV-negative participants dropped out compared to 33% of HIV-positive men (p=.10). This differential retention rate may indicate a need for more content about secondary prevention explicitly aimed at HIV-positive young BMSM.

This study is not without limitations. We did rely on self-reports of sexual risk behavior, which are open to recall bias and socially desirable responding. However, having participants complete surveys online offers relative anonymity which may increase the authenticity of participant responses and limit this bias (Houston & Fiore, 1998; Pealer, Weiler, Pigg, Miller, & Dorman, 2001; Zabinski et al., 2001). In prior studies, subjects were found to be more truthful about HIV-risk factors when reporting to a computer compared to a clinician (Locke et al., 1992). While we are unable to verify usage on control sites we found no statistical differences in the number of log-ins or reported time spent online between the control and intervention groups. Additionally, our control condition, while serving as a comparison to the intervention, may not have been as controlled as we would have liked. Unlike the HMP website, participants in the control condition may have visited different combinations of websites to different extents, and the assumption that they were a uniform group may not be a valid one. In addition, we only recruited a convenience sample of young BMSM who had internet access and responded to our advertisements thus limiting our ability to know the characteristics of those declining and the generalizability of our findings. Next steps include developing and testing methods and intensified core components of the intervention to strengthen long-term effects and to assess the effectiveness of HMP on a more geographically diverse population.

Overall, the findings support the acceptability and feasibility of delivering this prevention program to a group that has few interventions despite bearing a significant burden of the epidemic. Tailored online interventions for this target group are long overdue, making this a very timely study. Further, we believe that future interventions for young BMSM should combine internet and mobile phone technologies. Blacks are the most active users of the mobile internet – and their use of it is also growing the fastest ("Wireless Internet Use," Pew Internet and American Life Project). This phenomenon serves to decrease the Black-White digital divide. Half of Blacks with cell phones, on a typical day, use at least one of 10 non-voice data applications, such as texting. Mobile access could give users immediate access to interventions, which would likely increase interventions’ relevance, effectiveness and use. Moreover, use of technology that is already fully integrated into people’s lives, would facilitate delivery and dissemination of interventions – as well as consistent support for motivation for behavior change and instant feedback and support.