Abstract
HealthMpowerment.org (HMP), is a mobile optimized, online intervention to reduce sexual risk behaviors among HIV-positive and HIV-negative young Black men who have sex with men (BMSM) by providing information and resources, fostering social support, and including game-based elements. A randomized controlled trial with 474 young BMSM compared HMP to an information-only control website. The rate of self-reported condomless anal intercourse (CAI) at 3-months was 32% lower in the intervention group compared to the control group (IRR 0.68, 95% CI: 0.43, 0.93), however this effect was not sustained at 12 months. Among HIV-positive participants, the rate of CAI at 3-month follow-up was 82% lower among participants with detectable viral loads in the intervention group compared to the control group (IRR 0.18, 95% CI: 0.04, 0.32). In a secondary analysis, when we limited to those who used HMP for over 60 minutes during the 3-month intervention period (n=50, 25.8%), we estimated 4.85 (95% CI: 2.15, 7.53) fewer CAI events than we would have expected in control participants, had they used the intervention at the same rate as the intervention group. Findings suggest that exposure to an online intervention can reduce the rate of CAI among young BMSM, at least in the short term. Given the stronger effect seen among those participants who complied with HMP, additional intervention engagement strategies are warranted.
INTRODUCTION
Young black men who have sex with men (BMSM) bear a disproportionate burden of the HIV epidemic in the United States (US). BMSM have a one-in-two chance of becoming infected in their lifetime compared with one in five among Hispanic/Latino MSM; both of which are higher than risk among white MSM (1 in 11) (1). Modeling studies estimate that based on annual incidence of 4.2% per year, 40% of BMSM would be HIV-positive by age 30, and 61% by age 40 (2). While rates of condomless anal intercourse (CAI) do not explain the disparity (3–6), CAI remains the primary means of HIV acquisition and spread of other sexually transmitted infections (STIs). To date, few efficacious interventions that address engagement in condomless sex exist for young BMSM.
A 2013 systematic review to identify effective HIV prevention interventions specifically designed for adult BMSM (7), identified five randomized, controlled efficacy interventions (RCTs) aimed at reducing HIV sexual risk behaviors. All were delivered in group settings, and none were tailored to address the unique contexts of HIV risk among young BMSM (e.g. the mean age of participants from these five studies ranged from 30-51 years of age). Only two of these interventions were found to reduce HIV sexual risk behavior (8, 9). Wilton, et al. reported reductions in CAI with casual partners after receipt of a six session group intervention (3MV) compared to a delayed control group (8) and Peterson, et al. demonstrated the superiority of a triple session over a single session intervention compared to a wait-list control group in reducing CAI out to 18 months (9). An additional four RCTs (10–13) that enrolled only BMSM were identified in a 2016 systematic review of effective interventions for black and minority ethnic men who have sex with men (14). While, none of these studies were efficacious at increasing condom use with male partners, a three-session, one-on-one counseling-based intervention for BMSM (mean age 41.6 years) did find a reduction in the frequency of anal receptive sex (11). An additional study of a culturally tailored HIV prevention intervention (Unity in Diversity) consisting of six group-based sessions and one individual session did show efficacy in increasing the odds of zero male sex partners. (12).
The efficacy of these studies notwithstanding, the group-based and multiple session designs of the in-person interventions, limit scalability and dissemination. Technology-based interventions, delivered online, allow for intervention delivery with high fidelity and can be more easily scaled if found efficacious (15, 16). While previous research indicates the efficacy of computer-based interventions in reducing HIV related sexual risks, these interventions have not been developed specifically for young BMSM (17). To that end, an interactive theory-based website, HealthMpowerment.org (HMP), was developed and tailored to be relevant to HIV prevention issues currently faced by young BMSM.
HMP was developed based on extensive formative work with the target population to address the unique developmental and cultural needs of young BMSM (18–20). A pilot study showed intervention feasibility and acceptability (21–23). This paper describes the results of a two-group randomized trial that was conducted to compare the efficacy of the HMP intervention to an information-only control condition. The primary outcome was the relative rate of self-reported CAI in the past 3 months in those randomized to the intervention group compared to those randomized to the control group at 3 months post-randomization, following intention-to-treat (ITT) principles.
METHODS
Intervention Description
HMP is a mobile phone and internet-based HIV intervention designed to increase safer sex behaviors among HIV-positive and HIV-negative young BMSM (aged 18-30 years). HMP was created based on the Integrated Behavioral Model (IBM) (18, 24). A knowledge library included 322 articles on HIV/STI prevention, dating, personal growth and health and wellness. Interactive features within the intervention include: forums for discussion of relevant topics, such as HIV prevention and treatment, dealing with stigma, safer sex behaviors and relationships (The Forum); a space to upload and share personal videos, audio, pictures or prose (Getting Real); the availability of an online doctor who would respond to questions within 72-hours (Ask Dr. W); and decision support tools (e.g. brief surveys, with feedback and referral to prevention services based on results, HIV testing and care locators) for assessing and addressing risk behaviors. A control website was developed that included 110 culturally tailored articles focused on HIV and STI prevention, a subset of the 322 articles provided to the intervention participants.
Sample
An RCT comparing the HMP intervention to the information-only control website enrolled 474 young BMSM between November 2013 and October 2015. Participants completed a computer-assisted self-interviewing (CASI) baseline survey at an in-person enrollment visit and follow-up survey assessments were conducted online at 3, 6 and 12-months post enrollment. Both the intervention and control conditions were 3 months. At the baseline enrollment visit, participants were given a tour of their assigned website (HMP or control) and told to use the site at their discretion for 3 months. However, access was allowed for the full 12-months of trial participation.
Recruitment, Enrollment and Retention
Participants were recruited from 11/13/2013-10/5/2015 through social media websites (n=124; 26.2%), venue and community-based flyers and palm cards (n=52; 11.0%), healthcare-based settings including HIV/STI clinics, case management organizations (n=167; 35.2%), and through friends or word of mouth (n=131; 27.6%). Individuals meeting eligibility criteria based on a phone screener were invited to attend an in-person enrollment visit. Those who provided informed consent were enrolled in the study. Upon enrollment, a CASI baseline survey using Qualtrics software (Qualtrics, Provo, UT) was completed. After participants completed the survey they were randomized in a 1:1 equal allocation into the HMP intervention or control group. Follow-up surveys were conducted online using Qualtrics. Participants were compensated for survey completion at each time point ($50 at baseline and 3-months, $30 at 6- and 12 months); a retention bonus ($50) was awarded to those who completed all follow-up time point surveys (3, 6 and 12-month). Study procedures were approved by [redacted for anonymity] Institutional Review Board.
Eligibility Criteria
Study eligibility criteria at enrollment were: (i) age 18 to 30; (ii) born biologically male; (iii) self-identify as Black; (iv) currently reside in North Carolina; (v) currently have access to a mobile device (e.g. smartphone, tablet) that connects to the internet and has texting capabilities; and (vi) any of the following in the past six months: (a) CAI with a male partner, (b) any anal sex with more than three male sex partners, (c) exchange of money, gifts, shelter, or drugs for anal sex with a male partner, or (d) anal sex while under the influence of drugs or alcohol (i.e., high or drunk within two hours of sex).
Measures
Sociodemographic items
assessed age, education, income, arrests in the past 3 months, health insurance, HIV-status and sexual identity. Homelessness was assessed by asking if in the past 6 months, there had been any period of one or more nights when homeless.
Sexual Risk Behaviors:
At baseline and each follow-up survey, the number of acts of CAI in the preceding 3 months was derived from each participant’s responses to the questions in the sexual risk domain of the survey. Participants were first asked for the total number of male partners they had anal sex with in the past 3 months. They then were asked to enumerate how many of these partners were HIV-positive, negative or unknown HIV status. They then enumerated the number of times they had insertive (with and without a condom) and receptive (with and without a condom) anal sex with their HIV-positive, negative or unknown status partners. The total number of acts of anal intercourse was calculated as the sum of the number of self-reported instances of receptive or insertive anal sex with a male partner; the number of condomless acts was then derived as this value subtracted by the number of times the participant reported using a condom. The number of acts of serodiscordant CAI was determined similarly, but restricted to those instances of anal sex where the partner was reported as having a different HIV status than the participant.
Detectable Viral Load:
Baseline and follow-up surveys asked HIV-positive participants about whether they had their viral load tested in the prior three months. If the participant answered yes, they were asked if their viral load was undetectable.
Psychosocial variables
Depressive symptomatology
Depressive symptoms were assessed with the Center for Epidemiologic Studies Depression Scale (CES-D), a 20-item validated survey of clinically significant distress as a marker for clinical depression (alpha=0.90) (25). Respondents indicated the frequency of each symptom over the past week on a 4-point Likert scale. A dichotomous depression variable was created using a CES-D score of >16, which suggests clinically relevant depressive symptomology.
Social support
The Medical Outcomes Study Social Support Survey (MOS-SSS) was used to measure perceived social support (26). An overall support index (alpha = 0.98) and subscales for emotional/informational (alpha = 0.97), tangible (alpha = 0.92), affectionate (alpha = 0.94), and positive social interaction (alpha = 0.96) subscales were calculated. Higher scores indicate greater perceived social support.
Social isolation
Social isolation was derived from the 6-item version of the Lubben Social Network Scale (alpha=0.83) (27). The scale assesses social network size and the ease and frequency of contact with network members (range=0-30) with a higher score indicating a lower likelihood of social isolation. Individuals with a score <12 were considered socially isolated.
Substance use
Substance use items assessed any use of alcohol, marijuana or synthetic marijuana, crack or powder cocaine, methamphetamine, club drugs (e.g., ecstasy, ketamine, Molly, GHB), opiates and inhalants in the past 3 months.
All measures (except age) were asked at the baseline and all follow-up surveys.
Analysis
Baseline descriptive characteristics were calculated to summarize sociodemographic variables, sexual behaviors and drug use, HIV status and select psychosocial variables among men in the two conditions.
The primary effect measure of interest was the relative rate of self-reported CAI in the past 3 months in those randomized to the intervention group compared to those randomized to the control group at 3 months post-randomization, following intention-to-treat (ITT) principles. Durability effects were determined by examining the relative rate of CAI at 6 and 12 months post-randomization. A secondary analysis was conducted to determine the rate of serodiscordant CAI at 3, 6, and 12 months. In addition, we investigated whether the effect of the intervention on rates of CAI at 3 months post-randomization was moderated by viral load detectability. A cut-off value of 200 CAI events was selected for all models, which resulted in exclusion of two individuals from the analysis. Furthermore, due to the extreme outliers in the reported number of CAI events, a sensitivity analysis was performed using different outlier-cutoffs.
A generalized linear mixed modelling framework was used to estimate the effect of the HMP intervention on the change in the rate of CAI from baseline. Since the primary outcome was a counting process, diagnostics were performed on the baseline count data to select the most parsimonious probability distribution, which was determined to be marginalized zero-inflated Poisson (mZIP) (28). A random intercept term was introduced to account for correlation between repeated measures on each subject. Since the model was fitted without an offset term, rate is defined with respect to the absolute number of CAI episodes an individual engaged in over the 3 month study period. The parameter of primary interest was the multiplicative interaction between study group (binary) and the 3-month time point (binary), which estimates the relative change from baseline to month 3 in the incidence rate ratio comparing the two groups. A separate model was also fitted to the data and included a three-way interaction between study group (binary), time (categorical) and baseline detectable viral load (binary), which estimates the difference in the change in the rate of CAI between the two groups from baseline. In a secondary analysis, the same models were fitted to data with acts of condomless serodiscordant anal intercourse as the outcome. For all models, an inverse probability weighting (IPW) approach was used to account for loss to follow-up at each time point. Briefly, the probability of loss to follow-up at each time point was estimated using a logistic regression model, conditioning on measured covariates. The inverse of this probability was then used to weight the sample for non-response.
To further investigate the efficacy of the HMP intervention among only those participants who complied with the intervention, we estimated the compiler averaged causal effect (CACE) (29); defined as the difference between the CAI rates of individuals who used the intervention and individuals in the control group who would have used the intervention if they had been exposed to it, i.e., the counterfactual. Because subjects were randomized in this study, the counterfactual framework is reasonable, and under certain regularity assumptions, this is equivalent to adjusting the difference between the CAI rates of the intervention and control groups by the proportion of compilers in the intervention group. Specifically, to estimate the CACE, we calculated the difference in rates of CAI between the intervention and the control group and then divided this by the proportion of subjects that were deemded ‘users’ in the intervention group. The CACE was calculated using a threshold of 60 minutes spent on the site to classify individuals as users or non-users. As a sensitivity analysis, the time-on-site threshold was varied over a wide range of values to evaluate how much the CACE changed as a function of the threshold. The demographic characteristics of users and non-users were compared using appropriate univariate statistical tests (i.e. Kruskal-Wallis rank-sum tests for continuous variables and Chi-square tests for categorical variables) to determine whether there were identifiable subgroups of individuals with greater propensity to engage with the intervention.
RESULTS
Description of sample
The baseline characteristics of the study population by intervention group are shown in Table 1. Participants had a mean age of 24.33 (SD=3.22). Most identified as gay (66.7%) and 42.0% (n=199) were HIV-positive at baseline. The majority had a high school degree or less (75.8%) and an income under $11,000 (53.0%). The control group tended to have more individuals recruited from online sources (30.5% vs. 21.9% for the intervention arm) and fewer from healthcare-based venues (31.8% vs. 38.7% in the intervention arm). Overall retention was 85.2%, 80.2% and 78.3% at 3, 6 and 12 months, respectively (Figure 1). Forty-four (18.5%) and 26 (11.0%) study participants did not complete 3-month follow-up surveys in the intervention and control groups, respectively. Irrespective of randomization status, individuals who did not complete 3-month follow-up had lower social support scores and were less likely to have health insurance than individuals who were retained. The mean (SD) social support score among the 70 individuals who did not complete 3-month follow-up was 65.5 (29.4), versus 73.1 (26.9) among the 404 retained (Kruskal-Wallis p-value 0.03). Among participants not retained, 43 (61.4%) had health insurance, while 296 (73.3%) of retained individuals had health insurance (Chi-square p-value 0.04).
Table 1.
Baseline characteristics of study population by intervention group
| Total Sample (N=474) | Intervention (N=238) | Control (N=236) | |
|---|---|---|---|
| Age | |||
| Mean (SD) | 24.33 (3.22) | 24.30 (3.15) | 24.37 (3.29) |
| Education | |||
| < High school | 43 (9.1%) | 28 (11.8%) | 15 (6.4%) |
| High school/GED, some technical/college | 316 (66.7%) | 141 (59.2%) | 175 (74.2%) |
| College degree or more | 115 (24.2%) | 69 (29.0%) | 46 (19.5%) |
| Income | |||
| Less than $10,999 | 248 (53.0%) | 120 (50.6%) | 128 (55.4%) |
| $11,000 - $20,999 | 87 (18.6%) | 46 (19.4%) | 41 (17.7%) |
| $21,000 - $30,999 | 68 (14.5%) | 36 (15.2%) | 32 (13.9%) |
| More than $31,000 | 65 (13.9%) | 35 (14.8%) | 30 (13.0%) |
| Health Insurance | 339 (71.5%) | 173 (72.7%) | 166 (70.3%) |
| Sexual Identity | |||
| Gay | 316 (66.7%) | 152 (63.9%) | 164 (69.5%) |
| Bisexual | 95 (20.0%) | 49 (20.6%) | 46 (19.5%) |
| Other | 63 (13.3%) | 37 (15.5%) | 26 (11.0%) |
| HIV-Status | |||
| HIV-positive | 199 (42.0%) | 107 (45.0%) | 92 (39.0%) |
| HIV-Negative / Unknown | 275 (58.0%) | 131 (55.0%) | 144 (61.0%) |
| Viral Load | |||
| Detectable | 57 (12.0%) | 28 (11.8%) | 29 (12.3%) |
| Undetectable | 105 (22.2%) | 56 (23.5%) | 49 (20.8%) |
| Arrested (last 3 months) | 28 (5.9%) | 18 (7.6%) | 10 (4.2%) |
| Homeless (last 6 months) | 104 (21.9%) | 59 (24.8%) | 45 (19.1%) |
| Depressive symptoms (last 3 months) | 232 (49.7%) | 115 (49.1%) | 117 (50.2%) |
| Social Support (Standardized Score) | |||
| Mean (SD) | 71.98 (27.34) | 72.10 (27.10) | 71.85 (27.63) |
| Social Isolation | 144 (30.4%) | 70 (29.5%) | 74 (31.4%) |
| # anal sex acts (past 3 months) | |||
| Mean (SD) | 11.84 (30.88) | 12.65 (31.97) | 11.04 (29.79) |
| Median (IQR) | 5.00 (9.00) | 5.00 (9.00) | 5.00 (10.00) |
| # condomless anal sex acts (past 3 months) | |||
| Mean (SD) | 6.11 (24.76) | 5.74 (19.75) | 6.48 (28.97) |
| Median (IQR) | 1.00 (5.00) | 1.00 (4.00) | 2.00 (5.00) |
| # serodiscordant anal sex acts | |||
| Mean (SD) | 4.51 (10.28) | 5.27 (12.29) | 3.73 (7.70) |
| Median (IQR) | 1.00 (4.00) | 1.00 (4.00) | 1.00 (4.00) |
| # condomless serodiscordant anal sex acts | |||
| Mean (SD) | 1.73 (5.68) | 1.68 (5.32) | 1.78 (6.02) |
| Median (IQR) | 0.00 (1.00) | 0.00 (1.00) | 0.00 (1.00) |
| Substance use (last 3 months) | |||
| Alcohol | 394 (83.3%) | 202 (85.2%) | 192 (81.4%) |
| Marijuana | 291 (61.4%) | 149 (62.6%) | 142 (60.2%) |
| Other drugs | 108 (22.8%) | 64 (26.9%) | 44 (18.6%) |
| None | 45 (9.5%) | 17 (7.1%) | 28 (11.9%) |
| Recruitment Source | |||
| Friend/Word of mouth | 131 (27.6%) | 64 (26.9%) | 67 (28.4%) |
| Healthcare-based settings | 167 (35.2%) | 92 (37.7%) | 75 (31.8%) |
| Venue and community recruitment | 52 (11.0%) | 30 (12.6%) | 22 (9.3%) |
| Social media | 124 (26.2%) | 52 (21.9%) | 72 (30.5%) |
Numbers may not sum to column totals due to missing values for certain variables.
Figure 1:
HealthMpowerment Consort Diagram
Usage of HMP
The mean total time spent on HMP (n=194) and the control site (n=210) was 102.6 minutes [SD 224.97] and 23.90 minutes [SD 61.87], respectively. The median number of total log-ins was 10.21 (range 1-222) on HMP and 2.79 (range 1-572) on the control site. Overall, 40.7% of intervention participants read any articles on HMP, compared to 53.3% who read any article on the control site. More than a quarter (26.3%) of participants posted at least once to either The Forum, Getting Real or Ask Dr. W. Participant posts in The Forum and Getting Real covered a wide range of topics from commenting on current events (e.g. Black Gay Pride), to sharing health and relationship advice (e.g. getting on Pre-Exposure Prophylaxis), to coping with stigma and discrimination related to HIV, race/ethnicity, and gender and sexual identity. Questions posed in the Ask Dr. W section of the site included specific medical querries (e.g. “What medications are ok for people who are dual diagnosed with severe depression and HIV that will not interfere with one another?”; “What is your advice for someone who has had discharge but tested negative for HIV, Syphilis, Gonorrhea and Chlamydia?”), but also extended to broader questions about health and wellness (e.g. “Why isn’t HIV curable?”; “I’ve read several articles that say that douching isn’t safe and I’ve also read some that say that it’s ok. What’s the truth?”).
Changes in CAI over time
Table 2 shows the incidence rate ratio estimates, with 95% confidence intervals at each follow-up time point, for both CAI and serodiscordant CAI. The rate of CAI at 3-months post-randomization was 32% lower in the intervention group compared to the control group (incidence ratio 0.68, 95% CI: 0.43, 0.93). This result was unchanged when including the two participants with the most extreme (i.e., >200) CAI events. Loss to follow-up of study participants was found to be a function of both treatment arm, as well as HIV status, education, and any illicit drug use in the past 3 months. When we weighted the participants who completed 3-months of follow-up for their inverse probability of selection to account for a potential selection bias induced by loss to follow-up at 3 months, the incidence rate ratio was slightly attenuated to 0.74 (95% CI: 0.46 – 0.99). There was no difference observed in the change in the rate of serodiscordant CAI in the intervention group compared to the control group.
Table 2:
Incidence ratios (and 95% confidence intervals) for CAI and serodiscordant CAI, comparing HMP exposed to unexposed, at 3, 6, and 12 months post-randomization. Results are shown both with and without IPW adjustment for loss to follow-up.
| Month (Post-randomization) | CAI Incidence Ratio (95% CI) | Serodiscordant CAI Incidence Ratio (95% CI) |
|---|---|---|
| 3 | 0.68 (0.43, 0.93) | 1.37 (0.64, 2.10) |
| 6 | 1.41 (0.88, 1.94) | 1.59 (0.74, 2.44) |
| 12 | 1.09 (0.68, 1.50) | 1.23 (0.57, 1.89) |
| Inverse Probability Weighted Estimates | ||
| 3 | 0.74 (0.46, 0.99) | 1.53 (0.70, 2.36) |
| 6 | 1.42 (0.88, 1.95) | 1.74 (0.78, 2.70) |
| 12 | 1.11 (0.69, 1.53) | 1.14 (0.51, 1.77) |
| 3 | 0.74 (0.46, 0.99) | 1.53 (0.70, 2.36) |
The predicted incidence rates are plotted in Figure 2. Both the intervention and control groups had a decrease in CAI over time. From baseline to 12 months, the rate of CAI in the control group decreased by 32.0% (95% CI: 19.9%, 44.1%). From baseline to 12 months, the rate of CAI in the intervention group decreased by 39.1% (95% CI: 28.3%, 50.0%).
Figure 2.
Predicted rates of CAI, and 95% confidence intervals, at each time point from a mixed-effects marginalized zero-inflated Poisson model.
Among HIV-positive participants, the effect of the intervention on CAI was moderated by baseline detectable viral load. At 3-months post-randomization, the rate of CAI was 82% lower among HIV-positive participants with detectable viral loads in the intervention group compared to the control group (incidence ratio 0.18, 95% CI: 0.04, 0.32). However, the rate was only 11% lower among participants who were either HIV-positive with undetectable viral loads or HIV-negative/unknown in the intervention group compared to the control group (incidence ratio 0.88, 95% CI: 0.51, 1.26). When weighted for loss-to-follow-up, the incidence ratio increased to 0.50 (95% CI: 0.12, 0.89) for detectable viral loads, but remained similar for undetectable viral loads/HIV negatives (incidence ratio 0.87, 95% CI: 0.51, 1.23).
CACE analysis
Fifty participants (25.8%) had time on site greater than or equal to 60 minutes over the 3-month intervention period and were classified as being fully compliant with the intervention. Table 3 compares the baseline characteristics of intervention compliers and non-compliers. Those who used the site more were more likely to have at least a college degree, health insurance, and self-identify as gay than non-compliers. Users were less likely to report marijuana use in the prior 3 months.
Table 3:
Baseline Characteristics of HMP Compilers, defined as a total time using HMP of 60 minutes or more.
| Compliers (N=50) | Non-Compliers (N=144) | p value | |
|---|---|---|---|
| Age | 0.159 | ||
| Mean (SD) | 23.84 (2.57) | 24.60 (3.20) | |
| Education | 0.035 | ||
| < High school | 7 (14.0%) | 15 (10.4%) | |
| High school/GED, some technical/college | 22 (44.0%) | 93 (64.6%) | |
| College degree or more | 21 (42.0%) | 36 (25.0%) | |
| Income | 0.661 | ||
| Less than $10,999 | 23 (46.0%) | 72 (50.0%) | |
| $11,000 - $20,999 | 12 (24.0%) | 25 (17.4%) | |
| $21,000 - $30,999 | 9 (18.0%) | 23 (16.0%) | |
| More than $31,000 | 6 (12.0%) | 24 (16.7%) | |
| Viral Load | 0.729 | ||
| Detectable | 5 (10.0%) | 17 (11.8%) | |
| Undetectable / HIV-Negative / Unknown | 45 (90.0%) | 127 (88.2%) | |
| Sexual Identity | 0.036 | ||
| Gay | 40 (80.0%) | 88 (61.1%) | |
| Bisexual | 7 (14.0%) | 29 (20.1%) | |
| Other | 3 (6.0%) | 27 (18.8%) | |
| Homeless (last 6 months) | 8 (16.0%) | 39 (27.1%) | 0.115 |
| Arrested (last 3 months) | 3 (6.0%) | 12 (8.3%) | 0.595 |
| Health Insurance | 46 (92.0%) | 99 (68.8%) | 0.001 |
| Depression (last 3 months) | 24 (48.0%) | 70 (49.3%) | 0.875 |
| # anal sex acts (past 3 months) | 0.842 | ||
| Mean (SD) | 9.70 (14.71) | 11.56 (22.09) | |
| Median (IQR) | 4.50 (9.00) | 5.50 (9.00) | |
| # condomless anal sex acts (past 3 months) | 0.786 | ||
| Mean (SD) | 4.36 (10.57) | 5.53 (18.29) | |
| Median (IQR) | 1.00 (4.00) | 1.00 (5.00) | |
| # serodiscordant anal sex acts | 0.860 | ||
| Mean (SD) | 3.66 (8.02) | 5.52 (12.47) | |
| Median (IQR) | 1.00 (4.00) | 1.00 (5.00) | |
| # condomless serodiscordant anal sex acts | 0.498 | ||
| Mean (SD) | 0.92 (3.08) | 2.11 (6.37) | |
| Median (IQR) | 0.00 (1.00) | 0.00 (1.00) | |
| Social Support (Standardized Score) | 0.767 | ||
| Mean (SD) | 74.95 (24.02) | 73.25 (27.44) | |
| Social Isolation | 15 (30.0%) | 39 (27.1%) | 0.692 |
| Substance use (last 3 months) | |||
| Alcohol | 44 (88.0%) | 124 (86.1%) | 0.736 |
| Marijuana | 23 (46.0%) | 98 (68.1%) | 0.006 |
| Other drugs | 11 (22.0%) | 40 (27.8%) | 0.424 |
| None | 4 (8.0%) | 9 (6.3%) | 0.690 |
When the analysis was limited to those participants in the intervention group who used HMP (i.e., 60 minutes or more on the site), we observed a substantially stronger effect of the intervention. Specifically, from our ITT analysis, we observed 1.25 (95% CI: 0.34, 2.17) fewer CAI events in the intervention group, compared to the control group, over the 3-month period after baseline. However, from the CACE analysis, we estimated 4.85 (95% CI: 2.15, 7.53) fewer CAI events in the intervention group compared to what we would have expected in control participants had they used the intervention at the same rate as the intervention group over the same time period (i.e., 4.85=1.25/0.258, the usage proportion among the intervention group). The CACE estimate was consistent under a range of cut-off values, from 0 to 100 minutes spent on site. Specifically, and as expected, the rate difference from the CACE analysis became larger the higher the cut-off value, indicating that those who spent more time exposed to HMP, had a lower rate of CAI.
DISCUSSION
Findings from this study suggest that exposure to a theory-based online intervention can reduce the rate of CAI among young BMSM, at least in the short term. Furthermore, this effect was much stronger among those participants who complied with the intervention and among those HIV-positive participants with detectable viral loads. Delivered online, HMP represents a new risk reduction prevention technology for young BMSM, however, ensuring consistent and sustained engagement with the intervention is critical to achieve maximum potential effects. Features of HMP, including those designed to support community building among HIV-positive, negative, and status-unknown young BMSM as well as the inclusion of game-based elements, were novel at the time of study launch.
Both the intervention and control groups had a decrease in CAI over time, as has been seen in other prevention interventions targeting BMSM (30, 31). A reduction in CAI over time among control participants may represent regression to the mean or could indicate that the control condtion, consisting of a large number of tailored prevention articles may have addressed a critical unmet need among young BMSM. While the information-only control may have diluted the measurable effect of the full HMP intervention, provision of this tailored information to all study participants created a more ethical study design given the lack of culturally-responsive prevention resources for young BMSM at the time of HMP launch.
Prior Internet-based HIV prevention interventions, particularly those enrolling minority men who have sex with men have experienced high attrition rates (32–35). Our study is notable for maintaining high overall retention (78-85%), during the 12 month follow-up period, suggesting the feasibility of scaling up this intervention approach. Enrollment at an in-person baseline visit may have increased participant buy-in and has been suggested as a means to decrease fraud (36) and to increase engagement among participants for online studies (37, 38). The impact of the monetary incentives employed in this study to encourage survey completion must be considered before moving to replication and wider scale dissemination. Further, using multiple retention mechanisms, including texting for survey completion (39) and retention incentive bonuses for completion of all surveys were likely contributors to success.
HMP is a unique intervention as it was, to our knowledge, one of the first online interventions developed specificially for both HIV-positive and HIV-negative young BMSM. However, we could not have predicted the significant evolution that impacted the HIV prevention landscape soon after HMP was developed and implemented. While behavior change including CAI remains important, biomedical prevention interventions such as pre-exposure prophylaxis (PrEP) for HIV-negative individuals and treatment as prevention using antiretroviral therapy for positive individuals now has a more central role, among researchers and clinical practice (40–42). Unfortunately, questions about perceptions and usage of these strategies were not included in this study. While we only collected self-report of viral suppression, it is encouraging that 3-months post-randomization, the rate of CAI was 82% lower among participants with detectable viral loads in the intervention group compared to the control group.
We chose to define “usage” of HMP as 60 minutes of cummulative use, however, the optimal usage time to exact a prevention benefit from an online intervention is unknown. Given the breadth of material included in HMP, both information and activities focused both on prevention as well as general health and wellness for BMSM, exploring intervention usage in more nuanced ways could identify mechanisms driving the intervention’s effectiveness (e.g. possible impact of HMP on other health outcomes given the breadth of the material).
Second, there may be an explanation for how HMP impacted CAI directly via specific usage patterns within the site (e.g. the more articles I read about the risks of CAI, the less I engaged in this behavior) or via mediation through one of the other health outcomes (e.g. the more I participate in the forums, the less depressed I am, and the less depressed I am, the less CAI I have). Thus, future analyses of usage of HMP components is needed to explore the impact of the intervention across additional mental health and psychosocial domains including depression, social support and stigma.
We identified large variance in usage among participants in the intervention group, with some participants not using HMP at all and others using HMP almost daily. Given that higher use resulted in stronger intervention effects, this suggests that HMP may work well to change high risk sexual behavior, but that the intervention may need to be better targeted to people who are likely to comply with the intervention. In this study, compilers with the intervention were more likely to have at least a college degree. While we did not measure eHealth literacy in this sample, a recent online intervention found that compared to young MSM with high eHealth literacy who received a tailored intervention, those with low eHealth literacy who received a non-tailored intervention reported lower intervention system quality scores and the intervention had less influence on their sexual health decision making (43). Thus, future online interventions should ensure that users have the skills needed to use or navigate them. Non-gay identified participants were less likely to use the site. Prior research has found that bisexually identified men may be less attached to gay communities (44, 45) and therefore may experience less community support. Stigma experienced by bisexual Black men may influence not only the disclosure of their sexuality to friends, female sex partners and health care providers (46–48) but also their receptivity to prevention messages (49–51). Notably, among the seven bisexually identified men who complied with HMP, none reported female sex partners in the three months prior to their baseline interview, while 12 of the 29 bisexual non-compliers (41.4%) did report female sex partners. Thus interventions, such as HMP, designed to facilitate online support, must ensure that the environment is welcoming, affirming and supportive of a range of sexual identities. Future research will focus on better classifying these participants at baseline to develop additional engagement strategies to increase motivation for use. Consideration of more directed interventions or stepped care studies that provide intensified, tailored interventions for those with low usage may also be needed.
HMP was developed and launched during a time of rapid and signifcant technlogical advances in internet usage, particularly around how individuals utilized social media for networking with others. The forum-based social networking features constructed within HMP were less user-friendly compared with the more responsive social media communication mechanisms offered by platforms such as Twitter and Instagram that emerged after HMP was developed. Despite limitations of these design features, HMP’s user-contribution areas were some of the most popular features of the site, suggesting their importance for intervention engagement.
This study had several limitations. Data collected within HMP were self-reported and could be subject to social desirability bias and recall bias. However, data were collected using CASI, which should reduce social desirability bias (52–55). Further, in a study comparing self-report of viral load suppression, there was high agreement (88%) between self-report and clinical records of viral load <50 copies/mL (56). HMP only included those BMSM with internet access which may limit generalizability. However, as of 2015, 96% of those ages 18-29 had access to the internet (57) with 86% of those ages 18-29 having a smartphone (58).
CONCLUSION
Our study provides evidence for the efficacy of HMP to reduce sexual risk behaviors among young BMSM, particularly among the subset of participants who utilized the intervention for a greater amount of time. Behavioral changes were not sustained over 12 months. These findings suggest that while HMP can be effective for some BMSM, a non-directed online intervention may not be enough to sustain behavior changes. Future online interventions could target specific groups more likely to engage in online interventions and consider addressing a combination of behavioral, biomedical and structural issues that impact sustained engagement in prevention activities.
ACKNOWLEDGMENTS
We thank the participants of HMP for their time, insight and wisdom. We also give thanks to all our collaborating community members and organizations for their support and assistance.
Sources of Support: Funding for this study was provided by: NIMH R01MH093275-01
Footnotes
Data including these findings have been presented at the following meeting:
L.B. Hightow-Weidman, S. LeGrand, R. Simmons, J. Egger, S.K. Choi, K. Muessig. healthMpowerment: Effects of a Mobile phone-optimized, Internet-based Intervention on Condomless Anal Intercourse Among Young Black Men who have Sex with Men and Transgender Women. Abstract# WEPEC1001, 2017 International AIDS Society Conference. Paris, France, July 23-26, 2017.
Conflict of interest: All authors of this manuscript report no conflicts of interest.
Ethical approval: All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Informed consent: Informed consent was obtained from all individual participants included in the study.
REFERENCES
- 1.Hess KL, Hu X, Lansky A, Mermin J, Hall HI. Lifetime risk of a diagnosis of HIV infection in the United States. Ann Epidemiol. 2017;27(4):238–43. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Matthews DD, Herrick AL, Coulter RW, Friedman MR, Mills TC, Eaton LA, et al. Running Backwards: Consequences of Current HIV Incidence Rates for the Next Generation of Black MSM in the United States. AIDS Behav. 2016;20(1):7–16. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Millett GA, Flores SA, Peterson JL, Bakeman R. Explaining disparities in HIV infection among black and white men who have sex with men: a meta-analysis of HIV risk behaviors. AIDS. 2007;21(15):2083–91. [DOI] [PubMed] [Google Scholar]
- 4.Millett GA, Peterson JL, Flores SA, Hart TA, Jeffries WLt, Wilson PA, et al. Comparisons of disparities and risks of HIV infection in black and other men who have sex with men in Canada, UK, and USA: a meta-analysis. Lancet. 2012;380(9839):341–8. [DOI] [PubMed] [Google Scholar]
- 5.Millett GA, Peterson JL, Wolitski RJ, Stall R. Greater risk for HIV infection of black men who have sex with men: a critical literature review. Am J Public Health. 2006;96(6):1007–19. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Maulsby C, Millett G, Lindsey K, Kelley R, Johnson K, Montoya D, et al. HIV among Black men who have sex with men (MSM) in the United States: a review of the literature. AIDS Behav. 2014;18(1):10–25. [DOI] [PubMed] [Google Scholar]
- 7.Maulsby C, Millett G, Lindsey K, Kelley R, Johnson K, Montoya D, et al. A systematic review of HIV interventions for black men who have sex with men (MSM). BMC public health. 2013;13:625. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Wilton L, Herbst JH, Coury-Doniger P, Painter TM, English G, Alvarez ME, et al. Efficacy of an HIV/STI prevention intervention for black men who have sex with men: findings from the Many Men, Many Voices (3MV) project. AIDS Behav. 2009;13(3):532–44. [DOI] [PubMed] [Google Scholar]
- 9.Peterson JL, Coates TJ, Catania J, Hauck WW, Acree M, Daigle D, et al. Evaluation of an HIV risk reduction intervention among African-American homosexual and bisexual men. AIDS. 1996;10(3):319–25. [DOI] [PubMed] [Google Scholar]
- 10.Harawa NT, Williams JK, McCuller WJ, Ramamurthi HC, Lee M, Shapiro MF, et al. Efficacy of a culturally congruent HIV risk-reduction intervention for behaviorally bisexual black men: results of a randomized trial. AIDS. 2013;27(12):1979–88. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Jemmott JB 3rd, Jemmott LS, O’Leary A, Icard LD, Rutledge SE, Stevens R, et al. On the Efficacy and Mediation of a One-on-One HIV Risk-Reduction Intervention for African American Men Who Have Sex with Men: A Randomized Controlled Trial. AIDS Behav. 2015;19(7):1247–62. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Tobin K, Kuramoto SJ, German D, Fields E, Spikes PS, Patterson J, et al. Unity in diversity: results of a randomized clinical culturally tailored pilot HIV prevention intervention trial in Baltimore, Maryland, for African American men who have sex with men. Health Educ Behav. 2013;40(3):286–95. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Williams JK, Glover DA, Wyatt GE, Kisler K, Liu H, Zhang M. A sexual risk and stress reduction intervention designed for HIV-positive bisexual African American men with childhood sexual abuse histories. Am J Public Health. 2013;103(8):1476–84. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Fish J, Papaloukas P, Jaspal R, Williamson I. Equality in sexual health promotion: a systematic review of effective interventions for black and minority ethnic men who have sex with men. BMC public health. 2016;16(1):810. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Hightow-Weidman LB, Muessig KE, Bauermeister J, Zhang C, LeGrand S. Youth, Technology, and HIV: Recent Advances and Future Directions. Curr HIV/AIDS Rep 2015;12(4):500–15. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Muessig KE, Nekkanti M, Bauermeister J, Bull S, Hightow-Weidman LB. A systematic review of recent smartphone, Internet and Web 2.0 interventions to address the HIV continuum of care. Curr HIV/AIDS Rep 2015;12(1):173–90. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Schnall R, Travers J, Rojas M, Carballo-Dieguez A. eHealth interventions for HIV prevention in high-risk men who have sex with men: a systematic review. J Med Internet Res 2014;16(5):e134. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Hightow-Weidman LB, Fowler B, Kibe J, McCoy R, Pike E, Calabria M, et al. HealthMpowerment.org: development of a theory-based HIV/STI website for young black MSM. AIDS Educ Prev 2011;23(1):1–12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Muessig KE, Pike EC, Fowler B, LeGrand S, Parsons JT, Bull SS, et al. Putting prevention in their pockets: developing mobile phone-based HIV interventions for black men who have sex with men. AIDS patient care and STDs. 2013;27(4):211–22. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.LeGrand S, Muessig KE, Pike EC, Baltierra N, Hightow-Weidman LB. If you build it will they come? Addressing social isolation within a technology-based HIV intervention for young black men who have sex with men. AIDS care. 2014;26(9):1194–200. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Hightow-Weidman LB, Muessig KE, Pike EC, LeGrand S, Baltierra N, Rucker AJ, et al. HealthMpowerment.org: Building Community Through a Mobile-Optimized, Online Health Promotion Intervention. Health Educ Behav 2015;42(4):493–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Hightow-Weidman LB, Pike E, Fowler B, Matthews DM, Kibe J, McCoy R, et al. HealthMpowerment.org: feasibility and acceptability of delivering an internet intervention to young Black men who have sex with men. AIDS care. 2012;24(7):910–20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Muessig KE, Baltierra NB, Pike EC, LeGrand S, Hightow-Weidman LB. Achieving HIV risk reduction through HealthMpowerment.org, a user-driven eHealth intervention for young Black men who have sex with men and transgender women who have sex with men. Digital culture & education. 2014;6(3):164–82. [PMC free article] [PubMed] [Google Scholar]
- 24.Montano D, Kasprzyk D. Theory of reasoned action, theory of planned behavior, and the integrated behavioral model In: Glanz K, Rimer B, Lewis F, editors. Health behavior and health education: Theory, research, and practice. San Francisco, CA: Jossey-Bass; 2008. p. 67–92. [Google Scholar]
- 25.Radloff LS. The CES-D scale: A self-report depression scale for research in the general population. Applied Psychological Measurement. 1977;1:385–401. [Google Scholar]
- 26.Sherbourne CD, Stewart AL. The MOS social support survey. Social science & medicine. 1991;32(6):705–14. [DOI] [PubMed] [Google Scholar]
- 27.Lubben J, Blozik E, Gillmann G, Iliffe S, von Renteln Kruse W, Beck JC, et al. Performance of an abbreviated version of the Lubben Social Network Scale among three European community-dwelling older adult populations. Gerontologist. 2006;46(4):503–13. [DOI] [PubMed] [Google Scholar]
- 28.Long DL, Preisser JS, Herring AH, Golin CE. A marginalized zero-inflated Poisson regression model with overall exposure effects. Statistics in medicine. 2014;33(29):5151–65. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Yau LHY, Little R. Inference for the Complier-Average Causal Effect From Longitudinal Data Subject to Noncompliance and Missing Data, With Application to a Job Training Assessment for the Unemployed. Journal of the American Statistical Association. 2001;96:1232–44. [Google Scholar]
- 30.Koblin BA, Bonner S, Powell B, Metralexis P, Egan JE, Patterson J, et al. A randomized trial of a behavioral intervention for black MSM: the DiSH study. AIDS. 2012;26(4):483–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Williams JK, Wyatt GE, Rivkin I, Ramamurthi HC, Li X, Liu H. Risk reduction for HIV-positive African American and Latino men with histories of childhood sexual abuse. Arch Sex Behav. 2008;37(5):763–72. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Carpenter KM, Stoner SA, Mikko AN, Dhanak LP, Parsons JT. Efficacy of a web-based intervention to reduce sexual risk in men who have sex with men. AIDS Behav. 2010;14(3):549–57. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Christensen JL, Miller LC, Appleby PR, Corsbie-Massay C, Godoy CG, Marsella SC, et al. Reducing shame in a game that predicts HIV risk reduction for young adult MSM: a randomized trial delivered nationally over the Web. J Int AIDS Soc 2013;16(3 Suppl 2):18716. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Schonnesson LN, Bowen AM, Williams ML. Project SMART: Preliminary Results From a Test of the Efficacy of a Swedish Internet-Based HIV Risk-Reduction Intervention for Men Who Have Sex With Men. Arch Sex Behav. 2016;45(6):1501–11. [DOI] [PubMed] [Google Scholar]
- 35.Bull SS, Vallejos D, Levine D, Ortiz C. Improving recruitment and retention for an online randomized controlled trial: experience from the Youthnet study. AIDS care. 2008;20(8):887–93. [DOI] [PubMed] [Google Scholar]
- 36.Teitcher JE, Bockting WO, Bauermeister JA, Hoefer CJ, Miner MH, Klitzman RL. Detecting, preventing, and responding to “fraudsters” in internet research: ethics and tradeoffs. J Law Med Ethics. 2015;43(1):116–33. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Bull SS, Lloyd L, Rietmeijer C, McFarlane M. Recruitment and retention of an online sample for an HIV prevention intervention targeting men who have sex with men: the Smart Sex Quest Project. AIDS care. 2004;16(8):931–43. [DOI] [PubMed] [Google Scholar]
- 38.Gustafson DH, Robinson TN, Ansley D, Adler L, Brennan PF. Consumers and evaluation of interactive health communication applications. The Science Panel on Interactive Communication and Health. Am J Prev Med 1999;16(1):23–9. [DOI] [PubMed] [Google Scholar]
- 39.Khosropour CM, Johnson BA, Ricca AV, Sullivan PS. Enhancing retention of an Internet-based cohort study of men who have sex with men (MSM) via text messaging: randomized controlled trial. J Med Internet Res. 2013;15(8):e194. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Cohen MS, Smith MK, Muessig KE, Hallett TB, Powers KA, Kashuba AD. Antiretroviral treatment of HIV-1 prevents transmission of HIV-1: where do we go from here? Lancet. 2013;382(9903):1515–24. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Freeborn K, Portillo CJ. Does Pre-exposure prophylaxis (PrEP) for HIV prevention in men who have sex with men (MSM) change risk behavior? A systematic review. J Clin Nurs 2017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Reyniers T, Hoornenborg E, Vuylsteke B, Wouters K, Laga M. Pre-exposure prophylaxis (PrEP) for men who have sex with men in Europe: review of evidence for a much needed prevention tool. Sex Transm Infect 2017;93(5):363–7. [DOI] [PubMed] [Google Scholar]
- 43.Horvath KJ, Bauermeister JA. eHealth Literacy and Intervention Tailoring Impacts the Acceptability of a HIV/STI Testing Intervention and Sexual Decision Making Among Young Gay and Bisexual Men. AIDS Educ Prev. 2017;29(1):14–23. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Dodge B, Schnarrs PW, Reece M, Goncalves G, Martinez O, Nix R, et al. Community involvement among behaviourally bisexual men in the Midwestern USA: experiences and perceptions across communities. Cult Health Sex. 2012;14(9):1095–110. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Dodge B, Schnarrs PW, Reece M, Martinez O, Goncalves G, Malebranche D, et al. Individual and Social Factors Related to Mental Health Concerns among Bisexual Men in the Midwestern United States. J Bisex. 2012;12(2):223–45. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Friedman MR, Bukowski L, Eaton LA, Matthews DD, Dyer TV, Siconolfi D, et al. Psychosocial Health Disparities Among Black Bisexual Men in the U.S.: Effects of Sexuality Nondisclosure and Gay Community Support. Arch Sex Behav. 2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Dodge B, Jeffries WLt, Sandfort TG. Beyond the Down Low: sexual risk, protection, and disclosure among at-risk Black men who have sex with both men and women (MSMW). Arch Sex Behav. 2008;37(5):683–96. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.Shearer K, Khosropour C, Stephenson R, Sullivan PS. Do Bisexual Men Tell Their Female Partners About Having Male Partners? Results from a National Online Hiv Prevention Survey in the United States. Int J Sex Health. 2012;24(3):195–204. [Google Scholar]
- 49.Shoptaw S, Weiss RE, Munjas B, Hucks-Ortiz C, Young SD, Larkins S, et al. Homonegativity, substance use, sexual risk behaviors, and HIV status in poor and ethnic men who have sex with men in Los Angeles. J Urban Health. 2009;86 Suppl 1:77–92. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50.Schrimshaw EW, Downing MJ Jr., Cohn DJ. Reasons for Non-Disclosure of Sexual Orientation Among Behaviorally Bisexual Men: Non-Disclosure as Stigma Management. Arch Sex Behav. 2018;47(1):219–33. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Henny KD, Drumhiller K, Sutton MY, Nanin J. “My Sexuality...It Creates a Stress”: HIV-Related Communication Among Bisexual Black and Latino Men, New York City: Arch Sex Behav; 2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52.Butler SF, Villapiano A, Malinow A. The effect of computer-mediated administration on self-disclosure of problems on the addiction severity index. J Addict Med 2009;3(4):194–203. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53.Newman JC, Des Jarlais DC, Turner CF, Gribble J, Cooley P, Paone D. The differential effects of face-to-face and computer interview modes. Am J Public Health. 2002;92(2):294–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 54.Kissinger P, Rice J, Farley T, Trim S, Jewitt K, Margavio V, et al. Application of computer-assisted interviews to sexual behavior research. Am J Epidemiol. 1999;149(10):950–4. [DOI] [PubMed] [Google Scholar]
- 55.Turner CF, Ku L, Rogers SM, Lindberg LD, Pleck JH, Sonenstein FL. Adolescent sexual behavior, drug use, and violence: increased reporting with computer survey technology. Science. 1998;280(5365): 867–73. [DOI] [PubMed] [Google Scholar]
- 56.Sewell J, Daskalopoulou M, Nakagawa F, Lampe FC, Edwards S, Perry N, et al. Accuracy of self-report of HIV viral load among people with HIV on antiretroviral treatment. HIV Med. 2017;18(7):463–73. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 57.Pew. Americans’ Internet Access: 2000-2015 2015. [Available from: http://www.pewinternet.org/2015/06/26/americans-internet-access-2000-2015/. Last accessed July 10, 2017.
- 58.Pew. Technology Device Ownership: 2015 2015. [Available from: http://www.pewinternet.org/2015/10/29/technology-device-ownership-2015/. Last accessed July 10, 2017.