Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2020 Jun 1.
Published in final edited form as: AIDS Behav. 2019 Jun;23(6):1668–1679. doi: 10.1007/s10461-019-02396-w

A Preliminary Randomized Controlled Trial of Game Plan, A Web Application to Help Men Who Have Sex with Men Reduce their HIV Risk and Alcohol Use

Tyler B Wray 1, Christopher W Kahler 1, Erik M Simpanen 1, Don Operario 1
PMCID: PMC6536322  NIHMSID: NIHMS1524970  PMID: 30671682

Abstract

Alcohol use is a key risk factor for HIV infection among men who have sex with men (MSM). Past studies show that brief motivational interventions (BMI) can increase the use of prevention methods (e.g., condoms), reduce alcohol use, and can be adapted for web-based delivery. However, few studies have explored these interventions’ effects in MSM. Forty high-risk, heavy drinking MSM who sought rapid HIV testing were randomly assigned to receive either (1) standard post-test counseling (SPC) alone, or (2) SPC plus Game Plan (GP), a tablet tablet-based BMI for alcohol use and HIV risk. Over three months of follow-up, GP participants reported 24% fewer heavy drinking days, 17% fewer alcohol problems, and 50% fewer new anal sex partners than controls. GP participants also reported fewer high-risk condomless anal sex (CAS) events than controls, but these differences were not significant. These initial results suggest that web-based BMIs may be promising tools to help MSM reduce health risk behaviors.

Keywords: Alcohol, men who have sex with men, HIV risk behavior, condom use, brief intervention, web-based intervention

Introduction

In the United States, HIV incidence continues to increase among certain subgroups of men who have sex with men (MSM) (1), including young MSM aged 25-34 (2). MSM accounted for 67% of all new infections in 2015 (3), and recent analyses show that, if these rates continue, 1 in 6 MSM will be diagnosed with HIV in their lifetimes (4). Although new biomedical prevention methods like pre-exposure prophylaxis (PrEP) show promise for reducing incidence (5, 6), uptake is currently < 5% of all eligible MSM (7). Rates of condom use during anal sex also continue to be inadequate among many MSM (8, 9), contributing to new HIV infections and record high rates of other sexually transmitted infections (STIs) (10-14).

Alcohol use is a key risk factor for HIV infection among MSM (15-17) due in large part to alcohol’s tendency to interfere with the use of effective prevention methods like condoms (18-21). Binge drinking (5+ drinks on a single occasion) is of particular concern, since drinking at this level is consistently associated with failing to use a condom during anal sex with high-risk partners among MSM (21-23). Innovative and scalable interventions are needed to encourage MSM to adopt effective prevention methods (e.g., condoms, PrEP) and use them consistently, and to address binge drinking as a co-occurring transmission risk factor.

Meta-analyses have shown that behavior change interventions can increase recipients’ use of HIV/STI prevention methods like condoms, and that their effects may be especially pronounced among MSM (24-26). Brief interventions for alcohol use have received broad support, with several meta-analyses showing that these interventions are often as effective as their longer, more extensive counterparts (27-29). In particular, brief interventions inspired by motivational interviewing have some of the most robust empirical support (27, 30). Recent studies have shown greater reductions in alcohol use among heavy drinkers who received a brief motivational intervention (BMI) in HIV primary care settings, when compared with brief education/advice (31, 32). Monti et al. (33) showed that a dual-behavior BMI reduced both heavy drinking and high-risk sexual behavior compared to brief advice over nine months of follow-up, among at-risk heterosexual male and female emergency department patients. Kahler et al. (34) also found that, in a sample of MSM living with HIV, a BMI compared to assessment only significantly reduced alcohol consumption and condomless sex with non-steady partners among men who reported this behavior prior to the intervention.

Despite evidence of efficacy, face-to-face interventions like these have a number of challenges that have so far limited their impact (35, 36). In particular, implementing these interventions in busy healthcare settings where they are often most needed is cost and resource-intensive, in part due to the need for highly-trained staff capable of delivering them with fidelity. Personalized feedback interventions (PFIs) are one type of BMI that has been adapted for digital and web-based delivery using a self-guided format (37), and could overcome many of these challenges. PFIs are inspired by principles of MI and aim to develop discrepancy between recipients’ current and desired behavior by providing personalized feedback about current behavior and comparing it with a relevant social group (38, 39). Although direct comparisons between these digitally-delivered alcohol PFIs and those delivered face-to-face show that, as a whole, the effects of face-to-face interventions may be more durable, both types of PFIs show reductions in alcohol use (40-43), and some well-designed digital PFIs for alcohol use may perform as well as similar face-to-face interventions (44). Although digitally-delivered interventions have also been shown to increase condom use among MSM (45), no studies have yet explored whether digitally-delivered alcohol or dual-behavior interventions can help MSM reduce heavy drinking and HIV risk behavior.

The effects of brief alcohol and HIV interventions may be most profound when they are delivered soon after recipients experience negative consequences of a behavior (i.e., “teachable moments”) (46-48). With respect to alcohol use, studies have shown that those who receive a BMI after an alcohol-related incident (an event that required medical, police, or administrative attention) showed greater reductions in drinking than those who received them at an unrelated time (48-50). With respect to HIV, voluntary testing is often sought after a possible sexual exposure to HIV (51, 52); up to 60% of MSM in one sample listed this as a key reason for testing (53). As such, HIV testing could be an optimal time to intervene in order to reduce alcohol use and HIV risk. Counseling is already offered alongside voluntary HIV testing in many settings, and there are guidelines on its content (54). However, in practice, no specific approach to counseling is used consistently (55), and alcohol use is rarely addressed. Moreover, Metsch et al. (56) found that a broad, “person-centered” post-test counseling approach that aligned with CDC recommendations did not reduce STI incidence or sexual risk behavior among MSM compared to information alone. Although these findings have widely been construed as evidence that providing counseling alongside testing does not reduce risk, they may instead suggest that for post-test counseling to be effective, it must involve a more consistent, theoretically-informed, and empirically-supported approach to behavior change that addresses key factors that contribute to risk behavior. Internet-facilitated approaches could help capitalize on these “teachable moments,” while also delivering consistent, theoretically-informed content that helps MSM change to reduce their risk.

Given these needs, we developed Game Plan, an interactive web application that is optimized for tablet computers and aims to help high-risk MSM consider reducing their HIV-risk behavior and alcohol use after they test negative for HIV at a clinic. A full description of Game Plan’s content has been reported elsewhere (57). The core features of Game Plan include: providing personalized, digestible, and engaging feedback about recipients’ level of drinking and risk for HIV, and comparing this with other gay/bisexual men in their age group; incorporating reflective exercises similar to those often used in MI sessions that are intended to elicit “change talk”; engaging recipients in game-like activities to develop further discrepancy and promote personalized HIV and alcohol risk reduction goals and change planning. Game Plan was also developed using a thorough user-centered design process (58, 59) that helped ensure it was engaging to its intended users (heavy drinking, high-risk MSM) and generally aligned with the “spirit” of MI.

In this preliminary study, we explored initial evidence of Game Plan’s promise by testing whether MSM who used the Game Plan app on a tablet after completing HIV testing and post-test counseling reduced their alcohol use and sexual risk behavior compared with HIV who received HIV testing and post-test counseling alone. Specifically, we examined descriptive data on a number of important antecedents of change for both alcohol and sexual risk behavior that were collected within the app, such as changes in participants’ motivation to change and behavior change goals. Next, we explored whether those who used Game Plan reported fewer drinking days, binge drinking days (i.e., days on which 5+ drinks were consumed), and a lower average number of drinks per drinking day, compared to those receiving post-test counseling alone. We also explored whether more participants in the Game Plan condition would consult with a medical provider about starting PrEP, receive PrEP prescriptions, and report a fewer number of anal sex partners, as well as condomless anal sex (CAS) events with new and unknown status partners.

METHODS

Participants

Forty MSM in the Northeastern US were recruited to receive free rapid HIV testing from gay-oriented smartphone dating apps (e.g., Grindr, Scruff) and enrolled between October and December 2017. Eligible participants were (1) assigned male sex at birth, (2) over age 18, (3) able to read in English, (4) reported condomless anal sex (CAS) with male partner of unknown HIV status in the past three months, (5) had not tested for HIV in the last 6 months, and (6) were HIV-negative or unsure of their status. They also were (7) classified as “heavy drinkers” using NIAAA criteria, meaning that they reported consuming at least 14 drinks on an average week or drinking ≥ 5 drinks in a single sitting at least once in the past month. Those who (8) were currently receiving medications or counseling for an alcohol or drug-related problem were excluded, since ongoing treatment may have interfered with the study’s primary outcomes. Those who reported (9) currently taking pre-exposure prophylaxis (PrEP) were also excluded, since a key goal of the study was to explore whether those who used Game Plan were more likely to seek and start PrEP. As this was an exploratory RCT, a priori power analyses were not conducted to determine the final sample size. However, given that past studies have shown small-to-medium effects for similar interventions on related outcomes (33, 43, 60), we chose a sample size of N=40 a priori to align with existing recommendations for two-arm exploratory trials with effects of this size (61, 62).

Procedures

Figure 1 shows the Consolidated Standards of Reporting Trials (CONSORT) flow diagram of participants through study milestones (Moher et al., 2010). Advertisements offering free HIV testing were presented to users of selected dating apps who logged in within a 30-mile radius of Providence. Those interested were directed to a study landing page, where they completed a brief online survey before scheduling a testing appointment using an online scheduling system. At these appointments, all participants were first tested for HIV using a rapid test (OraSure’s OraQuick Rapid HIV Antibody Test; Bethlehem, PA). All participants tested negative, and after receiving these results, those who met basic eligibility criteria based on their online surveys were then provided with more information about the study. Informed consent was then acquired from those interested in enrolling, before participants completed baseline assessments. Participants were randomly assigned to receive either (1) standard post-test counseling (control), or (2) standard post-test counseling plus the Game Plan app using a simple random assignment procedure. Those in the control condition received HIV testing, standard post-test counseling and referrals only. Standard counseling adopted a person-centered approach commonly used in clinic-based settings (56). This approach involves discussing participants’ patterns of risk and key obstacles to safer behavior, before offering tailored options for reducing risk. Those in the Game Plan condition first received HIV testing, post-test counseling and referrals, but then were provided with a tablet computer (iPad Air) and allowed to use the Game Plan app for as long as they wished. On average, participants used Game Plan for 33 minutes (Range = 23-39 min.). Neither participants nor research staff were blind to condition.

Figure 1.

Figure 1.

Open in a new tab

CONSORT diagram of pilot study.

After the session was complete, all intervention and control participants were then sent a link to complete an online survey via email every 30 days for the next three months. Reminder emails were sent each day for 5 days until it was completed. If the surveys were not completed within 5 days of their due date, research staff followed up with participants over the phone and helped to troubleshoot issues. Participants were paid $60 for completing their in-person appointments, and $30 for each monthly assessment they completed, with a bonus of $50 for completing all assessments, for a total of $200. Payments were issued in cash or on reloadable debit cards within 24 hours of completing study procedures. The use of convenient online surveys, together with issuing substantial bonuses for completing all study procedures, were intended to ensure high response rates. All procedures were approved by the Brown University Institutional Review Board, and this study was registered on ClinicalTrials.gov (NCT03435783).

Game Plan app.

Game Plan provides basic feedback and normative comparisons about users’ risk for HIV (based on their recent sexual behavior) and alcohol use, similar to many existing web-based personalized feedback interventions. However, Game Plan also goes beyond these interventions by incorporating reflective exercises similar to those often used in MI sessions that are intended to elicit “change talk.” These exercises were designed to be similar to the thought exercises commonly used to develop discrepancy between current and desired behavior, such as the “looking forward/looking back” exercise (63), values card sort (64), and pros/cons “decisional balance” (38). Game Plan also incorporates a substantive change planning component. This section presents users with a set of change goals that are color-coded based on their potential for reducing risk, with those reducing risk the most presented first (e.g., get on PrEP, use condoms with all partners, stop drinking entirely, reduce how much [they are drinking]). Once a goal is chosen, users can select from a number of specific steps tailored to their personal barriers, as well as details about who else can help them and when they will start. These goals are then added to a change plan with information about local resources (e.g., PrEP clinics, STD testing, LGBT-friendly alcohol/drug treatment), which they can then print or email to themselves. All content is tailored to users’ current motivation to change each behavior, and “planting a seed” sections are available for those especially resistant to change.

Measures

In-app data.

Participants’ responses to content within the app (e.g., change goals selected) were collected and exported using the app’s database. Participants’ motivation to change to reduce their risk for HIV and alcohol use were assessed before and after receiving feedback for each section (sex risk, alcohol), and was assessed on a 1 (not at all) to 8 (extremely) scale reflecting the extent to which they were interested in making a change to be safer (sexually) or to drink less. We also collected data about participants’ goal choices in the change planning section, including (1) whether or not they elected to set any goal to change each behavior and (2) the type of goal they set.

Alcohol use and sexual behavior.

Daily drinking and sexual behavior were assessed at baseline and over the three-month study period using an online Timeline Followback (65-69). In this survey, participants were first presented with a calendar of the last 30 days, and asked to note any days on which they drank or engaged in oral, anal, or vaginal sex. Details about the behaviors on each day were then assessed using a “detail view.” For alcohol use, participants reported the number of standard drinks they consumed each day they drank (with a visual key provided) and the number of hours over which they drank. For sexual behavior, participants reported the number of partners they had that day (up to 4), and for each partner, their gender, whether it was a new partner, someone they were in a sexually exclusive relationship with, whether they had ever asked about their HIV status prior to having sex with this partner, and if so, what their status was. They also reported which sex acts they engaged in with each partner (oral, insertive/receptive anal sex, vaginal sex) and whether they used a condom for each act. From this data, we generated the number of total drinks, drinking days, binge drinking (5+ drinks) days, and the average number of drinks on drinking days across each 30-day period to serve as primary drinking outcomes. High-risk CAS events were those that involved CAS with partners who (1) were new/casual, or (2) whose HIV status was unknown, either because participants had never asked, or reported that they did not know. Number of CAS events in the past 30 days was the primary sexual behavior outcome.

Alcohol-related problems.

The number of alcohol-related problems participants reported over the last 30 days was assessed using the Brief Young Adult Alcohol Consequences Questionnaire (B-YAACQ; 70) as a secondary outcome. The B-YAACQ assesses 24 consequences of alcohol consumption, each rated either no (0) or yes (1). Example items include “I have passed out from drinking,” and “While drinking, I have said or done foolish things.” The B-YAACQ has shown excellent psychometric properties in longitudinal studies (70).

PrEP consultation and uptake.

Monthly surveys assessed whether participants had (1) consulted with a medical provider (e.g., doctor, nurse, or other prescriber) about starting PrEP over the past month, and (2) had received a prescription for PrEP in the past 30 days. These variables were coded to reflect as a secondary outcome whether participants had consulted about or started PrEP at any time across the three-month study period.

Analyses

To explore Game Plan’s effects on alcohol and sexual risk outcomes after testing compared to standard post-test counseling alone, we estimated several Generalized Estimating Equations (GEEs). Given that most outcomes were count in nature (e.g., number of binge drinking days, number of new partners, number of CAS events), Poisson distributions and log-link functions were specified. Each model controlled for month (represented as a dummy-coded categorical variable), the respective dependent variable at baseline, and included a dummy variable reflecting condition assignment with the control group as the reference. All analyses were conducted in intent-to-treat fashion using all available data (71). As such, we assumed that participants had not consulted with a provider or been prescribed PrEP in a given month, for example, when data were missing for that month. Although we used these models to explore between-group differences using conventional values of statistical significance (α = .05), given the exploratory nature of this study, we also interpreted the size and direction of effects in all models.

RESULTS

Attrition and Missing Data

Of the 40 participants originally recruited, only one stopped responding to monthly surveys prior to month three, for an overall dropout rate of 2.5%. The overall response rate to monthly surveys was 97.5%, with only four total monthly surveys missing. This provided a total of 156 non-missing person-months of data. At baseline, participants randomly assigned to the Game Plan condition drank less frequently (χ2[1]=4,99, p = .026), but had more anal sex partners (χ2[1]=5.85, p = .016) than did control participants. However, the two conditions did not significantly differ on any other primary outcome. See Table 1 for demographic characteristics by study condition.

TABLE 1.

Demographic and Behavioral Characteristics of the Study Sample (N = 40)

Characteristics Control
(N = 20)		 Game Plan
(N = 20)		 χ2 or F1
N (%) N (%)
Age (Range: 18 – 53, M ± SD)2 27.7 (8.3) 28.8 (9.9) 0.4
Race
 White 13 (65.0) 15 (75.0) 0.1
 Black or African American 4 (20.0) 1 (5.0)
 Asian 0 (0.0) 2 (10.0)
 Multiracial 3 (15.0) 2 (10.0)
Ethnicity (Hispanic or Latino) 1 (5.0) 3 (15.0) 1.7
Currently in Exclusive Relationship3 0 (0.0) 1 (5.0) 1.2
Low income4 6 (30.0) 4 (20.0) 0.1
No college degree 12 (60.0) 6 (30.0) 2.2
Unemployed 4 (20.0) 2 (10.0) 0.5
Identity other than gay or bisexual 3 (15.0) 1 (5.0) 1.9
Days since last CAS2 47.2 (18.9) 47.5 (23.1) 0.1
Years since most recent HIV test2 0.6 (0.7) 2.3 (6.1) 1.2
AUDIT5 ≥ 8 13 (65.0) 14 (70.0) 1.2
Open in a new tab

Note.

1

All p > .05.

2

Shown in M and SD.

3

Represents participants who reported currently being in a sexually exclusive, monogamous relationship with one partner.

4

Represents those with a household annual income <$30,000/year.

5

Alcohol Use Disorders Identification Test.

In-App Responses.

On average, participants’ in-app responses about their motivation to start making sexual choices increased 9.0% (SD = 12.6%) after viewing their HIV risk feedback and completing “reflective activities” designed to develop discrepancy. Similarly, participants’ motivation to make changes to their drinking increase an average of 6% (SD = 19.6%) after viewing their alcohol feedback. However, 85% of those in the Game Plan condition elected to set a goal to reduce their sexual risk in the change planning section, and 75% chose goals capable of reducing risk considerably (e.g., consulting with medical providers about PrEP, using condoms with all partners, using condoms with new/casual partners). Similarly, 80% of these non-treatment-seeking, heavy-drinking MSM also elected to set a goal to change their drinking, with 70% overall choosing to reduce “how much [they were] drinking.” See Table 2.

TABLE 2.

Top Change Plan Goals Selected by Participants Assigned to use Game Plan (N = 20)1

Goals2 %
Sexual Risk Reduction
 Take a once-daily pill called “pre-exposure prophylaxis” or “PrEP” 30.0
 Use condoms with all my partners 30.0
 Have anal sex with only one partner, someone I know well 25.0
 use condoms with any new or casual partners 40.0
 Ask my partners about the last time they were tested 5.0
 Make sure I get tested for HIV and other STDs more regularly 5.0
 Meet fewer partners on “hookup” apps or websites 5.0
 Only “top” with partners I don’t know that well 5.0
 Prepare better (e.g., use plenty of lube) when “bottoming” 10.0
Alcohol Use
 Stop drinking altogether 0.0
 Reduce or “cut down” on how much I’m drinking 70.0
 Change how I’m drinking to keep bad things from happening as often 10.0
Open in a new tab

Note.

1

Users could select multiple change goals.

2

Goals are listed in the order they were presented within the app, which also reflected how much each goal could reduce risk.

Alcohol Use

Alcohol use outcomes are depicted in Figure 2. In the GEE model of the number of drinking days (see Table 3), the overall effect of condition was significant, and suggested that those in the Game Plan condition drank approximately 27% fewer days than those in the control condition across the follow-up period. Planned contrasts comparing the conditions within each month of follow-up showed that these between-group differences were also statistically significant across each month of follow-up. A similar model of total binge drinking days also showed that while the overall effect of intervention condition did not reach statistical significance (p = .051), those in the Game Plan condition reported 24% fewer binge drinking days than control condition participants across the follow-up period. Planned contrasts suggested that while Game Plan participants reported fewer binge drinking days than control participants across all months, this difference was only statistically significant in month 2. Models of the average number of drinks participants consumed per drinking day showed no differences between Game Plan and control participants. Finally, in GEE models of alcohol-related problems, the overall effect of intervention condition was significant, and showed that Game Plan participants reported 17% fewer alcohol-related problems than control participants across the follow-up period. Planned contrasts showed that these between-group differences were statistically significant specifically in months 2 and 3.

Figure 2.

Figure 2.

Open in a new tab

Between-group differences in the number of drinking days (top left), binge drinking days (top right), average number of drinks per drinking day (bottom left) and alcohol-related problems (bottom right) across three months of follow-up

TABLE 3.

GEE models of between-group differences in past 30-day alcohol outcomes across the three-month study period.

Variable Total Drinking Days Total Binge Drinking
Days		 Average
Drinks/Drinking Day		 Alcohol-Related Problems
IRR p 95% CI IRR p 95% CI IRR p 95% CI IRR p 95% CI
Behavior at baseline1 1.05 <.001 1.04-107 1.14 <.001 1.09-1.19 1.09 .002 1.03-1.15 1.01 .081 0.99-1.03
Time 1.02 .504 0.97-1.06 1.04 .429 0.94-1.15 0.95 .225 0.87-1.03 0.95 .117 0.89-1.01
Intervention condition 0.73 .002 0.61-0.89 0.76 .051 0.48-1.01 0.97 .777 0.77-1.21 0.83 .047 0.68-0.99
Within-month contrasts
 Month 1 0.82 .017 0.70-0.97 1.03 .861 0.71-1.50 1.01 .948 0.72-1.42 1.12 .406 0.86-1.45
 Month 2 0.65 <.001 0.54-0.76 0.68 .047 0.46-0.99 1.06 .753 0.74-1.50 0.96 .762 0.74-1.25
 Month 3 0.81 .009 0.69-0.95 0.86 .412 0.59-1.24 1.08 .674 0.76-1.53 0.71 .017 0.54-0.94
Open in a new tab

Note.

1

Term represents participants’ baseline values of each outcome.

Condom Use and PrEP Uptake

Sexual risk behavior outcomes are depicted in Figure 3. Models of the total number of new anal sex partners showed that a significant between-group difference between the intervention conditions, and suggested that although Game Plan participants reported a greater number of anal sex partners at baseline, they reported 50% fewer new partners than control participants over the course of the follow-up period when controlling for number of partners at baseline (see Table 4). Planned contrasts comparing the conditions within each month of follow-up showed significant between-group differences in months 2 and 3. However, in models of both the number of CAS events participants had with any partner and CAS events specifically with casual/unknown HIV status partners, the effect of intervention condition was not significant, and the effect sizes were small or nil. Finally, 40% of Game Plan participants (n = 8) reported having consulted with a medical provider about starting PrEP over the three-month study period, versus 30% of control participants (n = 6). However, 15% of those in the control condition (n = 3) actually received a PrEP prescription during the three-month follow-up period, versus 5% of Game Plan participants (n = 1).

Figure 3.

Figure 3.

Open in a new tab

Between-group differences in the number of new anal sex partners (left), total CAS events (center) and total high-risk CAS events (right) across three months of follow-up

TABLE 4.

GEE models of between-group differences in past 30-day sexual risk outcomes across the three-month study period.

Variable Total # of New Anal
Sex Partners		 Total CAS Events Total High-Risk CAS1
Events
IRR p 95% CI IRR p 95% CI IRR p 95% CI
Behavior at baseline2 1.11 .003 1.04-1.19 1.17 <.001 1.08-1.26 1.15 .002 1.05-1.25
Time 1.23 .003 1.07-1.40 1.08 .380 0.91-1.29 1.15 .126 0.96-1.37
intervention condition 0.50 <.001 0.35-0.72 0.88 .619 0.54-1.44 1.00 .989 0.60-1.68
Within-month contrasts
 Month 1 0.55 .040 0.31-0.97 0.73 .341 0.38-1.39 0.93 .833 0.46-1.86
 Month 2 0.37 .001 0.21-0.67 0.60 .114 0.32-1.13 0.69 .255 0.36-1.31
 Month 3 0.29 <.001 0.17-0.51 0.96 .905 0.52-1.77 0.88 .280 0.47-1.64
Open in a new tab

Note.

1

High-risk CAS events were those that involved CAS with partners who (1) were new/casual, or (2) whose HIV status was unknown, either because participants had not ever asked before, or reported that they did not know.

2

Term represents participants’ baseline values of each outcome.

DISCUSSION

In this preliminary study, we tested whether using Game Plan after receiving HIV testing and standard post-test counseling reduced alcohol use and sexual risk behavior among high-risk, heavy drinking MSM over three months of follow-up, compared to post-test counseling alone. Overall, this preliminary study produced mixed findings. Game Plan participants reported significantly fewer drinking days and fewer binge drinking days across the 3-month follow-up period compared to controls. Game Plan participants also reported significantly fewer alcohol-related problems than controls, and these differences appeared stronger in the later months of the follow-up period. This pattern of results is consistent with some other studies testing of brief alcohol interventions among MSM, which show that the effects of these interventions on drinking outcomes may strengthen over time (34). Contrary to our hypotheses, however, Game Plan participants did not consume fewer drinks when they drank than controls. Still, that those in the Game Plan condition exhibited improvement across three of the four alcohol-related outcomes examined is consistent with past exploratory studies of personalized feedback for alcohol use among MSM (72), and provides considerable optimism about the potential web-based versions of these interventions have for reducing alcohol use among heavy drinking, HIV-negative MSM. The study demonstrates the potential for conducting alcohol intervention opportunistically in the context of HIV testing.

Support for Game Plan’s effects on sexual health outcomes was less clear. Despite having slightly more new anal sex partners at baseline, Game Plan participants reported substantially fewer such partners across the post-intervention follow-up period compared to controls. Given that a quarter of Game Plan participants selected “having sex with only one partner at a time, someone [they] knew well” as a change goal, this could reflect their success toward initiating that change. Participants in both groups reported a reduction in the number of any CAS events over the follow-up period relative to baseline, and although this difference was not statistically significant, Game Plan participants reported a 12% greater reduction in these events than controls, providing some evidence of benefit. Similarly, both groups showed a comparable reduction in the number of CAS events with casual/unknown HIV status partners during the follow-up relative to baseline, and the effect size for between-group differences was close to zero across the follow-up period, suggesting that Game Plan had little effect on this outcome. Finally, although there was some evidence that Game Plan may have prompted more high-risk MSM to consult with a medical provider about starting PrEP, more participants in the control condition reported actually having started PrEP over the course of the three-month follow-up period, providing mixed results about Game Plan’s potential to help connect these men with PrEP.

These results show that the size of Game Plan’s effects varied across outcome, but the overall direction and pattern of effects are consistent with some benefit for both alcohol use and sexual risk reduction. Although one cannot establish efficacy in a preliminary study like this, Game Plan participants generally reduced all outcome behaviors after using the app when compared to baseline and appeared to mostly fare as well or better than controls (although not always significantly so). Given that delivering such a brief intervention in HIV/STI testing clinics would be inexpensive and unburdensome to both to testing providers and to patients themselves, future research on Game Plan could involve conducting preliminary research on implementation approaches, in addition to more rigorous tests of efficacy.

Limitations

The biggest limitation of this study was its small sample size (N = 40). Although this sample size is consistent with a Stage 1b pilot randomized controlled trial (73), a larger sample would have allowed more confident conclusions about Game Plan’s potential for helping high-risk, heavy drinking MSM reduce their alcohol use and risk for HIV. Similarly, the study’s relatively short follow-up period prevented us from exploring the durability of Game Plan’s effects on outcomes. Relying solely on self-report to assess the study’s primary outcomes may also have been a limitation. Although past studies largely support the accuracy of self-reported alcohol use and sexual risk behavior when appropriately assessed (74-77), collecting clinically-relevant biomarkers of these behaviors, such as bacterial STIs and phosphatidylethanol (PEth, a biomarker of alcohol use; 78, 79), could provide a stronger test of the intervention’s potential to improve health outcomes. Next, our comparison of those who received standard post-test counseling alone to those who received post-test counseling plus Game Plan meant that control participants received an “active” intervention (i.e., one that addressed sexual risk behavior, which the intervention also addressed) and that Game Plan participants received “more” of an intervention than controls; our goal was primarily to explore whether using Game Plan could provide additional benefits for behavior change beyond standard practice. Current CDC guidelines suggest that those providing HIV testing also offer risk reduction counseling after testing, but do not mandate that patients receive it in order to get tested to avoid it serving as a barrier to testing (80). However, Game Plan was not intended to replace post-test counseling in these clinics, but to serve as either a compliment to it or another option for those uninterested in meeting with a counselor. As such, we believe that these conditions appropriately reflect how Game Plan might be most often used in community HIV test clinics. Finally, Game Plan was specifically developed to address sexual health among heavy drinking, high-risk MSM. Given that both the intervention content and this study focused specifically on this population, these results may not generalize to other groups.

In summary, findings from this initial study of Game Plan’s effects provide considerable optimism about its potential to help heavy drinking, high-risk MSM reduce their drinking. Its potential to help reduce HIV-risk behavior was less clear, but those who used Game Plan still showed meaningful reductions in key outcomes (e.g., number of new sex partners, condomless sex overall). Given the strong association that is consistently observed between heavy drinking and sexual risk behavior in across populations (including MSM; 18, 19, 22, 23) and robust evidence that alcohol interventions reduce risk behavior (81), further and more rigorous testing is clearly warranted.

Acknowledgements

This manuscript was supported by R34AA023478 and L30AA023336 from the National Institute on Alcohol Abuse and Alcoholism.

Footnotes

Conflicts of interest: The authors declare that they have no conflicts of interest.

Informed consent: Informed consent was obtained from all individual participants included in the study.

Ethical approval: All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

REFERENCES

  • 1.Centers for Disease Control and Prevention . Trends in HIV diagnoses, 2005-2014 Atlanta, GA: U.S. Department of Health and Human Services; 2016. [Available from: http://www.webcitation.org/6vAPQZG6c. [Google Scholar]
  • 2.Centers for Disease Control and Prevention. HIV among African American gay and bisexual men. Atlanta, GA: National Center for HIV/AIDS, Viral Hepatitis, STD, and TB Prevention; 2017. [Google Scholar]
  • 3.Centers for Disease Control and Prevention. HIV Surveillance Report, 2015 Atlanta, GA: U.S. Department of Health and Human Services; 2016. [Available from: http://www.webcitation.org/6vAPZPU3R. [Google Scholar]
  • 4.Centers for Disease Control and Prevention. Lifetime risk of HIV diagnosis Atlanta, GA: National Center for HIV/AIDS, Viral Hepatitis, STD, and TB Prevention; 2016. [Available from: https://www.cdc.gov/nchhstp/newsroom/2016/croi-press-release-risk.html. [Google Scholar]
  • 5.Punyacharoensin N, Edmunds WJ, De Angelis D, Delpech V, Hart G, Elford J, et al. Effect of pre-exposure prophylaxis and combination HIV prevention for men who have sex with men in the UK: a mathematical modelling study. The lancet HIV. 2016;3(2):e94–e104. [DOI] [PubMed] [Google Scholar]
  • 6.Centers for Disease Control and Prevention. As many as 185,000 new HIV infections in the U.S. could be prevented by expanding testing, treatment, PrEP February, 2016. [Available from: https://www.cdc.gov/nchhstp/newsroom/2016/croi-press-release-prevention.html. [Google Scholar]
  • 7.Siegler A, Mouhanna F, Giler R, McCallister S, Yeung H, Jones J, et al. , editors. Distribution of active PrEP prescriptions and the PrEP-to-need ratio, US, Q2 2017. Conference on Retroviruses and Opportunistic Infections (CROI); 2018; Boston, MA. [Google Scholar]
  • 8.Reece M, Herbenick D, Schick V, Sanders SA, Dodge B, Fortenberry JD. Condom use rates in a national probability sample of males and females ages 14 to 94 in the United States. The journal of sexual medicine. 2010;7(s5):266–76. [DOI] [PubMed] [Google Scholar]
  • 9.Cohen MS, Chen YQ, McCauley M, Gamble T, Hosseinipour MC, Kumarasamy N, et al. Antiretroviral therapy for the prevention of HIV-1 transmission. New England Journal of Medicine. 2016;375(9):830–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Garofalo R, Herrick A, Mustanski BS, Donenberg GR. Tip of the iceberg: Young men who have sex with men, the Internet, and HIV risk. American Journal of Public Health. 2007;97(6):1113. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Gold RS, Skinner MJ, Grant PJ, Plummer DC. Situational factors and thought processes associated with unprotected intercourse in gay men. Psychology and Health. 1991;5(4):259–78. [Google Scholar]
  • 12.Blumenthal J, Mulvihill DE, Jain S, Graber S, Hanashiro M, Ellorin E, et al. , editors. HIV risk perception among men who have sex with men: A randomized controlled trial. Conference on Retroviruses and Opportunistic Infections (CROI); March, 2018; Boston, MA. [Google Scholar]
  • 13.MacKellar DA, Valleroy LA, Secura GM, Behel S, Bingham T, Celentano DD, et al. Unrecognized HIV infection, risk behaviors, and perceptions of risk among young men who have sex with men: opportunities for advancing HIV prevention in the third decade of HIV/AIDS. JAIDS Journal of Acquired Immune Deficiency Syndromes. 2005;38(5):603–14. [DOI] [PubMed] [Google Scholar]
  • 14.MacKellar DA, Valleroy LA, Secura GM, Behel S, Bingham T, Celentano DD, et al. Perceptions of lifetime risk and actual risk for acquiring HIV among young men who have sex with men. AIDS and Behavior. 2007;11(2):263–70. [DOI] [PubMed] [Google Scholar]
  • 15.Koblin BA, Husnik MJ, Colfax G, Huang Y, Madison M, Mayer K, et al. Risk factors for HIV infection among men who have sex with men. Aids. 2006;20(5):731–9. [DOI] [PubMed] [Google Scholar]
  • 16.Sander PM, Cole SR, Stall RD, Jacobson LP, Eron JJ, Napravnik S, et al. Joint effects of alcohol consumption and high-risk sexual behavior on HIV seroconversion among men who have sex with men. AIDS (London, England). 2013;27(5):815. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Baliunas D, Rehm J, Irving H, Shuper P. Alcohol consumption and risk of incident human immunodeficiency virus infection: a meta-analysis. International journal of public health. 2010;55(3):159–66. [DOI] [PubMed] [Google Scholar]
  • 18.Rehm J, Shield KD, Joharchi N, Shuper PA. Alcohol consumption and the intention to engage in unprotected sex: systematic review and meta-analysis of experimental studies. Addiction. 2012;107(1):51–9. [DOI] [PubMed] [Google Scholar]
  • 19.Scott-Sheldon LA, Carey KB, Cunningham K, Johnson BT, Carey MP, Team MR. Alcohol use predicts sexual decision-making: a systematic review and meta-analysis of the experimental literature. AIDS and Behavior. 2016;20(1):19–39. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Maisto SA, Palfai T, Vanable PA, Heath J, Woolf-King SE. The effects of alcohol and sexual arousal on determinants of sexual risk in men who have sex with men. Archives of sexual behavior. 2012;41(4):971–86. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Shuper PA, Joharchi N, Monti PM, Loutfy M, Rehm J. Acute Alcohol Consumption Directly Increases HIV Transmission Risk: A Randomized Controlled Experiment. J Acquir Immune Defic Syndr. 2017;76(5):493–500. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Kahler CW, Wray TB, Pantalone DW, Kruis RD, Mastroleo NR, Monti PM, et al. Daily associations between alcohol use and unprotected anal sex among heavy drinking HIV-positive men who have sex with men. AIDS Behav 2015;19(3):422–30. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Vosburgh HW, Mansergh G, Sullivan PS, Purcell DW. A review of the literature on event-level substance use and sexual risk behavior among men who have sex with men. AIDS and Behavior. 2012;16(6):1394–410. [DOI] [PubMed] [Google Scholar]
  • 24.Noar SM. Behavioral interventions to reduce HIV-related sexual risk behavior: review and synthesis of meta-analytic evidence. AIDS Behav. 2008;12(3):335–53. [DOI] [PubMed] [Google Scholar]
  • 25.Johnson WD, Holtgrave DR, McClellan WM, Flanders WD, Hill AN, Goodman M. HIV intervention research for men who have sex with men: A 7–year update. AIDS education & prevention. 2005;17(6):568–89. [DOI] [PubMed] [Google Scholar]
  • 26.Johnson WD, Diaz RM, Flanders WD, Goodman M, Hill AN, Holtgrave D, et al. Behavioral interventions to reduce risk for sexual transmission of HIV among men who have sex with men. Cochrane Database Syst Rev. 2008;3(3). [DOI] [PubMed] [Google Scholar]
  • 27.Moyer A, Finney JW, Swearingen CE, Vergun P. Brief interventions for alcohol problems: a meta-analytic review of controlled investigations in treatment-seeking and non-treatment-seeking populations. Addiction. 2002;97(3):279–92. [DOI] [PubMed] [Google Scholar]
  • 28.Bien TH, Miller WR, Tonigan JS. Brief interventions for alcohol problems: a review. Addiction. 1993;88(3):315–35. [DOI] [PubMed] [Google Scholar]
  • 29.Carey KB, Scott-Sheldon LA, Carey MP, DeMartini KS. Individual-level interventions to reduce college student drinking: a meta-analytic review. Addict Behav. 2007;32(11):2469–94. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Vasilaki EI, Hosier SG, Cox WM. The efficacy of motivational interviewing as a brief intervention for excessive drinking: a meta-analytic review. Alcohol and Alcoholism. 2006;41(3):328–35. [DOI] [PubMed] [Google Scholar]
  • 31.Hasin DS, Aharonovich E, O'leary A, Greenstein E, Pavlicova M, Arunajadai S, et al. Reducing heavy drinking in HIV primary care: a randomized trial of brief intervention, with and without technological enhancement. Addiction. 2013;108(7):1230–40. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Aharonovich E, Hatzenbuehler ML, Johnston B, O'Leary A, Morgenstern J, Wainberg ML, et al. A low-cost, sustainable intervention for drinking reduction in the HIV primary care setting. AIDS Care. 2006;18(6):561–8. [DOI] [PubMed] [Google Scholar]
  • 33.Monti PM, Mastroleo NR, Barnett NP, Colby SM, Kahler CW, Operario D. Brief motivational intervention to reduce alcohol and HIV/sexual risk behavior in emergency department patients: A randomized controlled trial. Journal of consulting and clinical psychology. 2016;84(7):580. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Kahler CW, Pantalone DW, Mastroleo NR, Liu T, Bove G, Ramratnam B, et al. Motivational interviewing with personalized feedback to reduce alcohol use in HIV-infected men who have sex with men: A randomized controlled trial. Journal of consulting and clinical psychology. 2018;86(8):645. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Coates TJ, Richter L, Caceres C. Behavioural strategies to reduce HIV transmission: How to make them work better. Lancet. 2008;23(372):669–84. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Higa DH, Crepaz N, Marshall KJ, Kay L, Vosburgh HW, Spikes P, et al. A systematic review to identify challenges of demonstrating efficacy of HIV behavioral interventions for gay, bisexual, and other men who have sex with men (MSM). AIDS and Behavior. 2013;17(4):1231–44. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Lustria ML, Cortese J, Noar SM, Glueckauf RL. Computer-tailored health interventions delivered over the Web: review and analysis of key components. Patient Educ Couns. 2009;74(2):156–73. [DOI] [PubMed] [Google Scholar]
  • 38.Miller MB, Leffingwell T, Claborn K, Meier E, Walters S, Neighbors C. Personalized feedback interventions for college alcohol misuse: an update of Walters & Neighbors (2005). Psychology of addictive behaviors : journal of the Society of Psychologists in Addictive Behaviors. 2013;27(4):909–20. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Walters ST, Neighbors C. Feedback interventions for college alcohol misuse: What, why and for whom? Addictive behaviors. 2005;30(6):1168–82. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Wagener TL, Leffingwell TR, Mignogna J, Mignogna MR, Weaver CC, Cooney NJ, et al. Randomized trial comparing computer-delivered and face-to-face personalized feedback interventions for high-risk drinking among college students. Journal of substance abuse treatment. 2012;43(2):260–7. [DOI] [PubMed] [Google Scholar]
  • 41.Carey KB, Carey MP, Henson JM, Maisto SA, DeMartini KS. Brief alcohol interventions for mandated college students: comparison of face-to-face counseling and computer-delivered interventions. Addiction. 2011;106(3):528–37. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Pedersen ER, Parast L, Marshall GN, Schell TL, Neighbors C. A randomized controlled trial of a web-based, personalized normative feedback alcohol intervention for young-adult veterans. J Consult Clin Psychol. 2017;85(5):459–70. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Carey KB, Scott-Sheldon LA, Elliott JC, Garey L, Carey MP. Face-to-face versus computer-delivered alcohol interventions for college drinkers: A meta-analytic review, 1998 to 2010. Clinical psychology review. 2012;32(8):690–703. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Rooke S, Thorsteinsson E, Karpin A, Copeland J, Allsop D. Computer-delivered interventions for alcohol and tobacco use: a meta-analysis. Addiction. 2010;105(8):1381–90. [DOI] [PubMed] [Google Scholar]
  • 45.Schnall R, Travers J, Rojas M, Carballo-Diéguez A. eHealth interventions for HIV prevention in high-risk men who have sex with men: a systematic review. Journal of medical Internet research. 2014;16(5). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46.Barnett NP, Monti PM, Wood MD. Motivational interviewing for alcohol-involved adolescents in the emergency room. Innovations in adolescent substance abuse interventions. 2001:143–68. [Google Scholar]
  • 47.Barnett NP, Goldstein AL, Murphy JG, Colby SM, Monti PM. “I’ll Never Drink Like That Again”: Characteristics of Alcohol-Related Incidents and Predictors of Motivation to Change in College Students. Journal of studies on alcohol. 2006;67(5):754. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.Mun EY, White HR, Morgan TJ. Individual and situational factors that influence the efficacy of personalized feedback substance use interventions for mandated college students. Journal of consulting and clinical psychology. 2009;77(1):88. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49.Carey KB, Scott-Sheldon LA, Garey L, Elliott JC, Carey MP. Alcohol interventions for mandated college students: A meta-analytic review. American Psychological Association; 2016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50.Walton MA, Goldstein AL, Chermack ST, McCammon RJ, Cunningham RM, Barry KL, et al. Brief alcohol intervention in the emergency department: moderators of effectiveness. J Stud Alcohol Drugs. 2008;69(4):550–60. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51.Mimiaga MJ, Goldhammer H, Belanoff C, Tetu AM, Mayer KH. Men who have sex with men: perceptions about sexual risk, HIV and sexually transmitted disease testing, and provider communication. Sexually transmitted diseases. 2007;34(2):113–9. [DOI] [PubMed] [Google Scholar]
  • 52.Spielberg F, Branson BM, Goldbaum GM, Lockhart D, Kurth A, Celum CL, et al. Overcoming barriers to HIV testing: preferences for new strategies among clients of a needle exchange, a sexually transmitted disease clinic, and sex venues for men who have sex with men. Journal of Acquired Immune Deficiency Syndromes. 2003;32(3):318–27. [DOI] [PubMed] [Google Scholar]
  • 53.Kellerman SE, Lehman JS, Lansky A, Stevens MR, Hecht FM, Bindman AB, et al. HIV testing within at-risk populations in the United States and the reasons for seeking or avoiding HIV testing. Journal of acquired immune deficiency syndromes (1999). 2002;31(2):202–10. [DOI] [PubMed] [Google Scholar]
  • 54.Centers for Disease Control and Prevention. Revised guidelines for HIV counseling, testing, and referral. MMWR Recommendations and reports: Morbidity and mortality weekly report Recommendations and reports/Centers for Disease Control. 2001;50(RR-19):1. [PubMed] [Google Scholar]
  • 55.Kamb ML, Dillon BA, Fishbein M, Willis KL. Quality assurance of HIV prevention counseling in a multi-center randomized controlled trial. Project RESPECT Study Group. Public Health Rep. 1996;111 Suppl 1(Suppl 1):99–107. [PMC free article] [PubMed] [Google Scholar]
  • 56.Metsch LR, Feaster DJ, Gooden L, Schackman BR, Matheson T, Das M, et al. Effect of risk-reduction counseling with rapid HIV testing on risk of acquiring sexually transmitted infections: the AWARE randomized clinical trial. JAMA : the journal of the American Medical Association. 2013;310(16):1701–10. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 57.Wray TB, Kahler CW, Simpanen EM, Operario D. Game Plan: A web application to help men who have sex with men reduce their HIV risk and alcohol use. JMIR Formative Research. 2018;Online ahead of print. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 58.Cooper A, Reimann R, Cronin D. About Face 3: The Essentials of Interaction Design: John Wiley & Sons; 2007. [Google Scholar]
  • 59.Goodwin K Designing for the digital age: How to create human-centered products and services: John Wiley & Sons; 2011. [Google Scholar]
  • 60.Noar SM, Black HG, Pierce LB. Efficacy of computer technology-based HIV prevention interventions: a meta-analysis. Aids. 2009;23(1):107–15. [DOI] [PubMed] [Google Scholar]
  • 61.Moore CG, Carter RE, Nietert PJ, Stewart PW. Recommendations for planning pilot studies in clinical and translational research. Clinical and translational science. 2011;4(5):332–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 62.Whitehead AL, Julious SA, Cooper CL, Campbell MJ. Estimating the sample size for a pilot randomised trial to minimise the overall trial sample size for the external pilot and main trial for a continuous outcome variable. Statistical methods in medical research. 2016;25(3):1057–73. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 63.Miller WR, Rollnick S. Motivational interviewing: Preparing people to change addictive behavior. New York: Guilford Press; 1991. [Google Scholar]
  • 64.Miller WR, C’de Baca J, Matthews D, Wilbourne P. Personal values card sort 2001. [Available from: http://www.webcitation.org/6vAPu1yfc. [Google Scholar]
  • 65.Sobell L, Sobell M. Timeline followback user’s guide: A calendar method for assessing alcohol and drug use. Toronto: Addiction Research Foundation; 1996. [Google Scholar]
  • 66.Sobell LC, Sobell MB. Timeline follow-back Measuring alcohol consumption: Springer; 1992. p. 41–72. [Google Scholar]
  • 67.Schroder KE, Johnson CJ, Wiebe JS. Interactive Voice Response Technology applied to sexual behavior self-reports: a comparison of three methods. AIDS Behav. 2007;11(2):313–23. [DOI] [PubMed] [Google Scholar]
  • 68.Carey MP, Carey K, Maisto S, Gordon C, Weinhardt L. Assessing sexual risk behaviour with the Timeline Followback (TLFB) approach: continued development and psychometric evaluation with psychiatric outpatients. International journal of STD & AIDS. 2001;12(6):365–75. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 69.Wray TB, Adia AC, Perez AE, Simpanen E, Woods L-A, Celio MA, et al. Timeline: A web application for assessing the timing and details of health behaviors. The American Journal of Drug and Alcohol Abuse 2018;Online ahead of print. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 70.Kahler CW, Hustad J, Barnett NP, Strong DR, Borsari B. Validation of the 30-day version of the Brief Young Adult Alcohol Consequences Questionnaire for use in longitudinal studies. Journal of Studies on Alcohol and Drugs. 2008;69(4):611–5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 71.Little R, Yau L. Intent-to-treat analysis for longitudinal studies with drop-outs. Biometrics. 1996:1324–33. [PubMed] [Google Scholar]
  • 72.Kuerbis AN, Schaumberg K, Davis CM, Hail L, Morgenstern J. Unpacking personalized feedback: an exploratory study of the impact of its components and the reactions it elicits among problem drinking men who have sex with men. Substance use & misuse. 2014;49(4):383–94. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 73.Rounsaville BJ, Carroll KM, Onken LS. A stage model of behavioral therapies research: Getting started and moving on from stage I. Clinical Psychology: Science and Practice. 2001;8(2):133–42. [Google Scholar]
  • 74.Hjorthøj CR, Fohlmann A, Larsen AM, Arendt M, Nordentoft M. Correlations and agreement between delta-9-tetrahydrocannabinol (THC) in blood plasma and timeline follow-back (TLFB)-assisted self-reported use of cannabis of patients with cannabis use disorder and psychotic illness attending the CapOpus randomized clinical trial. Addiction. 2012;107(6):1123–31. [DOI] [PubMed] [Google Scholar]
  • 75.Hjorthøj CR, Hjorthøj AR, Nordentoft M. Validity of timeline follow-back for self-reported use of cannabis and other illicit substances—systematic review and meta-analysis. Addictive behaviors. 2012;37(3):225–33. [DOI] [PubMed] [Google Scholar]
  • 76.Simons JS, Wills TA, Emery NN, Marks RM. Quantifying alcohol consumption: self-report, transdermal assessment, and prediction of dependence symptoms. Addictive behaviors. 2015;50:205–12. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 77.Wray TB, Kahler CW, Monti PM. Using ecological momentary assessment (EMA) to study sex events among very high-risk men who have sex with men (MSM). AIDS and Behavior. 2016;20(10):2231–42. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 78.Viel G, Boscolo-Berto R, Cecchetto G, Fais P, Nalesso A, Ferrara SD. Phosphatidylethanol in blood as a marker of chronic alcohol use: a systematic review and meta-analysis. International journal of molecular sciences. 2012;13(11):14788–812. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 79.Stewart SH, Law TL, Randall PK, Newman R. Phosphatidylethanol and alcohol consumption in reproductive age women. Alcoholism, clinical and experimental research. 2010;34(3):488–92. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 80.Branson BM, Handsfield HH, Lampe MA, Janssen RS, Taylor AW, Lyss SB, et al. Revised recommendations for HIV testing of adults, adolescents, and pregnant women in health-care settings. Morbidity and Mortality Weekly Report: Recommendations and Reports. 2006;55(14):1-CE–4. [PubMed] [Google Scholar]
  • 81.Scott-Sheldon LA, Carey KB, Johnson BT, Carey MP, Team MR. Behavioral interventions targeting alcohol use among people living with HIV/AIDS: a systematic review and meta-analysis. AIDS and behavior. 2017;21(2):126–43. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES