Do Diary Studies Cause Behavior Change? An Examination of Reactivity in Sexual Risk and Substance Use in Young Men Who Have Sex with Men

Abstract

Behavioral diaries are frequently used for observing sexual and substance use behaviors, but participating in diary studies may cause behavior change. This study examined change in sexual and substance use behaviors among young men who have sex with men (YMSM) in a two-month diary study compared to control. An analytic sample of 324 YMSM was randomized to receive daily diaries, weekly diaries, or no diaries (control) for 2 months. Half of the diary participants were randomized to receive automated weekly feedback. Between-subjects analyses found no evidence of change in sexual or substance use behaviors from baseline to 2-month follow-up when comparing the diary conditions to control. Within-persons growth mixture models of all diary data showed significant decreases in condomless anal sex (CAS) and illicit drug use. Weekly automated feedback had no effect on behavior change. Findings provide evidence of change in CAS and illicit drug use amongst diary participants.

Introduction

Men who have sex with men (MSM) accounted for 70% of new HIV infections in 2015, and adolescent and young adult MSM (YMSM) are the only group in which rates of new infections are increasing [1]. Further, YMSM have been found to use alcohol and drugs at substantially higher rates than their heterosexual counterparts [2, 3], and substance use has been linked to HIV risk behavior among YMSM, particularly when used before or in concert with sex [4, 5]. Given rising HIV incidence among YMSM, there is an urgent need to establish innovative and scientifically sound prevention strategies.

Behavioral diaries are frequently used as a method for observing various health-related behaviors and their correlates [6], and this approach has been successfully utilized to understand substance use and sexual risk behaviors in young and adult MSM populations [7–12]. Sexual behavior diaries in particular have been found to more accurately estimate frequency of condom use than retrospective surveys [12–14]. Further, this methodology has helped to clarify whether alcohol and drug use are indeed associated with condomless sex by matching the timing of the substance use to the occurrence of the sexual encounter [5, 9, 10, 12]. This literature has clarified that illicit drug use is a consistent event-level predictor of condomless sex in MSM [5], but the association between drinking before sex and condomless sex is less straightforward [5, 10]. Importantly, the accuracy of observational diary studies rests on the assumption that this method does not produce participant reactivity (i.e., systematic change in responding resulting from repeated assessment). For example, if diary studies lead to change in sexual risk behavior over time, then this may also alter the association between relevant independent variables (e.g., substance use) and sexual risk. Several studies have found no evidence of reactivity in diary studies of young and adult MSM [10, 15, 16], but others have found evidence of systematic behavior change [8, 17, 18]. For example, New-comb and Mustanski [17] found that MSM enrolled in a weekly diary study decreased their sexual risk behavior if they had high levels of certain social cognitive variables, such as HIV risk reduction motivation.

Several variations of diary administration schedules have been used across various health domains, including daily and once-weekly diaries (e.g., a weekly timeline followback of the prior 7 days) [15, 18–22]. Recently, Glick et al. [15] investigated reactivity in sexual risk behavior among MSM enrolled in a web-based sexual behavior diary study by comparing diary participants to a no diary control condition. Control condition participants increased their number of condomless anal sex (CAS) acts relative to those who completed diaries. While this study indicates that participation in an observational diary study did not cause behavior change, more research is needed to determine whether different diary schedule frequencies (e.g., daily, weekly) are more or less likely to influence behavior change relative to a control condition. Furthermore, given the central role that substance use plays in HIV transmission risk behavior, more research is needed to better understand substance use reactivity, as well as how change in sexual risk and substance use behavior may co-vary in diary studies. Understanding these effects will provide a more nuanced understanding of the accuracy of using the diary methodology as an observational approach to understand health-behavior patterns.

Behavioral diaries are also frequently used by researchers and practitioners as self-monitoring tools to encourage health-behavior change [20–22]. The use of self-monitoring diaries for enacting behavior change is fairly ubiquitous, particularly as technology platforms have contributed to the ease of administration of these intervention strategies. According to the Pew Research Center, 64% of U.S. adults owned a smartphone in 2015 [23], and in another survey conducted that same year, 58% of smartphone users had specifically downloaded an app for the purpose of tracking and changing a health-related behavior [24]. This methodological approach to behavior change assumes that the act of documenting a behavior leads to self-monitoring, and therefore maximizes behavioral reactivity. This assumption opposes that which is made when using this same method as an observational tool. Whether diary approaches do indeed lead to self-monitoring and behavior changes remains unclear. Several reviews of self-monitoring interventions aimed at encouraging health-behavior change have questioned how well self-monitoring tools actually achieve clinically meaningful behavior change, even when statistically significant change is present [20, 22].

Additionally, self-monitoring tools often provide feed-back to participants about their behavior change progress, which is thought to maximize reactivity by increasing motivation to engage in behavior change. Indeed, self-monitoring and feedback have been used to encourage behavior change in countless interventions, such as those based on the transtheoretical model of change [25]. Social cognitive theory of self-regulation describes the processes by which individuals monitor their behavior over time for the purpose of enacting and maintaining behavior change [26]. Self-regulation is dependent upon self-monitoring, including the fidelity, consistency and proximity of self-monitoring to the target behavior. Receipt of consistent feedback that clearly illustrates behavior patterns is necessary in order to change targeted health behaviors. Applied to sexual risk and substance use behaviors, Social Cognitive Theory would posit that individuals who receive feedback illustrating their individualized link between their substance use and sexual risk behaviors would likely report larger decreases in sexual risk than those who do not receive such feedback. However, to our knowledge no existing studies have randomly assigned participants in a diary study to receive feedback or to complete behavioral diaries without feedback, so it is unclear the degree to which feedback improves behavior change.

Furthermore, little research has evaluated differences between daily and weekly diary approaches with regard to risk behavior change. Daily diaries may provide more accurate data than weekly diaries because they minimize memory bias, particularly when reporting on symptoms that may be less anchored by discrete events compared to more concrete behaviors (e.g., it may be more difficult to recall an affective state compared to a substance use event). On the other hand, recent research suggests that some participants may be more compliant with diary studies when on a weekly compared to daily schedule [27]. It remains unclear whether one diary frequency may lead to more behavioral reactivity than another, though Social Cognitive Theory of self-regulation would predict that daily approaches, or those that assess the behavior in closer proximity to the occurrence of the behavior, would result in more behavior change, particularly when feedback is provided to the participant about their engagement in the target behavior [26].

The goal of this study was to examine the extent to which sexual and substance use behaviors change in a sample of YMSM as a result of participating in a behavioral diary study in order to maximize their value as a tool for both observation and intervention. The current study randomized binge-drinking and/or drug-using YMSM to a daily, weekly or no diary control condition for a period of two months. Further, half of the YMSM in the daily and weekly conditions were randomized to receive automated feedback about their substance use and sexual risk behaviors each week. Based on Social Cognitive Theory of self-regulation [26], we hypothesized that: (a) YMSM in both diary conditions would reduce their substance use and sexual risk behaviors relative those in the control condition; (b) YMSM in the daily condition would experience more behavioral reactivity than those in the weekly condition; and (c) YMSM receiving feedback each week would reduce their risk behaviors more than those who did not. In exploratory analysis, we examined whether any observed change in sexual risk behavior during the course of the diary study was associated with a similar change in alcohol and drug use behaviors.

Methods

Participants

Participants were 370 YMSM enrolled in a 2-month prospective diary study of substance use and sexual behavior. In order to be included in the analytic sample of the current study, participants either had to have: (1) both baseline and 2-month follow-up data; or (2) have answered at least one diary survey (N = 324). The mean age of the analytic sample at the start of the study was 23.5 (SD = 3.2). The largest racial group in the analytic sample was White (45.1%), followed in order of size by Hispanic/Latino (23.5%), Black/African American (19.7%), multi-racial (7.8%), participants who reported their race/ethnicity as other (1.6%), Asian/Pacific Islander (1.3%), and Native American (0.9%). The majority of participants identified as gay (87.7%) with a minority identifying as bisexual (12.3%). At the start of the study, 42.2% of the sample reported being in a serious romantic relationship.

Procedures and Design

Participants were recruited online via national advertisements posted on Facebook from August 2014 to April 2015. Enrolled participants were fairly evenly distributed across the United States; the largest percentage was from the South (34.0%), followed by the West (26.3%), Midwest (22.2%), and Northeast (17.3%). Inclusion criteria were: (1) assigned male at birth and current male gender identity; (2) oral or anal sex with another man during the past 6 months; (3) any binge-drinking (i.e., 5 or more drinks on a single occasion) or illicit drug use during the past 30 days; (4) between the ages of 16 and 29; (5) HIV-negative or unknown serostatus. The protocol was approved by the Institutional Review Board (IRB) at Northwestern University with a waiver of parental permission under 45 CFR 46.408(c) for participants aged 16–17 [28].

YMSM who clicked on the Facebook ads were linked to a landing page that described the study and interested individuals were directed to a screener that assessed the above inclusion criteria and collected various forms of demographic and contact information to aid in identifying potential duplicate or fake participants. Eligible participants were sent a link to complete a baseline assessment after which we used block randomization to randomize YMSM to one of three diary conditions. In the full sample, 121 YMSM were assigned to weekly diaries, 120 to daily diaries, and 129 to the no diary control condition. Daily diary participants reported their substance use and sexual behaviors from the preceding day, while weekly participants reported these behaviors retrospectively day-by-day for each day of the preceding week (i.e., 7-day timeline followback). Further, within each of these two active diary conditions, half of the participants were randomly assigned to receive automated feedback each week (61 participants in the weekly condition and 60 participants in the daily condition; total feedback in full sample N = 121). Feedback was delivered in the form of data visualizations (i.e., graphs and charts) of their frequency of substance use and sexual behavior, as well as change in these risk behaviors from the prior weeks. Feedback was provided on a weekly basis immediately after completing their weekly diary (weekly condition) or their seventh daily diary of the week (daily condition).

Participants in each of the active diary conditions were allotted 48 h to complete each diary survey. Automated emails were sent to participants as a reminder to complete diary surveys but very little personalized contact was utilized to encourage participation. The average diary completion rate across both the daily and weekly conditions was 56.11% (SD = 37.95%), and the median completion rate was 64.29%. There were 39 participants (16.2%) who never completed a diary (29 in the daily condition and 10 in the weekly condition). Excluding those who never began their diaries, the mean completion rate for the entire diary sample was 66.94% (SD = 31.47%). We examined demographic characteristics and randomization condition as predictors of diary compliance. Only sexual orientation was a statistically significant predictor of compliance, such that bisexual-identified participants had a higher completion rate that gay-identified participants. More details about patterns and predictors of diary compliance in this study are reported elsewhere [27]. All participants (including control condition) completed a 2-month follow-up survey to assess change in risk behaviors. Participants were paid up to $60 for participation, pro-rated for participation level. Participants were paid $15 for completing the baseline assessment and $15 for the 2-month follow-up. With regard to diary completion, participants were paid $15 if they completed at least 50% of their diary surveys (regardless of condition) and an additional $15 if they completed at least 85% of their diary surveys. All payments were made upon study completion.

Measures

General Demographics

The demographic questionnaire assessed participants’ age, race/ethnicity, self-reported sexual orientation, and relationship status.

Baseline Substance Use and Sexual Risk Behavior

We created quantity-frequency (QF) indices of alcohol and marijuana use utilizing two items from the Alcohol Use Disorders Identification Test (AUDIT; [29]) and Cannabis Use Disorders Identification Test—Revised (CUDIT-R; [30]), respectively. The frequency item asked: “How often do you [have a drink containing alcohol/use marijuana]?” Response options ranged from never (coded 0) to 4 or more times a week (coded 4). The alcohol use quantity item asked: “How many drinks containing alcohol do you have on a typical day when you are drinking?” Response options ranged from “1 or 2” (coded 1) to “10 or more” (coded 5). The marijuana use quantity item asked: “How many hours were you ‘stoned’ on a typical day when you had been using marijuana?” Response options ranged from “less than 1” (coded 1) to “7 or more” (coded 5). QF indices were calculated by multiplying the quantity items by the frequency items [31]. We created a dichotomous variable to indicate past 2-month use of any of the following other drugs: cocaine/crack, heroin, meth, opiates, prescription depressants (e.g., Xanax, Valium), prescription stimulants (e.g., Ritalin, Adderall), ecstasy/MDMA, GHB, ketamine, poppers, erectile medications, inhalants, or other illicit drugs.

With regard to sexual behavior, we assessed several indicators of risk. First, we measured the total number of sexual partners during the last 2 months with the item: “In the past 2 months, that is since [DATE], with how many different males have you had oral or anal sex?” Next, we assessed frequency of CAS with the item: “On average, how often did you use condoms when you had anal sex with male partners in the past two months?” Response options ranged from “none of the time” (coded 1) to “all of the time” (coded 5). Finally, we asked several questions about their most recent anal sex encounter. If participants reported condomless insertive or receptive anal sex at this encounter, and the encounter occurred within the last 2 months, we coded the variable as “1.” Participants were coded “0” if they used a condom during their most recent anal sex encounter during the last 2-months.

Diary Measure

Participants reported their daily substance use and sex behaviors, either every day (daily condition) or weekly using a 7-day timeline follow-back (weekly condition) for 2 months. We administered three entry questions for each day to assess substance use: “Did you [drink/use marijuana/ use any other drugs] on [insert date]?” Participants who endorsed drinking on a given day were asked the number of drinks they consumed. We created two daily drinking variables: (1) total number of drinks consumed each day; and (2) a dichotomous binge-drinking variable (five or more drinks on a given day). Daily marijuana use was code as “1” for any use and “0” for no use. Participants who endorsed “any other drug use” were provided with a list of illicit drugs and asked to indicate which they used each day (see list in baseline substance use measure). Participants who endorsed use of any of those drugs were coded “1” while those who endorsed no other drug use were coded “0.” In terms of daily sexual behavior, participants reported on specific sexual behaviors (e.g., oral and anal sex) that occurred each day and whether or not a condom was used for each behavior. Daily sexual risk behavior was defined as occurrence of anal sex with a male partner without a condom (either insertive or receptive) on a given day.

Statistical Analyses

In order to compare self-reported risk behaviors measured at baseline and 2-month follow-up, we ran generalized linear mixed models in SPSS. These models were chosen in order to control for any baseline differences on the outcome variables that may have existed between the different research conditions. Models were run separately for alcohol use (QF), marijuana use (QF), illicit drug use, number of sexual partners, frequency of CAS, and CAS at most recent sexual encounter. To test the effects of diary condition on change from baseline to the 2-month follow up, we included diary condition as a moderator. Within all models, we controlled for the effects of age, baseline relationship status, sexual orientation, and race/ethnicity. We then re-ran all of the models with feedback condition as the moderator instead of diary condition to test the effect of feedback on behavior change from baseline to 2-month follow-up.

We followed up the baseline and 2-month comparisons by examining growth curves for the participants in the active diary conditions. We modeled latent growth curves in MPlus in order to understand the effects of participating in the diary portion of the study had on engagement in risk behaviors. We first modeled growth independent of predictors or covariates for each of the daily primary risk behaviors: CAS, alcohol use, binge-drinking, marijuana use, and illicit drug use. Growth models were formed by creating sum scores for each week of the study, which resulted in eight time points for each participant. The decision to aggregate data at the week level for participants in the daily condition was made to address convergence issues associated with attempts to model growth models using dichotomous daily data and 56 time points with our sample size. CAS, binge-drinking, marijuana use, and drug use were measured as the number of days in each week that participants engaged in that particular behavior. Alcohol use was measured as the total number of drinks participants reported for each week. MPlus uses a Poisson distribution to account for count data. In order to address missingness, missing data was imputed using multiple imputation in the MICE R package. In addition to the primary outcome variables, participant demo-graphics (age, race/ethnicity, sexual orientation, student status, employment status, education level, living situation, relationship status, and HIV status), partner demographics (age, gender, race/ethnicity, relationship status with partner, and HIV status), and study condition were included in the imputation process to improve predictions of missing data. Non-linear growth terms were initially tested but were dropped from growth models because there was no evidence of significant non-linear effects.

After examining the initial growth models, we added diary and feedback condition in the models to test for differences associated with study condition. In these models, we also included demographic covariates, including age at base-line, race, relationship status at baseline, and sexual orientation. Race was coded into three separate dummy codes: one representing White participants (White = 1, all others = 0), Black participants (Black = 1, all others = 0), and Latino/ Hispanic participants Latino/Hispanic = 1, all others = 0).

The last growth models we ran were parallel process models. In a parallel process model, the change in multiple measures is modeled simultaneously and the growth terms for both measures are allowed to correlate in order to test how related the growth in the two measures are. Specifically, we were interested in associations between the growth in CAS over the course of the study and growth in substance use. Even for cases in which the substance use behavior did not demonstrate significant change in prior models, we included it in the parallel process models. Correlated slopes between CAS and substance use could still provide important information about the co-occurrence of these behaviors, even in the absence of change over time for the sample as a whole. We created four separate parallel process models to test for those associations: CAS with alcohol use, CAS with binge-drinking, CAS with marijuana use, and CAS with drug use.

In follow-up analyses, we examined correspondence between data reported in the diaries and 2-month follow-up in order to assess whether any differences in patterns of reactivity across models was due to differences in self-report between survey types. We compared alcohol frequency, alcohol quantity, marijuana frequency, illicit drug use (dichotomous), and frequency of CAS. For alcohol and marijuana frequency, we calculated the number of days participants reported use and recoded that number to fit within the 2-month follow-up response options. For alcohol quantity, we took the median number of drinks participants reported on a given day and recoded those into the 2-month follow-up response options. For illicit drug use, participants were coded as having used substances during the diary period if they reported using a substance on at least 1 day. Frequency of CAS was created from the diary data by calculating the percentage of the total number of anal sex acts reported in the diaries that were condomless and recoding those values to fit within the 2-month follow up response options. We tested for differences between the diary and 2-month follow-up data using repeated measures t-tests and Chi square tests.

Results

Baseline and 2‑Month Follow‑Up Comparisons

Generalized linear mixed models comparing differences based on diary condition and feedback condition on change between the baseline and 2-month follow up assessments are presented in Tables 1 and 2. We found no differences in change over time in alcohol QF, marijuana QF, illicit drug use, number of sex partners, frequency of CAS, or CAS with most recent partner between the diary conditions. We also observed no differences between feedback conditions in these substance use and sexual behaviors.

Table 1.

Generalized linear repeated measure model of diary condition moderating change from baseline to 2-month follow up

	Beta (SE)
	Alcohol QF	Marijuana QF	Illicit drug use (dichotomous)	Number of sex partners	Frequency of CAS	CAS w/last partner
Intercept	1.42 (0.23)***	1.60 (0.26)***	− 2.07 (0.89)*	0.94 (0.26)***	1.22 (0.24)***	− 0.42 (0.44)
Time point
Baseline (referent)	–	–	–	–	–	–
2 month	0.00 (0.17)	0.03 (0.16)	0.06 (0.34)	− 0.09 (0.17)	0.09 (0.16)	0.09 (0.31)
Diary
Daily	0.15 (0.18)	− 0.28 (0.18)	0.40 (0.40)	− 0.02 (0.18)	− 0.05 (0.18)	− 0.34 (0.34)
Weekly	0.09 (0.18)	− 0.07 (0.17)	0.66 (0.40)	− 0.02 (0.18)	0.02 (0.17)	− 0.14 (0.33)
Control (referent)	–	–	–	–	–	–
Time point * diary
2 month * daily	0.15 (0.18)	− 0.25 (0.25)	0.29 (0.59)	− 0.27 (0.26)	− 0.12 (0.25)	− 0.37 (0.48)
2 month * weekly	0.09 (0.18)	− 0.04 (0.23)	− 0.85 (0.53)	0.22 (0.25)	− 0.06 (0.24)	− 0.15 (0.46)
2 month * control (referent)	–	–	–	–	–	–

Post-hoc comparisons revealed no differences between daily and weekly conditions in any of the above-presented effects. All models included race/ethnicity, age, sexual orientation, and baseline relationship status as covariates

QF quantity-frequency, CAS condomless anal sex

^*p < 0.05

^**p < 0.01

^***p < 0.001

Table 2.

Generalized linear repeated measure model of feedback condition moderating change from baseline to 2-month follow up

	Beta (SE)
	Alcohol QF	Marijuana QF	Illicit drug use (dichotomous)	Number of sex partners	Frequency of CAS	CAS w/last partner
Intercept	1.43 (0.23)***	1.63 (0.26)***	− 2.15 (0.89)*	0.95 (0.26)***	1.22 (0.24)***	− 0.43 (0.44)
Time point
Baseline (referent)	–	–	–	–	–	–
2 month	0.00 (0.17)	0.03 (0.16)	0.06 (0.34)	− 0.09 (0.17)	0.09 (0.16)	0.10 (0.31)
Feedback
Feedback	0.21 (0.18)	− 0.23 (0.17)	0.35 (0.38)	0.07 (0.18)	− 0.05 (0.18)	− 0.30 (0.34)
No feedback	0.03 (0.18)	− 0.11 (0.17)	0.76 (0.42)	− 0.13 (0.18)	0.02 (0.17)	− 0.17 (0.33)
Control (referent)	–	–	–	–	–	–
Time point * feedback
2 month * feedback	0.05 (0.25)	− 0.08 (0.24)	− 0.46 (0.53)	− 0.13 (0.25)	− 0.15 (0.25)	− 0.23 (0.48)
2 month * no feedback	− 0.01 (0.25)	− 0.16 (0.24)	− 0.28 (0.58)	0.15 (0.25)	− 0.04 (0.25)	− 0.28 (0.46)
2 month * control (referent)	–	–	–	–	–	–

Post-hoc comparisons revealed no differences between daily and weekly conditions in any of the above-presented effects. All models included race/ethnicity, age, sexual orientation, and baseline relationship status as covariates

QF quantity-frequency, CAS condomless anal sex

^*p < 0.05

^**p < 0.01

^***p < 0.001

Risk Behavior Growth Curves

Results for latent growth curve models before the addition of predictors and demographic covariates are presented in Table 3. For the overall diary sample (i.e., excluding control group participants), there was a statistically significant decrease in the number of days each week that participants used drugs and the number of days they engaged in CAS across the 2 month course of the study. Further, the number of days on which participants used marijuana each week increased over the course of the study. There was no statistically significant change in alcohol use or binge-drinking during the study.

Table 3.

Individual growth models of risk behaviors by week

	Standard intercept	Standard slope
Alcohol use	1.16 (0.16)***	− 0.15 (0.12)
Binge drinking	− 0.64 (0.09)***	1.23 (1.11)
Marijuana use	− 0.34 (0.08)***	0.70 (0.35)*
Drug use	− 1.38 (0.10)***	− 1.32 (0.52)*
Condomless anal sex	− 0.96 (0.09)***	− 1.16 (0.27)***

These analyses did not adjust for predictors (i.e., randomization arms) or demographic covariates (i.e., age at baseline, race, relationship status at baseline, and sexual orientation)

^*p < 0.05

^**p < 0.01

^***p < 0.001

Effects of Diary and Feedback Conditions on Growth in Risk Behavior

The effects of diary and feedback conditions in predicting risk behavior intercepts and slopes after controlling for demographic factors are presented in Table 4. Participants in the weekly diary condition began the first week of the study with significantly lower alcohol use, binge drinking, marijuana use, illicit drug use, and CAS compared to the daily condition. Participants in the weekly condition also had a significantly steeper decline in CAS compared to the daily participants. There was no difference in risk behavior growth based on diary condition for alcohol use, binge-drinking, marijuana use, or illicit drug use. Participants who received feedback during the study began the first week with significantly higher illicit drug use compared to participants who did not receive feedback. There was no difference in risk behavior growth for alcohol use, binge-drinking, marijuana use, illicit drug use, or CAS based on feedback condition. Of note, the observed reduction in illicit drug use over time noted above became non-significant in these adjusted models, but the decline in CAS retained significance.

Table 4.

Individual growth models with diary and feedback as predictors and demographic covariates

	Std. intercept	Std. slope	Diary on intercept	Diary on slope	Feedback on intercept	Feedback on slope
Alcohol use	− 1.51 (0.61)*	0.78 (0.88)	− 0.28 (0.06)***	− 0.02 (0.09)	0.11 (0.07)	0.10 (0.09)
Binge drinking	− 2.54 (0.74)***	3.71 (2.39)	− 0.27 (0.08)***	− 0.53 (0.29)	0.13 (0.09)	− 0.02 (0.30)
Marijuana use	0.81 (0.65)	− 1.83 (1.56)	− 0.20 (0.08)*	0.03 (0.24)	− 0.09 (0.08)	− 0.11 (0.19)
Drug use	− 1.61 (0.84)*	− 1.56 (2.01)	− 0.34 (0.10)***	0.03 (0.27)	0.30 (0.11)**	0.43 (0.35)
Condomless anal sex	− 1.39 (0.72)*	− 3.52 (1.41)*	− 0.37 (0.09)***	− 0.47 (0.23)*	0.14 (0.09)	0.17 (0.19)

Diary condition is coded as daily = 0 and weekly = 1, feedback condition is coded as 0 = no feedback and 1 = feedback. These analyses adjusted for demographic covariates (i.e., age at baseline, race, relationship status at baseline, and sexual orientation)

^*p < 0.05

^**p < 0.01

^***p < 0.001

Parallel Process Models

Results for parallel process models are presented in Table 5 (note that the final row within each model shows the correlation between change in CAS and each substance use behavior). We observed a statistically significant positive correlation between change in engagement in CAS and growth in alcohol use, marijuana use, and binge-drinking. In other words, as CAS decreased, alcohol use, binge-drinking and marijuana use also decreased. There was no statistically significant association between change in CAS and change in illicit drug use.

Table 5.

Parallel process models of sexual risk and substance use behaviors

	Std. estimate	p value
Alcohol use and CAS
Alcohol intercept (AI)	1.16 (0.16)	< 0.001
Alcohol slope (AS)	− 0.15 (0.12)	0.199
CAS intercept (CI)	− 0.95 (0.09)	< 0.001
CAS slope (CS)	− 0.79 (0.30)	0.008
AI with AS	− 0.25 (0.13)	0.056
AI with CI	0.50 (0.10)	< 0.001
AI with CS	0.17 (0.23)	0.464
CI with CS	0.52 (0.25)	0.038
CI with AS	− 0.06 (0.13)	0.655
AS with CS	0.82 (0.16)	< 0.001
Marijuana use and CAS
Marijuana intercept (MI)	− 0.39 (0.07)	< 0.001
Marijuana slope (MS)	0.52 (0.32)	0.105
CAS intercept (CI)	− 1.03 (0.10)	< 0.001
CAS slope (CS)	− 1.03 (0.34)	0.003
MI with MS	− 0.44 (0.25)	0.084
MI with CI	0.12 (0.03)	< 0.001
MI with CS	0.05 (0.08)	0.507
CI with CS	0.62 (0.36)	0.081
CI with MS	− 0.06 (0.17)	0.717
MS with CS	0.62 (0.24)	0.009
Illicit drug use and CAS
Drug intercept (DI)	− 1.36 (0.10)	< 0.001
Drug slope (DS)	− 1.18 (0.52)	0.023
CAS intercept (CI)	− 0.95 (0.09)	< 0.001
CAS slope (CS)	− 0.94 (0.32)	0.003
DI with DS	0.23 (0.46)	0.611
DI with CI	0.72 (0.10)	< 0.001
DI with CS	0.42 (0.26)	0.102
CI with CS	0.70 (0.27)	0.010
CI with DS	− 0.24 (0.31)	0.441
DS with CS	0.31 (0.39)	0.432
Binge-drinking and CAS
Binge-drinking intercept (BI)	− 0.64 (0.09)	< 0.001
Binge-drinking slope (BS)	0.17 (0.36)	0.642
CAS intercept (CI)	− 0.95 (0.09)	< 0.001
CAS slope (CS)	− 0.82 (0.29)	0.005
BI with BS	− 0.32 (0.27)	0.230
BI with CI	0.57 (0.10)	< 0.001
BI with CS	0.33 (0.24)	0.180
CI with CS	0.53 (0.25)	0.039
CI with BS	− 0.01 (0.26)	0.977
BS with CS	0.77 (0.22)	< 0.001

The final row within each model presents the covariance between growth in CAS and each substance use behavior

CAS condomless anal sex

Diary and 2‑Month Follow Up Comparisons

Comparisons between the diary and 2-month follow up data are presented in Table 6. We found no significant differences between the two measures on reports of alcohol frequency, alcohol quantity, or CAS frequency. Participants reported a significantly higher frequency of marijuana use in the diaries compared to the 2-month follow-up (diary M = 2.91, 2-month M = 2.51, t = − 4.80, p < 0.001). Participants were also inconsistent in their reports of drug use in the diary and 2-month follow-up. While a similar percentage endorsed having used drugs over the course of the study in both measures (14.7% in diaries, 15.3% in 2-month follow-up), only 32.4% of those who reported use in at least one measure consistently reported use in both.

Table 6.

Comparison of diary data and follow up survey data for the same 2-month period

	Diary % (N)	2 month follow up % (N)	t	DF	p value
Alcohol frequency
Never	3.3 (5)	1.3 (2)	− 1.65	152	0.100
Monthly or less	15.7 (24)	21.6 (33)
2–4 times a month	32.0 (49)	38.6 (59)
2–3 times a week	41.2 (63)	29.4 (45)
4 + times a week	7.8 (12)	9.2 (14)
Alcohol quantity
1 or 2 drinks	31.8 (47)	34.5 (51)	0.67	147	0.504
3 or 4 drinks	38.5 (57)	35.1 (52)
5 or 6 drinks	20.3 (30)	16.2 (24)
7, 8, or 9 drinks	6.8 (10)	8.8 (13)
10 or more drinks	2.7 (4)	5.4 (8)
Marijuana frequency
Never	29.6 (47)	40.3 (64)	− 4.80	159	< 0.001
Monthly or less	13.8 (22)	23.3 (37)
2–4 times a month	11.9 (19)	6.3 (10)
2–3 times a week	25.2 (40)	5.7 (9)
4 + times a week	19.5 (31)	24.5 (39)
Frequency of CAS
None	36.9 (48)	30.0 (39)	− 0.94	129	0.348
Less than half the time	11.5 (15)	20.8 (27)
About half the time	12.3 (16)	13.8 (18)
More than half the time	9.2 (12)	15.4 (20)
All the time	30.0 (39)	20.0 (26)
Drug use	2 month follow up		χ2	DF	p value
Diary	No	Yes	26.04	1	< 0.001
No	91.3 (126)	52.0 (13)
Yes	8.7 (12)	48.0 (12)

Discussion

The current analyses reveal several important considerations for the use of sexual behavior and substance use diaries for observation and intervention. First, analysis of baseline and 2-month follow-up data revealed no differences in change in risk behaviors across diary frequency groups. In contrast, growth curve analyses of the two active diary conditions utilized all available diary data and found significant decreases in CAS and illicit drug use across the 2-month study period. Furthermore, our analyses found no differences in change in risk behavior between those participants who received weekly feedback about their engagement in risk behaviors and those who did not. Finally, parallel process models found that change in CAS was associated with change in alcohol use, binge-drinking and marijuana, indicating that those who did decrease their risk behaviors tended to decrease engagement in multiple behaviors.

Contrary to hypotheses, though consistent with prior diary studies of YMSM [15, 18], these data observed no differences in risk behavior change between the three diary conditions at 2-month follow-up. Upon first glance, this points to an absence of substantial behavioral reactivity in the diary relative to control conditions. However, analysis of baseline to follow-up change relative to control necessitates the use of global items to measure risk behavior that average risk behavior engagement during a specified period of time. Importantly, research has found that within-persons approaches (e.g., behavioral diaries) more accurately capture the frequency and timing of sexual and substance use behaviors [13, 14, 19], which is corroborated by the current analyses which found different patterns of reporting of marijuana and illicit drug use between diaries and 2-month follow-up global items. While the use of a no diary control condition is necessary in order to conclude with certainty that any observed behavior change is not simply due to naturally occurring change over time, it also eliminates the ability to utilize more accurate event-level approaches to measuring change in risk behavior within-persons.

Due to the inherent limitations of between-subjects comparisons of risk behavior change from baseline to 2-month follow-up, we conducted several within-persons analyses using all available data points from the diary surveys to get a more fine-grained picture of behavior change in this study. Growth curve analyses using data from both active diary conditions (i.e., daily and weekly) found that across all participants, illicit drug use and CAS decreased each week across the 2-month study period. Importantly, the only other known study that compared behavior change among YMSM in a sexual diary study to a control condition used a between-subjects approach similar to that described in the preceding paragraph [15]. In contrast, these within-persons analyses suggest that between-subjects approaches may mask behavior change by utilizing measures that average risk behavior engagement across multiple risk episodes. If indeed self-monitoring is at the root of these within-persons changes, then these findings indicate that YMSM may perceive drug use and CAS as more acutely risky or dangerous than marijuana or alcohol use. YMSM may be more motivated to reduce behaviors they perceive as riskier, and because of that, participating in a diary study may highlight the frequency with which participants are engaging in CAS and illicit drug use. It is important to note that the decrease in illicit drug use over time became non-significant in adjusted models. Given that our prior work with these data found that Black participants were less compliant over time [27], differential patterns of compliance may have influenced this finding. We discuss implications of diary compliance in more detail below.

Furthermore, growth curve analyses found a steeper declining slope of change in CAS in the weekly compared to daily diary condition. This finding contrasts with hypotheses that we would observe more behavioral reactivity in the daily condition due to more frequently completed diary entries. Social Cognitive Theory of self-regulation would predict that more frequent monitoring of behaviors (e.g., daily) would lead to more behavior change because the monitoring occurs in close proximity to the actual behavior [26], but our data do not support this hypothesis. On the other hand, participants in the weekly condition completed seven daily diaries in a single survey one time per week (i.e., time-line followback), which may have allowed YMSM an opportunity to reflect on the risk behaviors they engaged in during the prior week. In other words, it may be more impactful to document 7 days of risk behavior in one sitting compared to documenting this behavior once per day, which may lead some YMSM to become more motivated to enact behavior change. It is important to note, though, that YMSM in the weekly condition on average reported less use of alcohol, marijuana and other illicit drugs at baseline. Given that sub-stance use has been found to be associated with sexual risk behavior [5], the lower rates of use in the weekly condition may have facilitated more effective behavior change relative to the daily condition. It is also possible that completing 7 daily reports of risk behaviors in one sitting (i.e., a weekly diary) may lead to more social desirability bias or increased underreporting of risk behaviors over time. As such, this effect should be replicated in future studies.

To further understand how change in sexual risk and sub-stance use behaviors co-varied within-persons during the course of the 2-month study period, we conducted parallel process models. These analyses found that decreases in CAS each week were associated with similar decreases in alcohol use, binge-drinking, and marijuana use. In other words, those YMSM who did decrease their engagement in CAS also successfully decreased their alcohol and marijuana use. These findings indicate that there may be a group of individuals who are able to self-monitor effectively, and these individuals do so across multiple risk behaviors, including alcohol and marijuana use, which showed no change over time for the sample as a whole. Future research should focus on determining who these individuals are that are able to reduce their risk behaviors using a simple automated self-monitoring platform. It is possible that certain demographic, personality or pre-existing motivational factors may enhance the ability to self-monitor effectively across several risk behaviors.

One possible explanation for the fact that we observed no evidence of reactivity in the baseline to 2-month comparisons, yet we observed change in CAS and illicit drug use in the diary data, is that the data collected across the two methodologies are inherently different from one another. As mentioned previously, if diary data is indeed more accurate than global self-report data, then our growth curve analyses of diary data may provide a more accurate analysis of reactivity. However, it is also plausible that decreasing compliance over time in the diary conditions may appear as reactivity in analyses. The present analyses found good correspondence between diary and 2-month follow-up data for sexual behavior. If decreasing compliance were a contributing factor to the observed change in CAS the diary data, one would expect different patterns of reporting across methods. In contrast, our analyses found worse correspondence between methods for illicit drug use data. More research is needed to understand how these differences in reporting may be associated with observed reactivity. It would also be informative to conduct qualitative exit interviews with participants at varying levels of reactivity and compliance in order to understand what is at the root of behavior change (or lack thereof) and to specifically assess whether self-monitoring or under-reporting is occurring amongst those whose risk behaviors reduced over time.

In addition to examining behavioral reactivity, this study aimed to understand the effect of delivering individualized feedback on behavior change, and we randomly assigned half of the participants in both the weekly and daily diary conditions to receive feedback once per week. Feedback was delivered in the form of three data visualizations demonstrating each participant’s risk behaviors during the prior week and across the entire 2-month study period. We found no evidence that this type of automated feedback led to significant decreases in sexual or substance use behaviors. Moreover, growth curve analyses of diary data found that feedback condition was not associated with the slope of change in any of the target risk variables, meaning there was no difference in change in risk behavior between those who received feedback and those who did not. Importantly, while the visualizations were tailored to each individual’s risk behavior patterns through the use of graphs and charts of their own data, we did not provide participants with concrete behavior change strategies. Further, the receipt of feedback was completely automated and involved no contact with a real-life person. Our null findings indicate that it may be necessary to provide more concrete behavior change strategies and/or utilize more personalized contact in order to engage participants and motivate them to enact behavior change. This has important implications for self-monitoring interventions. Given that most publically-available online or mobile app programs are fully-automated in this manner, many currently-available self-monitoring programs may be insufficient for leading to meaningful behavior change.

Alternatively, it may be that some individuals with certain characteristics are able to engage in behavior change using an automated feedback platform, while others need a higher intensity intervention that involves teaching specific risk reduction strategies. Stepped-care or adaptive randomized trial designs provide an innovative method through which to evaluate an increasingly intensive stepped-care self-monitoring intervention. Stepped-care designs may be evaluated via a Sequential Multiple Assignment Randomized Trial (SMART), which can evaluate which intervention sequences are most effective for which types of people [32, 33]. In a SMART design, participants can be randomized to increasingly intensive interventions based on response to a prior intervention. Applied to self-monitoring, an individual who does not respond initially to a basic automated self-monitoring protocol may then be “stepped-up” to receive more tailored behavior change strategies or health educator coaching sessions.

The present findings should be interpreted in the context of study limitations. First, despite a strong trial design in which we randomly assigned participants to a daily, weekly or no diary control condition, our study design necessitated the use of global items to assess average engagement in risk behaviors during the follow-up period. This between-subjects approach to examining change in risk behaviors between randomization groups is less sensitive to change than within-persons approaches utilizing diary data. While we were able to observe change in risk behaviors using growth curve analyses that utilized all diary data, these analyses could not compare findings to a control condition. Second, more individuals in the daily than weekly condition had to be excluded from analyses because they never completed any diaries. This may have affected our results because our analyses were not based on the total number of participants randomized. Intercept differences by diary condition were significant for all risk behaviors which may be a product of this attrition or an indication that randomization was not completely successful. Third, 2 months provides a substantial amount of data on sexual and substance use behaviors for an observational study, but it may not be enough time to observe meaningful behavior change that results from self-monitoring. Future analyses should examine behavioral reactivity using longer follow-up periods.

Finally, we note that patterns of diary compliance may have affected or findings. Compliance with the diary protocol was modest, so we may have detected larger changes in risk behaviors if compliance had been higher. As reported elsewhere [27], one of the other primary aims of this study was to examine how diary condition was associated with compliance and attrition, so we purposefully did not utilize many strategies for keeping participants engaged in the diary protocol or re-engaging lost participants, aside from automated email reminders. Most diary and self-monitoring studies use resource-intensive strategies for maintaining participant engagement, and we assert that our approach likely better mimics an individual’s use of a self-monitoring program in the real world, such as a smart phone app. In this way, our findings may be more generalizable to the manner in which self-monitoring protocols perform in real-world settings.

Despite these limitations, this study provides valuable data about behavior change in the context of behavioral diary studies. This methodology is frequently used for observation of health-related behaviors prospectively, yet in this context, the design assumes an absence of behavioral reactivity. The diary methodology is also frequently used for self-monitoring such behaviors for the purpose of engaging in behavior change. Using within-persons analyses, this study found some evidence for systematic behavior change in CAS and illicit drug use, particularly amongst those in the weekly diary condition, but we found little evidence of reactivity for alcohol, binge-drinking or marijuana use. Furthermore, we found no evidence that automated feedback was associated with behavior change, which indicates that self-monitoring interventions should employ more personalized feedback strategies in order to maximize behavior change. More work is clearly needed to understand for whom and under what conditions diary studies become self-monitoring interventions across multiple health outcomes.