Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2018 Apr 1.
Published in final edited form as: J Subst Abuse Treat. 2017 Jan 25;75:43–48. doi: 10.1016/j.jsat.2017.01.007

Race/ethnicity, education, and age are associated with engagement in ecological momentary assessment text messaging among substance-using MSM in San Francisco

Caitlin M Turner a, Phillip Coffin a,b, Deirdre Santos a, Shannon Huffaker a, Tim Matheson a, Jason Euren a, Anna DeMartini a, Chris Rowe a, Steven Batki b, Glenn-Milo Santos a,b
PMCID: PMC5367385  NIHMSID: NIHMS848601  PMID: 28237053

Abstract

Background

Ecological momentary assessments (EMA) are data collection approaches that characterize behaviors in real-time. However, EMA is underutilized in alcohol and substance use research among men who have sex with men (MSM). The aim of this analysis is to explore the correlates of engagement in EMA text messages among substance-using MSM in San Francisco.

Methods

The present analysis uses data collected from the Project iN pilot study (n=30). Over a two-month period, participants received and responded to EMA daily text messages inquiring about their study medication, alcohol, and methamphetamine use. Baseline characteristics including demographics, alcohol use, and substance use were examined as potential correlates of engagement in EMA text messages in logistic regression and proportional hazards models.

Results

Participants had a 74% response rate to EMA text messages over the study period. MSM of color had significantly lower adjusted odds of responding to EMA texts 80% of the time or more, compared to white MSM (adjusted odds ratio=0.05, 95%CI=0.01-0.38). College-educated MSM had a lower adjusted hazard of week-long discontinuation in EMA texts (adjusted hazard ratio=0.12, 95%CI=0.02-0.63). Older MSM had a higher adjusted hazard of week-long discontinuation in EMA texts (adjusted hazard ratio=1.15, 95%CI=1.01-1.31).

Conclusion

Differences in engagement in EMA text prompts were discovered for MSM with different racial/ethnic backgrounds, ages, and education levels. Substance use variables were not correlated with engagement in text messages, suggesting that EMA may be a useful research tool among actively substance-using MSM in San Francisco.

Keywords: Men who have sex with men, text messaging, ecologic momentary assessment, methamphetamine, alcohol, substance use

1. Introduction

Current methamphetamine (meth) and alcohol use are more prevalent among men who have sex with men (MSM) compared to the general U.S. population (Schoenborn et al., 2013; Finlayson et al., 2011; Durell et al., 2008). Most MSM who report use of these substances are non-dependent and use episodically (Santos et al., 2013; Santos et al., 2011); however, these behavior patterns remain independently associated with HIV risk (Plankey et al., 2007; Read et al., 2007; Koblin et al., 2003). Studies that survey substance use at a limited number of time points fail to capture the episodic nature of these behaviors and are subject to bias (Shiffman, 2009). Ecological momentary assessments (EMA) are data collection approaches that characterize behaviors “in subjects' natural environments” (Shiffman, 2008). By capturing real-time assessments of dynamic behaviors such as alcohol and substance use, EMA methods have the potential to reduce recall bias and strengthen ecological validity (Shiffman, 2008; Smyth & Stone, 2003). Ecological momentary assessments may be employed in the form of electronic diaries, text messages, and other mobile technologies (Shiffman et al., 2008). Widespread availability of these new technologies makes EMA a feasible option for data collection (Portell, 2015). Previous studies utilized EMA to gather social and behavioral observations in topic areas such as tobacco use (Waters et al., 2014; Watkins et al., 2014; Berkman et al., 2011), eating disorders (Kraus et al., 2015), HIV/AIDS (Hensel et al., 2012; Cook et al., 2010), and mood disorders (Ebner-Priemer & Trull, 2009).

EMA is an underutilized data collection method for substance use research among men who have sex with men. However, it may prove useful for capturing the “situational cues and social contexts” that influence substance use behavior (Shiffman, 2009). Although researchers are concerned that compliance with EMA is questionable among substance-using populations, studies have demonstrated that this is not the case (Yang et al., 2015; Serre et al., 2012; Freedman et al., 2006; Hopper et al. 2006; Collins et al., 2003). Recent studies have also shown that there is no strong correlation between EMA and change in behaviors over time, allaying concerns that reporting behaviors on a day-to-day basis in EMA could lead to assessment reactivity (Cook et al. 2010; Rowan et al., 2007; Hufford et al., 2002). A study by Yang and colleagues (2015) was the first to show that smartphone-based EMA methods were feasible and acceptable among MSM who use alcohol. However, the study did not have the statistical power to analyze differences in characteristics between participants who had high EMA text response rates and those with low response rates (Yang et al., 2015). Analyses such as these would inform targeted trainings to improve participant engagement in EMA technologies, especially in diverse samples. The aim of this secondary analysis is to examine the sociodemographic, behavioral, and psychological correlates of engagement in EMA text messaging among active substance-using MSM (SUMSM) enrolled in a pharmacological intervention in San Francisco.

2. Material and Methods

2.1. Study design and participants

Data for the analysis came from the Project iN study conducted at San Francisco Department of Public Health from June 2013 to September 2014. The study was approved by the Committee on Human Research affiliated with University of California, San Francisco (IRB Number: 12-09809). Project iN is a double-blind, 1:1 randomized control trial designed to assess the feasibility, acceptability, and tolerability of a pharmacologic intervention to substance use among non-dependent meth- and alcohol-using MSM in San Francisco (clinicaltrials.gov=NCT01723384). The objectives and methods of the parent study are described elsewhere (Santos et al., 2015). In brief, the study recruited MSM between the ages of 18 and 70 via community outreach and social media. Eligibility criteria included: self-reported meth use of two times per month or more and binge-drinking at least once weekly in the past 3 months; concurrent meth or alcohol use and anal intercourse in the past 3 months; desire to reduce or discontinue meth/alcohol use; and absence of acute psychiatric or medical ailments that would preclude safe study participation (e.g., depression with suicidal ideation or known allergy to naltrexone). Potential participants were excluded if they were determined by study clinicians to be dependent on meth or alcohol, as defined by the Diagnostic and Statistical Manual of Disorders, Fourth Edition (DSM-IV). Thirty participants were enrolled and randomized to either placebo or naltrexone, to be taken on a targeted, intermittent basis (i.e., while experiencing cravings or before planned alcohol or meth use). Informed consent was given by all participants.

2.2. Data collection and analysis

Participants came in for study visits at two-week intervals for a period of 2 months. At these visits, data were collected using audio computer-assisted self-interview (ACASI) surveys. Each participant's ACASI survey contained information about socio-demographic, behavioral, and psychological characteristics. Included in the present analysis as potential demographic correlates of engagement in EMA were: race/ethnicity, annual income, education level, employment status, and health insurance status. Participants also responded to items that captured their behaviors in the month prior to completing their baseline ACASI, such as: substance use (e.g., marijuana or cocaine use), polysubstance use (use of substances in addition to meth or alcohol), alcohol use, binge drinking frequency, and average meth use. Severity of alcohol and meth dependence were each measured with scales that contained five items on a 4-point Likert scale (e.g., “Did you worry about your meth use?”), with higher scores denoting a higher degree of dependence (Gossip et al., 1995). Although MSM in this sample were determined by study clinicians to be non-dependent for both meth and alcohol according to the DSM-IV, severity of dependence scores for meth and alcohol were included in this analysis because they afforded a granular, recent, and psychological characterization of self-reported substance dependence. Finally, participants were asked to guess the treatment group to which they belonged (naltrexone or placebo) and about acceptability of EMA texting, motivation to join the study (e.g., to reduce meth or alcohol use), and depression status (Cheng et al., 2006).

Ecological momentary assessments were gathered by sending automated text messaging prompts to participants through an online platform called Capito Health. A HIPAA-compliant, cloud-based data server stored text message prompts and responses from participants. At the baseline visit, study staff registered participants' phones with Capito Health and trained participants to respond to text messaging prompts. Every participant enrolled in the study chose the time at which they would receive the text prompts each day and used their own cell phones to text their responses. Every day for the 60-day study period, participants were asked to respond to up to three text prompts inquiring about their behaviors in the previous day, including: 1) If they took their study medication, 2) If they used meth and/or alcohol; and 3) If they took their study medication before meth or alcohol use. As a confidentiality safeguard, abbreviations (e.g., “ma” for methamphetamine, “al” for alcohol, and “tx” for study medication) were used in text messaging prompts. Participants were trained to provide numerical responses (e.g., 1=yes and 2=no) to these prompts to further mask their substance and study medication use. Participants were paid a dollar for each day that they completed text messages, up to a maximum of 60 dollars. Additionally, participants earned up to $125 for completing study visits for the pharmacologic trial ($25 for screening, $35 for enrollment, $10 for visits every 2 weeks, and $35 for final visits).

Since text prompts were sent to participants every 24 hours, participant responses were considered complete in a given day if they replied to all text message questions within 22 hours after the first message was sent that day. This provided a two-hour buffer between receiving text responses and sending out the next set of prompts. Incomplete responses included situations in which participants either failed to reply to text questions or replied late. Once responses were categorized into either complete or incomplete, outcome variables unique to multivariable logistic regression and Cox proportional hazards models were created.

The dependent outcome for the logistic regression model was engagement in EMA text messages, characterized a priori as “high” (i.e., complete responses to 80% or more of texts) or “low” (i.e., complete responses to less than 80% of texts) for each participant. This categorization was based on recent studies utilizing EMA methods among substance users, in which high compliance to EMA was defined between 75% and 80% (Yang et al., 2015; Serre et al., 2012). The dependent outcome for the Cox proportional hazards model was time-to-failure (in days) for the event of interest: the first week-long discontinuation of responding to EMA text messages. Had participants failed to respond to text prompts for more than a week, then there would be no benefit to collecting day-to-day data in addition to ACASI data (i.e., all the information could be gathered retrospectively via survey interviews instead of daily assessments).

Potential socio-demographic, psychological, and behavioral correlates were first evaluated with bivariable analyses using Wilcoxon rank-sum tests, Fisher's exact tests, and Cox proportional hazards models for single predictors, as appropriate. Covariates with p-values less than 0.25 were included in initial multivariable models (Bursac et al., 2008). A parsimonious model was reached for multivariable logistic regression and Cox proportional hazards analyses using a backward elimination approach. Likelihood ratio tests confirmed that variables dropped from the nested models did not make significant contributions to the larger model. Departures from the proportional hazards assumption were verified using Therneau and Grambsch's test. All analyses were conducted in STATA version 12.0 (College Station, TX).

3. Results

All thirty participants completed the baseline survey; 28 (93%) completed the final, 2-month follow-up survey. Overall, the sample of MSM in Project iN was ethnically diverse (30% Black, 7% Asian/Pacific Islander, 17% Latino/Hispanic, 40% white, 3% mixed and 3% other) with an average age of 43 (SD=9.3). A majority of participants (80%) reported earning incomes of less than $40,000 per year. Moreover, most participants (70%) were unemployed. However, 87% of participants reported completing at least some college.

3.1. Alcohol and substance use prevalence

Thirty-seven percent of MSM in the sample reported consuming alcohol 2–3 days per week. On average, participants reported binge drinking on about 6 different occasions per month. A majority of participants used methamphetamine an average of 1–2 days per week or more (60%), and approximately 33% injected meth in the month prior to completing their baseline survey. A high proportion of MSM reported using poppers/nitrates (40%) and powdered cocaine (37%) in the month prior to completing their baseline survey. Almost half (43%) of the sample reported marijuana use in the past month. Most MSM in the sample (80%) used one or more substances in addition to meth and alcohol.

3.2. Cross-sectional and longitudinal analysis results

On average, participants responded to approximately 69% of the EMA texts they received. Forty-three percent of participants were classified as high engagers in EMA texts (responding completely to 80% or more of texts). Cumulatively, participants had responded to all text message prompts they received on 1,111 days out of 1,503 total days of follow-up (74% response rate). By the end of the 2-month follow-up period, 17 of the 30 MSM in the sample (57%) discontinued their responses to texts for one week or longer (see Figure 1). Among those who experienced this failure event, the mean and median time to first week-long discontinuation of responses to text messages was 27 days (SD=20.7; IQR=10-41).

Figure 1. Kaplan-Meier Survival Curve.

Figure 1

Open in a new tab

Table 1 demonstrates the results of bivariable and multivariable analyses. Age, health insurance status, and race/ethnicity were included in the final multivariable logistic regression model. According to the final model, race/ethnicity significantly predicted engagement level in EMA texting among MSM (p=0.004); however, age and health insurance did not. The adjusted odds of high engagement in texts among MSM of color was significantly lower compared to that of white MSM (adjusted OR=0.05; 95%CI=0.01-0.38).

Table 1.

Correlates of engagement in and first week-long discontinuation of EMA text responses among substance-using MSM, Project iN, San Francisco, CA, 2013–2014.

Engagement Level
Multivariable Logistic Regression Bivariable Analysis Multivariable Cox Proportional Hazards
Low High
N (%) N (%) aOR (95% CI) HR (95% CI) aHR (95% CI)
Demographic Characteristics
Age, mean (SD) 45.76 (8.00) 39.38 (9.88) 0.95 (0.84 – 1.07) 1.09 (1.00 – 1.17)† 1.15 (1.01 – 1.31)†
Race/ethnicity
 MSM of color 2 (17%) 10 (83%) 1 1
 White MSM 15 (83%) 3 (17%) 0.05 (0.01 – 0.38)* 4.15 (0.89 – 19.29)
Yearly income
 Less than $40,000 14 (58%) 10 (42%) 1
 $40,000 or more 1 (25%) 3 (75%) 0.58 (0.07 – 4.64)
Highest education level achieved
 Less than college 4 (100%) 0 1 1
 College and beyond 13 (50%) 13 (50%) 0.15 (0.04 – 0.63)† 0.12 (0.02 – 0.63)†
Health insurance
 Uninsured 2 (29%) 5 (71%) 1 1
 Insured 15 (65%) 8 (35%) 0.19 (0.02 – 2.27) 3.06 (0.39 – 24.14)
Employment status
 Unemployed or student 14 (64%) 8 (36%) 1 1
 Employed full or part time 3 (38%) 5 (62%) 0.20 (0.02 – 1.57) 0.15 (0.01 – 1.62)
Psychosocial Characteristics
CESDa, mean (SD) 19.12 (11.26) 19.38 (11.17) 0.99 (0.93 – 1.06)
Polysubstance useb
 0 additional substances 2 (33%) 4 (67%) 1
 1 additional substance 4 (67%) 2 (33%) 1.68 (0.28 – 10.08)
 2-3 additional substances 7 (78%) 2 (22%) 1.49 (0.27 – 8.15)
 ≥ 4 additional substances 4 (44%) 5 (56%) 0.52 (0.07 – 3.70)
Alcohol use
 4 days per month or less 4 (67%) 2 (33%) 1
 2-3 days per week 5 (45%) 6 (55%) 0.60 (0.08 – 4.34)
 4-5 days per week 7 (70%) 3 (30%) 2.28 (0.45 – 11.57)
 Every day 1 (33%) 2 (67%) 0.81 (0.07 – 8.92)
Binge drinkingc, mean (SD) 7.35 (8.89) 5.54 (4.07) 1.05 (0.99 – 1.12) 1.05 (0.99 – 1.13)
Average methamphetamine use
 1-3 days per month 5 (50%) 5 (50%) 1
 1-2 days per week 6 (60%) 4 (40%) 1.53 (0.30 – 7.92)
 3-4 days per week 6 (75%) 2 (25%) 1.92 (0.35 – 10.49)
Injected methamphetamine
 No use 10 (56%) 8 (44%) 1
 Any use 7 (70%) 3 (30%) 0.63 (0.16 – 2.51)
Marijuana use
 No use 10 (59%) 7 (41%) 1
 Any use 7 (54%) 6 (46%) 0.62 (0.18 – 2.16)
Poppers/Nitrates use
 No use 10 (56%) 8 (44%) 1
 Any use 7 (58%) 5 (42%) 0.63 (0.18 – 2.15)
Crack/cocaine use
 No use 11 (52%) 10 (48%) 1
 Any use 6 (67%) 3 (33%) 1.27 (0.33 – 4.94)
Powdered cocaine use
 No use 11 (58%) 8 (42%) 1
 Any use 6 (55%) 5 (45%) 0.95 (0.28 – 3.30)
GHB use
 No use 12 (55%) 10 (45%) 1
 Any use 5 (63%) 3 (38%) 0.41 (0.09 – 1.95)
Ecstasy use
 No use 14 (58%) 10 (42%) 1
 Any use 3 (50%) 3 (50%) 1.00 (0.21 – 4.74)
Viagra, Levitra, Cialis use
 No use 13 (54%) 11 (46%) 1
 Any use 4 (67%) 2 (33%) 0.59 (0.13 – 2.73)
Recreational prescription drug use
 No use 15 (63%) 9 (37%) 1
 Any use 2 (33%) 4 (67%) 0.30 (0.04 – 2.37)
SDS scored, mean (SD) 7.12 (4.09) 6.77 (3.96) 1.02 (0.87 – 1.19)
SDSA scoree, mean (SD) 5.59 (3.92) 6.85 (3.72) 1.02 (0.87 – 1.18)
Intervention arm
 Naltrexone 9 (60%) 6 (40%) 1
 Placebo 8 (53%) 7 (47%) 1.03 (0.42 – 2.55)
Acceptability Factors
Satisfaction with daily texting
 Dissatisfied/neutral 6 (55%) 5 (45%) 1
 Satisfied 9 (53%) 8 (47%) 1.19 (0.32 – 4.45)
Motivation to Join
To stop using meth
 No 5 (71%) 2 (29%) 1
 Yes 12 (52%) 11 (48%) 0.91 (0.19 – 4.26)
To reduce meth use
 No 9 (50%) 9 (50%) 1
 Yes 8 (67%) 4 (43%) 1.21 (0.36 – 4.01)
To stop using alcohol
 No 10 (56%) 8 (44%) 1
 Yes 7 (58%) 5 (42%) 1.12 (0.34 – 3.72)
To reduce alcohol use
 No 11 (73%) 4 (27%) 1
 Yes 6 (40%) 9 (60%) 1.05 (0.32 – 3.47)
Other treatment/counseling has not worked
 No 14 (61%) 9 (39%) 1
 Yes 3 (43%) 4 (57%) 0.75 (0.16 – 3.51)
Need the money
 No 11 (58%) 8 (42%) 1
 Yes 6 (55%) 5 (45%) 1.47 (0.45 – 4.88)
To try the medication
 No 8 (47%) 9 (53%) 1
 Yes 9 (69%) 4 (31%) 0.86(0.25 – 2.90)
To help the community
 No 15 (54%) 13 (46%) 1
 Yes 2 (100%) 0 1.26 (0.37 – 4.35)
Open in a new tab

Notes: OR, odds ratio; aOR, adjusted odd ratio; HR, hazard ratio; aHR, adjusted hazard ratio; CI, confidence interval; substance use behaviors reported in the month prior to the baseline ACASI visit.

†

p-Value < 0.05.

*

p-Value < 0.01.

a

CESD, Center for Epidemiologic Studies Depression Scale.

b

Any substance use in addition to meth and alcohol.

c

Frequency of consuming five or more alcoholic drinks on a single occasion.

d

SDS, severity of dependence scale score for methamphetamine.

e

SDSA, severity of dependence scale score for alcohol.

The final Cox proportional hazards model was comprised of age, binge drinking frequency, employment, and education level. Two participants were excluded from the model due to the onset of a week-long discontinuation event at the first day of follow-up. There was no evidence of departure from the proportional hazards assumption in the final model. Participants with a college education had a significantly lower hazard (p=0.012) of week-long discontinuation from EMA text messages compared to those with at least some college education, holding all other correlates constant (adjusted HR=0.12, 95%CI=0.02-0.63). Age was also significant after adjusting for other covariates (p=0.042). For every one-year increase in age, the adjusted hazard of week-long discontinuation increased by 15 percent (adjusted HR=1.15, 95%CI=1.01-1.31). Binge drinking frequency and employment were not significant predictors of discontinuing text responses for a week or more.

3.3. Qualitative feedback about EMA text messaging

A majority of participants (60%) reported being satisfied or highly satisfied with the daily EMA text messages, and 85% of MSM in the sample reported participating in the EMA portion of the study with little to no difficulty. Participants' most frequently reported positive aspect of the EMA component of the study was that the text prompts served as reminders (n = 9) and helped them reflect on and monitor their use (n = 11). Two participants reported that they liked having contact with someone who cared about their use. Eighteen participants reported that there were no negative aspects to the text messaging component of the study. Of those who responded to less than 80% of their text prompts (n = 17), 3 participants reported technical issues (e.g., study text prompts not being received, participant text responses not being received, and worry about texts prompts expiring) and 3 reported logistical issues (e.g., “learning how to text”, stolen phone, and “not having a working phone on some of the occasions”). To improve the EMA portion of the study, two participants recommended that study staff set up an online alternative to texting (i.e., online texting platforms) and two other participants suggested sending reminders to respond to texts. Most (n = 19) advised making no changes to the text messaging portion of the study.

4. Discussion

The present analysis shows that there are differences in engagement with EMA texting technology for this sample of SUMSM by racial/ethnic background, education level, and age. White MSM had higher odds of responding to EMA text messages. Older MSM and those in the sample with less than a college education were more likely to experience a weeklong discontinuation of responding to texts. In bivariable and multivariable analyses, binge drinking frequency, severity of dependence, polysubstance use, type of substance used, and frequency of meth/alcohol use were not associated with engagement in EMA. A majority of participants were satisfied and reported little to no difficulty with the EMA texting component of the study. Feedback about the EMA texts was favorable. Most participants (n=19) liked the monitoring/reminder aspect of the text prompts. Some reported logistical or technical issues and suggested improvements to the EMA texting methodology implemented in the present study.

In any case, MSM in the sample had a 74% response rate to EMA text message prompts over the two-month study period. Of note, this replicated the response rate to daily surveys calculated in a study of African American MSM who use alcohol (Yang et al., 2015). As demonstrated by Figure 1, the proportion of participants who experienced a week-long discontinuation event was about 57% by the end of the study period. In addition, there was a steady change in the proportion of the sample experiencing a week-long discontinuation event, indicating that there were no significant drop-offs in responding to texts at the beginning or end of the study. Of note, in both the cross-sectional and longitudinal analyses, we considered late responses to text messages as incomplete. This provided a more conservative definition of text message completion. Had the analyses been less conservative in defining late responses as complete instead of incomplete, the overall average response rate would have been higher.

To our knowledge, Project iN is one of the first pharmacologic studies to examine EMA among SUMSM. EMA is an underutilized data collection tool in substance use research among MSM even though it is useful for studying episodic risk behaviors such as alcohol and methamphetamine use (Shiffman, 2009), particularly in stigmatized populations. Traditional surveillance methods of HIV-related risk behaviors among high-risk MSM collected data cross-sectionally and had long recall windows (e.g., Finlayson et al., 2011; Koblin et al., 2003). However, these methods may lead to participant recall error when the risk behaviors of interest occur frequently and are variable (Shiffman et al., 2008), which is particularly important for this sample of polysubstance-using MSM. A recent study of high-risk MSM indicated that prospective, smartphone-based EMA methods improved data accuracy of HIV-related risk behaviors such as substance use (Wray et al., 2016). Overall, substance use variables were not significantly correlated with engagement in EMA text messaging in the present study, suggesting that intoxication may not influence text messaging response rates. These observations suggest that EMA is a promising method for prospectively characterizing behaviors in substance-using populations with characteristics similar to the ones in this sample.

Although our findings pointed to the utility of EMA within this study population, the generalizability of results to other substance-using populations was limited due to the nature of the Project iN sample. Only 30 SUMSM participated in the study. Due to the small sample, the study may have been underpowered and unable to detect some important correlates of engagement in EMA. However, in examining multiple time points for the time-to-event analysis, there was increased the power to detect associations between socio-demographic variables and time to week-long discontinuation of text responses. The other limitation of this study was that participants were gathered via convenience sampling. Given that participants were enrolled in a pharmacologic intervention, they may have been characteristically different from the overall population of SUMSM. Most earned annual incomes less than $40,000 and were unemployed. Although polysubstance use was prevalent, self-reported marijuana use was surprisingly less common in this sample compared to the 52% prevalence estimate for marijuana use among MSM in San Francisco (Finlayson et al., 2011). Moreover, EMA was used in conjunction with a research study comprising in-person visits to collect data. It was unclear whether using EMA alone would lead to similar participation rates. Additionally, since the study period for Project iN was only 2 months, it was not clear if EMA would be useful for studies with longer follow-up times. Finally, outcome misclassification may have occurred in using weeklong discontinuation as a proxy for participant disengagement in EMA (e.g., some participants may have temporarily stopped responding if they were traveling or had a busy week, but became engaged in texts later in the follow-up period).

In light of these limitations, future analyses among substance-using populations would benefit from examining EMA correlates within larger, more representative samples with longer follow-up periods. Indeed, the San Francisco Department of Public Health is utilizing EMA in a larger, ongoing study of alcohol-using MSM in a pharmacologic trial (Santos, 2014). The data collected from this study may be used to further explore the potential barriers to engagement in EMA text platforms for SUMSM. More comprehensive qualitative studies should be conducted to further understand these and other barriers to high engagement in EMA.

4.1. Conclusions

Findings from the present analysis suggest that EMA implementation should be sensitive to differences in engagement levels based on SUMSM's ages and cultural and educational backgrounds. However, given the small sample size, results must be interpreted with caution. If the associations observed hold for larger samples of SUMSM, this would suggest that they may benefit from additional technology support or training with EMA. The lack of association between substance use and text messaging engagement suggests that EMA may be a valuable tool for future studies that aim to capture nuances in behaviors among SUMSM.

Highlights for Review.

  • MSM of color had significantly lower adjusted odds of responding to ecological momentary assessment (EMA) texts 80% of the time or more

  • College-educated MSM and young MSM had significantly lower adjusted hazard of week-long discontinuation in EMA

  • Substance use variables were not correlated with engagement in text messages

  • EMA is a potentially useful research tool among active substance users

  • Future studies should further explore barriers to engagement in EMA text platforms for diverse populations of substance-using men who have sex with men

Footnotes

Conflicts of Interest: none.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  1. Berkman ET, Dickenson J, Falk EB, Lieberman MD. Using SMS text messaging to assess moderators of smoking reduction: validating a new tool for ecological measurement of health behaviors. Health Psychology. 2011;30:186–194. doi: 10.1037/a0022201. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Bursac Z, Gauss CH, Williams DK, Hosmer DW. Purposeful selection of variables in logistic regression. Source Code for Biology and Medicine. 2008;3 doi: 10.1186/1751-0473-3-17[doi]. 17-0473-3-17. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Cheng ST, Chan AC, Fung HH. Factorial structure of a short version of the center for epidemiologic studies depression scale. International Journal of Geriatric Psychiatry. 2006;21(4):333–336. doi: 10.1002/gps.1467[doi]. [DOI] [PubMed] [Google Scholar]
  4. Collins RL, Kashdan TB, Gollnisch G. The feasibility of using cellular phones to collect ecological momentary assessment data: application to alcohol consumption. Experimental and Clinical Psychopharmacology. 2003;11(1):73. doi: 10.1037//1064-1297.11.1.73. [DOI] [PubMed] [Google Scholar]
  5. Cook PF, McElwain CJ, Bradley-Springer LA. Feasibility of a daily electronic survey to study prevention behavior with HIV-infected individuals. Research in Nursing & Health. 2010;33(3):221–234. doi: 10.1002/nur.20381[doi]. [DOI] [PubMed] [Google Scholar]
  6. Durell TM, Kroutil LA, Crits-Christoph P, Barchha N, Van Brunt DL. Prevalence of nonmedical methamphetamine use in the United States. Substance Abuse Treatment, Prevention, and Policy. 2008;3:19. doi: 10.1186/1747-597X-3-19. http://doi.org/10.1186/1747-597X-3-19. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Ebner-Priemer UW, Trull TJ. Ecological momentary assessment of mood disorders and mood dysregulation. Psychological Assessment. 2009;21(4):463–475. doi: 10.1037/a0017075[doi]. [DOI] [PubMed] [Google Scholar]
  8. Finlayson TJ, Le B, Smith A, Bowles K, Cribbin M, Miles I, et al. HIV risk, prevention, and testing behaviors among men who have sex with men–National HIV Behavioral Surveillance System, 21 US cities, United States, 2008. MMWR Surveill Summ. 2011;60(14):1–34. [PubMed] [Google Scholar]
  9. Freedman MJ, Lester KM, McNamara C, Milby JB, Schumacher JE. Cell phones for ecological momentary assessment with cocaine-addicted homeless patients in treatment. Journal of Substance Abuse Treatment. 2006;30(2):105–111. doi: 10.1016/j.jsat.2005.10.005. doi:S0740-5472 (05)00203-5[pii] [DOI] [PubMed] [Google Scholar]
  10. Gossop M, Darke S, Griffiths P, Hando J, Powis B, Hall W, Strang J. The Severity of Dependence Scale (SDS): psychometric properties of the SDS in English and Australian samples of heroin, cocaine and amphetamine users. Addiction. 1995;90(5):607–614. doi: 10.1046/j.1360-0443.1995.9056072.x. [DOI] [PubMed] [Google Scholar]
  11. Hensel DJ, Fortenberry JD, Harezlak J, Craig D. The feasibility of cell phone based electronic diaries for STI/HIV research. BMC Medical Research Methodology. 2012;12 doi: 10.1186/1471-2288-12-75[doi]. 75-2288-12-75. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Hopper JW, Su Z, Looby AR, Ryan ET, Penetar DM, Palmer CM, Lukas SE. Incidence and patterns of polydrug use and craving for ecstasy in regular ecstasy users: An ecological momentary assessment study. Drug and Alcohol Dependence. 2006;85(3):221–235. doi: 10.1016/j.drugalcdep.2006.04.012. doi:S0376-8716 (06)00162-1[pii] [DOI] [PubMed] [Google Scholar]
  13. Hufford MR, Shields AL, Shiffman S, Paty J, Balabanis M. Reactivity to ecological momentary assessment: An example using undergraduate problem drinkers. Psychology of Addictive Behaviors: Journal of the Society of Psychologists in Addictive Behaviors. 2002;16(3):205–211. [PubMed] [Google Scholar]
  14. Koblin BA, Chesney MA, Husnik MJ, Bozeman S, Celum CL, Buchbinder S, et al. High-risk behaviors among men who have sex with men in 6 US cities: baseline data from the EXPLORE Study. American Journal of Public Health. 2003;93(6):926–932. doi: 10.2105/ajph.93.6.926. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Kraus N, Lindenberg J, Zeeck A, Kosfelder J, Vocks S. Immediate effects of body checking behaviour on negative and positive emotions in women with eating disorders: An ecological momentary assessment approach. European Eating Disorders Review: The Journal of the Eating Disorders Association. 2015;23(5):399–407. doi: 10.1002/erv.2380[doi]. [DOI] [PubMed] [Google Scholar]
  16. Plankey MW, Ostrow DG, Stall R, Cox C, Li X, Peck JA, Jacobson LP. The relationship between methamphetamine and popper use and risk of HIV seroconversion in the multicenter AIDS cohort study. Journal of Acquired Immune Deficiency Syndromes. 2007;45(1):85. doi: 10.1097/QAI.0b013e3180417c99. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Portell M, Anguera MT, Hernandez-Mendo A, Jonsson GK. Quantifying biopsychosocial aspects in everyday contexts: An integrative methodological approach from the behavioral sciences. Psychology Research and Behavior Management. 2015;8:153–160. doi: 10.2147/PRBM.S82417[doi]. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Read TR, Hocking J, Sinnott V, Hellard M. Risk factors for incident HIV infection in men having sex with men: a case-control study. Sexual Health. 2007;4(1):35–39. doi: 10.1071/sh06043. [DOI] [PubMed] [Google Scholar]
  19. Rowan PJ, Cofta-Woerpel L, Mazas CA, Vidrine JI, Reitzel LR, Cinciripini PM, et al. Evaluating reactivity to ecological momentary assessment during smoking cessation. Exp Clin Psychopharmacol. 2007;15:382–389. doi: 10.1037/1064-1297.15.4.382. [DOI] [PubMed] [Google Scholar]
  20. Santos GM, Coffin PO, Das M, Matheson T, DeMicco E, Raiford, et al. Dose-response associations between number and frequency of substance use and high-risk sexual behaviors among HIV-negative substance-using men who have sex with men (SUMSM) in San Francisco. Journal of Acquired Immune Deficiency Syndromes (1999) 2013a;63(4):540. doi: 10.1097/QAI.0b013e318293f10b. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Santos GM, Das M, Colfax GN. Interventions for non-injection substance use among US men who have sex with men: what is needed. AIDS and Behavior. 2011b;15(1):51–56. doi: 10.1007/s10461-011-9923-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Santos GM, Coffin P, Santos D, Huffaker S, Matheson T, Euren J, et al. Feasibility, acceptability and tolerability of targeted naltrexone for non-dependent methamphetamine-using and binge-drinking men who have sex with men. Journal of Acquired Immune Deficiency Syndromes. 2015c doi: 10.1097/QAI.0000000000000922[doi]. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Santos GM. NIH High-Risk High-Reward Research Symposium. Bethesda, MD: 2014d. The Better THAN Study: Targeting Heavy Alcohol use with Naltrexone. Abstract # 35. [Google Scholar]
  24. Schoenborn CA, Adams PF, Peregoy JA. Vital and health statistics Series 10, Data from the National Health Survey. 257. 2013. Health behaviors of adults: United States, 2008-2010; pp. 1–184. [PubMed] [Google Scholar]
  25. Serre F, Fatseas M, Debrabant R, Alexandre JM, Auriacombe M, Swendsen J. Ecological momentary assessment in alcohol, tobacco, cannabis and opiate dependence: A comparison of feasibility and validity. Drug and Alcohol Dependence. 2012;126(1-2):118–123. doi: 10.1016/j.drugalcdep.2012.04.025[doi]. [DOI] [PubMed] [Google Scholar]
  26. Shiffman S, Stone AA, Hufford MR. Ecological momentary assessment. Annual Review of Clinical Psychology. 2008a;4:1–32. doi: 10.1146/annurev.clinpsy.3.022806.091415. [DOI] [PubMed] [Google Scholar]
  27. Shiffman S. Ecological momentary assessment (EMA) in studies of substance use. Psychological Assessment. 2009b;21(4):486. doi: 10.1037/a0017074. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Smyth JM, Stone AA. Ecological momentary assessment research in behavioral medicine. Journal of Happiness Studies. 2003;4(1):35–52. [Google Scholar]
  29. Waters AJ, EH S, Wetter DW, Cinciripini PM, Robinson JD, Li Y. Cognition and craving during smoking cessation: an ecological momentary assessment study. Nicotine Tob Res. 2014;16(Suppl. 2):S111–S118. doi: 10.1093/ntr/ntt108. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Watkins KL, Regan SD, Nguyen N, Businelle MS, Kendzor DE, Lam C, et al. Advancing cessation research by integrating EMA and geospatial methodologies: associations between tobacco retail outlets and real-time smoking urges during a quit attempt. Nicotine Tob Res. 2014;16(Suppl. 2):S93–S101. doi: 10.1093/ntr/ntt135. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Wray TB, Kahler CW, Monti PM. Using Ecological Momentary Assessment (EMA) to Study Sex Events Among Very High-Risk Men Who Have Sex with Men (MSM) AIDS and Behavior. 2016:1–12. doi: 10.1007/s10461-015-1272-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Yang C, Linas B, Kirk G, Bollinger R, Chang L, Chander G, et al. Feasibility and acceptability of smartphone-based ecological momentary assessment of alcohol use among African American men who have sex with men in Baltimore. JMIR mHealth and uHealth. 2015;3(2) doi: 10.2196/mhealth.4344. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES