Feasibility and Acceptability of Text Messaging to Assess Daily Substance Use and Sexual Behaviors among Urban Emerging Adults

Abstract

Background

Daily process research can help distinguish causal relationships between substance use and sexual risk behaviors in high-risk groups, such as urban emerging adults. We employed text messaging to assess 18–25 year-olds’ daily substance use and sexual risk behaviors over 28 days. We describe the implementation of this method, attitudes regarding the daily surveys, and correlates of survey completion.

Method

We recruited 111 emerging adults from an urban Emergency Department in a resource-limited area who reported recent drug use and unprotected sex (M_age=22.0; 53.2% female; 45.1% African American; 43.2% receiving public assistance).

Results

Respondents completed M=18.0 (SD = 8.7) of 28 daily surveys (27 items each). Participants completing a 1-month follow-up found the surveys not at all/only a little annoying (90.3%) and were comfortable with questions about drugs/alcohol (97.9%) and sex (94.6%). Completion was higher on weekdays versus weekends, and earlier in the study. Daily survey completion was unrelated to same-day substance use measured by the Timeline Follow Back at follow-up; polysubstance use and drinks consumed were associated with lower odds of next-day completion. School enrollment, public assistance, unlimited texting plan, lower baseline alcohol use, and depression symptoms at follow-up were associated with higher completion. Technology difficulties were commonly mentioned barriers to completion.

Conclusions

Participants in this urban, resource-constrained sample found the daily text message methodology acceptable for reporting sensitive information. With rapid advancements in technologies and increased accessibility, text messaging remains a promising methodology for the study of daily processes in substance use and HIV risk behaviors. Keywords: text messaging; assessment; emerging adults; substance use; risky sex; mobile technology

Introduction

Substance use and risky sex remain important public health concerns in emerging adults (i.e., ages 18–25) as both increase risk for transmission of sexually transmitted infections (STIs), including HIV/AIDs. Substance use and sexual risk behaviors (SRBs) peak during emerging adulthood (Arnett, 1992, 2000, 2005; Center for Behavioral Health Statistics and Quality, 2015; Dariotis et al., 2008; A. L. Stone, Becker, Huber, & Catalano, 2012). Respectively, the past-year prevalence rates of any illicit drug use and alcohol use are 37.7% and 63.5% among 18–20 year-olds and 35.1% and 84.2% among 21–25 year-olds (Center for Behavioral Health Statistics and Quality, 2015). National data indicate that 35–45% of unmarried 21–24 year-olds never or seldom use condoms (Johnston et al., 2015); 50% of 12^th grade students used condoms at their last sexual intercourse (Kann et al., 2014).

Although risk behaviors are theorized to be a product of factors at multiple socio-ecological levels (e.g., individual, peer/family relationships; DiClemente, Salazar, Crosby, & Rosenthal, 2005; Hawkins, Catalano, & Miller, 1992; Petraitis, Flay, Miller, Torpy, & Greiner, 1998; Voisin, Jenkins, & Takahashi, 2011) individual-level data can help untangle causal, within-person relationships among substance use and sexual risk behaviors. Daily and event-level assessments can reduce memory biases, increase ecological validity, and provide high temporal resolution for assessing behaviors unfolding in the real world (A. A. Stone, Shiffman, Atienza, & Nebeling, 2007). These methods can aid in understanding how cognitions, quantity and frequency of substance use, and sexual risk co-occur in order to inform interventions targeting these behaviors.

In the present study, we employed Short Message Service (SMS) text messaging to assess daily substance use and sexual behaviors among urban, emerging adults from a low socioeconomic status (SES) community. Previous research has employed technology-driven methods for daily prospective assessments, such as web-based diaries, ecological momentary assessment (EMA), and interactive voice response (IVR). These methods require internet access, a smartphone, and/or a voice telephone line, with assessments typically completed in an uninterrupted session. IVR studies in substance use samples that include from 20 to 55 questions (lasting about 10 minutes) have high daily compliance (over 90%) for two-week reporting periods (Collins, Kashdan, & Gollnisch, 2003; Cranford, Tennen, & Zucker, 2010). Notably, many daily studies were implemented prior to the widespread adoption of texting and generally were not conducted among samples from low-SES communities possibly facing barriers to engagement. Few researchers using IVR for substance use data have enrolled youth or emerging adults, with the exception of Collins and colleagues (2003) who sampled 10 emerging adults.

Many emerging adults prefer texting over voice calls (A. Smith, 2011). Therefore, we expected that an SMS methodology may be acceptable to emerging adults, 98% of whom have a cell phone and, on average, send and receive over 100 texts per day and 3,200 texts per month (Anderson, 2015; Duggan, 2013; A. Smith, 2011). SMS is advantageous because it does not require the data plan or internet access necessary for web-based surveys or mobile applications. While, nearly all American cell phone owners use SMS, only 67% (75% of young adults 18–29) have broadband internet access at home, with lower rates among individuals without a high school education (47%) and those with incomes less than $20,000 (41%) (Horrigan & Duggan, 2015). About 75% of teenagers have unlimited text messaging plans; thus, we would expect similar rates among emerging adults (Lenhart, Ling, Campbell, & Purcell, 2010).

Recent studies have employed SMS given its availability and advantages discussed (Phillips, Phillips, Lalonde, & Dykema, 2014). Yet, the only study we could identify deploying assessments via SMS with substance using young adults took place with college students over a 14-day period, wherein 9 questions were texted three times per day resulting in response rates greater than 85% (Phillips et al., 2014). College students are not necessarily representative of urban emerging adults in resource-constrained communities who may have increased rates of substance use and may differ in motivations for engaging in research and/or barriers to using SMS. The extent to which SMS is feasible in such populations is unknown, but is important to investigate because they could potentially benefit from the increasing use of technology tools in research and healthcare in order to reduce barriers to engagement and increase access, particularly with regard to socially sensitive behaviors.

In the present study, we examined the feasibility and acceptability of SMS assessments of drug use and sexual behaviors among 18–25 year-old urban emerging adults. We also explored demographic and substance use factors associated survey compliance. We hypothesized that participants would find the text messaging modality acceptable and that substance use would be negatively associated with compliance, potentially due to the acute intoxication-related impairments and/or other psychosocial issues that complicate participants’ daily lives.

Method

Study Setting

We recruited participants from the Emergency Department (ED) at Hurley Medical Center in Flint, Michigan between November 2014 and September 2015 (see Bonar et al., in press for further detail). This ED is the only Level-One trauma center and public hospital in Flint. Similar to other urban mid-sized cities, Flint is a mid-sized city with poverty rates that are three times the national average (U.S. Census Bureau, 2015a, 2015b, 2015c, 2015d, 2015e). We recruited from the ED because young people attending EDs have increased rates of risk behaviors (Bonar et al., 2016; Cunningham et al., 2006; Walton et al., 2011; Wilson & Klein, 2000). This study was approved by the Institutional Review Boards at both The University of Michigan and Hurley Medical Center. A Certificate of Confidentiality was obtained from the National Institutes of Health.

Study Population

Patients aged 18–25 years attending the ED during recruitment shifts were eligible for screening for this longitudinal study. Screening exclusion criteria were: insufficient cognition or a physical condition precluding informed consent (e.g., psychosis, altered mental status, intoxication/overdose), presenting for a reason requiring social work intervention (e.g., sexual assault, suicidal ideation), having a live-in significant other already enrolled, hearing/visual impairment precluding participation, illiteracy and/or inability to understand English, or other longitudinal study participation.

Study Protocol

Research Assistants (RAs) recruited during day (typically 8:00 a.m. to 6:00 p.m.) and evening shifts (typically 2:00 p.m. to 12:00 a.m.) across all days of the week, including some national holidays, covering an average of six shifts per week. Potential participants were identified via electronic medical record and asked to self-administer a 15-minute screening survey on a tablet computer. Interested patients gave written, informed consent and received a gift valued at $1 (e.g., candles, headphones).

Eligibility criteria for the longitudinal study were: past 4-week illicit drug use or non-medical use of prescription drugs (opioids, sedatives, stimulants), past 4-week sexual intercourse without a condom, and having a cell phone with SMS capability. RAs described the longitudinal study to those eligible. Interested participants provided written consent and completed additional baseline surveys followed by a Timeline Follow Back Interview (TLFB) for substance use and sexual behaviors on the current and previous 28 days (Scott-Sheldon, Carey, & Carey, 2010; Weinhardt et al., 1998). Urine drug screens (UDS) were also obtained. Next, participants were oriented to the SMS surveys (including written and verbal instructions for maximizing privacy), and immediately received a practice version via SMS. Participants received $20 cash for the baseline survey and the interview, $5 cash for the UDS, and a $10 credit card gift card to offset the cost of SMS surveys. All baseline procedures took approximately 1–1.5 hours.

On the morning following their ED visit, participants received an automated text message at 10:00 a.m. local time prompting them to text “START” to begin the daily survey. These messages were initially delivered by a 6-digit short-code. When some participants (primarily those with government subsidized phones) reported problems receiving them, we switched to using a 10-digit local number. They received an automated reminder at 6:00 p.m. for incomplete surveys which could be completed up until midnight. Participants received the morning prompts and evening reminders (when necessary) daily for 28 days (texting back “STOP” would cease all prompts, but no participants did this). Our SMS assessment was developed and managed by Mosio Incorporated (San Francisco, CA).

Each day, 27 survey items texted in sequential order, one at a time, assessed: cannabis use, alcohol use, other drug use, risky sexual behaviors, and affect for the previous day (see Appendix for items). Nearly all drug-using youth at this ED report cannabis use (Bohnert et al., 2015), thus, our daily assessments focused on cannabis. If participants did not report a target behavior (e.g., cannabis use) on a given day they did not receive relevant follow-up questions (e.g., cannabis motives). Instead, they received an equal number of items querying other behaviors (e.g., exercise, working). Occasionally, participants contacted RAs about problems sending/receiving text messages. To alleviate some system errors, midway through the study our programmers installed a feature to re-send the most recent question received. When problems were not immediately remedied, we collected participants’ daily data via phone interview (N=19 of 3,108 possible surveys).

Participants received $2 for each completed survey and a $5 bonus for each week they completed 6 of 7 daily surveys, loaded onto their gift card weekly (total possible survey compensation = $76). Participants received text reminders of weekly survey completion and payment status. RAs attempted to contact participants if they missed two consecutive surveys. Upon survey completion, participants received thank-you messages stating the study phone number, the compensation structure, and a reminder to delete their messages. From Day 14 onward, closing messages included the number of remaining surveys. RAs also sent a weekly text message with a progress update.

After the 28-day survey period, participants completed a follow-up survey and a TLFB interview mirroring baseline measures ($25 cash compensation), including the UDS ($5 cash compensation), typically at the study hospital or a community location (e.g., coffee shop) where privacy was ensured.

Measures

Demographics

Items adapted from national surveys and prior research (Cunningham et al., 2014; Harris, 2009; Johnston, Bachman, O'Malley, & Schulenberg, 2011; G.R. Smith et al., 1996; United States Department of Health and Human Services, National Institutes of Health, & National Institute on Drug Abuse, 2008) were administered at screening to characterize participants’ age, gender, race, educational status, receipt of public assistance and other background characteristics.

Cell phone availability and characteristics

To assess eligibility, at screening, participants were asked (yes/no) if they had a cell phone and, if so, whether the phone had text messaging capability, was a smartphone, and if it had an unlimited text message and/or data plan (Ranney, Choo, Spirito, & Mello, 2013).

Substance use

At screening, items modified from the Substance Abuse Outcomes Module (SAOM; G.R. Smith et al., 1996; G. R. Smith et al., 2006) and the NIDA-Modified Alcohol Smoking and Substance Involvement Screening Test (ASSIST) version 2.0 (Humeniuk et al., 2008; WHO ASSIST Working Group, 2002), assessed substance use-related eligibility. Participants reported how many days in the past 4 weeks (0–28 days) that they used cannabis, cocaine/crack, street opioids, hallucinogens, inhalants, and/or methamphetamines, or had non-medical use of prescription sedatives, opioids, and stimulants. A modified TLFB assessed these substances over the previous 28 days at baseline and at follow-up for the 28 day survey period (Scott-Sheldon et al., 2010; Weinhardt et al., 1998).

Baseline and follow-up measures included the 16-item Short Rutgers Alcohol Problems Index (Earleywine, LaBrie, & Pedersen, 2008) and parallel versions we created for cannabis use (based on White, Labouvie, & Papadaratsakis, 2005) and other drug use to assess past 4-week consequences of substance use. On each scale, we summed the number of items that the participant reported occurring at least once. Participants only completed consequence measures for the specific type of substance used (i.e., either alcohol, cannabis, or other drugs); for participants who used more than one substance, their highest scale score was used for analysis.

Sexual partners and condom use

At screening, participants reported lifetime sexual activity (Centers for Disease Control and Prevention (CDC), 2012) and number of main and/or casual partners in the previous 4 weeks using definitions informed by the literature (Weinhardt et al., 1998) and previous research from the study site (Bonar et al., 2016). Items adapted from Monitoring the Future (Johnston, O'Malley, Bachman, & Schulenberg, 2010) assessed past 4-week condom use with each partner type. As a conservative indication of risk, any sex without a condom comprised the sexual risk eligibility criterion for the study. At baseline and follow-up, the modified TLFB assessed sexual partners and condom use (Scott-Sheldon et al., 2010; Weinhardt et al., 1998). For analyses, we coded participants at each time-point as having multiple partners (yes/no). If they had a main or casual partner we coded them based on presence of inconsistent condom use (yes/no).

Mental health

At screening and follow-up, the Generalized Anxiety Disorder-7 (GAD-7; Spitzer, Kroenke, Williams, & Löwe, 2006) and the Patient Health Questionniare-2 (PHQ-2; Kroenke, Spitzer, & Williams, 2003) assessed past 2-week anxiety and depression symptoms. Higher scores indicate more severe symptoms (GAD-7 range = 0 to 21; PHQ-2 range = 0 to 6). Cronbach’s alpha for the GAD-7 was 0.92 at screening and .88 at follow-up. The Spearman Brown correlation assessing reliability for the PHQ-2 was .65 at screening and .66 at follow-up.

Text message survey acceptability and difficulties

We created several follow-up survey items to assess participants’ attitudes and experiences with the SMS surveys, including: time spent completing surveys, perceived annoyance of the surveys, perceived ease of providing accurate information about substance use and sex, and comfort level with the text messages on substance use and sex, separately. Participants reported whether they used a password/code on their phones, if they deleted the surveys, and if others saw the survey messages. Mid-way through the study, we received approval to ask additional semi-structured questions to the sub-sample of participants who had not yet completed their one-month follow-up assessment. Participants were asked if they had difficulty sending or receiving the study text messages and their responses were audio-recorded and transcribed.

Data Analysis

We conducted descriptive analyses characterizing the feasibility and acceptability of the SMS methodology. Means, standard deviations, and proportions were calculated for variables of interest. Correlational analyses and t-tests were used to assess relationships between demographic, substance use, sexual risk, and mental health and number of daily surveys completed. Chi-square analyses assessed the distribution of total completed surveys based on week and day of the week and number of incentive bonuses received by week. We used multilevel binary logistic regression models to evaluate the daily-level relationships between daily text message survey completion and substance use (as measured by the TLFB at follow-up) while controlling for within-individual dependence. Separate models were fit to assess four substance use measures (quantity of cannabis, alcohol use, other drug use, polysubstance use). Polysubstance use was categorized into no use, one substance, and at least two substances. The first author reviewed responses to the qualitative items assessing text message completion difficulties and coded them into 8 categories. A single participant could mention multiple difficulties, thus receiving more than one code.

Results

Participants

Figure 1 displays the number of participants approached, screened, and enrolled. Table 1 displays mobile characteristics, substance use, and sexual behavior data at baseline for the 111 individuals who enrolled in the longitudinal study. Their mean age was 22.0 (SD=2.2). Participants were 53.2%% female and 45.1% reported their ethnicity was African American (37.8% European American, 17.1% Other). The majority were single (81.1%) and 18.9% were married or engaged, 20.7% were currently in school (73.0% were high school graduate or higher), and 43.2% received public assistance. Nearly all (95.5%) had a smartphone, 66.7% had an unlimited text message plan, and 64.0% had an unlimited data plan.

Figure 1.

Study Flow Chart (November 2014 – September 2015)

Table 1.

Participants’ Characteristics and Relationships with Number of Daily Surveys Completed

	Sample Characteristics	# Completed Surveys	Significance
	M (SD) or % (n)	M (SD)	T-value or r	p-value
Mobile phone characteristics
Has smartphone
Yes	95.50% (106)	18.25 (8.52)	t(109)= −1.38	0.17
No	4.50% (5)	12.80 (11.39)
Has unlimited text message plan
Yes	66.67% (74)	19.38 (8.83)	t(109)= −2.40	0.02
No	33.33% (37)	15.27 (7.78)
Unlimited data plan
Yes	63.96% (71)	19.07 (8.25)	t(109)= −1.73	0.09
No	36.04% (40)	16.13 (9.19)
Baseline variables
Cannabis use days	16.19 (11.09)	--	r= −0.14	0.11
Alcohol days	3.85 (6.11)	--	r= −0.19	0.04
Illicit drug use days	0.41 (1.69)	--	r= −0.08	0.40
Prescription drug misuse days	1.16 (3.50)	--	r= −0.10	0.30
Poly-substance use days	3.94 (6.53)	--	r= −0.19	0.051
Alcohol/drug consequences	4.21 (4.41)	--	r= −0.09	0.35
Multiple sex partners
Yes	18.92% (21)	19.33 (7.00)	t(109) = −0.78	0.44
No	81.05% (90)	17.70 (9.02)
GAD-7 score	7.56 (6.18)	--	r= 0.08	0.37
Baseline PHQ-2 score	1.57 (1.66)	--	r= 0.10	0.29
Follow-up variables
Cannabis use days	14.54 (10.37)	--	r= −0.03	0.79
Alcohol days	3.30 (4.70)	--	r= −0.07	0.48
Illicit drug use days	0.63 (3.19)	--	r= −0.18	0.08
Prescription drug misuse days	0.59 (2.03)	--	r= −0.10	0.33
Poly-substance use days	3.06 (4.87)	--	r= −0.15	0.15
Alcohol/drug consequences	2.68 (3.39)	--	r= −0.07	0.49
Multiple sex partners
Yes	22.78% (18)	19.72 (6.20)	t(95) = −0.31	0.76
No	81.44% (79)	19.10 (8.05)
Follow-up GAD-7 score	5.34 (4.49)	--	r= 0.06	0.50
Follow-up PHQ-2 score	1.49 (1.51)	--	r= 0.20	0.04

Over the 4 weeks before baseline, participants used cannabis on M=16.2 days (SD=11.1), alcohol on M=3.9 days (SD=6.1), prescription drugs on M=1.2 days (SD=3.5), and other illicit drugs on M=0.4 days (SD=1.7). Polysubstance use occurred on M=3.9 days (SD=6.5). On the baseline TLFB, most participants (81.1%) reported one sexual partner; 18.9% had at least two partners. Most (90.1%) had a “main” partner and 99.0% of those with a main partner reported inconsistent/no condom use. Fewer individuals (23.4%) had a casual partner, with 73.1% of them reporting inconsistent/no condom use. Among 97 participants at follow-up, 22.8% had multiple partners, 80.4% had main partners (92.3% with inconsistent/no condom use), and 23.7% had casual partners (65.2% with inconsistent/no condom use).

Text Message Survey Completion

The 111 baseline participants completed M=18.0 (SD=8.7) of 28 possible surveys, with five participants (4.5%) completing none. Surveys that were started, but not completed (i.e., “partial completions”) occurred on M=1.8 days (SD=2.2). The average number of surveys not started and not completed was 8.2 (SD=8.5). Among the 95.5% finishing any surveys, they completed M= 18.9 (SD=7.9), had M=1.9 (SD=2.2) partial completions, and M=7.3 (SD=7.5) were not started. The average percentage of surveys completed was 64.3% (67.4% after removing participants who completed zero surveys). Among the 111 participants, the average total compensation earned for the daily surveys and bonuses was M= $46.20 (SD=$24.62).

Table 2 shows daily completion rates. The number of completed versus partially completed and not started (combined) surveys was not significant based on day of the week (χ²[6]=4.96, p=0.54). Weekday (65.4%) completion rates were slightly higher than weekend (61.6%) rates (χ²[1]=4.00, p<0.05). Week One had the highest completion rate (76.1%) followed by Week 2 (66.2%), Week 3 (58.7%), and Week 4 (56.4%; χ²[3]=79.95, p<.001). Completion rates were higher in Week Two versus Weeks Three and Four, but Weeks Three and Four were not significantly different. Receipt of weekly $5 bonuses was highest in Week One (64.0%), with 54.1% receiving bonuses in Week Two, 41.4% in Week Three, and 44.1% in Week Four (χ²[3]=14.02, p<.01).

Table 2.

Of a Possible Total of 28 Surveys, Percentage Completed, Partially Responded to, and Not Started by Day of Week (N = 111).

Day of the Week	Survey Completion Status, % (n)
	Completed		Partial		Not Started
	n	%	n	%	n	%
Monday	294	68.21	25	5.80	112	25.99
Tuesday	297	68.91	32	7.42	102	23.67
Wednesday	286	66.36	29	6.73	116	26.91
Thursday	287	66.59	30	6.96	114	26.45
Friday	288	66.82	24	5.57	119	27.61
Saturday	272	63.11	35	8.12	124	28.77
Sunday	275	63.81	29	6.73	127	29.47

A total of 2,203 surveys were partially or fully completed. Cannabis use was reported on 50.7% (1086/2,144 non-missing responses), alcohol use on 17.6% (366/2,079), and other drug use on 6.3% (129/2,058). Participants reported having sex on 28.9% of days (594/2,055 non-missing responses), with 88.7% (525/592) of those having sex with a main partner. Condoms were used on 11.7% of days when sex occurred (69/592). Days of sex with multiple partners were infrequent (8.2% of sex days, 49/594).

Regarding demographics, participants receiving public assistance (M=19.7, SD=7.9) completed more surveys than those without public assistance (M=15.8, SD=9.3; t[109]=−2.38, p=.02). Those currently in school completed almost five more surveys (M=21.7, SD=5.7) than those not in school (M=17.0, SD=9.1, t[109]= −2.38, p=.02). Completion rates did not significantly differ by age group (M_18-21=17.6, SD=8.8 vs. M_22-25=18.3, SD=8.7), gender (M_females=19.4, SD=8.3 vs. M_males=16.4, SD=9.0), race (M_{African American}=18.1, SD=8.1 vs. M_Other=18.0, SD=9.2), educational attainment (M_less than high school=17.7, SD=9.1 vs. M_higher=18.1, SD=8.6), or marital status (M_{married/engaged}=15.0, SD=10.3 vs. M_single=18.7, SD=8.2).

As shown in Table 1, participants with unlimited text messaging completed more surveys than those without (M=19.4 vs. 15.3), but other phone characteristics were non-significant. Baseline, but not follow-up, alcohol use days were negatively correlated (r=−0.19, p=0.04) with total completions. Similarly, baseline, but not follow-up, poly-substance use days were negatively correlated (r=−0.19, p=.051) with surveys completed, which may be accounted for by alcohol use. Baseline and follow-up substance use consequences were not associated with completions. Completion rates did not differ based on having multiple sex partners at either time point (completion rates by condom use with each partner type were not examined to due to cell size). Depression symptom scores at follow-up were associated with more surveys completed (r=0.20, p=0.04).

Using the follow-up TLFB data, completion of the daily surveys was not significantly associated with same-day daily quantity of cannabis consumed (OR=0.99, 95% CI=0.98=1.01), number of drinks consumed (OR=1.00, 95% CI=1.00–1.01), or presence of other drug use (OR=1.04, 95% CI=0.94–1.16). Similarly, compared to days with no use, poly-substance use was not associated with daily survey completion (OR=1.02, 95% CI=0.98–1.05). Cannabis use and other drug use on the day prior to the survey was not associated with survey completion (cannabis OR=0.99, 95% CI=0.98–1.01; other drug OR=1.01, 95% CI=0.93–1.11). Both higher number of drinks consumed and polysubstance use on the day prior to the survey were associated with lower odds of survey completion (number of drinks OR=0.99, 95% CI=0.99–1.00, p<.001; polysubstance use OR=0.96, 95% CI=0.92–0.99, p<.01).

Text Message Survey Feasibility, Acceptability, and Difficulties

Among the 97 participants completing follow-up, 95.9% (n=93) provided responses on feasibility and acceptability of the surveys (see Table 3). Of participants completing at least one survey, 77.4% indicated the surveys took 15 minutes or less on average. The majority (90.3%) said the surveys were “not at all” or “a little” annoying. Nearly all said it was “somewhat” or “very” easy to answer accurately (97.9% for substance use; 95.7% for sexual behaviors) on the surveys. Almost all felt “very” or “somewhat” comfortable answering questions about substance use (97.9%) and sexual behavior (94.6%). The majority (80.6%) indicated they would participate in another similar study.

Table 3.

Participants’ Attitudes at Follow-Up About Daily Text Message Surveys (N=93)

Survey Item	n	%
Average minutes spent on daily survey
1–5 minutes	13	13.98
6–10 minutes	30	32.26
11–15 minutes	29	31.18
16–20 minutes	10	10.75
21+ minutes	11	11.83
Did surveys become annoying?
Not at all	52	55.91
A little	32	34.41
Somewhat	7	7.53
Very annoying	2	2.15
Difficulty of providing accurate information about alcohol/drug use
Very easy	79	84.95
Somewhat easy	12	12.90
Somewhat difficult	2	2.15
Very difficult	0	0.00
Difficulty of providing accurate information about sexual behaviors
Very easy	79	84.95
Somewhat easy	10	10.75
Somewhat difficult	4	4.30
Very difficult	0	0.00
Would participate in another daily text message survey study
Yes	75	80.65
No	3	3.23
Maybe	15	16.13
Comfort with text questions about alcohol and drugs
Very comfortable	78	83.87
Somewhat comfortable	13	13.98
Somewhat uncomfortable	2	2.15
Very uncomfortable	0	0.00
Comfort with text questions about sexual behavior
Very comfortable	74	79.57
Somewhat comfortable	14	5.05
Somewhat uncomfortable	5	5.38
Very uncomfortable	0	0.00
Deleted messages after each survey
No, never	9	9.68
Yes, always	67	72.04
Sometimes	17	18.28
Has password/passcode on phone
Yes	60	65.22
No	32	34.78
Did anyone else see the messages?
No	83	89.25
Yes	10	10.75
Boyfriend/girlfriend/spouse/fiancé	9	8.10
Other sex partner	1	0.90
Friend	1	0.90
Sibling	2	1.80
Other family member	1	0.90

Regarding strategies to manage survey privacy, 72.0% always deleted the messages after each survey and 65.6% used a phone passcode. All 10.8% who responded that another person saw the study texts were “not at all” bothered by this and indicated that most often the messages were viewed by a significant other.

Among the 55 participants who received the semi-structured items asking about difficulties receiving or responding to the text messages, 60.0% were male. Their mean age was 22.3 (SD=2.2); 47.3% were African American, 34.6% were European American, and 18.2% were other races. Among those, 29.1% had no problems with the texts. Several (25.4%) indicated they had technical problems (e.g., questions being sent twice, the system not registering responses), 18.2% complained of delays in receiving messages (possibly due to low-tier wireless providers), and 14.5% described problems with their mobile service provider (e.g., service disruptions). Others described sending an incorrect response due to making a typographical error (9.9%), having difficulty completing a survey before the midnight cut-off (e.g., due to working late; 9.9%), their phone service being cut-off or their phone breaking (7.3%), or other reasons (e.g., forgetting; 7.3%). Anecdotally, RAs who were in contact with participants also indicated that some occasionally had their phone service shut off due to lack of payment, but that they resumed surveys when their phones were re-activated. RAs also learned that some participants shared their phone with another person (e.g., sibling, partner) and did not complete surveys when the other person had the phone.

Discussion

Our findings indicate that the vast majority of respondents reported comfort with and easy completion of automated two-way SMS text messages assessing substance use and sexual behaviors. This is notable for a sample comprised of resource-constrained urban emerging adults who have not been included in such research. Most were willing to participate in similar research in the future even though the surveys were perceived as lasting several minutes. These findings are promising for future event-level research because such methods help researchers to identify temporal associations among risk behaviors, as well as understand contextual factors influencing risk behaviors during a developmental period of peak engagement in sexual risk and substance use. Consistent with precision medicine approaches, understanding these factors can inform the development of just-in-time interventions (Nahum-Shani et al., 2016) that are tailored to the unique contextual factors of the individual in real time. For example, knowing when individuals tend to engage in high risk sex and/or substance use allows for tailoring just-in-time intervention messages for high risk periods.

Regarding individual factors associated with compliance, one might expect that substance use could impact daily completion (e.g., potentially due to intoxication), although previous research suggests higher compliance on cannabis use days (Buckner, Crosby, Silgado, Wonderlich, & Schmidt, 2012). Results from daily data analysis, adjusting for the dependence of observations within participants, indicated that substance use as assessed by the TLFB at follow-up was not associated with whether participants completed surveys on the same day, but polysubstance use and number of drinks consumed were associated with lower odds of next-day completion. We also found evidence of lower compliance when examining the correlations between baseline (but not follow-up) alcohol and polysubstance use days and completion rates. Given that drug use, not alcohol use, was an entry criterion for the study, it may be that individuals using alcohol or multiple substances represent a higher risk group with unique barriers to engaging in daily process research. However, alcohol was infrequently used by participants in this sample, thereby reducing concerns about the impact of consumption on survey completion. Given these findings, future researchers collecting daily data from samples with heavier alcohol consumption should be advised to consider strategies to increase daily reporting.

Next, it is unclear why participants with greater depression symptoms at follow-up would complete more surveys. Those with greater symptoms may have suffered from anhedonia or waning motivation to engage in activities, thus allowing time to complete surveys, or they may have answered more surveys due to altered time perceptions. Specifically, subjective reports among depressed individuals indicate they perceive time as passing relatively slowly (Thönes & Oberfeld, 2015). Alternatively, it may be that increased attention to risk behaviors via daily surveys increased participants’ perceived depressive symptoms.

Few other quantitatively assessed factors were associated with completions. Those in school finished more surveys, suggesting that a routine schedule could facilitate daily reporting. Consistent with prior research (Phillips et al., 2014), rates differed by weekend versus weekdays; potentially indicating different activity patterns. Individuals receiving public assistance completed more surveys, potentially suggesting that they perceived the modest incentives as beneficial, that they had more flexibility in their time, or that they are more inclined to engage with research for altruistic purposes. Not surprisingly, participants with unlimited texting plans had higher completion rates whereas those with limited plans may have incurred more costs if completing more surveys. Thus, additional compensation for participants without unlimited texting may raise compliance in future studies. Further, the proportion of participants with unlimited texting plans was lower than national data from teenagers, which may reflect the lower SES of our participants, and/or their older age, where participants may be responsible for paying their own phone bills as opposed to sharing a bundled family plan, like many teens, particularly those from higher income households (Lenhart et al., 2010).

To our knowledge, this study is the first to use SMS to collect daily data from a high-risk, low SES sample of drug-using emerging adults. Given the nature of our sample and the relatively demanding research protocol, the overall completion rate of 64% provides a benchmark for future use of similar methods in resource-constrained populations. While compliance was lower than some IVR studies (Collins et al., 2003; Cranford et al., 2010), rates were similar to or improved compared to others using EMA. For example, Buckner and colleagues (2012) reported similar rates among cannabis using college students completing EMAs for two weeks (61–64%). Shorey and colleagues found slightly lower rates (57–61%) for 90-day web-based diaries completed by college students (Shorey, Stuart, McNulty, & Moore, 2014; Shorey, Stuart, Moore, & McNulty, 2014).

Research using SMS for daily process methods has not been conducted among such a socioeconomically disadvantaged group of emerging adults who may face considerable daily challenges, particularly after just presenting for emergency medical care. Encouragingly, 78.5% of ineligible participants had cell phones with SMS, indicating the potential feasibility of future studies using SMS to assess other behaviors among urban, low-SES emerging adults. To potentially increase completions, several points should be considered by future researchers. First, our survey period was 28 days, and similar studies were of shorter duration (e.g., 14 days). We also found that compliance declined after Week 1, suggesting that 28 days was too long for some participants to sustain engagement. Survey length and relative cumbersomeness may have been a deterrent for some, although most reported they were not annoyed by the surveys. We cannot calculate participants’ true response times because they were not required to complete surveys in a single sitting, which may be a strength of our design because it allowed flexibility. Questions may have become repetitive after several daily administrations; thus, future researchers could consider ways to modify daily question format, presentation, or order to increase engagement, although these modifications must be balanced against validity concerns. Because most participants reported it was easy to recall their daily behaviors accurately, researchers could consider methods to reduce response burden, such reducing the frequency of assessments (e.g., alternating days, weekly).

Although designed to increase completion rates, our weekly bonus incentives may have paradoxically reduced compliance. For example, if a participant missed the first two surveys in a week they may have been less likely to complete additional surveys because they would not receive the weekly bonus. Research testing how different compensation structures (including reminders of compensation earned or lost) impact completion rates within this population could provide useful data on strategies for maximizing engagement. Lack of a regular schedule due to low rates of employment or schooling or financial constraints associated with text message costs and cell phone bills may have reduced compliance. Any ongoing medical problems associated with the baseline ED visit could have altered participants’ routines in a way that affected survey participation. Further, to support validity, our study included “filler” items to balance survey length for participants not reporting target behaviors each day. Although this procedure likely increases the accuracy of self-report, this approach is relatively unique. Prior researchers have avoided such skip patterns (Cranford et al., 2010), or have used fewer questions to reduce burden and potentially enhance compliance.

Qualitative interview data combined with anecdotal reports from participants and RAs aid in understanding compliance challenges and may be helpful to future researchers as they consider study design. For example, early in the study we learned via participant contact that government-subsidized phones had problems receiving messages from a short code (these responses were not mentioned in the qualitative data collected for the latter half of participants). Thus, we began sending messages from a local 10-digit number. We also learned that some participants experienced system errors when responding (e.g., duplicate messages, responses not registering) so our programmers developed a feature where participants could text a keyword to re-send the most recent question in order to clear the problem. Occasionally, we learned of service disruptions where participants with a specific wireless provider were without texting service for several hours and when possible we collected participants’ data via phone interview. Participants attributed some problems with the study messages to lower-tier wireless service providers (e.g., those other than large national providers) which may have also contributed to some of the delays in sending/receiving messages. During the 28-day survey period, we only learned of concerns such as service disruptions or survey errors when participants either contacted us for help or if we reached them after at least two days of incomplete surveys, thus piloting assessment systems and checking in with participants regularly is recommended in order to identify problems early. Participants’ schedules and their ability to maintain consistent wireless service during the longitudinal study phase are also possible barriers to compliance that may be remedied by customizing assessment delivery schedules and/or providing incentives for phone service.

Despite the useful information provided on feasibility of these methods in this population, this study has limitations. Generalizability to other emerging adults or hospital patients requires replication. Because we did not anticipate all potential problems with SMS delivery, we did not create a systematic assessment for tracking all participants’ barriers to compliance, thus we relied on anecdotal reports and questions added midway through the study to inform some conclusions. Nonetheless, we believe this information may be helpful to future researchers attempting to employ these methods with similar populations. Providing study-funded mobile devices may have enhanced compliance, however, because most individuals already carry a phone, a second device may have been burdensome, and would have limited ecological validity.

Conclusions

We conclude from the present analysis that at-risk emerging adults are generally willing to engage in daily process studies of sensitive behaviors using text messaging, but that likelihood of daily survey completion appears unrelated to acute substance use in this sample. Participants were willing to share information to help elucidate the relationships between substance use and sexual risk and reported minimal discomfort in doing so. Future studies may benefit from employing different strategies to enhance compliance such as: increasing financial remuneration (balancing against the need to avoid coercion), fewer daily questions or assessments, longer daily completion windows, and/or allowing participants to choose preferred assessment schedules based on routines. Although shorter study duration may increase compliance, researchers may consider balancing the burden of a greater number assessments with increasing remuneration over time. Additionally, if larger sample sizes are possible, fewer daily assessments may be needed to obtain sufficient data. Finally, as technology access continues to improve, particularly among socio-economically disadvantaged samples, it is possible that some barriers to compliance found in this study will be remedied.

Appendix

Survey Item	Response Options
Marijuana use and motives
How many times did you use weed YESTERDAY, separated by at least 1 hour in between each use? Reply with a number: 0–24.	0–24
How much WEED did you have YESTERDAY?	<.5g (<1 joint), .5g (1 joint), 1g (2 joints),1.5g (3 joints), 2g (4 joints),2.5g+ (5+ joints)
Did you use weed yesterday TO ENJOY THE FEELING?	Not at all, a little, somewhat, a great deal
Did you use weed yesterday TO AVOID FEELING LEFT OUT?	Not at all, a little, somewhat, a great deal
Did you use weed yesterday TO EXPAND YOUR AWARENESS?	Not at all, a little, somewhat, a great deal
Did you use weed yesterday TO HELP YOU FEEL LESS DEPRESSED OR NERVOUS?	Not at all, a little, somewhat, a great deal
Did you use weed yesterday TO MAKE A SOCIAL GATHERING MORE FUN?	Not at all, a little, somewhat, a great deal
Alcohol use and motives
How many standard drinks containing ALCOHOL did you have yesterday? Reply with a number: 0–50 (type 50 if more than 50).	0–100
Did you use alcohol yesterday TO MAKE A SOCIAL GATHERING MORE FUN?	Not at all, a little, somewhat, a great deal
Did you use alcohol yesterday TO AVOID FEELING LEFT OUT?	Not at all, a little, somewhat, a great deal
Did you use alcohol yesterday TO HELP YOU FEEL LESS DEPRESSED OR NERVOUS?	Not at all, a little, somewhat, a great deal
Did you use alcohol yesterday TO ENJOY THE FEELING?	Not at all, a little, somewhat, a great deal
Other drug use
Did you use street drugs or prescription drugs to get high YESTERDAY?	Yes, No
Sexual behaviors and motives
How many people did you have sex with YESTERDAY? Reply with a number: 0–10 (type 10 if more than 10)	0–10
Was a condom used from start to finish EVERY TIME you had sex yesterday?	Yes, No
Did you have sex with a spouse, boyfriend, girlfriend, or fiancé YESTERDAY?	Yes, No
Did you use weed in the 3 hours before having any sex YESTERDAY?	Yes, No
Did you use street or prescription drugs to get high in the 3 hours before having any sex YESTERDAY?	Yes, No
Did you drink alcohol in the 3 hours before having any sex YESTERDAY?	Yes, No
Did you have sex yesterday TO FEEL CLOSER TO A PARTNER?	Not at all, a little, somewhat, a great deal
Did you have sex yesterday TO EXPERIENCE THE EXCITEMENT OF IT?	Not at all, a little, somewhat, a great deal
Did you have sex yesterday TO FEEL MORE SELF–CONFIDENT?	Not at all, a little, somewhat, a great deal
Did you have sex yesterday TO FEEL BETTER WHEN YOU WERE FEELING LOW?	Not at all, a little, somewhat, a great deal
Did you have sex yesterday TO AVOID BEING PUT DOWN BY OTHERS FOR NOT HAVING SEX?	Not at all, a little, somewhat, a great deal
Did you have sex yesterday TO MAKE SURE YOUR PARTNER WOULD STILL WANT TO BE WITH YOU?	Not at all, a little, somewhat, a great deal
Affect
How much of the day YESTERDAY did you feel unhappy, angry, or irritable?	None, some, about half, almost all or all
How much of the day YESTERDAY did you feel happy, friendly, or cheerful?	None, some, about half, almost all or all

Note. Capitalization within items reflects how items were presented on participants’ cell phones. Items about marijuana (Simons, Correia, Carey, & Borsari, 1998), alcohol (Grant, Stewart, O'Connor, Blackwell, & Conrod, 2007), sex (Cooper, Shapiro, & Powers, 1998) and affect (Hensel, Fortenberry, & Orr, 2010) were modified from previous research to fit the parameters of text messaging.