Measuring Daily Activity of Emerging Adults: Text Messaging for Assessing Risk Behavior

Abstract

We evaluated the validity of the use of an SMS text messaging survey for measuring daily life activity in a sample of emerging adults. Short Message Service (SMS) text messaging is a prevalent form of everyday communication in the lives of emerging adults, yet there is limited research on the use of automated text messaging as a data collection method in clinical research. Study participants were 274 ethnically diverse emerging adults (54.4% female, baseline age = 17–21 years), and constructs included alcohol use, substance use, school activity, peer interaction, mood, and interaction with parents. Participants responded to “bursts” that included multiple surveys during the course of 2 weeks, 6 months apart (a total of 13 texting surveys). Most of the questions were strongly associated across bursts. Findings revealed response stability for participating subjects across the 6 months and across the texting and self-report survey methodologies. Paired sample t-tests indicated that participants reported differently across data methodologies, which suggests that some data collection methodologies are best suited for certain types of constructs, such as alcohol consumption. Study results encapsulate the daily life of emerging adults and highlight the importance of evaluating the validity of SMS text messaging as a potential data collection device in future research.

Emerging adulthood represents the developmental period between late teens and early mid-20s for people living in industrialized societies (Arnett, 2007). Emerging adults in the early twenty-first century are a particularly relevant population for studying texting methodology because cell phones, common devices used in texting, have become important tools for many individuals in the population. Among Smith’s (2010) nationally representative sample, 96% of U.S. residents ages 18–29 years owned a cell phone. Further, texting is most prevalent among cell owners ages 18–29; 95% of these cell owners use cell phones to send texts (Smith, 2011).

Cell phones also became useful and effective data collection instruments during the past decade (Kuntsche & Labhart, 2014). Traditional self-report measures are not always ideal for data collection because they rely on memory recall (Phillips, Phillips, Lalonde, Dykema, 2014). According to research by Ferguson and Shiffman (2011), the passage of time can mitigate participant recall and lead to inaccurate reporting. Therefore, cell phones are powerful data collection tools because most people carry them regularly, so repeated measurements can conveniently be conducted (Kuntsche & Labhart, 2014).

Research on the use of automated text messaging in clinical studies is limited (Richmond et al., 2015). Few studies have evaluated the reproducibility and stability of texting as a method for collecting data in comparison with that of other sources, such as self-report. The bulk of literature about Short Message Service (SMS) text messaging in clinical and health research is focused on monitoring and data collection (Richmond et al., 2015). Studies have investigated the use of SMS text messaging in clinical domains such as alcohol use (Kuntsche & Robert, 2009), bulimia nervosa (Shapiro et al., 2010), and smoking cessation (Spohr et al., 2015). Although few studies have evaluated use of text messaging to collect data, particularly about substance use (Phillips et al., 2014), some studies have demonstrated its efficacy. In a study that investigated the use of SMS technology in alcohol research, Kuntsche and Robert (2009) found it to be an appropriate methodology for assessing alcohol use, because it is convenient to use, economical, and suitable for large surveys. In a study by Berkman, Dickenson, Falk, and Lieberman (2011) to assess the use of SMS for collecting data about smoking cessation, findings supported the use of this technology because of its low-cost and real-time measurement capabilities.

Collecting accurate data about risk behaviors is not a simple task (van Heerden, Norris, Tollman, Stein, & Richter, 2013). Collection of data about a participant’s substance use has traditionally relied on biological measures and/or self-report and recall of substance use (Phillips et al., 2014). Social desirability may affect reporting and lead to reporting biases (Harris, Griffin, McCaffrey, & Morral, 2008). For instance, participants are more likely to give socially desirable responses in face-to-face interviews as opposed to self-report questionnaires, because of a lack of perceived anonymity (Schwarz, Strack, Hippler, & Bishop, 1991). Further, parental reports used in the assessment and evaluation of adolescent behavior are not always correct because parents’ perceptions often contrast with the perceptions of their children (Hadley, Smith, Gallo, Angst, & Knafl, 2008).

SMS text messaging as a data collection method offers many advantages in clinical research. Modern cell phones present a solution to the challenges posed by self-report and measurement of irregularly occurring behaviors (van Heerden et al., 2013). According to Kuntsche and Labhart (2014), current texting technologies facilitate collection of data in real time with little bias. Rapid data collection and absence of interviewer bias are other advantages of cell phone use and automated text messaging in research (Johansen & Wedderkopp, 2010).

This study evaluated the reproducibility and stability of the use of SMS text messaging methodology for measuring overall daily life activity of emerging adults. Risk behavior was examined, along with daily activities, and assessments included alcohol use, substance use, school activity, interaction with peers, mood, and interaction with parents. The quality of all clinical research depends on the accuracy of data gathered from the research participants (Johansen & Wedderkopp, 2010); therefore, it is relevant to understand and evaluate the efficacy of SMS texting methodology in clinical research. To evaluate the acceptability and validity of using SMS to measure daily life activity of emerging adults, three primary study aims were addressed: (a) What are the response rates about risk behaviors for each item of the survey? (b) Is there response stability across texting surveys? (c) How do responses to text-messaged questions compare with responses to a one-time paper self-report questionnaire?

Method

Participants

Our study sample included 274 of the 593 emerging adults (117 male) ages 17–21 years (M = 19.4, SD = 0.71) who were participating in Project Alliance 2 (PAL 2), a large-scale longitudinal study. The study began when the participants were in sixth grade, when they were initially part of an intervention study to investigate the impact of a school-based, family-centered intervention on the development of risk behavior. The PAL 2 sample comprised families residing in urban neighborhoods in the Pacific Northwest region of the United States. Adolescents and their parents were recruited from three ethnically and socioeconomically diverse public middle schools. All parents of sixth grade students in three middle schools were invited to participate, and 80% of the parents agreed to do so. Participants were followed longitudinally from adolescence into emerging adulthood. For our study, participants were recruited as they completed a self-report survey and a texting assessment during emerging adulthood. Participants received $100 upon completion of the self-report questionnaire and were compensated $20 per burst of texting questions.

The data collection was approved by the appropriate institutional review board. The PAL 2 longitudinal study has maintained a high degree of retention, with 74% of the original sample (N = 593) participating during the emerging adult years (N = 441). All participants were offered an opportunity to participate in our SMS study, and 62% agreed to do so (N = 274). The sample continued to be an at-risk sample, with an average household income of $45,000 per year for a household of four. At age 20, 54% of young adults were living with their parents, 24% were attending a 4-year college, and 14% did not yet have a high school degree. The participants self-identified as follows: European American (35.8%), multiple ethnicities (23.0%), Hispanic-Latino (16.0%), African American (14.4%), Asian American (7.0%), American Indian-Native American (1.9%), Pacific Islander (1.9%). Rolling enrollment for participation in the texting assessment began in August 2013 and finished in February 2015.

Procedure

The texting questionnaire was administered by research assistants via a texting system. Participants used their own phones for the research project. Participants practiced the texting procedure once on personal cell phones at the research office. Two bursts of texting took place during a 1-year period. Each burst spanned a 2-week period. The survey, which consists of 11 questions, was administered every other day at 12:00 p.m. within the 2-week burst. The questions were administered at this time because participants were most likely to be reached then. The first burst included a total of seven surveys that were the same (except the first one is for enrollment). After that, one more burst occurred, with six total surveys. The bursts were automatically sent out every 6 months to collect data twice a year. The 6-month window of time was used between bursts to get a sense of participants’ behaviors at two particular time points. We decided to use “burst” methodology similar to that used in a study conducted by Ram et al. (2014), which used smartphone technologies to implement measurement bursts. In a measurement-burst design, a faster time scale is embedded in a slower time scale (Ram et al., 2014). A disadvantage of “burst” methodology is that it creates a 6-month window in which no data are being collected. An advantage of this design is that it allows researchers to analyze how short-interval and long-interval processes of change may be associated (Ram & Gerstorf, 2009).

Of the 274 participants, 269 completed part or all of the first burst (98%). Of these participants, the majority completed most of the texting bursts (70% completed at least six of the seven bursts). Part or all of the second burst was completed by 73% of the participants. Of these participants, 69% completed at least six of the seven bursts. Thus, the majority of participants completed most of the survey within bursts. For this study, we used scores based on at least three of the texts answered, which included scores from 91.82% of the 269 participants in Burst 1 (247) and 87.5% of the 200 participants in Burst 2 (175). Before receiving the first burst of the texting survey, participants concurrently completed a 45-minute paper questionnaire used to evaluate in depth various facets of a young adult’s life.

Measures

Texting survey questionnaire.

The texting survey questions measured the following constructs: alcohol consumption, marijuana consumption, drug use, time spent working or going to school, time spent interacting with friends, mood, time spent interacting with one or both parents, conflict and/or humor experienced with parent or parents, and parental support. The measures and the corresponding texting questions of the texting survey are shown in Table 1.

Table 1.

Matched Texting and Young Adult Survey Questions

Construct	Texting question	Young Adult Survey question
Alcohol consumption	1. In the past 24 hours, how many alcoholic drinks have you had? Reply with a number between 0 and 10. If you have had more than 10, please enter 10.	How often did you drink beer/wine/liquor in the last 3 months? When you drank beer/wine/liquor in the last 3 months, how much did you usually drink?
Marijuana use	2. In the past 24 hours, have you used marijuana? Reply with a Y for Yes or N for No.	How often did you use marijuana in the last 3 months?
Drug use	3. In the past 24 hours, have you used any other drugs to get high or buzzed? Reply with a Y for Yes or N for No.	Did you use any of these substances at least once in the past 3 months? (heroin, morphine, cocaine or crack, speed or meth, Ecstasy, angel dust or PCP, acid or LSD, DMT, mushrooms, gasoline, glue, other inhalants, over the counter meds for recreation, prescription meds for recreation)
Time spent working or going to school	4. In the past 24 hours, how many hours did you spend working or going to school? Reply with a number between 0 and 24.	For the average week, how many hours did you spend working for pay? For the average week, how many hours did you spend volunteering at a job for valuable work experience? For the average week, how many hours did you spend in structured educational activities such as vocational school, college, or university? For the average week, how many hours did you spend doing homework or other education-related activities outside of class time (e.g., internships)? For the average week, how many hours did you spend socializing on the telephone with friends or family members?
Time spent interacting with friends	5. In the past 24 hours, how many hours did you spend interacting with friends (in person, phone, text, social media)? Reply with a number between 0 and 24.	For the average week, how many hours did you spend online for fun or leisure? For the average week, how many hours did you spend hanging out with friends? For the average week, how many hours did you spend socializing on the telephone with friends or family members?
Depressive mood	6. In the past 24 hours, how sad, irritable, or depressed were you? Reply with a number between 1 (none or a little) to 10 (a lot).	Fill in the circle 0, 1, or 2 to describe yourself over the past 6 months. I am unhappy, sad, or depressed. (0 = not true, 1 = somewhat or sometimes true, 2 = very true or often).
Happy mood	7. In the past 24 hours, how happy/cheerful were you? Reply with a number between 1 (none or a little) to 10 (a lot).	Fill in the circle 0, 1, or 2, to describe yourself over the past 6 months. I am a happy person. (0 = not true, 1 = somewhat or sometimes true, 2 = very true or often) Please select the answer that best describes YOU over the past 3 months. Taking all things together, how would you say things are these days, would you say you’re happy, pretty happy, or not too happy these days?
Time spent interacting with parent(s)	8. In the past 24 hours, did you interact with one or both of your parents (phone, text, or in person)? Reply with a Y for Yes or N for No.	Have you had contact (e.g., in person, over the phone, text, Skype, social media, email) with mother in the past 3 months? (Yes/No) Have you had contact (e.g., in person, over the phone, text, Skype, social media, email) with father in the past 3 months? (Yes/No)
Conflict experienced with parent(s)	9. In the past 24 hours, was there conflict or tension between you and your parent? Reply with a Y for Yes or N for No.	We got angry at each other. (never, sometimes, about half the time, often, always) (mother) We got angry at each other. (never, sometimes, about half the time, often, always) (father) We argued or had a disagreement (over the telephone, email, or in person). (never, sometimes, about half the time, often, always) (mother) We argued or had a disagreement (over the telephone, email, or in person). (never, sometimes, about half the time, often, always) (father) We had a big argument about a little thing. (never, sometimes, about half the time, often, always) (mother) We had a big argument about a little thing. (never, sometimes, about half the time, often, always) (father) One of us got so mad we cut off communication with the other person. (never, sometimes, about half the time, often, always) (mother) One of us got so mad we cut off communication with the other person. (never, sometimes, about half the time, often, always) (father)
Humor experienced with parent(s)	10. In the past 24 hours, was there laughter, humor, or fun between you and your parent? Reply with a Y for Yes or N for No.	Over the last 3 months, how often did the following things happen between you and your mother: We enjoyed spending time together (over the telephone, email, Skype, social media, or in person). (never, sometimes, about half the time, often, always) Over the last 3 months, how often did the following things happen between you and your father: We enjoyed spending time together (over the telephone, email, Skype, social media, or in person). (never, sometimes, about half the time, often, always) Over the last 3 months, how often did the following things happen between you and your mother: I got along well with my mother. (never, sometimes, about half the time, often, always) Over the last 3 months, how often did the following things happen between you and your father: I got along well with my father. (never, sometimes, about half the time, often, always)
Parent support	11. In the past 24 hours, did your parent help you in any way? Reply with a Y for Yes or N for No.	In the past 3 months, to what extent would you seek or accept advice or guidance from your mother. (never, sometimes, about half the time, often, always) In the past 3 months, to what extent would you seek or accept advice or guidance from your father. (never, sometimes, about half the time, often, always) During the last year, how much of your financial support came from each of the following sources – your parents? (none, a little, some, about half, most, almost all, all)

Young Adult Survey questionnaire.

Participants were given a self-report questionnaire used to assess a wide variety of attitudes and behaviors in young adulthood. The Young Adult Survey (YAS) questionnaire was adapted from an earlier instrument developed by colleagues at the Oregon Research Institute (Metzler, Biglan, Ary, & Li, 1998) and from the Child and Family Center Youth Questionnaire (Child and Family Center, 2001a). The questionnaire was used to assess emerging adulthood (EA) risk behavior, school adjustment, achievement, and perception of parenting skills. Items were adapted in response to developmental changes likely to occur before or during EA. Constructs were operationalized based on theory and developmental research identifying risk behaviors and salient factors relevant to EA (Stormshak, DeGarmo, Chronister, & Caruthers, 2017). Questions were selected that corresponded with the constructs measured during texting (shown in Table 1).

Table 1 includes the measures, the corresponding texting questions, and the corresponding matching questions from the YAS. Alcohol consumption was specifically assessed through a self-report question adapted from the Teen Interview (CINT; Child & Family Center, 2001b). Marijuana consumption was specifically assessed through the self-report question from the CINT (Child & Family Center, 2001b). Drug use was assessed through a self-report question adapted from the CINT (Child & Family Center, 2001b). Time spent working or going to school was assessed through self-report questions from the Time Allocation Measure of Social Adaptation (TAMSA; Dishion & Caruthers, 2007). Time spent interacting with friends was assessed through self-report questions from the TAMSA (Dishion & Caruthers, 2007). Sad, irritable, or depressed mood was assessed through self-report questions from the Achenbach Adult SelfReport questionnaire (ASR; Achenbach, 2003). Happy or cheerful mood was assessed through self-report questions from the ASR questionnaire (Achenbach, 2003) and the General Social Survey (GSS; Smith, Marsden, & Hout, 2015). Time spent interacting with one or both parents, conflict or tension experienced with parents, and laughter/humor/fun experienced with parents were assessed through questions for each parent that were adapted from an earlier instrument developed by researchers at Oregon Research Institute (Metzler et al., 1998) and from the CFC Youth Questionnaire (Child and Family Center, 2001a). Parent support was measured through a self-report questionnaire that was adapted to assess social networks (Margolis, Fosco, & Stormshak, 2016). The construct was also assessed with a self-report question from the follow-up questionnaire for 19- to 30-year-olds in a national survey on drug use (Johnston, O’Malley, Bachman, Schulenberg, & Miech, 2016).

Converted Young Adult Survey questions.

To compare survey methodologies, each texting question from the first burst was matched with a corresponding YAS question. Only the first burst of texting questions was compared with the YAS survey questions because they were completed concurrently. To compare these questions by using concurrent time scales, the YAS questions were converted into composite variables with scales that matched the scales of the corresponding texting questions. Items and scales from the YAS were converted to map onto the dichotomous texting question for each construct of interest.

Data Analyses

Descriptive statistics for the response rates of the texting questions across the first two bursts of the study are reported in Table 2. Bivariate correlational analyses were conducted across Bursts 1 and 2 of the texting questions to test response stability across bursts. Bivariate correlational analyses also were conducted across the responses to texting questions and corresponding responses on self-report YAS questionnaires to test response stability across data collection methodologies. These results are reported in Table 3. Last, paired sample t-tests were conducted to compare reported responses in the texting survey and in the YAS questionnaire for reported alcohol consumption, marijuana use, drug use, hours spent working or going to school, hours spent interacting with friends, and conflict experienced with parents. The participants were recruited throughout all seasons of the year, and the majority of them were not in school. As a result, we do not have any hypotheses about seasonal differences in response rates. We controlled for time of day by texting respondents at the same time of day during each burst. However, sometimes they did not respond to the text until later in the day or evening.

Table 2.

Descriptive Statistics for Texting Questions Bursts 1 and 2

Survey texting question	Burst 1			Burst 2
	n	M	SD	n	M	SD
Alcohol consumption	249	0.52	0.96	172	0.52	0.85
Marijuana use (Y/N)	248	0.18	0.32	171	0.18	0.32
Drug use (Y/N)	248	0.01	0.03	170	0.02	0.10
Time spent working or going to school*	248	4.45	3.49	170	5.65	3.62
Time spent interacting with friends	247	7.75	5.08	170	7.52	4.53
Depressive mood*	245	2.60	1.46	169	2.91	1.58
Happy mood*	245	6.97	1.67	169	6.87	1.55
Time spent interacting with parent(s)*	243	0.85	0.25	168	0.77	0.31
Conflict experienced with parent(s)*	242	0.13	0.20	168	0.10	0.20
Humor experienced with parent(s)*	242	0.72	0.32	167	0.62	0.36
Parent support*	240	0.64	0.33	167	0.58	0.35

Note. Paired sample t-tests were conducted for each measure.

^*p < .05

Table 3.

Correlations Across Bursts 1 and 2 of the Texting Data Questions (r₁) and Across Texting Data Questions (Burst 1) and Young Adult Survey Questionnaire (r₂)

Texting question	r1	r2
Alcohol consumption	.592*	.467*
Marijuana	.789*	.699*
Drug use	.029	.112a
Hours worked in school	.491*	.294*
Hours interacting with friends	.615*	.397*
Irritable/depressed/sad	.566*	.363*
Happy/cheerful	.594*	.445*
Interacting with parents	.481*	.327*
Conflict with parents	.361*	.378*
Laughter/fun with parents	.473*	.306*
Help from parents	.475*	.284*

^aNote. Because the texting question and the converted YAS questions are categorical (yes/no), Cramer’s v is reported here instead of a correlation. The corresponding chi-square test reveals a significant relationship, χ² (1, N = 257) = 3.21, p > .05

^*p < .001

Results

Descriptive statistics for means, standard deviations, and frequencies of the response rates of each texting question across the two bursts of the study are reported in Table 2. Statistics are provided for the total sample and are based on participants’ self-report measures. Descriptive statistics indicate that reported alcohol use increased over time. Reported marijuana use, drug use, hours worked or in school, hours interacting with friends, depressed mood, happy mood, interaction with parents, conflict with parents, humor with parents, and help received from parents remained stable over time.

Overall, the texting survey provides a snapshot of the day-to-day behaviors of an emerging adult in this sample. In Burst 1, the average young adult, per day, consumed .52 drinks, spent 4.45 hours working or attending school, spent 7.75 hours interacting with friends, and spent about 1 hour interacting with one or both parents. An average of 18% of the young adults reported having smoked marijuana in the previous 24 hours; an average of 1% of the young adults reported other drug use. Regarding depressive mood, young adults reported an average of 2.60 on a Likert scale ranging from 1 (a little) to 10 (a lot). Regarding happy mood, young adults reported an average of 6.97 on a Likert scale ranging from 1 (a little) to 10 (a lot). An average of 85% of young adults reported that they had interacted with their parents during the previous 24 hours. An average of 13% of young adults reported conflict between them and their parents during the past 24 hours; an average of 72% of young adults reported that they had experienced laughter or fun with their parents during the previous 24 hours. Last, an average of 64% reported having received help from their parents in the previous 24 hours. Further, the survey provides an assessment of risk behaviors, such as alcohol and drug consumption.

Bivariate correlations across Bursts 1 and 2 of the texting questions are reported in the second column of Table 3. Response rates for all the measures across Burst 1 and Burst 2, excluding drug use, were significantly associated across Burst 1 and Burst 2 (p < .001). Results of the bivariate correlations across Burst 1 and Burst 2 indicate that the texting survey is a reliable assessment across time for the assessed measures, excluding drug use.

Bivariate correlations across the responses to texting questions and corresponding responses on self-report YAS questionnaires are reported in the third column of Table 3. In general, the texting and questionnaire data were correlated in this sample. Response rates for all the measures, excluding drug use, were significantly associated across Burst 1 and Burst 2 (p < .001). A chi-square test of independence was performed to examine the relation across response rates for drug use, for both data collection methodologies. The relationship between these variables was not significant, χ²(1, N = 274 = 3.21, p > .05. Results of the bivariate correlations across the texting survey and self-report survey measures indicate that the texting survey is a reliable assessment across methodologies.

Paired sample t-tests were conducted for reported responses of alcohol consumption, marijuana use, drug use, hours spent working/going to school, hours spent interacting with friends, and conflict with parents. Paired sample t-tests are reported in Table 4.

Table 4.

Results of Sample t-test and Descriptive Statistics for Texting (Burst 1) and Young Adult Survey Questions

	Texting		YAS			95% CI for mean difference
	M	SD	M	SD	n		t	df
Alcohol consumption	0.59	1.01	0.44	0.93	219	0.02, 0.29	2.29**	218
Marijuana use	0.25	0.36	0.21	0.37	177	−0.00, 0.08	1.73	176
Drug use	0.08	0.28	0.09	0.28	257	−0.04, 0.04	−0.17	256
Working or going to school	4.45	3.48	5.34	3.73	248	−1.43, −0.35	−3.26**	247
Interaction with friends	7.75	5.08	1.29	1.32	247	5.85, 7.03	21.50*	246
Irritable sada	0.00	1.00	−0.00	0.99	245	−0.14, 0.14	0.01	244
Happy/cheerfula	0.00	1.00	0.00	0.99	245	−0.13, 0.13	−0.08	244
Interaction with parentsa	0.00	1.00	−0.00	1.02	243	−0.14, 0.15	0.10	242
Conflict with parents	0.13	0.20	0.22	0.20	240	−0.12, −0.06	−6.71*	239
Laughter/fun with parentsa	0.02	0.97	0.01	1.00	239	−0.12, 0.16	0.27	238
Help from parentsa	0.00	1.00	−0.01	0.99	240	−0.14, 0.16	0.16	239

^aNote. Paired sample t-tests were conducted using the z-scores of the texting and YAS measures.

^*p < .001

^**p < .05

Findings from the paired sample t-tests suggest that participants answered differently on specific measures, depending on the type of data collection tool. Specifically, participants reported more alcohol use and time spent interacting with friends on the texting survey than on the self-report paper survey. Further, participants reported more hours spent in school and more conflict with parents on the self-report survey than on the texting survey. Paired sample t-tests were conducted using the z-scores of the variables to compare reports of mood, interaction with parents, laughter/fun with parents, and help received from parents; however, these paired sample t-tests were not statistically significant (p < .05).

Discussion

Current and developing smartphone technology provides more possibilities than ever before for collecting data in real time with mitigated bias and more accuracy in clinical research (Kuntsche & Labhart, 2014). Our study sought to evaluate the reliability of this data collection tool and also compare it with a concurrent self-report measure. We collected data from emerging adult participants and evaluated the validity of the use of SMS text messaging for assessing daily life activity of this age cohort, including risky behaviors. Study findings contribute to developing research about the validity of SMS texting as a data collection tool for assessing risk behaviors in clinical and prevention research (Berkman et al., 2011; Kuntsche & Robert, 2009; Lim, Sacks-Davis, Aitken, Hocking, & Hellard, 2010; Phillips et al., 2014). Further, this research provides insight into the daily life of emerging adults in a highly diverse sample.

The study addressed three primary questions: (a) What are the response rates about risk behaviors for each item of the survey? (b) Is there response stability across texting surveys? (c) How do responses to text-messaged questions compare with responses to a one-time paper self-report questionnaire? First, descriptive statistics across the first two bursts demonstrated response stability of reported measures over time. Although reported alcohol use increased over time, reported marijuana use, drug use, school/work activity, peer interaction, mood, and parent–youth interaction remained stable over time across the first two bursts. It is interesting to note high levels of reported marijuana use in this sample across the two bursts of the sample (18%) in the context of the recent legalization of marijuana in the state of Oregon, where the study took place. These use rates are also much higher than national rates for this age, which suggest about 4% of young adults use marijuana daily (Hedden, 2015; Johnston et al., 2016). In addition, there were very high reported rates of parent–youth interaction in the first burst (85%), illuminating the high level of contact that emerging adults in this sample have with their parents.

Second, there were significant bivariate correlations across the first two bursts for the majority of the texting questions (except for drug use), indicating response stability of participants’ responses over time. There may not have been response stability over time of drug use because of low base rates of drug use in the sample. In line with previous research, this study demonstrated the efficacy of SMS texting methods to collect data (Kuntsche & Robert, 2009; Phillips et al., 2014).

Third, a comparison was made between texting survey data and YAS report data to determine whether participants reported similarly across data collection methods. Because previous research has noted that collecting accurate data about risk behaviors can be difficult (van Heerden et al., 2013), it was important to recognize how participants reported on varying measures with different data collection methodologies. The high number of significant correlations between the texting survey responses and comparable self-report questionnaire data demonstrate response stability across the two data collection methods. This finding suggests that the texting survey is a reliable assessment across methodologies, and the constructs have face validity when compared with self-report questionnaire data, which is the standard data collection methodology for most research.

Paired sample t-tests indicate that participants reported more alcohol use and time spent interacting with friends on the texting survey than on the self-report survey. Also, paired sample t-tests indicate that participants reported fewer hours spent in school and less conflict with parents on the texting survey than on the self-report questionnaire. Some data collection methodologies may be better suited for certain types of constructs. Participants may have reported more alcohol use on the texting survey than on the self-report questionnaire because of the absence of interviewer bias (van Heerden et al., 2013). Further, use of recall in self-report may limit accuracy and increase bias because of a potential lack of retained memory of experiences (Shiffman, Stone, & Hufford, 2008), which may play a part in differing responses across surveys.

Limitations and Future Research

Findings should be interpreted in light of the study’s limitations. First, for the texting questions about reported mood, interaction with parent, laughter/fun with parents, and help received from parents, it was difficult to match comparable questions from the YAS questionnaire because of differences in scales and time frames. Therefore, it is important to use caution while interpreting these correlations across the texting questions and YAS questions. Because of these differences in scales and time frames, the paired sample t-tests for these same measures (mood, interaction with parents, laughter/fun with parents, and help received from parents) were conducted using z-scores of the variables, as opposed to the converted composite variables.

Also, troubleshooting challenges occurred during the distribution of the texting survey that affected data collection, including the frequency of phone numbers changing and the difficulty of working with specific phone carriers, such as T-Mobile. Last, greater numbers of assessments produce more reliable outcomes (Stone & Shiffman, 1994). This study evaluated measures every other day for 2 weeks in one burst. Although this is a consistent repeated measure, it is important to note that more assessments may lead to more valid assessments of these behaviors. Social and geographical context may have had a significant impact on this study, given the recent legalization of marijuana in the state of Oregon and liberal laws about medical marijuana. Thus, the results may not be generalizable to other geographic locations.

This project has implications for future clinical and prevention research. Because cell phone ownership and cell phone use as a communication tool are so pervasive, it is relevant to further explore the appropriateness of this tool for data collection in clinical and prevention research. Valid and reliable methodological data collection tools could furnish accurate information about substance use and risk behavior in young adult and other populations. A more comprehensive understanding of how SMS text messaging can be applied to data collection can have favorable implications for how data are collected among emerging adult populations. In addition, further research is needed to identify underlying mechanisms that led to differing response rates across methodologies for these measures. Participants may have differed in their reports of substance use, hours spent in school, and conflict experienced across surveys because of the “on the spot” nature of the texting survey, as opposed to retrospective recall used in self-report questionnaires (Phillips et al., 2014).

Overall, findings support the use of SMS automated text messaging as a method of collecting data about various measures of daily life in a sample of emerging adults. The findings appear to align with those from other studies regarding the use of text messaging in clinical research. The data collection method was reliable over time and across methodologies (a self-report questionnaire). An advantage of this method of data collection over other methods is the convenience and ease with which self-rated items scores can be captured longitudinally (Richmond et al., 2015). Findings from this study also suggest that participants answered differently on specific measures, depending on the type of data collection tool, indicating that perhaps some data collection methodologies are more appropriate for different types of measures. Study results not only shed light on the daily lifestyle of a sample of emerging adults, they also provide a rationale for investigating the appropriateness of distinct data collection methodologies for different types of measures. Identifying mechanisms that may explain why the type of data collection methodology matters can help inform future clinical and prevention research designs.