Acceptability of Ecological Momentary Assessment Among Young Men Who Have Sex with Men

Abstract

The study evaluated the acceptability of text message- and voice-based ecological momentary assessment (EMA) methods among a sample (n=74) of young men who have sex with men (MSM). We assessed the acceptability of text message- and voice-based EMA methods. Almost all participants (96%) reported that they would be willing to accept texts on their smartphone to answer questions about their current mood, surroundings, or feelings. A large majority (89%) also reported being willing to accept phone calls to answer these questions. This work suggests that different EMA methods are acceptable for use among young MSM.

INTRODUCTION

Ecological momentary assessment (EMA) encompasses a suite of methods and approaches that aim to assess health behaviors and their contexts in real-time in real-world environments (Stone, Shiffman, Atienza, & Nebeling, 2007). These methods were developed in part in response to the limitations of retrospective recall, as recalled data assessed via self-report can frequently be unreliable and are often systematically biased (Coughlin, 1990). These methods recognized that behaviors and experiences are affected by context, so data on them must be collected in real-world settings rather than a controlled research setting for it to be representative of one’s lived experienced (Stone et al., 2007). Studies utilizing these methods often involve repeated measures over varying durations, affording the temporal resolution needed to assess the dynamics of within-subject changes in behavior and experience over time and across contexts (Stone et al., 2007). These methods have employed various modes of data collection, including palm-top computers (Porter et al., 2000), telephones (Collins, Kashdan, & Gollnisch, 2003), and smartphone applications (Spook, Paulussen, Kok, & Van Empelen, 2013). These methods have been employed in various populations to study a variety of constructs, including mood and affect (Porter et al., 2000), alcohol use (Collins et al., 2003), and physical activity and sedentary behavior (Spook et al., 2013). These methods are useful tools in studying health behaviors as they are often discrete and episodic (Stone et al., 2007). However, few studies utilizing EMA approaches have focused on vulnerable and marginalized populations that experience substantial health disparities, including gay, bisexual and other men who have sex with men (MSM). Young MSM in particular are disproportionately impacted by HIV, other sexually transmitted infections, substance use, and mental health conditions (Stall, Friedman, & Catania, 2008), and it is important to note that understanding their behaviors and contexts in real-time through EMA could potentially aide in the reduction these disparities. Many of the health disparities experienced by young sexual minority men (i.e., those who identify as male and identify as gay, bisexual, or some other non-heterosexual identity and/or engage in sexual behaviors with other male-identified individuals) are related to specific health behaviors that can be measured via EMA approaches. For example, in the United States, young men who have sex with men (MSM) are disproportionately impacted by the HIV epidemic and account for the highest number of new HIV infections each year (Control & Prevention, 2015). EMA approaches have been used successfully with older MSM to measure condomless sexual behavior to identify events that place the individual at increased risk for HIV infection (Wray, Kahler, & Monti, 2016). However, none of these studies have specifically focused on younger MSM. In addition, sexual minority individuals, in particular sexual minority youth, are at increased risk for substance use and misuse compared to their heterosexual peers (Hughes & Eliason, 2002). EMA approaches have been used successfully with older MSM to measure alcohol and other drug use (Turner et al., 2017; Yang et al., 2015), but none of these studies have been conducted with younger populations. As such, there is a need to assess the acceptability of these approaches prior to implementation in pilot projects. The objective of this study was to evaluate the acceptability of text message- and voice-based EMA methods among a racially and ethnically diverse sample of young MSM.

METHODS

Participant Recruitment

Participants were recruited from Project 18 (P18), a prospective cohort study of sexual behavior, substance use, and mental health burdens involving 600 diverse young MSM recruited between 2009 and 2011. The P18 cohort study has been described in detail elsewhere (Halkitis et al., 2013). Participants were eligible for participation in the parent study if they were 18 or 19 years old, assigned male sex at birth, lived in the New York City metropolitan area, reported having had sex with another male in the six months preceding screening (operationalized as any physical contact that could lead to orgasm), and self-reported their HIV status as negative or unknown.

Regarding the current study, participants were enrolled in a sub-study aiming to assess the acceptability and feasibility of using Global Positioning System (GPS) methods to measure spatial mobility and exposure to neighborhood contexts (n=75) (Duncan et al., 2016). Recruitment methods have been described previously. In brief, a sample of 200 active study participants were invited to be screened, of which 82 individuals called the research office to be screened, 81 individuals were screened, and 75 individuals were scheduled and enrolled. Targets for enrollment in the sub-study were set based on race/ethnicity so that there would be roughly equivalent numbers of White, Black, and Hispanic participants. In addition, the parent study (Project 18) oversamples racial/ethnic minority MSM to provide the opportunity to conduct analyses that compare the experiences and behaviors of individuals of different racial and ethnic backgrounds to identify their specific needs. Assessments for the sub-study were conducted in 2014. All protocols were approved by the New York University Committee on Activities Involving Human Subjects and written informed consent was obtained from each participant prior to participation.

Measures

The acceptability of text message- and voice-based EMA methods were assessed using two items: “Would you participate in a study that sent you texts via a smartphone asking you questions about your current mood, surroundings, and feelings?” and “Would you participate in a study that called you to ask questions about your current mood, surroundings, and feelings?” These items had two response options (“yes” and “no”).

Socio-demographic characteristics were collected. Participants reported their age (years), gender (male, transfemale, genderqueer) and race/ethnicity (White, Black, Hispanic, Asian, Multiracial/other). Participants also reported whether or not they were currently enrolled in school (yes, no) and their highest level of education completed (high school or less, some college/ technical school, college degree or more). In addition, participants were asked where they currently lived: in a family apartment/house, their own apartment/house, with friends/roommates or in temporary housing/shelter. They also reported their total individual annual income which is categorized here as <$15,000, $15,000-$35,000, and >$35,000. Data were collected on sources of income including: regular job, public assistance, odd jobs, own business, parent, selling drugs and sex for money/drugs. We assessed sexual identity and it was dichotomized as exclusively homosexual versus not exclusively homosexual. To assess for relationship status, participants reported whether or not they currently had a main/steady partner (yes, no). In addition, participants reported if they were foreign born (yes, no). Participants were also asked to report their HIV status as part of the parent study. If the participant reported being negative, unknown or positive without proof of status, their status was confirmed with the INSTI^™ HIV-1/HIV-2 Rapid Antibody Test from bioLytical^™ Laboratories. All positive test results were further verified with a confirmatory HIV test, and individuals who tested HIV-positive were linked to care services.

Statistical Analyses

In total, 74 participants responded to the EMA items, representing 98.7% of the sample for the sub-study. First, descriptive statistics (e.g. frequencies) were calculated for all items. Chi-square tests of independence and Fischer’s exact test (when expected cell counts were less than 5) were used to test the null hypothesis that the distribution of yes/no responses to each of the items varied by socio-demographic categories. Statistical significance was calculated at p < .05. All analyses were conducted in SPSS Version 23.0 (IBM Corporation, Armonk, New York) in 2016.

RESULTS

The socio-demographic characteristics of our sample are described in Table 1. The vast majority (92%) were either 22 or 23 years of age. Overall, 70.3% of the participants identified as Black or Hispanic, and a majority earned less than $35,000 annually. In this sample, participants’ responses supported the idea that EMA is a feasible tool for data collection (Table 2). Almost all participants (95.9%) reported they would be willing to accept texts on their smartphone to answer questions about their mood, surroundings or feelings. Moreover, a large majority (89.2%) also reported being willing to accept phone calls to answer these types of questions. As shown in Table 2, exploratory analyses examining the acceptability of text message- and voice-based EMA methods by various socio-demographic characteristics (e.g., race/ethnicity, educational status, sexual orientation) show no appreciable differences by sub-groups (p > .05).

Table 1.

Sample Demographics (n = 74)

	% (n)
Age
21 years old	8.2 (6)
22 years old	45.9 (34)
23 years old	45.9 (34)
Gender Identity
Male	93.2 (69)
Transgender female	1.4 (1)
Genderqueer	2.7 (2)
No Gender Identification	2.7 (2)
Sexual Identity (Exclusively Homosexual)	59.5 (44)
Race/Ethnicity
White or Caucasian	25.7 (19)
Black or African American	32.4 (24)
Hispanic or Latino	37.8 (28)
Asian or Pacific Islander	6.8 (5)
Multiracial or other	6.8 (5)
National Origin (Foreign Born)	12.2 (9)
Educational Attainment
High school or less	60.8 (45)
Some college/technical school	10.8 (8)
College degree or more	28.4 (21)
Enrolled in School Currently (Yes)	31.1 (23)
Current Housing
Family apartment or house	40.5 (30)
Own apartment or house	18.9 (14)
Friends/roommates	28.4 (21)
Temporary housing, SRO, or shelter	12.3 (9)
Individual Annual Income
Less than $15,000	51.3 (37)
$15,000 to $35,000	37.5 (27)
Greater than $35,000	11.0 (8)
Sources of Income
Regular job	81.1 (60)
Public assistance	20.3 (15)
Odd jobs	40.5 (30)
Own business	12.2 (9)
Parents	70.3 (52)
Selling drugs	5.4 (4)
Sex for money and/or drugs	6.8 (5)
Has a Partner Currently
Yes	47.3 (35)
No	52.7 (39)

Table 2.

Acceptability of Ecological Momentary Assessment (EMA) Among Young MSM Overall and by Socio-Demographic Characteristics

Text-Messaged-Based EMA	%(n)	%(n)	p value
	Yes	No
Overall Sample	95.9 (71)	4.0 (3)
Race/Ethnicity			.759
White or Caucasian	100.0 (17)	0.0 (0)
Black or African American	96.0 (24)	4.0 (1)
Hispanic or Latino	90.5 (19)	9.5 (2)
Asian or Pacific Islander	100.0 (4)	0 (0)
Multiracial or other	100.0 (6)	0.0 (0)
Individual Annual Income
Less than $15,000	94.6 (35)	5.4 (2)	.999
$15,000 to $35,000	96.3 (26)	3.7 (1)
Greater than $35,000	100.0 (8)	0.0 (0)
Educational Attainment			.999
High school or less	95.6 (43)	4.4 (2)
Some college/technical school	100.0 (8)	0.0 (0)
College degree or more	95.2 (20)	4.8 (9)
Has a partner currently			.242
Yes	100.0 (35)	0.0 (0)
No	92.3 (36)	7.7 (3)
Voice-Based EMA	%(n)	%(n)	p value
	Yes	No
Overall Sample	89.2 (66)	10.8 (8)
Race/Ethnicity			.583
White or Caucasian	82.4 (14)	17.6 (3)
Black or African American	84.0 (21)	16.0 (4)
Hispanic or Latino	95.2 (20)	4.8 (1)
Asian or Pacific Islander	100.0 (4)	0 (0)
Multiracial or other	100.0 (6)	0.0 (0)
Individual Annual Income
Less than $15,000	83.8 (31)	16.2 (6)	.473
$15,000 to $35,000	92.6 (25)	7.4 (2)
Greater than $35,000	100.0 (8)	0.0 (0)
Educational Attainment
High school or less	91.1 (41)	8.9 (4)	.645
Some college/technical school	87.5 (7)	12.5 (1)
College degree or more	85.7 (18)	14.3 (3)
Has a partner currently			.464
Yes	85.7 (30)	14.3 (5)
No	92.3 (36)	7.7 (3)

DISCUSSION

To our knowledge, this is the first study to examine the acceptability of EMA methods among a sample of young MSM and one of few studies to examine the acceptability of these methods in MSM populations overall. Findings from this study suggests that EMA methods are acceptable for use among a multi-ethnic sample of young MSM. Specifically, the lack of significant variations in EMA acceptability across sub-groups suggests that both text message- and phone-based methods can be implemented in diverse populations of young MSM.

These results corroborate the limited previous research in this area, including a recent study demonstrating the acceptability and feasibility of smartphone-based EMA protocol (i.e. survey prompts thru a smartphone app) to assess alcohol use among a small sample of Black MSM in Baltimore, who ranged from 27 to 62 years of age (n=15) (Yang et al., 2015). In addition, another recent study implemented a EMA protocol among a racially/ethnically diverse sample of 30 substance-using MSM in San Francisco with a mean age of 43 (standard deviation 9.3) (n=30) (Rowe et al., 2016). This study aimed to assess the concordance between daily reports of substance use collected by EMA text messages (short message service, SMS) and retrospective audio computer-assisted self-interview (ACASI) questionnaires and identify predictors of daily concordance in the sample. In this study, methamphetamine (EMA 20%, ACASI 11%, p<0.001) and alcohol use (EMA 40%, ACASI 35%, p=0.001) were reported significantly more frequently via EMA versus ACASI, which additionally highlights the importance of utilizing EMA methods over traditional survey methods.

Our study is subject to some limitations, including generalizability. Because non-probably recruitment methods were utilized, this sample may not be generalizable to the larger population of young MSM in New York City. In addition, this was a relatively small sample of young MSM in New York City. Sampling MSM may also include individuals who are at high-risk for HIV infection but may not identify as gay, bisexual, or some other non-heterosexual identity. Given the aims of the parent project, it was important to include all MSM on the basis of their sexual activity with other male-identified individuals, regardless of their self-identified sexual orientation. These findings may be affected by social desirability bias, where some participants may have felt uncomfortable answering ‘No’ or stating that they would not be willing to participate in a project using EMA methods. However, we believe that the influence of this bias has been minimized through our use of an audio computer-assisted self-interview (ACASI), which allowed participants to take the survey in private and on their own. Despite these limitations, this study adds a meaningful contribution to the literature by expanding to young MSM. It also utilizes the largest sample of MSM for any EMA feasibility and acceptability study to date.

EMA methods have the advantage of collecting data on behaviors in real-time in real-world environments, so that the influence of recall bias is minimized and so that the data collected are representative of an individual’s lived experiences outside of controlled research environments (Stone et al., 2007). In addition, these methods allow for the temporal resolution that allows for within-subject analyses of behavior change over time (Stone et al., 2007). However, it is possible that individuals may change their behavior in reaction to the research methods (referred to as reactivity bias) and may over-report healthful behaviors and under-report health risk behaviors.

Future research would benefit from utilization of this highly innovative method among MSM populations to overcome limitations of traditional survey-based research (e.g. recall bias) in assessments of recent sexual risk behaviors and tobacco, alcohol, and other drug use. Beyond addressing traditional limitations of research, EMA methods offer new avenues for engagement and inquiry for collecting data using mobile technologies real-time and tailoring behavioral health interventions to specific spatio-temporal contexts, including for young MSM populations.

CONCLUSION

Findings from this pilot feasibility study suggests that different EMA methods are acceptable for use among young MSM and more specifically, the lack of significant variations in EMA acceptability across sub-groups in this sample of MSM suggests that both text message- and phone- based methods can be implemented in diverse samples of young MSM.