A systematic review of recent methodological approaches for using ecological momentary assessment to examine outcomes in U.S. based HIV research

Abstract

Purpose of review:

In recent years, researchers have been adopting and using ecological momentary assessment (EMA) methods via technology devices for real-time measurement of exposures and outcomes in HIV research. To assess and critically evaluate how EMA methods are currently being used in HIV research, we systematically reviewed recent published literature (October 2017- October 2019) and searched select conference databases for 2018 and 2019.

Recent findings:

Our searches identified 8 published articles that used EMA via smartphone app, a handheld Personal Digital Assistant, and web-based survey programs for real-time measurement of HIV-related exposures and outcomes in behavioral research. Overall trends include use of EMA and technology devices to address substance use, HIV primary prevention (e.g., condom use and pre-exposure prophylaxis) and HIV treatment (medication adherence).

Summary:

This review supports the use of EMA methods in HIV research and recommends that researchers use EMA methods to measure psychosocial factors and social contexts and with Black and Latinx samples of gay and bisexual men, transgender women, and cisgendered women to reflect current HIV disparities in the US.

Introduction

Despite advances in HIV prevention and treatment, HIV persists and continues to disproportionately affect key populations (e.g., Black and Latinx gay and bisexual men, and Black cisgender women) in the United States (U.S.) [1]. The national call for “Ending the HIV Epidemic: A Plan for America” (EtHE) [2**] specifically targets populations such as Black and Latinx gay and bisexual men who have not been adequately reached in previous HIV prevention and treatment efforts in the U.S. which have successfully reduced the epidemic in populations such as gay and bisexual White men. This manuscript focuses on research in the U.S. as an initial step towards addressing this call.

A number of emerging research methodologies use technology devices to assess the context and timing of HIV risk and treatment behaviors. While much HIV-related behavioral research relies on participant recall, greater attention is being placed on capturing HIV-related behaviors as they occur in real-time and in real-word settings. [3–5]. Ecological momentary assessment (EMA) is a real-time data capture methodology that integrates the ubiquitous nature of technology (e.g., the availability of mobile phones and other wearable health tracking devices) with daily diaries. EMA is a unique method that may be used to capture nuances in how environmental contexts and psychological states affect HIV risk and treatment engagement. EMA also improves data accuracy and captures information on the proximal and dynamic determinants of HIV-related behaviors. HIV researchers are increasingly turning to EMA as an innovative counterpart to retrospective self-report data and as a way to engage key populations in HIV prevention and treatment [6–9*].

EMA uses repeated sampling of an individual’s episodic behavior (e.g., psychological state, sex events, and/or substance use) in real-time, longitudinally, and in various daily life settings outside of a controlled research laboratory [8*, 10]. EMA uses a blend of experience sampling, event-contingent responding, and daily diary methodologies to assess experiences and behaviors [10, 11]. Experience sampling methods investigate study participants’ subjective experiences and behaviors as they occur in real-time [12]. These methods prompt study participants to complete a survey at random times throughout the waking hours of each day [12]. Event contingent responding methods ask study participants to initiate a report each time they engage in the behaviors of interest (e.g., alcohol consumption, sex-event, and cigarette smoking) [7, 8, 10]. Lastly, daily diary methods consist of study participants’ written accounts of the behaviors of interest. Data collected using EMA methods have several advantages over methods that rely on retrospective self-report: 1) data are recorded in real time, minimizing recall bias; 2) data are time-tagged, maintaining the sequence and timing of events; 3) fluid and complex sampling schemes are possible; 4) participant burden is eased by skip patterns that permit individuals to opt-out of data collection while still providing researchers with multiple assessments over time; and 5) increases the ability to engage marginalized populations to capture highly stigmatized and/or illegal behaviors and activities [10, 13]. The consistency, depth, and accuracy of data captured using EMA has the potential to strengthen HIV-related behavioral research.

The data collection instruments used in EMA continue to evolve in parallel with advances in handheld technology. Most EMA data are now collected using handheld electronic devices such as smartphones, tablets, and wearable sensors [10, 14, 15]. More specifically, the proliferation of smartphones has encouraged the development of apps that facilitate EMA for data collection and intervention delivery. The ease to which EMA may be integrated into the everyday lives of individuals is a strength of this method that makes it particularly suited for HIV research. HIV-related behaviors are dynamic and are often influenced by a number of contextual and social factors. Using technology to integrate EMA into studies that use more traditional assessment methods (e.g., paper-pencil surveys) provides researchers with more opportunities to capture and examine these dynamic processes.

To date, reviews of the use of EMA to assess HIV risk, prevention, and care are limited. The primary aim of this review of recent literature is to characterize the use of EMA with technology devices to capture real-time measurements of exposures and outcomes in HIV prevention and treatment research. The secondary aim is to identify gaps in the reviewed literature and propose areas for future research.

Methods

This review was designed by a medical librarian (KN) in consultation with several domain experts (SLS, NM, and TT). We searched three databases of scholarly literature: MEDLINE ALL, PsycINFO, and CINAHL. The search used both text word searches and controlled vocabulary for two key concepts: ecological momentary assessment and the HIV prevention continuum. We excluded articles with subject indexing that indicated non-human or animal research as well as articles published before the start date (October 2017) of this review of the recent literature. The search was peer reviewed by an independent medical librarian for completeness and accuracy. We also hand-searched the American Public Health Association (APHA), Conference on Retroviruses and Opportunistic Infections (CROI), and Youth+ Tech+ Health (YTH) conference databases for conference years 2018 and 2019.

Articles retrieved from the bibliographic and conference databases were deduplicated in Covidence, a systematic review data management platform. We also used Covidence for title, abstract, and full-text article screening. We included publications in which the primary focus was on using EMA to investigate HIV exposures and outcomes related to HIV prevention and treatment (i.e., condom use, substance use, and medication adherence (including pre-exposure prophylaxis (PrEP) and antiretroviral therapy (ART)). We excluded publications that: 1) did not include a clear EMA approach, such as experience sampling, event-contingent responding, and daily diary methodology [10, 11], 2) did not include an HIV-specific exposure or outcome (e.g., publications that focused exclusively on other sexually transmitted infections or behavioral risks that were not HIV-specific), and 3) research that was conducted outside of the U.S. After removing duplicate publications, the remaining titles, abstracts, and full-text articles were evaluated for inclusion by two independent reviewers (SLS and TT). The interrater reliability between the reviewers was 0.95, indicating strong agreement. Discrepancies during full-text review were discussed with a third reviewer (NM) until a consensus was reached.

Data were abstracted from full-text articles using a set of 17 defined fields related to the design, sample size, sample characteristics, and location of the study; EMA type, mode, design, measures, and frequency; HIV exposures and outcomes; and key findings and implications. The first and senior authors (SLS and TT respectively) independently extracted data from each article and reviewed all extracted data for accuracy and completeness.

Results

In total, 8 studies met the selection criteria (Figure 1). The following sections summarize how EMA using technology-driven approaches were applied to the study populations along with methodological details (Table 1).

Figure 1: PRISMA Flow Diagram of Study Selection.

From: Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group (2009). Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med 6(7): e1000097. Doi: 10.1371/journal.pmed100097

Table 1:

Summary of recent EMA studies in U.S. based HIV prevention and medication adherence research (n=8)

Author, Publication year	Study purpose	Population, Location	Sample size, Age in years	EMA device type	EMA frequency, Duration, Alarm interval	EMA measures	EMA completion	HIV-related outcomes
Cook et al., 2018	To examine whether momentary motivation was a mechanism by which other everyday experiences affected antiretroviral therapy (ART) adherence in PLWH.	PLWH (43% White) Denver, CO	n= 87 Median age (40.0, SD=8.84)	Study-issued smartphone pre-loaded with Apptive® scheduling software Link to online survey Study-issued Medication Event Monitoring System (MEMS) Cap™ pill bottle	Once daily 10 weeks Random, no time specified	Control beliefs, mood, stress, coping, social support, and motivation for ART MEMS bottle	Not specified	ART adherence
Fazeli et al., 2019	To compare using a standard questionnaire and experience sampling method (ESM) to measure HIV-related stigma.	PLWH-men only (54% Black) Birmingham, AL	n= 109 Median age (41.36; SD=10.94)	Study-issued smartphone containing the random prompt survey link	3 times daily 7 consecutive days Random (10am-8pm)	Enacted stigma, internalized HIV stigma	74% of ESM random prompts were responded to	ART adherence HIV visit adherence/Engagement in care
Reback et al., 2018	Pilot study to evaluate the use of smartphone EMA for self-monitoring to optimize treatment outcomes among gay and bisexual men enrolled in an outpatient methamphetamine abuse treatment service program.	Gay and bisexual men (35% White) Los Angeles, CA	n= 34 Median age (40.6; SD=9.3)	Participant or study-issued smartphone with EMA app	5 intraday randomly-timed surveys 8 weeks Random (9am-12am)	Sexual risk behaviors, substance use and cravings, and environmental triggers	Not specified	Sexual risk behaviors Drug (methamphetamine) and alcohol use
Santa Maria et al., 2018	To use EMA data to examine realtime factors such as stress, urge, and substance use to determine the predictors of sexual activity among homeless youth.	Homeless youth (65% Black) Houston, TX	n= 66 Ages 18–24 (M=21.2)	Study-issued mobile phones	5 intraday randomly-timed surveys Once daily diary assessment 21 days Random assessments were scheduled to occur randomly in 4 epochs at waking Daily diary assessments were prompted once every 30 minutes at waking	Sexual risk behaviors, cognitions, stress, affect, environmental factors, and environmental circumstances	93% participation rate; Mean number of EMAs provided by each participant was 62% of daily EMAs and 40% of random EMAs	Sexual risk behaviors Drug use (injection drug use)
Shacham et al., 2019	To test the feasibility and acceptability of using technology to examine intraday variation in alcohol use, mood, and daily antiretroviral therapy (ART) adherence.	PLWH (74% Black) St. Louis, MO	n=34 ≥21 years of age	Study-issued smartphone with an app that was developed to measure mood and alcohol use Participants were given a MEMS Cap™ pill bottle	Once daily self-initiated 3 intraday randomly-timed surveys and event-specific surveys 28 days Morning report (8am-12pm) Randomly-timed surveys (12pm-8pm)	Alcohol use, affect and mood, and medication adherence	Participants completed 69% of the morning reports, 92% of the initial drinking survey, 66% of the drinking follow-up survey, and 57% of the three random daily surveys	ART adherence Alcohol use
Simons et al., 2019	To assess correlations between EMA assessed sexual arousal and sexual activity. To determine the effect EMA/daily self-monitoring has on condomless anal sex (CAS).	MSM (65% White) Two cities in the northeastern U.S.	n=165 Ages 21–50 (M=27.92, SD=7.31)	MetricWire (mobile EMA app) was installed on the participant’s smartphone or study-issued smartphone with MetricWire	Once daily self-initiated assessment 9 random prompt assessments 6 weeks, two 3-week measurement bursts with 3-weeks off Self-initiated (10am); Random (10am-2am)	Sexual activity, condom use, and sexual arousal	Participants completed 90% of the self-initiated morning assessments and 63% of the random prompt assessments	Sexual risk behaviors
Simons et al., 2018	To test associations between alcohol and sexual health outcomes using an EMA protocol.	Undergraduates (94% White) U.S. based	n=213 Ages 18–27 (M=19.86, SD=1.36)	Study-issued PDA with PMAT software	Once daily self-initiated assessment Random prompt assessments 6 measurement bursts, 49 days Self-initiated (completed at waking); Random (10am-12am)	Sexual risk behaviors, sexual partner characteristics, and alcohol use	Participants completed 95% of the self-initiated morning assessments and 79% of the random prompt assessments	Sexual risk behaviors Alcohol use
Wray et al., 2019	To use EMA methods to study high-risk condomless anal sex (CAS) events (i.e., those with non-exclusive or unknown HIV status partners) among HIV-negative MSM who are not on Pre-exposure prophylaxis (PrEP).	MSM (78% White) U.S. based	n= 80 Ages 18–53 (M=27.1, SD=7.8)	MetricWire (mobile EMA app) was installed on the participant’s smartphone	Once daily self-initiated assessment Up to 6 random prompt assessments 30 days Self-initiated (completed at waking); Random (9am-12am)	Sexual partner characteristics--are they known, HIV or PrEP status, duration of relationship, where they met, motivations for having sex, condom use at last sexual act, and desires for anal sex	Participants completed 95% of the self-initiated morning assessments and 79% of the random prompt assessments	Sexual risk behaviors Drug and alcohol use

Participant characteristics and study location

Various key populations were included in this review: young adults [16], including homeless young adults [17], HIV-negative or unknown status gay and bisexual men [18–20], and people living with HIV [21–23]. One study [23] collected EMA data with only HIV-positive men. In three [17, 22, 23] of the eight studies, participants who identified themselves as Black constituted a majority of the sample. All eight studies [16–23] included in the review were conducted in U.S. cities, including Los Angeles, California; Houston, Texas; and St. Louis, Missouri.

Momentary measurements and methodological details

Along the care continuum, five studies [16–20] targeted preventing primary acquisition and three [21–23] targeted supporting adherence to ART. For example, Simons et al [16] assessed alcohol intoxication and sexual behavior. Participants were provided Palm handheld devices which prompted participants to complete a brief (about two minutes) assessment at random times within two hour blocks between 10:00 am and midnight. Participants completed a self-initiated assessment each morning shortly after waking, which included assessments of sexual behavior not likely to be captured by the random prompts [16].

Several studies assessed psychosocial and personality factors and their association with HIV-related behaviors. Three [17–19] measured mood or affect; two [17, 21] measured mood or affect along with stress; and one [22] measured mood and alcohol use. Three studies [17–19] out of eight measured environmental factors along with mood, affect, or stress; and one [23] assessed enacted stigma and internalized stigma. For example, Reback et al [18] implemented an intervention to capture changes in methamphetamine use, cravings, sexual risk behaviors, external triggers, and internal triggers/affect over time. Smartphones with an EMA app, either the participant’s smartphone or one provided by the study, measured sexual risk behaviors, substance use, cravings and environmental triggers five times daily over an eight week period [18].

Three EMA studies [21–23] included people living with HIV (PLWH) to assess determinants of HIV medication adherence. Guided by a momentary motivation model, Cook et al’s study [21] used a study provided smartphone and MEMSCap pill bottles to measure control beliefs, momentary mood, stress, coping, and social support over 10 weeks to examine whether momentary motivation is a mechanism by which daily experiences affect adherence to ART. This study [21] used items from the Diary of Ambulatory Behavioral States including three items for mood (Cronbach alpha = .93) and six items for stress (Cronbach alpha = .67). Fazeli and Turan [23] conducted a study of HIV-positive men, using study-issued smartphones, evaluated (a) associations between questionnaire and experience sampling methods to measure internalized and enacted stigma, (b) psychosocial predictors (e.g., coping style, perceived HIV community stigma, helplessness) of discrepancies between questionnaire and experience sampling to assess internalized and enacted stigma, and (c) whether questionnaire or an experience sampling method better predicted HIV outcomes. The experience sampling method used in this study consisted of adapted items adapted items assessing internalized stigma and enacted stigma from validated measures, however the study investigators did not specify which measures were used. In a study [17] of homeless youth, a standardized measure for the 4-item Perceived Stress Scale was used to measure momentary stress with a study provided smartphone.

Only one study [22] reported on the feasibility and acceptability of using a mobile EMA app to measure alcohol consumption and mood patterns among a sample of PLWH while managing their medication adherence. Shacham et al [22] found that EMA via mobile technology was feasible among this population with 85% of enrolled participants completing the 28-day study protocol. Analyses indicated that the average medication adherence for the sample was 94.1%.

EMA device types ranged from study-issued smartphones [17, 18, 20–23] to personal smartphones [18–20]. Out of the eight studies, four used specific EMA apps (e.g., MetricWire) directly installed onto smartphones [18–20, 22]; and two studies [20, 23] used a Web-based online survey program hyperlinked from participants smartphones. For instance, Cook et al [21] used the Android operating system which was pre-loaded with Apptive^® scheduling software. Simons et al’s study [16] issued Palm handheld devices programmed with Purdue Momentary Assessment Tool (PMAT), an EMA software package developed specifically for EMA researchers.

Six studies [16–19, 21, 23] reported conducting a briefing or intake session to ensure that participants understood the EMA app before starting the survey. During the session, informed consent and baseline measurements were collected. Participants were also able to ask questions regarding the app, practice using the EMA app, and familiarize themselves with the data collection protocol.

All eight studies [16–23] used signal contingency to prompt EMA measurement. Four studies [16, 19, 20, 22] used both signal-based contingency and event-based self-report. Frequency of the signal varied from once per day [3] to nine times per day [20], and the study durations ranged from seven days [23] to 10 weeks, or 70 days [21].

Study completion rates ranged from 95% [16] to 57% [22] excluding studies with no reported completion rates. To maximize compliance with the EMA protocol, Cook et al [21] allowed participants to keep their study-issued smartphones at the end of the study if they completed at least one month of surveys over 10 weeks. A study [17] of homeless young adults incentivized participants up to $95 in gift cards for returning the study-issued smartphone at the final visit. Participants who completed 49.5% (52/105) to 75.2% (79/105) of EMAs received a $50 gift card, those who completed 76.2% (80/105) to 88.6% (93/105) received a $75 gift card, and those who completed 89.5% (94/105) or EMAs received a $95 gift card. The mean number of EMAs completed by each participant was 45 out of 105 possible observations.

Discussion

The advantages of EMA are numerous and include addressing limitations of retrospective self-report, capturing how behaviors and experiences are affected by social contexts, ensuring that data are representative of an individual’s lived experiences, and providing a mechanism for tailoring interventions to individual needs and contexts [10, 24*, 25*]. Moreover, advances in technology—including smartphone applications, health monitoring devices, and global positioning system (GPS) technologies—have decreased the risk of participant burden related to EMA and increased the research and intervention potential for this method.

The use of EMA in behavioral research (e.g., cigarette smoking, smoking cessation, physical activity, and diet) continues to grow. Findings from previous behavioral research studies have the potential to inform and strengthen the application of EMA in HIV research [26, 27]. Evidence confirms that EMA is well suited to investigations that seek to assess individual differences in dynamic behaviors (e.g., substance use or sexual risks) over a given time period, capture the temporal sequence or the antecedents and consequents of a behavior, and assess the social and environmental contexts in which a behavior occurs. Moreover, EMA approaches used in non-HIV related behavioral research may provide guidance to HIV researchers on: sampling strategies (e.g., time-based, interval contingent, event-based, random assessment, or some combination of these strategies); prompt and assessment device needs (e.g., two electronic devices—one to trigger a participant to complete an assessment and another device that administers the assessment, one electronic device that both prompts and assesses, or a combination of an electronic device with a paper-and-pencil diary); and strategies to increase response compliance [27, 28]. There are a number of differences that should be considered when drawing from EMA studies of the aforementioned behaviors to HIV-related behaviors. These factors include issues related to stigma, criminalization, engaging participants who may be under the influence drugs and/or alcohol, and frequency of the behavior. Nevertheless, there remains a number of opportunities for the HIV field to adapt and build upon existing EMA strategies.

We identified a number of studies that focused on gay, bisexual, and other MSM. This finding is consistent with other reviews and reflective of the epidemiology of the HIV epidemic in the U.S. [29, 30]. Surprisingly, most studies were conducted in samples that did not include a majority of racial/ethnic minority MSM. Given existing trends and patterns in HIV incidence and prevalence, it would be important for future research to focus on this population. We also identified gaps in the recent literature in the inclusion of racial/ethnic minority women and older PLWH. Both groups present unique challenges and barriers that are well-suited for EMA studies. For example, a study conducted by Moore and colleagues [31*] used EMA to assess daily functioning among older PLWH. Their findings showed that this group had excellent adherence to the EMA protocol, positive experiences with smartphone-based EMA methods, and that EMA data on mood and cognitions were associated with data from laboratory-based assessments [31*].

Integrating EMA with other existing technologies may be an effective strategy for developing HIV prevention interventions. Studies that combine EMA with GPS through a smartphone application have the potential to leverage the geolocation capabilities of a smartphone with the data driven strengths of EMA [32**, 33**]. Geographically-explicit ecological momentary assessment (GEMA) has the potential to advance the field substantially by integrating EMA measurement with both spatial and temporal data on HIV-related behaviors within an individual’s context. GEMA provides more nuanced information on the associations between spatial mobility and HIV-related behaviors. Additionally, research using GEMA may support the development of just-in-time adaptive interventions (JITAI) to address negative health behaviors. JITAI is a concept used to describe interventions in which intervention content is sent to an individual “just in time” to reduce risk behaviors. JITAI also adapts an intervention to the individual as their needs and supports for the particular behavior change. JITAI are still quite new in the HIV field, but promising studies in alcohol use and smoking cessation suggests that this intervention approach may have a significant effect on HIV outcomes and disparities [34–37].

There are a number of ethical issues that should be considered to ensure that human subjects’ rights, such as privacy and anonymity, are not compromised for study participants as HIV researchers continue to explore the use of EMA in research. Participant burden is one issue that will require future investigations to ensure that the inherent time commitment and inconvenience of EMA does not compromise participants’ well-being [38]. Privacy concerns are another important consideration when integrating EMA approaches in HIV-related behavioral research. Asking individuals to record their behaviors in great detail is more invasive than a retrospective survey and requires a level of trust that may be challenging for researchers to establish with those most vulnerable to HIV [27, 38]. Data collected using electronic EMA devices also raise concerns about confidentiality, data security, and other factors associated with using technology to wirelessly transmit sensitive data which may decrease participant anonymity. Moreover, as technology continues to advance EMA options, other ethical issues should be considered. For example, if an individual is enrolled in a GEMA study and reveals that they intend to harm themselves or others, are researchers required to notify law enforcement, should law enforcement or other emergency personnel be automatically alerted by the device, and should participants know about such precautions prior to enrolling in the study [38, 39]. Investigators will need to carefully examine these concerns in light of the number of strengths to using EMA in HIV-related behavioral research.

We identified a number of research gaps in the recent literature that should be addressed in future studies. Specifically, much of the research identified in this review assessed sexual risk behaviors, substance use, and ART adherence. There remains a vast potential for further development for how EMA may be utilized to examine other HIV outcomes like testing and linkage to care. For example, given the number of studies we identified that use EMA to measure ART adherence, similar strategies (e.g., MEMSCap data capture; assessing mood, affect, and timing of medication; and integrating telemedicine to support provider visit adherence) could be used to assess PrEP initiation and adherence. Lastly, our review found one study [22] which reported on the feasibility of using EMA to examine HIV-related exposures and outcomes.

In conclusion, this study provides researchers with informative evidence regarding methodological details for using EMA via technology devices, for real-time measurement of exposures and outcomes in HIV research. Future studies are needed to better understand the use of EMA as an intervention tool. That is, how does daily self-monitoring using EMA change participant behavior and how can these changes be leveraged to address HIV outcomes [18]. Lastly, when considering the use of EMA in HIV-related behavioral research, future research needs to focus on: (1) measuring psychosocial factors and social contexts, including in rural areas, and with Black and Latinx samples of gay and bisexual men, transgender women, and cisgendered women to reflect current HIV disparities in the U.S.; (2) examining the ethical issues related to asking individuals to identify when and where they engage in stigmatizing, and perhaps illegal behaviors; and (3) providing more information on the implementation of EMA for exposures and outcomes in HIV in study designs.