Combining ecological momentary assessment with objective, ambulatory measures of behavior and physiology in substance-use research

Abstract

Whereas substance-use researchers have long combined self-report with objective measures of behavior and physiology inside the laboratory, developments in mobile/wearable electronic technology are increasingly allowing for the collection of both subjective and objective information in participants’ daily lives. For self-report, ecological momentary assessment (EMA), as implemented on contemporary smartphones or personal digital assistants, can provide researchers with near-real-time information on participants’ behavior and mood in their natural environments. Data from portable/wearable electronic sensors measuring participants’ internal and external environments can be combined with EMA (e.g., by timestamps recorded on questionnaires) to provide objective information useful in determining the momentary context of behavior and mood and/or validating participants’ self-reports. Here, we review three objective ambulatory monitoring techniques that have have been combined with EMA, with a focus on detecting drug use and/or measuring the behavioral or physiological correlates of mental events (i.e., emotions, cognitions): (1) collection and processing of biological samples in the field to measure drug use or participants’ physiological activity (e.g, hypothalamic-pituitary-adrenal axis activity); (2) global positioning system (GPS) location information to link environmental characteristics (disorder/disadvantage, retail drug outlets) to drug use and affect; (3) ambulatory electronic physiological monitoring (e.g., electrocardiography) to detect drug use and mental events, as advances in machine learning algorithms make it possible to distinguish target changes from confounds (e.g., physical activity). Finally, we consider several other mobile/wearable technologies that hold promise to be combined with EMA, as well as potential challenges faced by researchers working with multiple mobile/wearable technologies simultaneously in the field.

1. Introduction

Technological advances are opening new frontiers in ecological momentary assessment (EMA). Paper diaries and questionnaires have given way to electronic versions delivered on smartphones that can timestamp and wirelessly upload entries. Mobile/wearable technology has also expanded the capacity to collect concurrently with real-time self-report a broad range of other types of data, such as biological samples, location, and physiological changes. This technology is enabling researchers to study substance use “in the moment,” monitoring both the individual and the environment to better understand its causes and consequences.

In this paper we review studies of substance use that combine EMA with objective measurements in the field. It has long been common in laboratory studies to combine self-report with objective measures; doing so in daily life is an important step forward. Objective measures can help confirm EMA entries, but they also provide unique insight into the spatiotemporal organization of mood and behavior and allow for novel tests of longstanding theories (e.g., about environmental influences on mood and drug use).

As the field develops, systematic reviews and meta-analyses will be needed to assess specific hypotheses. In this review, our aims are simply to introduce investigators to available techniques and to help mobile/wearable device developers combine their work with EMA.

2. Field collection/processing of biological samples

2.1 General considerations

Table 1 presents a summary of studies combining EMA with the field collection of biological samples in studies of substance use. Field collection of blood and urine, the biological matrices most commonly used in studies of substance users, presents challenges in terms of safety and participant acceptability. More progress has been made with breath, perspiration, and oral fluid/saliva. Other samples could be collected in the field (e.g., hair and nails, Krumbiegel et al. 2016); the time-frames of the information obtained from these may be less appropriate for matching with EMA reports, but they may be appropriate for characterizing longer-term patterns (G.A.A. Cooper et al. 2012; Short et al. 2016).

Table 1.

Field studies of substance use combining EMA with the field collection of participants’ biological samples

Reference	Substance(s)	Measure added to EMA	Participants	Monitoring duration	EMA device; other device(s)	Compliance/Feasibility	Notes
Garrison et al. 2015	tobacco	breath carbon monoxide (CO)	140 adult smokers with motivation to quit	22 days	Apple iPhone or Android smartphone ; piCO+ Smokerlyzer	N/A	protocol for planned/ongoing study; CO monitoring for verifying abstinence after 6 months sweat collected in the field for subsequent laboratory-based analysis
Linas et al. 2016	opioids, cocaine	sweat collection	109 adults with recent heroin or cocaine use	30 days	Palm Z22 PDA or Motorola Droid X2 smartphone ; PharmChek sweat patch	evaluable EMA data on 97% of participant-weeks; 91% of sweat patches returned
Simons et al. 2015	alcohol	transdermal alcohol	60 young adults (aged 18–25 years) with at least moderate drinking	3 × 1–2-week “bursts,” 1 per academic semester	palmtop computer (model N/A); Giner WrisTAS 7 alcohol sensor	82% EMA response rate; WrisTAS worn with no sensor failure on ~72% of days

Field monitoring may be particularly important for drugs with short windows of detectability or for drug-using situations that impede accurate self-reporting (e.g., Simons et al. 2015; Luczak & Rosen 2014). The latter include drug mixtures (e.g., alcoholic cocktails, many street-purchased drugs), especially those prepared by another person, as well as communal sources (e.g., a shared pipe). Depending on the technique, biological monitoring can reduce burden and maintain the naturalism of the use experience : it need not interrupt the normal “flow” of behavior as answering an EMA questionnaire does. Field detection of drug use may also be important for monitoring adherence to pharmacotherapies. Finally, studies of substance use may benefit from field monitoring of endogenous substances, including salivary cortisol as an indicator of hypothalamic-pituitary-adrenal (HPA) axis activity, as well as salivary alpha amylase and salivary flow rate as indicators of autonomic activity (for their combination with EMA in other populations, see e.g., Skoluda et al. 2016; Strahler & Nater 2017; Van Lenten & Doane 2016).

In choosing biological matrices and analyte(s), researchers should consider how they will verify the source, timing, and integrity of the sample, as well as whether sample collection will be time-based and/or event-based (Kudielka et al. 2012). It is also necessary to distinguish between techniques that collect samples in the field for later processing in the laboratory versus live processing. Several types of field collection of biological samples (without field processing) relevant to mHealth studies of substance use have been performed: collection of liquid perspiration to detect opioid and cocaine use in combination with EMA (Linas et al. 2016) or to detect alcohol without EMA (Phillips & McAloon 1980, but see also Phillips et al. 1984); collection of saliva/oral fluid to detect smoking in studies of mobile interventions (Abroms et al. 2014, Free et al. 2011); and collection of saliva/oral fluid for cortisol measurement in smokers and other substance users (al’Absi et al. 2004, 2007; Direk et al. 2011; Lovallo et al. 2000; Sorocco et al. 2006; Steptoe & Ussher 2006; see also Bauer et al. 2011). Although not yet performed with substance users specifically (see al’Absi et al. 2004, 2007 for paper questionnaires completed in the field), EMA has been successfully combined with the field collection of salivary cortisol in other populations (e.g., Damaske et al. 2016; Entringer et al. 2011; Giesbrecht et al. 2012; Huffziger et al. 2013; Kalpakjian et al. 2009; Skoluda et al. 2016; Strahler & Nater 2017; Van Lenten & Doane 2016). Below, we will focus on methods in which both sample collection and sample processing occur in the field.

2.2 Transdermal alcohol detection

Alcohol can be detected transdermally by electrochemical fuel cells that respond to excreted alcohol vapor in the air adjacent to the skin; monitoring devices seal with a gasket against the wrist or ankle and include additional sensors (e.g., skin temperature) to detect tampering/removal (reviewed by Leffingwell et al. 2013). Transdermal alcohol sensors can provide near-continuous monitoring, and at least one device and its associated website interface (SCRAM Systems web services, Alcohol Monitoring Systems, Inc., Littleton, CO; Leffingwell et al. 2013) offer near real-time access to results to researchers and, depending on the study design, participants.

To our knowledge, only one study has combined transdermal alcohol sensing with EMA (Simons et al. 2015), although others have combined it with paper diaries (Bond et al. 2014) or daily website-based questionnaires (Barnett et al. 2011, 2014, 2017) or e-mails (Marques & McKnight 2009). Simons and colleagues (2015) used transdermal sensors to capture the intra-day dynamics of drinking. Each of several types of self-report was significantly related to daily peak sensor values, although EMA random prompting appeared more likely to miss drinking episodes, and in exploratory analyses, dynamic aspects of the sensor data seemed to correspond to different styles of drinking (e.g., different speed/amount combinations). Agreement between transdermal sensing and self-report was also reported by Barnett and colleagues (2011, 2017) in the context of contingency management (see also Alessi et al. 2017; Dougherty et al. 2014, 2015). In their EMA study, Simons and colleagues (2015) showed that the sensor data and random-prompt self-reports each accounted for unique variance in dependence-symptom measures obtained at study baseline, illustrating another potential gain from combining methods.

2.3 Detection of alcohol by breath

Quantification of alcohol exposure by breath began with the police “drunkometer” of the 1930s (Holcomb 1938). It is a mainstay of assessment in research and forensic settings and, with Bluetooth-enabled portable breathalyzers and their associated smartphone apps, is available to consumers. Mass-marketed “smart” breatalyzers have been covered in the popular press in terms of self-monitoring (e.g., fitness to drive after a party; Cipriani 2014); however, to our knowledge, they have not been included in published research on alcohol use, with or without EMA. The commercial market for these devices also seems to have been in flux, as several models reviewed in a popular-press roundup published in late 2014 (Cipriani, 2014) now appear to have limited or no retail availability, and one manufacturer, Breathometer, Inc. (Burlingame, CA), reached a settlement with the Federal Trade Commission announced in January 2017 over unsupported accuracy claims (Ohlhausen 2017). Several other devices have been marketed with a criminal justice focus (e.g., the SL2 and Sobrietor models from BI Incorporated, Boulder, CO, USA; SCRAM Remote Breath model from Alcohol Monitoring Systems, Inc., Littleton, CO). These devices are mentioned without endorsement or criticism: to our knowledge, they have also not been used in published research. However, their manufacturers’ descriptions mention built-in compliance features that could be useful to researchers, including automated identification of the sample provider by facial or voice recognition.

Until commercial- grade, smartphone-paired breathalyzers and apps are more fully validated, researchers may prefer approaches that conform to standards of the US Department of Transportation or other external regulations. Standard breathalyzers can be deployed in combination with smartphones to document the time and authenticity of sample collection. This approach was taken with good feasibility and acceptability by Alessi and Petry (2013) in a study of contingency management. In response to a text message, non-treatment-seeking drinkers used the phone camera to take time-stamped videos of themselves using the breathalyzer. Videos were then sent using the phone’s text/video messaging service to the researchers, who replied with a text informing participants of their earnings. This study did not include EMA but is notable, like the smoking-cessation treatments reviewed below (Section 2.4), in showing how validated breath samples can be collected in the field and used for treatment.

2.4 Detection of smoking by breath

Like ethanol in breath, the concentration of carbon monoxide (CO) in breath falls relatively rapidly after smoking (Sandberg et al. 2011), making it a good candidate for field monitoring, either with “smart” devices or with standard monitors plus photo/video documentation. Meredith and colleagues (2014) have reported the development of a smartphone-paired CO detector that compared well with a commercial CO meter in laboratory tests, although such devices have not, to our knowledge, been studied in combination with EMA. In their published protocol, Garrison and colleagues (2015) include CO monitoring in a mobile intervention study (smartphone-based mindfulness training) with EMA measures of mood and craving: CO monitors will be mailed to participants, used during a video chat with researchers, then mailed back. CO testing with video verification has also been used in studies of several types of contingency management for smoking cessation (e.g., Dallery et al. 2007, 2013, 2015, 2017; Jarvis & Dallery 2017; Reynolds et al. 2015; Stoops et al. 2009; Sweitzer et al. 2016; see also Hertzberg et al. 2013; Hicks et al. 2017).

2.5 Conclusions and future directions for biological samples

Although drug use may be detectable by more indirect methods (discussed in Section 4), testing for drug/metabolite molecules in biological samples is likely to remain critical in substance use research. Major considerations continue to be low participant burden for collection and validation, as well as quick availability of results. With that, methods for collecting and processing samples in the field are developing rapidly. Oral fluid testing may become more prominent in field studies as methodological developments allow for detection of multiple drugs from single samples with simplified preparation and processing (Allen 2011; Øiestad et al. 2016; Scherer et al. 2017; Wiencek et al. 2017; Wille et al. 2014). Furthermore, future developments in electrochemical detection of drugs (e.g., Chen & Lu 2016, Huang et al. 2013) and cortisol (e.g., Singh et al. 2014) could make field processing practicable where laboratory processing is now required.

3. Global Positioning System (GPS) location information

3.1 General considerations

Table 2 presents a summary of studies combining EMA with GPS location information in field studies of substance use. In studies combining EMA and GPS, EMA provides information about participants’ momentary experiences, whereas GPS provides objective information about participants’ locations during those experiences, linked by electronic timestamps (e.g., Epstein et al. 2014; Kirchner & Shiffman 2016; Watkins et al. 2014). GPS location information can, in turn, be linked to information about the built, natural, and/or social environment in those places—as the value of knowing where participants are is enhanced by knowing what else is there. For drug users, environmental features may affect both the likelihood of use and the likelihood of negative consequences from use (e.g., Freisthler et al. 2014). Environmental information can be obtained from public records (e.g., census or tax information), commercial sources (e.g., streetscape photographs from commercial mapping applications), academic research (e.g., Cantrell et al. 2015; Furr-Holden et al. 2008), and crowdsourcing (Reichert et al. 2016).

Table 2.

Field studies of substance use combining EMA with objective measurement of participants’ location

Reference	Substance(s)	Measure added to EMA	Participants	Monitoring duration	EMA device; other device(s)	Compliance/Feasibility	Notes
Byrnes et al. 2017	alcohol	GPS	170 adolescents (aged 14–16 years)	1 month	Apple iPhone 5c smartphone with ActSoft Comet Tracker software	mean 68.4% EMA response rate; GPS details N/A
Epstein et al. 2014	opioids, cocaine	GPS	27 opioid-dependent adults receiving methadone maintenance	16 weeks	PalmPilot PDA; Qstarz BT-Q1000X GPS logger	mean 79.0% EMA response rate; GPS data available for 85.9% of completed EMA entries
Kirchner et al. 2013	tobacco	GPS	475 adult smokers making a quit attempt	Up to 1 month	Blackberry 9330 smartphone	79% EMA response rate; GPS tracking “successful” in 475/486 individuals consenting
Mennis et al. 2016	alcohol, tobacco, illicit drugs	GPS	139 adolescents (aged 13–14 years)	6 × 4-day periods, completed every other month for 1 year	“mobile phone with embedded GPS” (model N/A)	50% EMA response rate; GPS data available for 41% of not-at-home EMA entries	EMA to collect subjective responses to environment; drug use assessed by baseline questionnaire
Mitchell et al. 2014	tobacco	GPS	10 adult smokers with attention deficit hyperactivity disorder	7 days	Palm Treo 755P palmtop computer; Transystem i-Blue 747A+ GPS logger	EMA details N/A; 70% of participants met a priori GPS data quantity cutoff
Sarker et al. 2016	opioids, poly-drug use	ECG and respiration (controlling for physical activity), GPS	38 opioid-dependent poly-drug users receiving opioid agonist maintenance	4 weeks	smartphone (EMA, model N/A); wireless physiological sensor suite (ECG, respiration, accelerometry) + Android-based smartphone (model N/A)	88.0% EMA answer rate; 10.54 hours acceptable ECG data from 12.52 hours sensors wear per day; ~70% of physiologic al data available after excluding physical activity; GPS details N/A	stress inferred from physiological monitoring using the model of Hovsepian et al. 2015; GPS then used to assess environment al features associated with stress inference
Watkins et al. 2014	tobacco	GPS	47 adult smokers making a quit attempt (+ 55 additional parent-study participants with no GPS data)	1 week	LG Optimus P509 smartphone	84.2% EMA response rate; GPS data available for 29.9% of completed EMA entries	GPS recording scheduled to occur only when participants were responding to EMA

One major advantage of GPS data is the ability to investigate participants’ activity spaces: the places people visit in daily life and their routes among them (Freisthler et al. 2014; Martinez et al. 2014). People’s behaviors in and feelings about their activity spaces can differ from those associated with their residences (H.L. Cooper & Tempalski 2014). Without GPS, participants may be able to report their major locations throughout the day, or identify locations they use for major life purposes over longer periods (e.g., Martinez et al. 2014), but details about the momentary environment and routes among locations would be difficult or impossible to obtain. GPS allows for relatively passive collection of large amounts of detail.

In the substance use research published so far combining EMA with GPS, location information has been used to investigate (1) neighborhood disorder/disadvantage and (2) proximity to retail drug outlets.

3.2 Neighborhood disorder/disadvantage

Combinations of momentary environmental disorder/disadvantage and EMA have been used in studies of adolescents’ substance use (Byrnes et al. 2017; Mennis et al. 2016) and drug craving and mood in adult opioid/cocaine users (Epstein et al. 2014). In their studies of adolescents’ alcohol and/or drug use, both Mennis and colleagues (2016) and Byrnes and colleagues (2017) found positive associations between substance use and neighborhood disorder/disadvantage indices based on US Census data. In contrast, Epstein and colleagues (2014) reported inverse associations in a sample of methadone-maintained opioid-dependent outpatients between environmental disorder (as assessed by researchers’ previous observations of city blockfaces) in participants’ GPS-established locations and EMA-reported drug craving, stress, and mood. The difference between these studies may involve participants’ age, the substances involved, or treatment status. Using the same blockface observations of environmental disorder as Epstein and colleagues (2014), Sarker and colleagues (2016) combined EMA with both GPS and ambulatory physiological monitoring (reviewed in Section 4) to associate disorder with stress in opioid-dependent polydrug users: after inferring stress from participants’ physiological responses using machine learning, these researchers identified environmental features associated with both higher stress (e.g., graffiti) and lower stress (e.g., youth playing) likelihoods.

3.3 Retail drug outlets

A second approach to combining EMA and GPS concerns proximity to locations where drugs can be purchased. The proximity or density of retail outlets could affect drug use through cues, availability, or perceptions of norms (Freisthler et al. 2014; Kirchner et al. 2013; Mennis et al. 2016; Watkins et al. 2014; see also Pearson et al. 2016 for a protocol concerning rules and norms about smoking in GPS-determined locations). For legal drugs, these effects have been studied for both alcohol (Byrnes et al. 2017) and tobacco (Kirchner et al. 2013; Mitchell et al. 2014; Watkins et al. 2014), taking advantage of administrative databases (i.e., of licenses) for objective information about retail locations, although the information available varies by location (see, e.g., Mitchell et al. 2014 on using retail tobacco outlet information developed by Rose et al. 2013 in the absence of North Carolina state licensing of tobacco retailers). Objective information about marketplaces for illicit drugs would have to be obtained through different means, such as law-enforcement sources (e.g., Jennings et al. 2013; Linton et al. 2014; Moeller 2016). The legalization of recreational cannabis in several US states, with the associated regulation of transactions (e.g., Carnevale et al. 2017), should provide new opportunites for retailer monitoring.

For alcohol, Byrnes and colleagues (2017) found that adolescents’ EMA-reported drinking was associated with momentary proximity to objectively determined locations of alcohol outlets, but not to the adolescents’ own observations of alcohol outlets. This pattern may reflect the adolescents’ use of alcohol near, but not directly at, locations where alcohol is sold.

For tobacco, Kirchner and colleagues (2013) studied smokers’ EMA-reported craving and lapses during a quit attempt. Lapses were more likely on days with any retail encounter(s) vs. none, and on days with more vs. fewer encounters. However, retail exposure was not associated with increased daily average craving, as was expected. The authors suggest that the importance of smoking cues is diminished when “background” craving is high, so retail exposure could be an important risk factor on days with otherwise low craving. Subsequently, Watkins and colleagues (2014) found that proximity to retail outlets was associated with stronger urge to smoke during a quit attempt when participants were closer to home but considered their findings preliminary, in part due to missing location data (see Section 6 below). Finally, Mitchell and colleagues (2014) reported a proof-of-concept study showing the feasibility of combining EMA and GPS with information about tobacco retail outlets in smokers with attention deficit hyperactivity disorder.

3.4 Conclusion and future directions for GPS location information

Studies combining EMA and GPS have already helped elucidate how cues, craving, environmental disorder, and drug use are related in daily life. It may be useful to complement GPS with environmental geofencing, whereby virtual perimeters are established around key locations, allowing events to be triggered in those locations. Geofencing could ensure that participants are assessed in important but rarely or briefly visited places (Reichert et al. 2016), and interventions could be given near places participants identify as drug use or craving locations (Attwood et al. 2017; Gustafson et al. 2011, 2014; Naughton et al. 2016; see also Schick et al. 2018). It may also be possible to discern problematic locations through combinations of GPS and sensor-detected drug use or mood (as discussed in Section 2 and Section 4), obviating participants’ explicitly indicating them.

Researchers may increasingly want to link GPS data to aspects of the environment beyond disorder and retail outlets. This may call for sensors for light, sound, or air quality, among others. To integrate the resultant data, substance-abuse researchers may need training from or collaborations with outside experts. Even without these additional data, the number of GPS points collected in some studies can approach “big data” proportions. We will return to these considerations in Section 6.

4. Ambulatory physiological (cardiovascular) monitoring

4.1 General considerations

Table 3 presents a summary of studies combining EMA with electronic physiological monitoring in field studies of substance use. Interest in measuring physiology in daily life extends back at least to the development of the Holter monitor (Holter & Generelli 1949); initial challenges included the size and weight of the earliest equipment (e.g., Corday 1991, Figure 2), as well as issues with the transmission of data by radio. Although some difficulties with recording and transmission remain (e.g., sensor detachment, packet loss or delay), information can be reliably relayed from wearable sensors to smartphones by Bluetooth or other wireless protocols (e.g., Hovsepian et al. 2015; Rahman et al. 2014). Several systems allow near-continuous monitoring (e.g., Carreiro et al. 2015b; Hossain et al. 2014; Myrtek et al. 1988; Natarajan et al. 2016) upon which different EMA sampling schemes can be superimposed, akin to what is possible with GPS, although some studies have used more intermittent monitoring (e.g., cardiocascular readings every 45 min, synchronized with EMA; Kamark et al. 1998, 2002, 2005).

Table 3.

Field studies of substance use combining EMA with objective measurement of participants’ physiology

Reference	Substance(s)	Measure added to EMA	Participants	Monitoring duration	EMA device; other device(s)	Compliance/Feasibility	Notes
Bodin et al. 2017	tobacco	ECG	35 smokers and 114 non-smokers with high hostility	24 hours	Palm Pilot PDA; Marquette 8500 Holter ECG recorder	EMA and ECG data available for 5067 30-min periods in total, with 259 smoking episodes reported
Chatterjee et al. 2016	tobacco	ECG, respiration, wrist movement and orientation	61 adult smokers making a quit attempt	24 hours pre-quit, 72 hours post-quit	smartphone (model N/A); AutoSense physiologic al monitoring suite + smartwatches (model N/A)	EMA details N/A; 45/61 participants contributed data to model development (2,766 total participant-hours of AutoSense wear)	development of a model to detect cigarette craving based on relationships among craving, stress, and time of day
Hossain et al. 2014	cocaine	ECG, body movement (chest accelerometry)	38 poly-drug users in methadone maintenance treatment + 9 cocaine users in lab-based studies	4 weeks	smartphone (EMA, model N/A); AutoSense physiologic al monitoring suite + Sony Ericsson Xperia X8 smartphone	EMA details N/A; 922 participant-days of field data included 27 instances of cocaine use with acceptable sensor data from 13 participants (out of 142 reports from 20 people in total)	development of a model to detect cocaine use based on heart rate recovery after cocaine use vs. physical activity
Hovsepian et al. 2015	alcohol, tobacco	ECG and respiration (controlling for physical activity)	23 field study participants + 50 participants in lab-based studies	7 days	smartphone (EMA, model N/A); AutoSense physiologic al monitoring suite + Android smartphone (model N/A)	3/23 field participants excluded for poor quality/missing data; good quality physiologic al data available for 1,060 EMA reports in total	development of a model to detect stress; EMA self-reports of smoking and alcohol use mentioned in field study, but use details N/A
Kennedy et al. 2015	opioids, cocaine	heart rate (ECG RR interval)	40 opioid-dependent adults	Up to 4 weeks	smartphone (EMA model N/A); AutoSense physiologic al monitoring suite + Sony Ericsson Xperia X8 smartphone	overall EMA details N/A (heart rate available for 168 EMA drug use reports and 2,329 random prompt responses); 85.7% overall AutoSense data yield (hours acceptable data/hours sensor wear)	some data also used in model development by Hossain et al. 2014, Rahman et al. 2014, and Sarker et al. 2016
Natarajan et al. 2016	cocaine	ECG morphology (6 waveform features/proper ties)	5 cocaine-dependent persons (non-treatment-seekers) + 10 lab-based participants	37 total participant-days	Samsung Galaxy smartphone ; Zephyr BioHarness chest band	Only participant-initiated EMA (no answer %, but incomplete reports made on 8 days); ECG data lost on “some weekend days” due to device power issues	addresses several issues that can affect researchers generalizing lab-based measures to field studies
Rahman et al. 2014	tobacco, illicit drugs (Study 1); alcohol, tobacco (Study 2)	ECG, respiration, accelerometry, temperature (ambient and skin), galvanic skin response	40 “illicit drug users” (Study 1); 30 daily smokers and “social drinkers” (Study 2)	4 weeks (Study 1); 7 days (Study 2)	smartphone (EMA, model N/A); AutoSense physiologic al monitoring suite + Android smartphone (model N/A)	EMA details N/A; 75.3%–85.7% AutoSense data yields (acceptable data hours/sensor wear hours), with up to 65% data availability after excluding physical activity	development of a model to detect stress from multiple physiological parameters
Saleheen et al. 2015	tobacco	respiration, wrist movement and orientation	61 adult smokers making a quit attempt + 6 smokers for field-based training data	1 day with ad lib smoking, 3 days during quit attempt	smartphone (model N/A); AutoSense physiologic al monitoring suite + smartwatches (model N/A)	overall EMA details N/A (9/33 lapsers did not self-report lapse); 33 lapsers/61 participants included in algorithm development (2,766 total participant-hours of Autosense wear)	development of a model to detect smoking puffs from respiration and wrist movement/orientation; some data also used for craving detection by Chatterjee et al. 2016
Sarker et al. 2016	opioids, poly-drug use	ECG and respiration (controlling for physical activity), GPS	38 opioid-dependent poly-drug users receiving opioid agonist maintenance	4 weeks	smartphone (EMA, model N/A); wireless physiologic al sensor suite (ECG, respiration, accelerometry) + Android-based smartphone (model N/A)	88.0% EMA answer rate; 10.54 hours acceptable ECG data from 12.52 hours sensors wear per day; ~70% of physiologic al data available after excluding physical activity; GPS details N/A	stress inferred from physiological monitoring using the model of Hovsepian et al. 2015; GPS then used to assess environmental features associated with stress inference

For studies of substance use, physiological measurements—principally of cardiovascular parameters—combined with EMA show promise for detecting not only drug use, but also stress and correlates of mental events (e.g., emotions). Distinguishing target events/states from confounds (e.g., physical activity) is crucial, with progress made integrating inputs from multiple sensors, as reviewed below. With that, a number of the potential advantages (e.g., preserving the naturalism of behavior) and caveats (e.g., the importance of detecting sensor removal/tampering) that apply to the field collection and processing of biological samples (reviewed in Section 2) also apply to ambulatory physioligcal monitoring.

4.2 Detecting drug use by physiological changes

Algorithms to detect cocaine use in the field have been developed based on the observation that heart rate stays elevated after cocaine use longer than after accelerometer-detected physical activity (Hossain et al. 2014) or on ECG morphology (Natarajan et al. 2016). Using some of the same techniques as Hossain and colleagues (2014), Kennedy and colleagues (2015) found differences in heart rate depending on the drug used (EMA-reported heroin use < EMA-reported cocaine use), and for cocaine, differences depending on the amount used. Differences in heart rate variability associated with EMA-repored smoking have also been reported (Bodin et al. 2017).

Rather than cardiovascular parameters, Carreiro and colleages (2015a, 2015b, 2016) measured combinations of electrodermal activity, wrist accelerometry, and skin temperature as indices of autonomic activity. To our knowledge, this technique has not been combined with EMA in substance users; however, in field studies of cocaine users (2015a, 2015b) and emergency-department-based studies of patients given opioids for pain (2015a, 2016), these researchers identified changes associated with drug intake, as matched to urinalysis and/or timeline follow-back for cocaine or clinical care for opioids. Like combinations of cardiovascular measurements and accelerometry, these studies illustrate how multiple sensors in concert can facilitate detection of drug use.

4.3 Detecting physiological correlates of stress and mental events

In heroin and cocaine users, Kennedy and colleagues (2015) showed that heart rate changed with EMA-reported mood and stress (as well as drug use). Work has also been done to develop/apply algorithms to detect stress from cardiovascular and respiratory parameters in different populations of substance users (Hovsepian et al. 2015; Rahman et al. 2014; Sarker et al. 2016), as well as to detect craving in smokers by interactions of craving, stress, and time of day (Chatterjee et al. 2016). Considerable work in non-substance-users has also been done on stress and negative emotions, given their links to cardiovascular disease, as well as how potential protective factors (e.g., self-esteem, relaxation, positive social interactons) could buffer the cardiovascular effects of negative events (e.g., Brondolo et al. 2008; Enkelmann et al. 2005; Gump et al. 2011; Kamarck et al. 1998, 2002, 2005; Ottaviani et al. 2015; Plarre et al. 2011; Schwerdtfeger & Scheel 2012; Verkuil et al. 2015). These results could also be relevant to coping in substance users. Technically innovative early work on emotion by Myrtek and colleagues (reviewed by Myrtek & Brugner 1996; see also Loeffler et al. 2013; Myrtek et al. 1996, 1999, 2005) showed EMA/cardiovascular discrepancies, which could indicate either cardiovascular effects of unconscious processes or difficulty detecting cardiovascular correlates of self-reported emotions. This work highlights the nontrivial interplay of EMA and sensor readings in testing theories of emotion and behavior, and how each might validate the other (cf. Sarker et al. 2016).

4.4 Summary and future directions for physiological monitoring

As detection of target events/states continues to improve, future work may benefit from schemes that tie assessments or interventions to particular physiological values or changes (e.g., Myrtek et al. 1988 having heart rate changes trigger EMA; see also Ebner-Priemer et al. 2013 on EMA linked to accelerometry). Such procedures could be thought of as “physiofencing,” with virtual boundaries drawn around key values of physiological parameters (cf. Hossain et al. 2014; Sarker et al. 2016)..

Other physiological measurements may also be useful in studies of substance use. For example, respiration measurements, either alone or with hand/arm gestures, can indicate cigarette smoking (Ali et al. 2012; Lopez-Meyer et al. 2012; Saleheen et al. 2015). These algorithms may be adaptable to detect the use of other drugs taken by inhalation or insufflation. In these cases, respiration is essentially part of the drug taking behavior itself, in contrast to the techniques reviewed above used to detect the physioligcal consequences of intake. Respiration measurements are also currently being used in stress detection algorithms (e.g., Hovsepian et al. 2015), and they may assist in emotion detection (reviewed along with several other potentially useful parameters by Wilhelm & Grossman 2010). Sleep, a complex behavioral and physiological phenomenon, is also amenable to field monitoring by EMA (e.g., Whalen et al. 2008) and by mobile/wearable devices (e.g., Hasler et al. 2008; Sharkey et al. 2011). Sleep has profound links to substance use (e.g., Angarita et al. 2016); a full treatment of this topic is beyond our scope.

5. Other mobile technologies that could be combined with EMA in substance use studies

Other technologies have considerable promise to be combined with EMA, focusing on detecting drug use or correlates of mental events. Detection of bodily movement and orientation by accelerometers and gyroscopes is already an important part of physiological data collection (e.g., Hossain et al. 2014; Hovsepian et al. 2015). Wrist-mounted accelerometers and gyroscopes can detect puffing gestures in smokers (Parate et al. 2014; Raiff et al. 2014; Tang et al. 2014; see also Lopez-Meyer et al. 2012 and Saleheen et al. 2015 for both gesture and respiration measurements). Similar procedures may be able to detect other gestures used to take drugs (e.g., IV injection). In non-substance-using populations, accelerometry has been studied in association with EMA-reported mood (Kim et al. 2014; Powell et al. 2009; Schwerdtfeger et al. 2010; see also Dunton et al. 2015; Asselbergs et al. 2016; Kikuchi et al. 2007).

Drug-delivery systems with access sensors (e.g., MEMS caps for medication bottles) or more direct use sensors (e.g., “smart pills” that emit a radiofrequency signal upon exposure to the intragastric environment; Chai et al. 2017) could monitor medication adherence or self-reporting accuracy. To our knowledge, MEMS caps have not been combined with EMA in studies of substance use disorders; however, Sigmon and colleagues (2015, 2016) used a portable automated medication dispenser plus interactive voice response for buprenorphine maintenance. Smart pills have been used to study medication use in people prescribed opioids for pain (Chai et al. 2017). Outside of medication adherence, Pearson and colleagues (2017) used a Bluetooth-enabled e-cigarette that logged puff count and duration to corroborate EMA data.

Finally, photographs, videos, and/or audio could characterize participants’ physical and social environments, in addition to their own behavior. A distinction must be drawn between participants’ making occasional, purposive records (e.g., videos of sample provision; Section 2.3 and Section 2.4) versus “lifelogging” (Doherty et al. 2013 p. 320), the high-frequency, passive collection of audio or video from participants’ perspective (e.g., Brown et al. 2017; Doherty et al. 2013; Manson & Robbins 2017; Mehl 2017). Given the potential to record unconsenting bystanders, as well as intensely private or illicit activities, legal and ethical concerns will have to be carefully navigated (Brown et al. 2017; Kelly et al. 2013; Manson & Robbins 2017; Nebeker et al. 2016). It remains to be determined if these techniques will be widely acceptable to both participants and institutional review boards in studies of substance use. Some concerns may be ameliorated using systems that extract relevant information from audiovisual data without recording personal identifiers (e.g., extracting the tone but not the content of speech, Ooi et al 2014; or recognizing facial expressions apart from personal identity, Hernandez et al. 2012).

6. Challenges to combining EMA with other mobile technologies

All field research, compared with laboratory research, faces shared challenges (reviewed, e.g., by Carreiro et al. 2017; Trull & Ebner-Preimer 2013; Wilhelm & Grossman 2010). We will focus on areas that may be unique to, or especially prominent in, combinations of mobile/wearable technologies.

First, participant compliance may be particularly sensitive to the use of multiple technologies together, just as polypharmacy regimens can reduce medication adherence (Corsonello et al. 2009; Maddigan et al. 2003). Noncompliance may be unintentional—as participants are confused by and make honest mistakes in following monitoring schemes—but may also reflect frustration or burden. Researchers can address this in deciding when and how participants will trained (and, if needed, re-trained) on device usage (e.g., Lukasiewicz et al. 2005; Mitchell et al. 2014; Stone & Shiffman 2002), as well as attempting to maximize the “user friendliness” of devices and interfaces (e.g., Ehrler et al. 2015). Researchers also have to decide how to structure participant remuneration for compliance; it is common, but not universal, for participants to be paid for each device/data type (e.g., Mitchell et al. 2014).

In deciding what device(s) to use to collect their multiple forms of data, researchers should consider the potential for data loss of different kinds and amounts if devices are lost or damaged. Device loss has long been recognized as an obstacle to EMA (e.g., Lukasiewicz et al. 2005; cf. Gustafson et al. 2014), but survival rates can be good (e.g., 1 device not recovered for every 226 person-days of use; Epstein et al. 2009). Researchers need to balance the monetary cost of replacing equipment versus the scientific cost of not doing so (Gurvich et al. 2013). Combining multiple forms of data acquisition in a single device may also affect battery life (e.g., if continuous monitoring is done with a device that would otherwise be used only intermittently). Battery life is crucial to the successful use of wireless technology generally, and difficulties with participants keeping devices charged have been noted in several recent studies (Chai et al. 2017; Manson & Robbins 2017; Natarajan et al. 2016; Waters et al. 2014). Even with ideal user behavior, devices can fail to record or transmit data as intended (e.g., due to environmental conditions not encountered in laboratories; Marques & McKnight 2009; Rahman et al. 2014; Watkins et al. 2014).

The conspicuousness of device wear/use may also be an issue, as there are only so many socially discreet locations devices can be placed, and devices in certain locations or having certain appearances could resemble socially stigmatized devices/procedures (e.g., monitors used in criminal justice settings; Carreiro et al. 2015a, 2015b; Hossain et al. 2014; McKnight et al. 2012; cf. Boyer et al. 2012). For these social reasons, or due to the specifications of the devices themselves (e.g., one, but not all, devices being water-resistant or ruggedized), multiple devices may also interfere particularly severely with participants’ daily routines (e.g., bathing or wearing certain types of clothing; Alessi et al. 2017). It may be unavoidable scientifically to use comparatively fragile, or conspicuously or inconveniently placed devices (e.g., measuring light exposure at eye level; Figueiro et al. 2012), but researchers should be sensitive to the burdens of using such devices.

Some issues of data processing and analysis are also unique to or more prominent in studies combining multiple technologies. Techniques for analyzing EMA data are discussed elsewhere (Mehl & Conner 2012; Schwartz & Stone 1998; Shiffman 2014; Terhorst et al. 2017), and many of the issues encountered in those analyses apply to data collected from other field procedures as they are linked to EMA. Interpolation or imputation of missing values may be needed (e.g., Epstein et al. 2014), apart from the use of statistical procedures compatible with missing data. Data may also require different types of pre-processing (see, e.g., Hovsepian et al. 2015 and Rahman et al. 2014 on physiological monitoring; Bond et al. 2014 on transdermal alcohol sensing). Data-processing techniques developed for laboratory-based devices cannot necessarily be used for mobile sensor data (Natarajan et al. 2013), and aligning different kinds of data collected with different frequencies over different intervals can be challenging (e.g., Ebner-Preimer et al. 2013; Hovsepian et al. 2015). These issues should be considered at the design stage of a study.

Finally, for both data collection and analysis, the accuracy of all measures needs to be understood, from the limits of detection and margins of error in the analysis of biological samples (e.g., Marques & McKnight 2007 on transdermal alcohol sensing) to the maximum accuracy commitments of the US government to GPS system transmissions, apart from the actual accuracy of the end user’s receiver (“GPS Accuracy” 2017; Mennis et al. 2017). Detailed treatment of this topic is beyond our scope, but researchers should know the specifications of their chosen systems and consider the potential for some combinations of inaccuracy to be more problematic than others.

7. Future directions and conclusion

The value of combining EMA with other forms of mobile/wearable technology will likely only increase with the proliferation of mobile interventions. Mobile contingency management already includes technology to alert participants of the need for sample collection, to test samples, and to announce or deliver consequences (reviewed by Kurti et al. 2016). EMA could be added to this framework to study other changes in behavior and emotion during treatment. Cue-exposure therapies or extinction-based procedures could also be delivered in the field, with stimuli presented on mobile devices as EMA measures momentary effects (Gass et al. 2012; Tomko et al. 2017; Warthen & Tiffany 2009; Wray et al. 2011, 2015). Finally, as summarized in Table 4, neuropsychological or cognitive tests linked to EMA (Lukasiewicz et al. 2005; McCarthy et al. 2016; Schuster et al. 2016; Shiffman et al. 1995; Waters et al. 2014; Waters & Li 2008) could clarify contextual determinants of cognition and choice in people who use drugs, informing mobile interventions.

Table 4.

Other non-self-report responses collected from participants in combination with EMA in field studies of substance use

Reference	Substance(s)	Measure added to EMA	Participants	Monitoring duration	EMA device; other device(s)	Compliance/Feasibility	Notes
Lukasiewicz et al. 2005	alcohol	reaction time	18 alcohol-dependent adults (14 providing EMA data)	up to 3 weeks or until drinking lapse	Handspring Visor electronic diary	4 participants broke/lost electronic diary before any data could be transferred; 14 others had missing data rates of 5.6–26.8%	reaction time measured as time required to complete first EMA scale
McCarthy et al. 2016	tobacco	impulsive action (continuous performance test)	116 adult smokers making a quit attempt	4 weeks (1 week pre-quit, 3 weeks post-quit)	palmtop computer (model N/A)	N/A	impulsive choice also measured by delay discounting (self-reported money preferences)
Schuster et al. 2016	tobacco, alcohol, marijuana	visual working memory	287 adolescents, at five-year follow-up (9th or 10th grade students at baseline)	7 days	palmtop computer (model N/A)	92.6% EMA answer rate
Shiffman et al. 1995	tobacco	reaction time; timed finger tapping ; mental arithmetic; visuospatial logic	25 “regular smokers” + 26 “chippers” (i.e., lighter smokers)	2 × 4-day periods separated by 1 week (1 while smoking, 1 while abstinent)	PSION Organizer II model XP hand-held computer	72% of participants completed all EMA assessments; < 1% of behavioral tasks interrupted/neglect ed (i.e., to trigger a task restart protocol)	hand-held computer also outfitted with mouthpiece for “bogus pipeline” sham field CO testing
Waters & Li 2008	tobacco	reaction time	22 adult smokers + 22 adult non-smokers	1 week	HP iPAQ Pocket PC PDA	81.2% EMA answer rate; ~3% of reaction time measurements discarded as interrupted/from incorrect trials	Reaction times obtained in three versions of the Stroop task: “classic,” emotional words, and smoking-related words
Waters et al. 2014	tobacco	reaction time	129 adult smokers making a quit attempt	1 week	HP iPAQ Pocket PC PDA	119/129 participants provided data: Stroop data available for 1,322/2,441 EMA trials completed	Reaction times obtained in a smoking-related word version of the Stroop task

More generally, combining mobile assessments with mobile interventions opens up new kinds of experimental design, including microrandomization (Klasnja et al. 2015), which entails randomization at the level of the momentary event, not the entire person to a condition. Crucially, in microrandomization, the effects of the intervention are measured proximally (e.g., in the hour after each delivery). Combinations of EMA and other remote technologies may be particularly suitable for delivering and assessing microrandomized interventions.

A particularly compelling possibility is to intervene in the field before the target event occurs. In their pioneering description of ambulatory physiological monitoring, Holter and Generelli (1949) proposed using ambulatory EEG to develop an “epilepsy alarm” (p. 750) that would sound before seizures began. It may soon be possible to combine EMA with physiological information, as well as location/environmental information, to predict and prevent stress, drug craving and use (e.g., Chatterjee et al. 2016; Hossain et al. 2014; McClernon & Choudhury 2013; Sarker et al. 2016). One often-noted promise of mHealth is its giving clinicians the ability to intervene wherever and whenever needed—“just in time” (e.g., Carreiro et al. 2017). Prediction of future emotions and behavior may redefine what counts as “just in time” in compelling new ways.

Highlights.

• It is important to understand the momentary contextual determinants of drug use.

• Mobile technologies can assess participants and their environments during daily life.

• Progress has been made combining self-report, multiple sensors, and machine learning.

• Challenges include participant burden, device functionality, and data processing.

• Mobile assessment is leading to mobile intervention, including prediction/preemption.