Using Experience Sampling Methods/Ecological Momentary Assessment (ESM/EMA) in Clinical Assessment and Clinical Research: Introduction to the Special Section

Abstract

This article introduces the special section on experience sampling methods and ecological momentary assessment in clinical assessment. We review the conceptual basis for experience sampling methods (ESM; Csikszentmihalyi & Larson, 1987) and ecological momentary assessment (EMA; Stone & Shiffman, 1994). Next, we highlight several advantageous features of ESM/EMA as applied to psychological assessment and clinical research. We provide a brief overview of the articles in this special section, each of which focuses on 1 of the following major classes of psychological disorders: mood disorders and mood dysregulation (Ebner-Priemer & Trull, 2009), anxiety disorders (Alpers, 2009), substance use disorders (Shiffman, 2009), and psychosis (Oorschot, Kwapil, Delespaul, & Myin-Germeys, 2009). Finally, we discuss prospects, future challenges, and limitations of ESM/EMA.

Experience sampling methods (ESM; Csikszentmihalyi & Larson, 1987) and ecological momentary assessment (EMA; Stone & Shiffman, 1994) are important research tools that have come into their own over the past 2 decades (Fahrenberg, Myrtek, Pawlik, & Perrez, 2007; Piasecki, Hufford, Solhan, & Trull, 2007; Stone, Shiffman, Atienza, & Nebeling, 2007). Although ESM and EMA arose from different research traditions, they have in common the collection of self-reports or indices of behavior, cognition, or emotions in near real time in the daily lives of the participants, ideally with electronic devices. In this article, we introduce the conceptual basis for this assessment approach and highlight several advantageous features of ESM/EMA as applied to psychological assessment and clinical research. We provide a brief overview of the articles in this special section, each of which focuses on one of the following major classes of psychological disorders: mood disorders and mood dysregulation (Ebner-Priemer & Trull, 2009), anxiety disorders (Alpers, 2009), substance use disorders (Shiffman, 2009), and psychosis (Oorschot, Kwapil, Delespaul, & Myin-Germeys, 2009). Finally, we discuss future prospects and limitations of ESM/EMA.

Unlike traditional assessment approaches, ESM/EMA approaches are idiographic in their focus. This focus is credited to Gordon Allport (1937), who was convinced that the ultimate goal of personality psychology should be to derive the most complete understanding of the individual. Allport's work (as well as that of others who followed, e.g., Rosenzweig, 1950) emphasized that nomothetic approaches, though useful for summarizing general patterns or laws influencing behavior (e.g., the effects of personality traits, motivations, environmental events, or situations), are not well-suited for the rich description and understanding of the individual as he or she goes about daily life.

To appreciate the unique perspective and promise of ESM/EMA methods in this context, we provide a brief description of this approach. (For comprehensive overviews of these methods, see Fahrenberg & Myrtek, 1996, 2001; Fahrenberg et al., 2007; Shiffman, Stone, & Hufford, 2008; Stone & Shiffman, 1994; Stone et al., 2007).¹ As defined by Stone and Shiffman (1994), EMA is characterized by (a) collection of data in real-world environments; (b) assessments that focus on individuals’ current or very recent states or behaviors; (c) assessments that may be event-based, time-based, or randomly prompted (depending on the research question); and (d) completion of multiple assessments over time. Finally, it is important to note that ESM/EMA can be conducted using a wide variety of media, including paper diaries, electronic diaries, or telephones.

The latter issue deserves special comment. Investigators often collect field data using paper diaries, asking participants to complete one or more diary entries per day between visits to the clinic or laboratory. A chief limitation of this approach is that investigators cannot be sure that the ratings were actually completed at the times specified by the research design. Participants may neglect to make ratings at the scheduled time, then “back-fill” their diaries before reporting to the study center, presumably to avoid admitting not making the scheduled ratings (e.g., see Stone, Shiffman, Schwartz, Broderick, & Hufford, 2002). Because electronic diaries and smartphones, for example, are computerized, studies using this technology eliminate the back-filling problem. Participants complete ratings in response to prompts emitted by the device, and entries are electronically time-stamped. Back-filled entries, if participants tried to make them, would be time-stamped and easily identified. Fortunately, studies show that participants using electronic diaries show high rates of compliance at the time of the scheduled prompt. For example, participants drawn from a variety of clinical populations have provided timely responses to 85% or more of the delivered prompts (e.g., Collins et al., 1998; Hufford, Shields, Shiffman, Paty, & Balabanis, 2002; Shiffman, Paty, Gnys, Kassel, & Hickcox, 1996; Stone & Shiffman, 2002; Trull et al., 2008).

Although each article in this special section focuses on a class of disorders or syndromes, there are several general themes that each review shares in common. To orient readers to the articles that follow, we first highlight several advantageous features of ESM/ EMA as applied to psychological assessment and clinical research.

The Importance of Real-Time Assessment

ESM/EMA studies are designed to capture momentary ratings of experiences, and this makes them especially important for the assessment of moods, thoughts, symptoms, or behaviors believed to change over time (Ebner-Priemer, Eid, Stabenow, Kleindienst, & Trull, 2009). For example, mood is not a static process and may change due to the time of day or the events of the day. Historically, however, investigators have gathered data on dynamic psychological processes (like mood) using single-occasion retrospective self-reports. Unfortunately, this technique is subject to numerous biases because it often requires complex cognitive operations on the part of the subject (e.g., Hufford, 2007).

For years the field has relied on retrospective reporting as the primary means to gather information about the individual patient. Although efficient, there is strong evidence that retrospective reports are subject to biases that challenge both their reliability and validity. Studies have revealed significant discrepancies between real-time assessments and retrospective self-reports of mood, symptoms, and behaviors across a range of clinical problems (e.g., see Fahrenberg et al., 2007; Solhan, Trull, Jahng, & Wood, 2009; Stone & Broderick, 2007). Several possible sources of retrospective bias exist. For example, individuals are more likely to recall or report experiences that seem more personally relevant ( personal heuristics effect), that occurred more recently (recency effect), that stand out as significant or unusual (salience or novelty effect), or that are consistent with their current mood state (mood-congruent memory effect; e.g., Gorin & Stone, 2001; Hufford, Shiffman, Paty, & Stone, 2001).

Real-time data collection strategies such as ESM/EMA offer ways to reduce the effect of recall bias and provide a better picture of patients’ more immediate, moment-to-moment emotional, behavioral, and cognitive experiences in their natural environments. In contrast to traditional retrospective assessment approaches, ESM/EMA assessments usually require much simpler cognitive operations—questions such as “What is your level of depression right now” are straightforward and less subject to bias and forgetting. Thus, ESM/EMA data are less vulnerable to many sources of error that are inherent in many traditional assessment techniques.

The Importance of Ecological Assessment and Context

Relative to laboratory research, ESM/EMA has the advantage of being ecological—processes such as mood can be studied in participants’ “natural habitats” where they are subject to the many environmental and interpersonal factors that typify daily living but which cannot be recreated in the laboratory (Wilhelm & Roth, 1998). Clinical psychologists and clinical researchers are interested in the emotions, thoughts, behaviors, and experiences of patients in their day-to-day life. Such an understanding is necessary in order to provide an accurate evaluation of the relevant clinical problems, to recommend appropriate treatment or intervention, and ultimately to adequately evaluate treatment response. Assessment of patients in their natural environment serves to increase the construct, ecological, and external validity of our assessments.

In addition, ESM/EMA research can sample characteristics of the environment (e.g., location, time of day, presence of interpersonal conflict) that change over time. Context is an important consideration for many symptoms of psychopathology. Problematic emotions, thoughts, or behaviors may occur primarily under certain specific conditions, settings, or within certain relationships, to name just a few possibilities. Unfortunately, traditional clinical questionnaire and interview assessments rarely inquire about the context of symptoms. In contrast, ESM/EMA data collections can be tailored to reveal important contexts for symptoms.

For example, a very simple form of context is time of day. Data collected using electronic diaries are usually time-stamped so that symptoms, like mood, can be graphed along a temporal dimension. This is of great interest in the investigation of major depressive disorder, in which there often is a characteristic diurnal variation of mood (e.g., Peeters, Berkhof, Delespaul, Rottenberg, & Nicolson, 2006). Another example of important context is the emergence of symptoms in response to high levels of stress. Some symptoms of psychopathology, like dissociation in borderline personality disorder, are exacerbated under stressful conditions (Stiglmayr et al., 2007). A final example is within the context of specific dyadic relationships. An important Diagnostic and Statistical Manual of Mental Disorders (4th ed.; DSM–IV) borderline personality disorder criterion concerns unstable and intense interpersonal relationships. Borderline personality disorder patients who meet this criterion are prone to switch from positive feelings about a specific relationship to markedly negative feelings. To adequately assess such a pattern, it is necessary to analyze reports within specific dyadic relationships (Ebner-Priemer et al., 2009) rather than across different dyadic relationships.

Treatment Monitoring

One exciting clinical application of ESM/EMA is to monitor treatment progress. Often treatment progress is assessed using weekly or monthly questionnaires or interviews that require retrospection on the part of the patient. However, as noted above, retrospective self-report measures are notoriously context dependent and highly influenced by momentary accessible information (e.g., Fredrickson, 2000; Kahneman, Fredrickson, Schreiber, & Redelmeier, 1993; Kihlstrom, Eich, Sandbrand, & Tobias, 2000). Unfortunately, treatment studies using real-time data capture to monitor treatment success, such as the studies by Klosko, Barlow, Tassinari, and Cerny (1990) or Bauer et al. (2005), are still relatively rare in clinical psychology, even though some studies show that ESM/EMA might be better able to uncover relatively smaller treatment effects (Barge-Schaapveld & Nicolson, 2002). Mobile electronic devices have also been used successfully to administer treatment; however, the number of studies is still relatively small (for excellent examples, see Newman, Kenardy, Herman, & Taylor, 1997; Rodgers et al., 2005).

Articles in This Special Section

The articles in this special section focus on ESM/EMA research targeting major forms of psychological disorders: mood disorders and mood dysregulation, anxiety disorders, substance use disorders, and psychosis. In addition to presenting relevant research, each article highlights both conceptual and methodological issues in assessing these disorders, ways this methodology can address these issues, and future applications of these methods.

Ebner-Priemer and Trull (2009) review ESM/EMA studies that target mood disorders (e.g., major depressive disorder, bipolar disorder) and borderline personality disorder. Their review is organized around six major questions: (a) Do real-time assessments provided by ESM/EMA provide a different picture of mood patterns than do traditional retrospective self-reports? (b) How can we use repeated assessments to characterize dynamic processes like mood fluctuation? (c) How can ESM/EMA data complement and be integrated with other forms of ambulatory assessments (e.g., psychophysiological)? (d) How can ESM/EMA be used to reveal context-specific relations between mood and behavior? (e) How can ESM/EMA be used to guide real-time interventions and feedback? and (f) What evidence is there that ESM/EMA methods are more ecologically valid than traditional forms of assessment? The results of their review have several implications for clinical assessment. First, ESM/EMA data and retrospective data often are discrepant, and it will be important to examine the validity of each form of assessment using external criteria. Second, ESM/EMA data allow for an examination of dynamic processes like mood fluctuation, but it is important to match the sampling strategy and analytic techniques to the process of interest. Third, several ESM/EMA sampling strategies can be adopted to reveal context-specific associations, including event sampling for low base rate behaviors and concurrent and routine assessments of contexts and variables of interest so that lagged relationships can be revealed. Finally, individually tailored real-time feedback and intervention is a promising adjunct of ESM/EMA, and these methods appear especially well-suited to track treatment response.

Alpers's (2009) review of ambulatory assessment studies of panic disorder and specific phobias provides a focus on the contribution of physiological measurement to our understanding of these disorders as they manifest themselves in daily life. The term ambulatory assessment is most often used to refer to the continuous or near-continuous monitoring of physiological states, sometimes accompanied by diary reports of subjective states or contexts. Physiological measures are particularly well-suited to study these anxiety disorders because the symptoms themselves involve bodily processes and sensations. Further, without ambulatory assessment, it would be extremely difficult to observe or record spontaneous panic episodes or to be able to assess the context of symptom occurrence in specific phobia. Alpers notes interesting discrepancies between the retrospective reports of panic and phobia symptoms and ambulatory assessment reports, as well as between reports of both symptom occurrence and intensity and physiological indices related to these symptoms. There are intriguing developments in the use of feedback from ambulatory assessment as a form of intervention, as well as in the use of electronic diaries to both monitor and administer treatment. Finally, Alpers's review notes the promise of using these methods and technologies to better characterize self-exposure to feared situations, as well as the promise of using ambulatory assessment to elucidate cognitive processes associated with anxiety disorders.

Shiffman (2009) describes how EMA can facilitate the study of substance use and abuse. Substance use itself is an event that is best captured using event-based sampling by having participants initiate an assessment session when a predetermined event occurs (e.g., after consuming an alcoholic drink). By examining antecedent internal states (e.g., mood, craving) as well as external stimuli (e.g., substance cues, interpersonal influences), one can assess the viability of a number of theories regarding what influences the initiation of substance use. Likewise, proximal reinforcers of substance use can be evaluated by examining reports following substance use (e.g., stress reduction, positive affect). Because EMA is conducted in an individual's natural environment, a more ecologically valid assessment of naturally occurring antecedents and consequences of substance use can be studied. At the same time, Shiffman urges us to consider challenges to using EMA to assess substance use, including ensuring compliance as well as accurate reporting while sometimes in intoxicated states, assessing for possible reactivity, and validating EMA data. Many examples are provided to illustrate EMA applications to substance use: the association of mood and substance use, influences on relapse, and the use of EMA data to assess treatment outcome or process.

Oorschot et al. (2009) discuss the application of ESM/EMA to severe mental illness and to psychosis in particular. Their article first tackles several vexing issues for this vein of research, including whether those suffering from psychosis can provide reliable and valid reports and can complete the protocols in the specified manner (e.g., responding to random prompts, completing surveys when certain events or experiences occur in daily life). Their literature review is organized by the aims of the respective studies: describing the phenomenology of psychosis (e.g., positive and negative symptoms), investigating the influences on psychosis and related behavior or experience (e.g., stress, cannabis use), and testing psychological models of psychosis (e.g., paranoia, deficits in coping). Finally, Oorschot et al. present cutting-edge research on using EMA to elucidate Gene × Environment interactions, as well as the promise of EMA in treatment studies of psychosis.

Prospects and Challenges

ESM/EMA data can help address many of the goals of clinical assessment. Specifically, these data can provide a detailed account and understanding of an individual's problems as experienced in daily life. In turn, this information can both inform and enhance clinical treatment. In addition, this method can be quite valuable in the monitoring of treatment progress. Patients and clinicians alike can use the feedback from this form of assessment to gauge how treatment is progressing and whether modifications are necessary.

As the technology in electronic and ambulatory assessment devices becomes more sophisticated, the clinical applications in assessment, treatment, and research grow (Intille, 2007). Both PDAs and smartphones are now able to beam or transfer information in near real time. Further, these devices can serve not only as a platform for patients to answer queries but also as a means of collecting audio, video, geographical positioning, and (through attachments) some physiological and biological data (e.g., blood alcohol content). Data from wearable devices and biosensors are soon likely to be integrated with the information gathered via PDAs and smartphones to provide a comprehensive picture of patients’ emotional, psychological, behavioral, and physical functioning in their natural environment.

However, some of the excitement over these possibilities is tempered by several challenges that may affect how soon or how easily ESM/EMA assessment becomes a routine part of clinical practice or clinical research. A major issue is acceptance and compliance. Fortunately, acceptance and compliance on the part of patients does not seem to be a big concern, as studies routinely document that individuals are open to the use of electronic devices and are able to comply with prompts for assessments while in their natural environment (Ebner-Priemer & Sawitzki, 2007; Hufford, 2007; Mehl & Holleran, 2007). However, when using interactive assessment with individually tailored moment-specific feedback, patients not only have to comply with reports on their experience several times a day but also are asked to use specific and perhaps (to them) novel skills to alleviate their symptoms. This is likely to prove more challenging than simply reporting levels of distress or problems. Therefore, alternative assessment strategies (e.g., self-video gathered by a smartphone) may be helpful to document the ability of patients to follow through on the interventions that are suggested.

Interestingly, our impression is that clinicians and clinical researchers are more resistant to the idea of implementing real-time assessment methods than are clients or patients. There are some barriers to implementation that are quite understandable, including cost of devices, the need for knowledge about software and programming, and the challenge of how best to view or analyze the data gathered (even for a single case). Although these challenges are perhaps initially daunting, we do not believe them to be insurmountable. As more clinicians and clinical researchers integrate these methods into their work, more guidance and resources (e.g., freeware, how-to manuals) will likely be available.

In summary, real-time assessment and interactive treatment suggest several possibilities that may change the face of clinical assessment and clinical practice. This area, however, is still in its infancy, and several important questions need to be answered before these techniques are incorporated into routine practice. First, does EMA/ESM show added value or incremental validity? Do EMA/ESM data predict treatment response better than data from traditional assessment measures (e.g., questionnaire scores)? Second, is the cost–benefit ratio favorable? That is, are comparable outcomes obtained, and do ESM/EMA assessment and interactive treatment approaches save time and effort on the part of the clinician? Third, are clients and patients open to this form of assessment and treatment, and are they compliant with the coaching, advice, and practices prescribed electronically? Fourth, what is the best way to present or graphically display real-time data to both clients/patients and clinicians to maximize understanding of the clinical problems, aid appropriate formulation of treatment, and provide timely feedback on progress in order to improve the likelihood of a good outcome?

As users and advocates of ESM/EMA methods in clinical research and clinical work, we are excited about the prospects for this unique and important form of clinical assessment and intervention. These methods allow us to assess and evaluate individuals in their natural environment, adopting an idiographic perspective, and to determine whether certain patterns of mood, behavior, and experience can be generalized across groups of individuals. Therefore, the goals promoted by Allport (1937) and others who lamented the neglect of the individual seem closer to realization than before.