Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2024 Jan 19.
Published in final edited form as: J Affect Disord. 2022 May 7;311:63–68. doi: 10.1016/j.jad.2022.05.015

Ecological momentary assessment of affect in depression-prone and control samples: Survey compliance and affective yield

Andrew J Seidman a,*, Charles J George b, Maria Kovacs a
PMCID: PMC10798424  NIHMSID: NIHMS1872192  PMID: 35537542

Abstract

Background:

Ecological momentary assessment (EMA) is a high-frequency ambulatory data collection approach that has come to be widely used in emotion research. It therefore is timely to examine two features of EMA needed for a successful study: compliance with survey prompts and high affective yield (survey prompts that capture affect experience). We posit that compliance may be subject to temporal variation (time-of-day, days in study) and individual differences (depression history), and that affective yield may also differ by social context.

Methods:

We examined these issues in a sample of 318 young adults (Mage = 24.7 years, SD = 2.7), including those with current depression (n = 28), remitted depression (n = 168) and never-depressed controls (n = 122) who participated in a 7-day EMA protocol of negative and positive affect (NA and PA, respectively).

Results:

The overall compliance rate was 91% and remained stable across the survey week. However, subjects were significantly less likely to respond to the first daily prompt compared to those that followed. The likelihood of capturing NA and PA decreased with each EMA protocol day, and affective yield across social contexts differed by participants’ depression status.

Limitations:

The sample was largely comprised of White young adults. Relative to the remitted and control groups, the sample size for the currently depressed was unbalanced.

Conclusion:

Researchers can optimize compliance and affective yield within EMA by considering depression, time-of-day, study duration, and social context. Clinicians using EMA to monitor affect may benefit from considering these parameters.

Keywords: Ecological momentary assessment (EMA), Depression, Emotions, Survey compliance

1. Introduction

Ecological momentary assessment (EMA) is a high frequency sampling methodology that is increasingly used to study the experience and regulation of daily emotion and mood (Bolger and Laurenceau, 2013; Stange et al., 2019; Trull and Ebner-Priemer, 2020). By asking respondents to report on their affect when prompted in their natural environments, EMA helps circumvent the limitations of self-report questionnaires and lab-based assessments, including limited ecological validity and biased recall, which are particularly problematic in studies of depressed samples (e.g., Ben-Zeev et al., 2009). By sampling across multiple time frames, EMA can also reveal temporal variations in affect experience (e.g., Stange et al., 2019).

Given the popularity and growing use of EMA to study emotion in daily life, it is timely to examine how well this assessment approach has fared in two domains needed for a successful study, namely high rates of compliance with survey prompts and high affective yield. Compliance refers to responding to a prompt within the allocated time window; affective yield refers to the proportion of survey prompts that capture participants during an affect experience (Colombo et al., 2019; Rintala et al., 2019).

EMA studies of affect are typically implemented via smartphones: participants receive generally 5–10 daily prompts, which query about the presence and intensity of momentary negative and positive affect (NA and PA, respectively). Typical protocols last between 5 days and 2 weeks (e.g., Koster et al., 2015; Nelson et al., 2020; Stange et al., 2019; for a description of typical EMA characteristics, see Williams et al., 2021). Recent reviews that focused on studies of emotionally healthy adults have found that compliance with EMA prompts ranges between 78% and 85% (Rintala et al., 2019; Soyster et al., 2019; Williams et al., 2021). Partial compliance with prompts often results in the exclusion of participants’ data; for example, in recent studies, between 9 and 37% of prompts were excluded from analyses (e.g., Bos et al., 2019; Newman and Nezlek, 2021).

A diurnal variable that may affect compliance is the time of day a prompt is dispatched (e.g., Courvoisier et al., 2012). Indeed, in a meta-analysis of 10 EMA studies, Rintala et al. (2019) found that compliance was lowest during the first prompt in the morning (56%) and highest during the early afternoon (83%). A more extended temporal variable is study duration, which also may affect compliance: at the week-level, compliance decreased after the first day (83%) and reached its nadir (73%) five days into a six-day protocol (Rintala et al., 2019). However, another meta-analysis (Soyster et al., 2019) found no difference in compliance as a function of study duration.

Differences in compliance with EMA may also reflect individual difference variables, such as a history of depression. While in a meta-analysis of studies of currently depressed adults, overall compliance was 80% (Rintala et al., 2019), according to another review, about one-third of EMA studies with depressed samples had compliance rates lower than 80% (Colombo et al., 2019). Subsequent studies reported even lower rates, ranging between 63% and 76% (Panaite et al., 2020; Sheets and Armey, 2020; Thompson et al., 2021; for notable exceptions, see Nelson et al., 2020; Vrijsen et al., 2021). Even past depression appears to interfere with EMA compliance: adults with remitted depression appear averse to participation (Slofstra et al., 2018), and when enrolled, respond to between 63% and 85% of prompts (Sheets and Armey, 2020; Thompson et al., 2021; van Winkel et al., 2015).

How well EMA of emotion succeeds in capturing affective experience is critical to a study’s success (e.g., Stange et al., 2019; Trull and Ebner-Priemer, 2020). However, information about this issue is scant and not encouraging. For example, in a ten-day EMA study of community adults, only 31% of prompts included endorsement of any NA or PA since the previous prompt (Livingstone and Isaacowitz, 2021). In another EMA study, only 14% to 38% of prompts captured NA and only 55% of the prompts captured PA (Simon et al., 2021). In a college student sample, 28% of nearly 2000 prompts over a ten-day EMA failed to capture NA or PA since the prior assessment (Heiy and Cheavens, 2014).

Depression is one individual difference variable that would be expected to influence affective yield: in an EMA, participants with depression should have higher rates of negative affect and lower rates of positive affect than those free of depression. Indeed, adults in an episode of major depression spent substantially more days in dysphoric moods (65%) than did healthy controls (7%) while those with a depressive disorder other than MDD spent close to 50% of the days being dysphoric (Chepenik et al., 2006); further, currently depressed participants were anhedonic most days (56%), followed by participants with an “other” MDD (48%) - such days were rare (8%) for healthy subjects.

The social context in which people find themselves at the time of EMA prompts also may impact on affective yield. For example, people are less willing to express NA in public and when among acquaintances and strangers (e.g., English et al., 2017). It is especially important to consider social context when studying depression-prone samples, as its assessment alongside NA and PA can elucidate in what environments they are likely to experience their worst and best mood, respectively (e. g., Rottenberg et al., 2005). There also is evidence that negative social context influences situation appraisals and thereby affect experience (Vrijsen et al., 2021). However, there are scant empirical data about how various social contexts impact on affective yield and even less about how NA and PA interact with context among currently or previously depressed participants.

1.1. Study aims

Information about compliance and affective yield in EMA studies of emotion may help researchers to optimize their experimental designs (Stange et al., 2019; Trull and Ebner-Priemer, 2020). Therefore, the present study had three aims. First, we sought to replicate and extend existing findings that compliance with EMA prompts is associated with temporal parameters (i.e., time-of-day, days in study) and varies by depression history (i.e., current, remitted, never depressed). Second, we sought to replicate existing findings regarding affective yield in EMA protocols. Third, by testing if NA and PA yield is subject to these same parameters and social context, we sought to elucidate parameters of EMA design and sample characteristics which could influence study success.

2. Methods

2.1. Sample

The current sample includes 318 young adults in Hungary enrolled in a study examining daily life use and efficacy of emotion regulation (Mage = 24.7 years, SD = 2.7, Range = 18–31), 46% female, and 97% Caucasian (3% Roma or biracial). At study entry, 28 subjects were in a current episode of major depression, 168 had remitted from a past episode of depression, and 122 were never-depressed controls. Sample size was based on the availability of prior samples and exceeded the necessary N determined by a multilevel power analysis for the primary analyses pertaining to emotion regulation. Participants were initially recruited between 1996 and 2007 as children for a larger study of psychosocial and genetic risk factors in depression (see Kiss et al., 2007). The original study entry criteria included being aged 7 to 14 years and having an available biological parent. Exclusion criteria included a major systemic medical disorder or intellectual disability.

The depressed sample was initially recruited from 23 child mental health treatment centers across Hungary and meet criteria for a DSM-IV major depressive disorder; healthy controls were eventually recruited from nearby schools (see Kiss et al., 2007). At original study entry, all cases were evaluated by two diagnostic interviewers about a month apart, who separately assessed both the parent and the child via the semi-structured Interview Schedule for Children and Adolescents-Diagnostic version (ISCA; Sherrill and Kovacs, 2000). Final diagnoses were derived by consensus among senior child psychiatrists using a best-estimate procedure (Kovacs et al., 1984; Maziade et al., 1992). Portions of the original samples have subsequently participated in one or more later studies, including the current one, each of which involved the psychiatric evaluation of cases (see Procedures), yielding a continuous record of psychiatric disorders (Kiss et al., 2007).

2.2. Procedures

The current study received IRB approval at the University of Pittsburgh and University of Szeged in Hungary; participation occurred in Hungary. After providing written informed consent, psychiatric diagnoses were determined as per the DSM-IV (First et al., 1995) via the semi-structured Interview Schedule for Young Adults (ISYA-D), an age-appropriate modification of a diagnostic interview used with children and adolescents (Kiss et al., 2007; Sherrill and Kovacs, 2000). Consistent with past research (Kovacs et al., 1984) remission from depression was defined as maintaining no or minimal symptoms for at least two months after recovery from a depression episode. Again, final diagnosis was based on a consensus best-estimate procedure (Maziade et al., 1992) among trained clinicians, with documented interrater reliability in using the semi-structured interview schedule (Kiss et al., 2007). As part of a larger study, participants in the present study also completed a series of separate psychophysiological tasks which are not reported.

2.2.1. EMA protocol

All participants completed an individual orientation to EMA by trained research staff and were scheduled to begin the following day. The protocol is representative of other EMA studies of affect; it entailed seven consecutive days during which participants received five pseudo-random prompts scheduled approximately 90 min apart between 10 am-10 pm on weekdays, and 10 am-11 pm on weekends. Participants were given 30 min to respond to the prompt and received automated reminders every 10 min if it remained unopened; after the third reminder, access to the link was automatically closed. Prompt delivery was managed using the Twillio cellular messaging service. Research staff contacted all participants on days 1 and 3 to inquire about general difficulties; additional contact was completed on an as-needed basis. If participants missed more than two prompts in a day, they were offered an extra day in the EMA.

Survey prompts included a link to a secure website where participants responded to a series of questions that probed for worst and best affect (NA, PA) in the past hour. The question stem was: “During the past hour, how strongly did you feel…?” The listed emotions reflected items from the Positive and Negative Affect Schedule (PANAS; Watson et al., 1988) and included four negative emotions (sad/blue, nervous/anxious, irritable, and angry) and two positive emotions (happiness, satisfaction). At each prompt, participants used a visual sliding scale (0 = Not at all; 100 = Very much) to indicate how strongly they felt each emotion.

For the present analyses, we regarded a rating of ≥1 for any of the emotions as indicating affective yield, consistent with previous research (e.g., Heiy and Cheavens, 2014; Simon et al., 2021). If any NA or PA was endorsed, the participant then was asked to report general social context (in private vs. public) and the presence of others (alone, with family or significant other, acquaintances, friends, or strangers); multiple contexts could be endorsed.

2.3. Data analytic plan

All analyses were conducted using PROC GLIMMIX within SAS 9.4 (SAS Institute, 2013). Statistical significance was defined at the p < .05 level with all confidence intervals excluding 0. Effect sizes were expressed in terms of Hodges d for t-tests, partial η2 for F-statistics, and odds ratios (OR) and corresponding confidence intervals (95% CI) for parameters of models with a binary outcome. For age effects, OR’s were computed for +1 SD difference (3 years in age). This study was not preregistered. Data and syntax are available upon request.

To examine if compliance is related to temporal and diagnostic parameters, we tested for patterns of within- and between-day prompt responding to the maximum 35 prompts. Using a repeated-measures logistic regression approach, we fit a generalized linear mixed model with an AR(1) covariance structure for each subjects’ 35 responses (yes, no). Independent variables were prompt number (1–5, categorical, 1st prompt as the reference group), which served as a surrogate for time-of-day due to its fixed schedule, protocol day (1–7, categorical, 1st day as reference group), group (control, remitted depression, current depression), and group-by-prompt and group-by-day interactions. The interaction terms were decomposed to identify diurnal (i.e., within-day) and weekly (i.e., across days) variation in compliance, respectively. We included participant sex and age in years (centered at age 25) as covariates, as women tend to respond to more prompts and compliance generally improves with age (Rintala et al., 2019). We also considered a generalized linear mixed model with day of week (Sunday through Saturday) instead of days in study, which tests the effects of the day-of-week on compliance, respectively. The AR(1) parameter describes the strength of relationship between subjects’ likelihood of response to successive prompts (i.e., autocorrelation). The fixed parameter estimates for the independent variables are interpreted as log-odds ratio of non-response and were used to test for group and prompt# differences in in least squares means.

To derive NA and PA affective yield, we calculated the proportion of prompts that identified affective experience (i.e., any NA or PA item rated ≥1). We conducted the same analyses as described above by testing two repeated-measures logistic regression models with the outcome variable defined as the likelihood that a prompt yielded NA or PA. We tested a generalized linear mixed model with day of week to test day-specific effects on affective yield (Stone et al., 2012). Since participants only reported on social context if they endorsed any affect, it could not serve as a predictor variable. The percentage of prompts including NA and PA per social context was then regressed on the temporal parameters (prompt, day) and group in a full-factorial linear mixed model where within-subject dependence of contexts and affects was accounted for via unstructured covariance matrices. Group differences for each context and affect were then tested via least-squares means slices of the model’s group-by-context-by-affect interaction.

3. Results

The three groups of participants differed in age (F2, 315 = 83.77, p < .001). Those currently depressed were older than those remitted from depression (Mdiff = 1.18, SE = 0.44, p = .03) and the never-depressed controls (Mdiff = 4.20, SE = 0.45, p = .000); remitted participants were older than never-depressed controls (Mdiff = 3.03, SE = 0.26, p = .000).

3.1. Compliance

Overall study compliance was 91%. On average, participants responded to approximately 32 prompts out of a possible 35 (SD = 4.35, Range = 3–35). Consistent with reporting guidelines regarding the proportion of participants who completed at least 80% of survey prompts (Trull and Ebner-Priemer, 2020), this criterion was frequently met in the present sample: never-depressed controls (95%), currently depressed (86%), remitted (82%). Most participants (92%) responded to at least one prompt on all seven days of the protocol. Prompt number emerged as a significant predictor of compliance (F4, 1260 = 7.48, p < .001). The first daily prompt was the most likely to be missed (86.9% compliance) while prompts 2–5 were equally likely to be completed (90.9–91.66%, 1st prompt vs. others: OR’s: 1.6–1.7, p’s < .001; 2nd–5th prompts’ pairwise OR’s: 1.0–1.1, p’s > .5). Compliance did not differ across protocol days (F1, 10,797 = 0.29, p > .05).

Compliance also differed as a function of depression history (F2, 313 = 8.70, p < .001). Currently depressed and remitted participants were less likely than were never-depressed controls to respond to EMA survey prompts (87.5% and 87.2% vs. 94.8%, OR’s = 2.59–2.67, p < .001). There were no differences in compliance between those currently depressed and in remission (p = .86).

There was a trend for a group-by-prompt interaction on compliance (F2, 1260 = 1.77 p = .08) but no evidence for a group-by-day interaction (F2, 10,797 = 0.24, p = .78). We also considered a generalized linear mixed model with day of week (Sunday through Saturday) instead of days in study. Day of week did not have a significant effect on compliance (F6, 1885 = 1.4, p > .2) nor did it interact with group (F12, 1885 = 1.3, p > .2).

Sex and age were significant covariates (F1, 313 = 11.23, p < .001, and F1, 313 = 17.40, p < .001, respectively); females were more likely to respond to survey prompts than were males (91.6% vs. 88.8%, OR = 1.36, p < .001) and older subjects were more likely to be compliant than younger ones (slope = −0.08, SE = 0.02, OR3 years = 0.8, 95% CI = [0.7, 0.9], p < .001).

3.2. Affective yield

3.2.1. NA

The overall affective yield rate was 85.8% for any NA: it did not significantly differ by prompt number (F4, 7726 = 0.66, p = .62, partial η2 < 0.01) but varied across study days (F1, 2614 = 8.39, p < .01, partial η2 < 0.01). Namely, with each day since study entry, the likelihood of any NA reported decreased (overall slope = −0.10, SE = 0.04, OR7th vs. 1st Day = 0.5, 95% CI = [0.4–0.8], p < .001).

There was a trend for a group effect on NA yield (F2, 2460 = 2.54, p = .07, partial η2 < 0.01): currently depressed adults tended to be more likely to report NA compared with the remitted and control groups (93% vs. 86% vs. 86%, respectively; both OR’s = 2.2, both 95% CI’s = [1.4, 3.5], t’s > 3.3, p’s < .001). Remitted and control participants had nearly identical NA yield rates (p > .7, OR = 1.0, both 95% CI’s = [0.8, 1.2]).

There was no group-by-prompt interaction on NA yield (F8, 7857 = 0.51, p = .85, partial η2 < 0.01) but there was a trend for a group-by-day interaction (F2, 2601 = 2.32, p = .098, partial η2 < 0.01). While NA yield rates remained stable among currently depressed participants (slope = −0.07, SE = 0.09, p > .4), the likelihood of reporting any NA decreased for controls most for each day in the study (slope = −0.16, SE = 0.03, p < .001), followed by the remitted subjects (slope = −0.08, SE = 0.03, p = .005).

Sex emerged as a significant covariate (F1, 2212 = 15.11, p < .001, partial η2 = 0.01) with females being more likely to report NA (91% vs. 87%, OR = 1.4, 95% CI = [1.2, 1.6], p < .001). Age in years was also significant (F1, 2212 = 42.51, p < .001); NA yield increased as a function of older age (slope = 0.12, SE = 0.02, OR3 years = 1.4, 95% CI = [1.3, 1.6], p < .001).

3.2.2. PA

The overall affective yield rate was 97.9% for any PA: it did not significantly differ by prompt number (F4, 7726 = 0.62, p = .65, partial η2 < 0.01) but varied across study days (F1, 2981 = 4.51, p = .04, partial η2 < 0.01). With each day since study entry, there was a decreased likelihood that a prompt would include PA (overall slope = −0.10, SE = 0.05, OR7th vs. 1st Day = 0.5, 95% CI = [0.3, 0.95], p = .04).

PA yield significantly differed by depression history (F2, 2810 = 9.11, p < .001, partial η2 = 0.01): controls were most likely to report PA, followed by the remitted and currently-depressed participants (99.3% vs. 98% vs. 92.06%, respectively; pairwise OR’s: 0.1–0.3, 95% CI’s: [0.04, 0.6], p’s < .001). The group-by-prompt interaction did not predict PA yield (F8, 8060 = 1.07, p = .38, partial η2 < .01), nor was there evidence for a group-by-day interaction (F2, 2959 = 0.07, p = .93, partial η2 < .01).

Sex emerged as a significant covariate (F1, 2585 = 4.90, p = .05), with females being more likely to endorse PA across prompts (98.2% vs. 97.2%, OR = 1.5, 95% CI = [1.04, 2.2], p < .001). Age in years was not significantly related to PA yield (F1, 2566 = 0.72, p = .40, partial η2 < 0.01).

Table 1 summarizes the proportion of prompts that yielded NA and PA as a function of social context and group membership. When reporting NA, currently depressed participants were less likely than controls to be in public (Mdiff = −12.1, t574 = 2.59, p = .03, d = 0.22) and more likely to be with family or a significant other (Mdiff = 12.9, t549 = −2.41, p = .04, d = 0.21). Remitted participants were less likely than were controls to report NA when with friends (Mdiff = −3.8, t515 = −2.47, p = .04, d = 0.22) and PA (Mdiff = −4.2, t = −2.65, p = .02, d = 0.22). However, the three groups did not differ in rates of reporting either NA or PA when in private, alone, with acquaintances, or with strangers (p’s > .05).

Table 1.

Social contexts (M ± SD % of prompts) of most intense affect reported for the prior hour.

Social context Affect type Control (N = 121) Remitted (N = 168) Currently depressed (N = 28) Group F-statistic Partial η2
In private Negative 57.0 ± 23.3 59.5 ± 25.8 68.3 ± 25.6 2.87 0.01
Positive 60.3 ± 21.4 64.7 ± 25.4 70.3 ± 23.7 2.63 0.01
In public Negative 45.0 ± 23.4 42.0 ± 25.7 32.6 ± 25.9 3.38* D v C 0.01
Positive 42.0 ± 21.6 37.6 ± 25.0 31.3 ± 24.3 2.91 0.01
Alone Negative 38.7 ± 23.5 35.4 ± 26.2 36.7 ± 25.9 0.64 <0.01
Positive 32.2 ± 21.5 28.8 ± 25.4 38.3 ± 26.1 2.05 0.01
Family/significant other Negative 31.5 ± 21.8 37.6 ± 27.9 44.7 ± 34.5 3.63* D v C 0.01
Positive 38.4 ± 23.1 45.0 ± 30.0 43.1 ± 31.1 2.22 0.01
Acquaintance Negative 20.9 ± 21.7 21.4 ± 23.3 18.1 ± 21.7 0.36 <0.01
Positive 21.0 ± 21.3 22.6 ± 23.7 15.7 ± 19.7 1.50 0.01
Friend Negative 10.9 ± 11.2 7.2 ± 12.4 6.5 ± 10.2 3.47* R v C 0.01
Positive 15.0 ± 14.8 10.8 ± 14.7 8.6 ± 12.2 4.65** R v C 0.02
Stranger Negative 9.4 ± 12.7 7.2 ± 12.6 5.5 ± 9.1 1.51 0.01
Positive 6.8 ± 9.0 4.5 ± 9.3 4.4 ± 6.4 1.14 0.01
Open in a new tab

Note. Means and standard deviations represent the proportion of prompts in which peak affect (“worst” for NA; “best” for PA) was observed across different social contexts.

*

p < .05

**

p < .01 of group in context-by-affect slices of a mixed effects, full factorial model. Post-hoc comparisons (D = currently depressed, R = remitted, C = control) are significant at p < .05.

4. Discussion

The present study sought to contribute to the literature on compliance and affective yield in studies of emotion using EMA protocols and examine the roles of time (diurnal, between days), individual difference variables, and contextual factors in the outcomes of interest. The high compliance rates (87–95%) in our study underscore that, notwithstanding the frequent sampling that characterizes EMA, it is possible to maintain a very high level of subject cooperation. The finding that our compliance rates exceeded by 10% to 20% those reported by more recent reviews (e.g., Rintala et al., 2019; Soyster et al., 2019) may reflect the benefits of early and “as-needed” contact from research staff, which was built into the protocol, and that participants had a long-term relationship with study researchers (e.g., Kiss et al., 2007).

We replicated results that, in general, EMA compliance does not worsen as a function of study duration (Soyster et al., 2019) and additionally showed that there are no day-of-week effects. However, in line with previous research (Courvoisier et al., 2012; Rintala et al., 2019), we found a diurnal pattern: participants were most likely to miss the first morning prompt compared with the remaining prompts that day. One way to counter such source of missing data is to offer a second morning prompt on an as-needed basis. Other options include having participants self-initiate the first prompt each day, after which they can be transitioned into a routine sampling schedule (e.g., Vrijsen et al., 2021), or slightly extending the prompt response window (e.g., Ben-Zeev et al., 2009).

In the present study, young adults with current or remitted depression had lower compliance rates than did never-depressed controls: this finding suggests that consideration be given to incentivizing depression-prone subjects to boost motivation by rewarding timely prompt-responding or offering personalized emotion-relevant feedback from the EMA (e.g., van Genugten et al., 2020). It is notable that while our finding of lower compliance among depression prone people is consistent with clinical experience and the documented detrimental effects of depression on motivation and task completion (Barge-Schaapveld et al. 1999; Silvia et al., 2013), it conflicts with the conclusions of recent meta-analyses (Rintala et al., 2019; Soyster et al., 2019). However, most of the studies included in the meta-analyses did not use psychiatric diagnostic interviews to determine depression status (but relied on self-rated measures of depressive symptoms) and employed community-based sampling: thereby they represent populations different from the clinically-referred one in our study.

Contrary to expectations, we found high rates of affective yield during a weeklong EMA: we captured NA at close to 86% of the prompts and PA at close to 98% of the prompts. In comparison, other studies have reported capturing NA only around 40% or fewer prompts (Livingstone and Isaacowitz, 2021; Simon et al., 2021). Given that very few studies have provided information on affective yield, this feature of EMA needs to be addressed by future investigations. Our finding of high affective yield may reflect birth cohort effects or the originally clinically referred nature of the sample. Due to shifts in cultural norms, younger cohorts may be more open to report affect experiences. Also, recall, that depression-prone subjects were originally ascertained because they had pediatric onset mood disorders, which, by definition, predicts higher rates of trait and state negative affect. These possible explanations should be explored in future studies.

Although we found that the likelihood of capturing NA or PA remained stable throughout each day, it declined as the number of days in the study increased. However, there was not a day-of-week effect. Reduced affective yield as a function of study duration is discrepant with previous research (De Vuyst et al., 2019; Vachon et al., 2016) but may reflect an unintended artifact of our study design. Namely, in our protocol, reporting any affect during a prompt triggered a series of further queries concerning social context and the subsequent course of the emotion (not reported here). Thus, subjects may have sought to shorten their involvement with some prompts by denying affect experience. Supporting this notion, longer questionnaires (but not increased sampling frequency) have been shown to contribute to increased perceptions of study burden and lower compliance (Eisele et al., 2022). However, another recent study found the opposite pattern regarding perceived burden, but reported no differences in compliance (Hasselhorn et al., 2021). Future work is needed to elucidate the relationships between EMA sampling characteristics, compliance, and affective yield.

We also found that depression history, as an individual differences variable, and social context influenced the likelihood of NA and PA. Currently depressed participants were more likely to report NA when they were in public or with family/significant others compared with never-depressed controls. Further, participants remitted from depression were particularly sensitive to being with friends: in their company, they were less likely to report any affect than were controls in similar situations. These findings suggest that researchers consider context when targeting affect experience; asking participants to self-initiate prompts when they are in the social context of interest may produce higher affective yield.

While not of central study interest, sex and age emerged as significant predictors of EMA compliance and affective yield. Women were more likely to respond to prompts, and compliance improved with age, as reported by some others (Rintala et al., 2019) but inconsistent with a meta-analysis (Soyster et al., 2019). Women also were more likely to report NA and PA than were men, which aligns with well-known findings that they are more likely to endorse affective experiences in daily life. Furthermore, older subjects reported higher rates of NA (but not PA). As the age range among subjects spanned from 18 to 31 years, this finding may reflect the effects of emerging adulthood, which has been described as particularly stressful (Arnett, 2000).

4.1. Limitations

Our study had some limitations. The present sample was comprised of young adults and was homogeneous in terms of racial background (97% Caucasian), which limits generalizability of the findings. The relatively low number of currently depressed participants constrains our conclusions regarding the impact of current versus remitted depression or no depression on compliance and affective yield. A further limitation is that we cannot make claims regarding social context as a predictor of affective yield because participants reported this information only after they endorsed NA or PA. However, positive features of our study (e.g., large sample sizes, a balanced gender distribution across the sample, the use of psychiatric evaluation to determine depression) outweigh its limitations and the results expand the literature on compliance and affective yield in the EMA study of affect.

Acknowledgments

We thank Drs. Krisztina Kapornai, Enikő Kiss, and Ildikó Baji at the University of Szeged, and clinicians across research sites in Hungary, for their contributions.

Role of the funding source

The study was made possible through financial support from the NIMH #R01 MH084938. Andrew Seidman was financially supported by NIMH T32 #MH08159 while writing this manuscript.

Footnotes

Conflict of interest

Andrew J. Seidman has no conflicts of interest.

Charles J. George has no conflicts of interest.

Maria Kovacs has no conflicts of interest.

CRediT authorship contribution statement

Andrew J. Seidman managed the literature review, wrote the manuscript, and helped with study analysis.

Maria Kovacs developed the survey protocol, secured grant funding, and wrote the manuscript.

Charles J. George conducted the study analysis, created the included table, and edited the manuscript.

References

  1. Arnett JJ, 2000. Emerging adulthood: a theory of development from the late teens through twenties. Am. Psychol. 55 (5), 469–480. [PubMed] [Google Scholar]
  2. Barge-Schaapveld DQCM, Nicolson NA, Berkhof J, deVries MW, 1999. Quality of life in depression: daily life determinants and variability. Psychiatry Res. 88, 173–189. [DOI] [PubMed] [Google Scholar]
  3. Ben-Zeev D, Young MA, Madsen JW, 2009. Retrospective recall of affect in clinically depressed individuals and controls. Cognit. Emot. 23, 1021–1040. [Google Scholar]
  4. Bolger N, Laurenceau J-P, 2013. Intensive Longitudinal Methods: An Introduction to Diary and Experience Sampling Research. Guilford Press. [Google Scholar]
  5. Bos E, de Jonge P, Cox R, 2019. Affective variability in depression: revisiting the inertia–instability paradox. Br. J. Psychol. 110, 814–827. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Chepenik LG, Ten Have T, Oslin D, Datto C, Zubritsky C, Katz IR, 2006. A daily diary study of late-life depression. Am. J. Geriatr. Psychiatr. 14, 270–279. [DOI] [PubMed] [Google Scholar]
  7. Colombo D, Semonella M, Kwiatkowska M, Garciá Palacios A, Cipresso P, Riva G, Botella C, Fernández -Álvarez J, Patané A, 2019. Current state and future directions of technology-based ecological momentary assessment and intervention for major depressive disorder: A systematic review. Journal of Clinical Medicine 8, 465. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Courvoisier DS, Eid M, Lischetzke T, 2012. Compliance to a cell phone-based ecological momentary assessment study: the effect of time and personality characteristics. Psychol. Assess. 24, 713–720. [DOI] [PubMed] [Google Scholar]
  9. De Vuyst H-J, Dejonckheere E, Van der Gucht K, Kuppens P, 2019. Does repeatedly reporting positive or negative emotions in daily life have an impact on the level of emotional experiences and depressive symptoms over time? PLoS One 14, 1–12. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Eisele G, Vachon H, Lafit G, Kuppens P, Houben M, Myin-Germeys I, Viechtbauer W, 2022. The effects of sampling frequency and questionnaire length on perceived burden, compliance, and careless responding in experience sampling data in a student population. Assessment 29, 136–151. [DOI] [PubMed] [Google Scholar]
  11. English T, Lee IA, John OP, Gross JJ, 2017. Emotion regulation strategy selection in daily life: the role of social context and goals. Motiv. Emot. 41, 230–242. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. First MB, Spitzer RL, Gibbon M, Williams JBW, 1995. Structured Clinical Interview for DSM-IV Axis 1 Disorders - Patient Edition (SCID-I/P, Version 2.0). Biometrics Research Department, New York State Psychiatric Institute. [Google Scholar]
  13. Hasselhorn K, Ottenstein C, Lischetzke T, 2021. The effects of assessment intensity on participant burden, compliance, within-person variance, and within-person relationships in ambulatory assessment. Behavior Research Methods 1–18. Advance online publication. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Heiy J, Cheavens JS, 2014. Back to basics: a naturalistic assessment of the experience and regulation of emotion. Emotion 14, 878–891. [DOI] [PubMed] [Google Scholar]
  15. Kiss E, Gentzler AM, George C, Kapornai K, Tamas Z, Kovacs M, Vetro A, 2007. Factors influencing mother-child reports of depressive symptoms and agreement among clinically referred depressed youngsters in Hungary. J. Affect. Disord. 100, 143–151. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Koster EHW, Fang L, Marchetti I, Ebner-Priemer E, Kirsch P, Huffziger S, Kuehner C, 2015. Examining the relation between mood and rumination in remitted depressed individuals: a dynamic systems analysis. Clin. Psychol. Sci. 3, 619–627. [Google Scholar]
  17. Kovacs M, Feinberg TL, Crouse-Novak MA, Paulauskas P, Finkelstein R, 1984. Depressive disorders in childhood I.: a longitudinal prospective study of characteristics and recovery. Arch. Gen. Psychiatry 41, 229–237. [DOI] [PubMed] [Google Scholar]
  18. Livingstone KM, Isaacowitz DM, 2021. Age and emotion regulation in daily life: frequency, strategies, tactics, and effectiveness. Emotion 21, 39–51. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Maziade M, Roy MA, Fournier JP, Cliché D, Mérette C, Caron C, Garnea Y, Montgrain N, Shriqui C, Dion C, Nicole L, Potvin A, Lavallaa JC, Pires A, Raymond V, 1992. Reliability of best-estimate diagnosis in genetic linkage studies of major psychoses: results from the Quebec pedigree studies. Am. J. Psychiatr. 149, 1674–1686. [DOI] [PubMed] [Google Scholar]
  20. Nelson J, Klumparendt A, Doebler P, Ehring T, 2020. Everyday emotional dynamics in major depression. Emotion 20, 179–191. [DOI] [PubMed] [Google Scholar]
  21. Newman DB, Nezlek JB, 2021. The influence of daily events on emotion regulation and well-being in daily life. Personal. Soc. Psychol. Bull. 48, 19–33. [DOI] [PubMed] [Google Scholar]
  22. Panaite V, Rottenberg J, Bylsma LM, 2020. Daily affective dynamics predict depression symptom trajectories among adults with major and minor depression. Affect. Sci. 1, 186–198. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Rintala A, Wampers M, Myin-Germeys I, Viechtbauer W, 2019. Response compliance and predictors thereof in studies using the experience sampling method. Psychol. Assess. 31, 226–235. [DOI] [PubMed] [Google Scholar]
  24. Rottenberg J, Gross JJ, Gotlib IH, 2005. Emotion context insensitivity in major depressive disorder. J. Abnorm. Psychol. 114, 627–639. [DOI] [PubMed] [Google Scholar]
  25. SAS Institute Inc, 2013. SAS ® 9.4 statements: Reference. SAS Institute, Cary, NC. [Google Scholar]
  26. Sheets E, Armey M, 2020. Daily interpersonal and noninterpersonal stress reactivity incurrent and remitted depression. Cogn. Ther. Res. 44, 774–787. [Google Scholar]
  27. Sherrill JT, Kovacs M, 2000. Interview Schedule for Children and Adolescents (ISCA). J. Am. Acad. Child Adolesc. Psychiatry 39, 67–75. [DOI] [PubMed] [Google Scholar]
  28. Silvia PJ, Kwapil TR, Eddington KM, Brown LH, 2013. Missed beeps and missing data: dispositional and situational predictors of nonresponse in experience sampling research. Soc. Sci. Comput. Rev. 31, 471–481. [Google Scholar]
  29. Simon SG, Sloan RP, Thayer JF, Jamner LD, 2021. Taking context to heart: momentary emotions, menstrual cycle phase, and cardiac autonomic regulation. Psychophysiology 58, 1–16. [DOI] [PubMed] [Google Scholar]
  30. Slofstra C, Nauta MH, Holmes EA, Bos EH, Wichers M, Batalas N, Klein SN, Bockting CLH, 2018. Exploring the relation between visual mental imagery and affect in the daily life of previously depressed and never depressed individuals. Cognit. Emot. 32, 1131–1138. [DOI] [PubMed] [Google Scholar]
  31. Soyster PD, Bosley HG, Reeves JW, Altman AD, Fisher AJ, 2019. Evidence for the feasibility of person-specific ecological momentary assessment across diverse populations and study designs. J. Pers.-Oriented Res. 5, 53–64. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Stange J, Kleiman EM, Mermelstein RJ, Trull TJ, 2019. Using ambulatory assessment to measure dynamic risk processes in affective disorders. J. Affect. Disord. 259, 325–336. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Stone AA, Schneider S, Harter JK, 2012. Day-of-week mood patterns in the United States: on the existence of ‘Blue Monday’, ‘Thank God it’s Friday’, and weekend effects. J. Posit. Psychol. 7, 306–314. [Google Scholar]
  34. Thompson RJ, Bailen NH, English T, 2021. Everyday emotional experiences in current and remitted major depressive disorder: an experience-sampling study. Clin. Psychol. Sci. 9, 866–878. [Google Scholar]
  35. Trull TJ, Ebner-Priemer UW, 2020. Ambulatory assessment in psychopathology research: a review of recommended reporting guidelines and current practice. J. Abnorm. Psychol. 129, 56–63. [DOI] [PubMed] [Google Scholar]
  36. van Genugten CR, Schuurmans J, Lamers F, Riese H, Penninx BWJH, Schoevers RA, Riper HM, Smit JH, 2020. Experienced burden of and adherence to smartphone-based ecological momentary assessment in persons with affective disorders. J. Clin. Med. 9, 322. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Vachon H, Bourbousson M, Deschamps T, Doron J, Bulteau S, Sauvaget A, Thomas-Ollivier V, 2016. Repeated self-evaluations may involve familiarization: an exploratory study related to ecological momentary assessment designs in patients with major depressive disorder. Psychiatry Res. 245, 99–104. [DOI] [PubMed] [Google Scholar]
  38. van Winkel M, Nicolson NA, Wichers M, Viechtbauer W, Myin-Germeys I, Peeters F, 2015. Daily life stress reactivity in remitted versus non-remitted depressed individuals. Eur. Psychiatry 30, 441–447. [DOI] [PubMed] [Google Scholar]
  39. Vrijsen JN, Ikani N, Souren P, Rinck M, Tendolkar I, Schene AH, 2021. How context, mood, and emotional memory interact in depression: a study in everyday life. Emotion 1–11. Advance online publication. [DOI] [PubMed] [Google Scholar]
  40. Watson D, Clark LA, Tellegen A, 1988. Development and validation of brief measures of positive and negative affect: the PANAS scales. J. Pers. Soc. Psychol. 54, 1063–1070. [DOI] [PubMed] [Google Scholar]
  41. Williams M, Lewthwaite H, Fraysse F, Gajewska A, Ignatavicius J, Ferrar K, 2021. Compliance with mobile ecological momentary assessment of self-reported health-related behaviors and psychological constructs in adults: systematic review and meta-analysis. J. Med. Internet Res. 23, 1–19. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES