Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2025 Dec 2.
Published in final edited form as: Death Stud. 2024 Dec 2;50(3):482–494. doi: 10.1080/07481187.2024.2433109

Ecological momentary assessment in prolonged grief research: Feasibility, acceptability, and measurement reactivity

Emily H Mintz 1, Emma R Toner 2, Alexa M Skolnik 3, Alicia Pan 4, Madelyn R Frumkin 1,5, Amanda W Baker 1,6, Naomi M Simon 7, Donald J Robinaugh 1,4,*
PMCID: PMC12239693  NIHMSID: NIHMS2037514  PMID: 39622793

Abstract

Ecological momentary assessment (EMA) is a method of data collection that entails prompting individuals to report their experiences (e.g., thoughts, feelings, and behaviors) in real time over the course of their day-to-day lives. By providing rich information about how these experiences unfold over time within an individual, EMA has the potential to substantially advance our understanding of grief. However, there is uncertainty about how bereaved adults will respond to EMA, especially among those with high prolonged grief symptom severity. Accordingly, we evaluated the feasibility and acceptability of an EMA protocol in bereaved adults with low and high prolonged grief severity. Participants completed six 12-item EMA surveys per day on their smartphones for 17 days. Adherence was high (mean survey completion=90%, median = 96%), and only 6% of participants withdrew. Adherence remained high in those with high prolonged grief symptom severity (mean = 86%; median = 96%). On average, participants reported agreement that survey frequency and length were acceptable. There was no evidence for systematic worsening of symptoms during EMA data collection. Together, these findings suggest that EMA is feasible, acceptable, and safe for bereaved adults, including those with high prolonged grief symptom severity.

Keywords: prolonged grief disorder, ecological momentary assessment, adherence, acceptability, measurement reactivity

Introduction

Prolonged Grief Disorder (PGD) is a bereavement-specific syndrome characterized by prolonged, distressing, and impairing grief. An estimated 3–10% of bereaved individuals experience this syndrome (Lundorff et al., 2017; Rosner et al., 2014; Treml et al., 2022), placing them at heightened risk for a wide range of adverse physical and mental health outcomes (Clayton, 1973; Jacobs & Ostfeld, 1977; Keyes et al., 2014; Zinzow et al., 2009; Zisook et al., 2014; Zisook, Reynolds III, et al., 2010; Zisook, Simon, et al., 2010). To better understand the nature, etiology, and treatment of prolonged grief, researchers have relied predominately on two types of measurement: (a) self-report measures, such as the Inventory of Complicated Grief (ICG; Prigerson et al., 1995) or Prolonged Grief Disorder Scale-13-Revised (PGD-13 Prigerson et al., 2021), in which individuals retrospectively report on the frequency or severity of symptoms over a specified period of time (e.g., the past week, or past month) and (b) interview-based measures, such as the Structured Clinical Interview for Complicated Grief (SCI-CG; Bui et al., 2015), in which trained raters ask individuals about their experiences over similar periods of time and determine the presence or severity of a given symptom. These assessments are typically used to calculate a sum score intended to reflect the overall severity of grief symptoms or are used in conjunction with a diagnostic algorithm to determine the presence or absence of prolonged grief. Together, these assessments are the foundation of most research on grief and, thus, information gleaned from these assessments represents much of what we know about prolonged grief.

Although self-report and interview-based measures have shed considerable light on prolonged grief, they also have limitations. First, because these assessments require participants to reflect on their experiences over extended periods of time, they are vulnerable to the limits and biases of memory recall. The risk of recall failure or bias is heightened by the tendency for these assessments to be completed in relatively artificial settings (e.g., a clinic or research lab), isolated from the environment where the experiences themselves occur (Shiffman et al., 2008). Second, traditional assessments provide relatively little information about prolonged grief. In most cases, traditional assessments are used to provide only a single piece of information: either a summed severity score or a binary variable indicating the presence or absence of prolonged grief disorder. Although several recent studies have conducted a more detailed examination of individual symptoms of the prolonged grief disorder syndrome by examining individual items of traditional measures (e.g., Djelantik et al., 2020; Maccallum et al., 2017), these assessments still provide relatively little information about those individual symptoms. For example, consider the item “I feel myself longing for the person who died,” rated on a 5-point scale ranging from “Never” to “Always” (Prigerson et al., 1995). Responses to this item provide coarse estimates regarding the frequency of yearning but fail to capture any information about the intensity or dynamics of that experience as it unfolds in daily life.

Ecological Momentary Assessment (EMA; also known as experience sampling methodology) addresses many of the limits of existing measures. EMA entails having participants repeatedly report on their experiences (e.g., thoughts, emotions, and behaviors) in their natural environment, with a focus on assessing the participant’s state at that moment or over a very short period of time (e.g., the past few hours; Shiffman et al., 2008; Stone & Shiffman, 1994). By assessing experiences as they occur, EMA reduces recall bias (Degroote et al., 2020; Moore et al., 2016; Walz et al., 2014). Moreover, sampling participants in their natural environments provides insight into life events or circumstances that may contribute to the occurrence or intensity of symptoms (Yim et al., 2020). In the context of prolonged grief, EMA allows researchers to assess individual components of the prolonged grief syndrome and the situational or social factors that may contribute to them as those experiences play out over the course of people’s day-to-day lives. In doing so, EMA presents an opportunity to go beyond retrospective reports of symptom frequency to understand the dynamics of individual components of the prolonged grief syndrome (e.g., the within-person variance, peak severity, or probability of acute increases in yearning). Further, by gathering rich data on symptoms as they unfold over time in relation to one another, EMA affords the opportunity to examine the within-person structure of relationships among symptoms and how variables of interest correlate with elements of the prolonged grief syndrome within individuals over time (e.g., how loneliness relates to social interaction; Borsboom, 2017; Bringmann et al., 2022; Fisher & Boswell, 2016; Robinaugh et al., 2022; Wichers et al., 2021).

EMA has broadly been found to be acceptable and feasible in clinical populations (Bentley et al., 2021; Glenn et al., 2021; Moitra et al., 2017; Rogers, 2021; Walz et al., 2014; Yim et al., 2020). However, acceptability and feasibility vary across populations and studies (Kim et al., 2020). Given that EMA in the context of grief entails repeatedly asking participants to reflect on potentially distressing thoughts, emotions, and behaviors during a time that can be uniquely painful and preoccupying, it is critical to evaluate whether EMA methods are acceptable and feasible among bereaved adults.

Tentative insight into the potential feasibility and acceptability of EMA comes from studies that have adopted a daily diary assessment approach to studying grief. Like EMA, daily diaries are an ambulatory assessment method that entails repeatedly assessing experiences over time, but the frequency of daily diary assessments (daily) is lower than that typically associated with EMA research (multiple times per day). Daily diary studies of grief have typically utilized paper and pencil measures mailed to participants that are then completed and returned to the researchers via mail (e.g., Eisma et al., 2022; Ong et al., 2004; 2005; Ryckebosch-Dayez et al., 2016). In these studies, many participants who complete initial baseline assessments and are sent daily diary materials for such studies do not return any completed assessments (e.g., 37% of participants in Ryckebosch-Dayez et al., 2016). Among those who return surveys, there is some evidence suggesting that this approach is feasible and acceptable (Eisma et al., 2022). Moreover, daily diary studies of grief suggest that repeated assessment of grief over time is not associated with aversive measurement reactivity, with participants tending to show modest reductions in grief severity over the course of these studies (Eisma et al., 2022, Buyukcan-Tetik et al., 2024). However, while these daily diary studies do provide some insight into how EMA assessments may be received by bereaved adults, it is important to note that the methods utilized in these studies (e.g., paper-and-pencil measures completed daily and surveys delivered via mail) differ from many modern EMA studies, which commonly administer surveys via smartphone several times per day. Accordingly, caution is warranted in drawing inferences from this daily diary work about the feasibility and acceptability of EMA as a tool for studying prolonged grief.

Recently, Lenferink and colleagues took a significant step forward in this line of research by providing the first thorough investigation of the feasibility and acceptability of EMA in this population. EMA surveys were sent to a sample of bereaved adults via smartphone app five times per day for 14 days. Each survey contained 17 items assessing core features of prolonged grief disorder (e.g., “In the past three hours, I found myself yearning for him/her”). EMA was found to be moderately feasible in this population (65% of the participants completed at least 50% of the assessments). In addition, EMA was found to be acceptable for most individuals, with approximately 50% percent of participants reporting having gained some insight about their grief because of their participation in the study, and relatively few (15%) reporting that the phone surveys prompted unexpected emotional issues. Finally, consistent with findings from daily diary research (Eisma et al., 2022, Buyukcan-Tetik et al., 2024), there was no evidence for adverse measurement reactivity, as prolonged grief disorder symptoms tended to decline on traditional self-report assessments of prolonged grief disorder from pre- to post-EMA.

The study by Lenferink and colleagues was an important first step toward evaluating the feasibility and acceptability of EMA among bereaved adults. To extend this work, there are two critical issues in need of further study. Most importantly, because only 2 of the 80 bereaved adults in the Lenferink study met criteria for prolonged grief disorder (PGD), it remains unclear whether EMA is feasible and acceptable in those with high prolonged grief symptom severity. In addition, the findings from Lenferink suggest some cause for concern about the feasibility of EMA in bereaved adults that warrants further study. In particular, Lenferink observed lower mean adherence to the EMA surveys (57% survey completion rate) than has been reported in other EMA studies with psychiatric populations (Porras-Segovia et al., 2020; Vachon et al., 2019). This raises a potential concern that bereaved adults may find EMA protocols more challenging to complete than other individuals. In their discussion of this adherence rate, Lenferink and colleagues note that incentives or alternative study designs may have promoted better adherence, but this possibility remains untested in bereaved adults.

In this paper, we aim to address these gaps in the literature by investigating the feasibility and acceptability of EMA as well as the potential reactivity to EMA in a sample of bereaved adults with low and high severity of prolonged grief symptoms, as assessed by traditional clinical assessments at baseline. We have three aims. First, we evaluate the feasibility of the EMA protocol by investigating the proportion of EMA surveys completed and the proportion of individuals who withdrew from the study during the EMA assessments. Second, we evaluate the acceptability of the EMA surveys by analyzing data collected from a self-report acceptability questionnaire. Third, we examine the potential reactivity to the EMA items by assessing the trajectory of prolonged grief symptoms during the EMA portion of the study. For each aim, we performed analyses for both the overall sample, and separately in those with low and high prolonged grief symptom severity.

Materials and Methods

Participants

Participants were drawn from a study investigating the structure and dynamics of prolonged grief symptoms and physiological activity in bereaved adults. Participants were recruited through social media advertisements, postings on our institution’s research recruitment page, word-of-mouth (e.g., from clinicians, friends, or family members who heard about the study through online advertisements), and referrals from local bereavement support groups and clinicians. The study sample used for this paper includes those participants who completed the EMA portion of the broader study as of January 2023, the point at which we began analyses for this paper. Participants who had not yet completed the EMA portion of the study at that time were excluded from the sample to ensure appropriate assessment of adherence. Participants in this sample were enrolled between May 2019 and December 2022.

Participants were between the ages of 18 and 65 and experienced a loss at least 6 months prior to enrollment. All participants had access to a smartphone for survey completion, access to a laptop or computer with Internet connection to upload psychophysiological data collected from an ambulatory device and had stable treatment for at least 8 weeks prior to enrollment. Exclusion criteria included a history of schizophrenia, schizoaffective disorder, any psychotic symptoms or disorder, symptoms of mania or bipolar disorder, neurological disorder, alcohol or substance use disorder within the proceeding 6 months, moderate or greater suicide risk, use of any anticholinergic medications, tricyclic antidepressants, or “as needed” benzodiazepines, 2 weeks before enrollment, or any unstable medical illness requiring intervention or likely to interfere with study participation.

Procedures

Individuals who responded to the recruitment efforts listed above were initially assessed for eligibility via phone by trained research coordinators. During this initial contact, the research coordinator also provided information about the study, including the ecological momentary assessment procedures. Eligible participants were then sent a copy of the consent form to review and were scheduled for an initial screening visit (Visit 1). All study procedures were reviewed and approved by Mass General Brigham’s Institutional Review Board (Protocol #2017P002775).

Visit 1

The initial study visit began with informed consent, during which participants were again informed about the EMA survey procedures and were given an opportunity to ask questions or express concerns about this portion of the study. Understanding of the EMA procedures was confirmed with a brief series of True/False questions immediately following consent, after which participants completed self-report measures, including a demographics questionnaire and the Inventory of Complicated Grief (Prigerson et al., 1995, note: the syndrome now referred to as prolonged grief was previously referred to as complicated grief). Doctoral-level study staff then administered three structured interviews assessing other mental health symptoms. For those eligible to participate, a research coordinator then reviewed further details regarding EMA procedures, including reviewing definitions for each EMA item, providing an instruction guide on EMA procedures for reference, and sending test surveys to participants’ smartphones to ensure that notifications were received.

Ecological Momentary Assessment

Following Visit 1, eligible participants completed 17 days of data collection. REDCap, a HIPAA compliant data capture platform, was programmed to send SMS (text) messages containing a survey link six times a day for 17 days between the hours of 9:00 AM and 9:00 PM, for a total of 102 assessments. Each survey contained 12 items in which participants rated the extent to which symptoms of bereavement-related psychopathology were present (since the last survey or, if it was the first survey of the day, since the beginning of their day) on a sliding scale from 0 to 100. Participants were given 60 minutes to complete the survey and were encouraged to complete the survey as soon as possible after they received their text message. If participants had not completed the survey after 30 minutes, they were sent a reminder to complete the survey. Throughout the EMA data collection period, a research coordinator checked EMA responses at the end of each day and would contact participants if compliance fell below 80% on consecutive days. During the EMA data collection period, participants also wore an Empatica E4 wristband to assess psychophysiology. Participants wore the device from 9:00 AM until 9:00 PM each day for the 17 days.

Visit 2

Once the 17-day data collection period concluded, participants completed a second visit in which a subset of the self-report and clinician-administered assessments were repeated alongside a 10-item self-report measure on the acceptability of the EMA procedures. For some participants, this visit also included an in-person assessment of physiological reactivity. Not all participants completed this in-person assessment of physiological reactivity due to COVID-19 restrictions. The in-person visit occurred at Massachusetts General Hospital in Boston, Massachusetts.

Additional Study Procedures and Study Compensation

Participants completed a brief online follow-up survey 6 months after the second study visit containing self-report measures of prolonged grief, post-traumatic stress disorder, and depression. No data from that follow-up visit will be reported here. Participants were compensated $200 for participation in the study ($50 for each of the two study visits, $75 for completing the EMA portion of the study, and $25 for completing the six-month follow-up online survey). In addition, participants who completed 95% or more of the EMA surveys received a $50 bonus.

Measures

Inventory of Complicated Grief

The Inventory of Complicated Grief (ICG; Prigerson et al., 1995) is a 19-item assessment that asks participants about their recent bereavement-related emotions, thoughts, and behaviors. If participants had experienced more than one loss at least 6 months ago, they were asked to identify the loss that was currently causing them the most distress on a day-to-day basis. Responses consist of a 5-point Likert-like scale ranging from 0 (“Never”) to 4 (“Always”). A score of greater than or equal to 30 on this inventory is commonly used to indicate probable prolonged grief (Shear et al., 2016). In this study, we use this cut-off to define low vs. high prolonged grief symptom severity.

Ecological Momentary Assessment Survey

During the EMA data collection period, participants were asked to complete a 12-item survey 6 times per day for 17 days. The survey queried five emotions (i.e., yearning, emotional pain, positive emotions, loneliness, anxiety), three thoughts (i.e., thoughts related to the deceased loved one, the loved one’s death, or plans for the future), and four behaviors (i.e., activities that make participants feel closer to their deceased loved one, avoidance of things that remind participants of their deceased loved ones, engagement in leisure activities, and engagement in social activities; Jordan and Litz, 2014). Participants responded to questions on a sliding scale ranging from 0 (complete absence of a particular thought, feeling, or behavior) to 100 (the greatest intensity of an emotion, preoccupation with a thought, or engagement with a behavior ever experienced). All EMA items and the definitions for each item provided to participants appear in Supplementary Materials A.

EMA Acceptability Questionnaire

To assess the acceptability of EMA, participants were asked 10 questions regarding their experience throughout the data collection period, including questions focused on the acceptability of the frequency and length of the EMA surveys, perceived benefits, and perceived distress associated with completing the EMA assessments. Participants provided their level of agreement with each question item on a 7-point Likert scale ranging from 1 (strongly disagree) to 7 (strongly agree). All items appear in Figure 2.

Figure 2:

Figure 2:

Open in a new tab

The proportion of individuals endorsing each response option for each item of the EMA Acceptability Questionnaire.

Results

Sample Characteristics

Participants (n=117) predominately identified as female (n=100;85%), White (n=101;86%), and non-Hispanic/Latino (n=111; 95%). The age range of the sample was 20–65 years old with an average age of 46.39 and median of 49.26. There were 62 participants who had low prolonged grief symptom severity, and there were 55 participants with high prolonged grief symptom severity.

Feasibility

The mean proportion of completed EMA surveys in the full sample (n=117) was 90% (SD=20), with a median completion rate of 96% and modal completion rate of 99%. Seven participants (6% of the sample, 5 of whom had high prolonged grief symptom severity) withdrew from the EMA portion of the study. Among those who withdrew, three reported a lack of time to complete EMA, two reported withdrawing because of the emotions associated with completing the EMA procedures, and two were lost to follow-up and did not provide a reason for why they withdrew. Those who withdrew tended to do so early in the EMA data collection period, thereby completing very few assessments and skewing the overall distribution. Excluding those who withdrew from the EMA portion of the study, the mean adherence was 94% (SD = 8 with median=97%, and mode=99%.

The distributions of adherence to the EMA surveys for those with low vs. high prolonged grief symptom severity appears in Figure 1 and is described further in Supplementary Materials B. Those with high prolonged grief symptom severity completed a lower, though still high, proportion of assessments (M=86%) than those without (M=93%), t(77.61) = 2.07, p=0.04. This difference was no longer present when excluding those who withdrew from the EMA portion of the study (high prolonged grief symptom severity M=93%; low prolonged grief symptom severity M=95%, t(63.17) =1.60, p=0.12), suggesting that this group difference is driven by the greater number of people in the high prolonged grief symptom severity group who withdrew participation (n=5) relative to the number low prolonged grief symptom severity who withdrew (n=2).

Figure 1:

Figure 1:

Open in a new tab

The proportion of surveys completed and its relationship to prolonged grief symptoms. Panel A depicts the histograms for participants with low and high prolonged grief symptoms (depicted in blue and purple, respectively). Purple areas of the histogram signify points of overlap between these groups. The vertical dashed line identifies the 95% survey completion mark. Participants in the current study received additional compensation if they completed at least 95% of EMA surveys. Panel B depicts the relationship between prolonged grief symptom severity and the proportion of surveys completed. The blue line is the regression line for prolonged grief as a predictor of survey completion.

When treating prolonged grief as a dimensional variable, we similarly found a small negative association between prolonged grief symptom severity and adherence, r(115)=−.20, 95% CI [−.37,.02], p=.03, that was no longer significant when removing those who withdrew from the study, r(108)=−.10, 95% CI, [−0.28,0.09], p=.31.

Acceptability

The proportion of individuals endorsing each response option for each item of the EMA Acceptability Questionnaire appears in Figure 2. There were three types of questions. First, three items were concerned with the acceptability of specific aspects of the EMA protocol, including application usage, frequency of assessment, and the number of questions. Participants reported an average response between “agree” and “strongly agree” for the ease of use (M=6.38, SD=0.88) and between “somewhat agree” and “agree” for assessment frequency and the number of questions per survey (M=5.43, SD=1.40 and M=5.86, SD=1.12, respectively). Second, three items concerned the personal or societal benefit of the surveys. On average, participants reported between being neutral toward or somewhat agreeing with the statement “responding to the survey was helpful” (M=4.47, SD=1.48) and agreed that “my responses are an important part of the study” and “my responses will provide valuable information” (M=5.92, SD=0.99, and M=5.65, SD=1.05, respectively). Finally, three items concerned the potential burden or distress caused by completing the EMA surveys. On average, participants were between “somewhat disagree” and “disagree” for the statements “responding to the survey was burdensome” and “responding to the questions took a lot of time out of my day” (M=2.96, SD=1.53 and M=2.50, SD=1.41, respectively) and between “disagree” and “neutral” for the item “responding to the survey made me feel more distressed” (M=3.35, SD=1.64). Please see Supplementary Materials for three tables that detail valid percentages for each answer category on the Acceptability Questionnaire. One table demonstrates these percentages for individuals with high prolonged grief symptom severity, one table demonstrates these percentages for individuals with low prolonged grief symptom severity, and one table demonstrates the valid percentages in the overall sample.

The proportion of individuals endorsing each response option broken down by those with low and high prolonged grief symptom severity appears in Supplementary Materials C. Those with high prolonged grief symptom severity rated stronger average agreement with the notion that surveys were personally relevant t(108.32)=−2.29, p=.02. There were no differences between groups for any other item.

When examining prolonged grief as a dimensional variable, prolonged grief symptom severity was positively correlated with the extent to which participants felt their responses provided valuable information (r(111)=0.26, 95% CI [0.08,0.42], p=0.01) and were important to the study (r(111)=0.22 [0.04,0.39], p=0.02), and the extent to which items were deemed personally relevant (r(110)=0.25 [0.07,0.41], p=0.01). There was also a weak, negative relationship between prolonged grief symptom severity and the acceptability of survey frequency (r(111)=−0.19 [−0.036,−0.01], p=0.04). There was not a statistically significant relationship between prolonged grief symptom severity and any of the items related to potential burden of completing the EMA protocol, including the item relating to survey-related distress |rs|<.09, ps>=.35.

Reactivity

A plausible concern with EMA is that repeated assessment will have iatrogenic effects, particularly for vulnerable populations (Chen et al., 2024; Glenn et al., 2022). It is, therefore, crucial to understand potential reactivity effects of repeatedly responding to EMA items each day that focus on grief symptoms. As such, we investigated this possibility using a multilevel model wherein each variable was regressed on time at the within-person level. For the sample as a whole, we did not observe significant time trends for yearning, positive emotions, thoughts about the future, or grief-related approach behaviors (e.g., spending time with things that remind one of the deceased; ps > .05). For the remaining symptoms, we saw small but significant reductions over time in psychological pain (b=−.01, p=.009), loneliness (b=−.03, p<.001), anxiety (b=−.01, p=.009), thoughts of the deceased (b=−.02, p<.001), thoughts about the death (b=−.01, p=.009), and grief-related avoidance behaviors (e.g., avoiding reminders of the loss; b=−.02, p<.001). These regression coefficients suggest a reduction of roughly 1–3 points on the 100-point scale over the course of the EMA data collection period. Similarly, we observed a small but statistically significant increase in social activity (b=.04, p<.001) and leisure activities (b=.02, p=.03), suggesting an improvement of roughly 4 points over the EMA data collection period).

We next investigated whether prolonged grief symptom severity moderated these findings by examining an interaction between prolonged grief symptoms and time as predictors of each variable. As shown in Figure 3, there were modest differences in the slope of change in emotional pain, loneliness, anxiety, thoughts about the future, avoidant behaviors (avoiding reminders of deceased loved one), positive emotions, and approach behaviors (seeking reminders of deceased loved one) in those with high prolonged grief symptom severity versus those without (ps<.01). However, in both groups, slopes suggest minimal change in any EMA item.

Figure 3:

Figure 3:

Open in a new tab

This figure depicts the estimated relationship between time and each item of the EMA survey for each participant as well as the group averages of those estimated relationships for those with high prolonged grief symptom severity (purple) and those without (blue). For each EMA survey item, the average slope is relatively flat for both those with and without high prolonged grief, suggesting minimal relationship between time and EMA responses. Accordingly, there is no evidence for any worsening of symptoms as a result of completing EMA.

Although there was no evidence for systematic worsening of symptoms, two participants who withdrew from the EMA portion of the study identified their psychological response to completing the surveys as the reason for withdrawal. The first such participant perceived a greater than anticipated increase in the extent to which grief had become a focus of their thoughts since beginning the surveys and felt it was in their best interest to withdraw. The second participant similarly reported that EMA surveys increased their awareness of how much grief and loneliness they were experiencing and that, as a result, the surveys had become a source of distress. In both cases, participants completed the remainder of the study procedures and there was no evidence of clinical worsening (e.g., prolonged grief symptom severity stayed the same or reduced from Visit 1 to Visit 2 for both participants).

Discussion

In this study, we examined EMA feasibility, acceptability, and measurement reactivity in bereaved adults with low and high prolonged grief severity. Our findings suggest that EMA in bereaved adults is feasible and acceptable. The mean proportion of assessments completed was 90% (median = 96%), and only 6% of participants withdrew from the EMA portion of the study. Most participants reported finding the logistics of the EMA procedures acceptable, denied burden associated with the EMA surveys, and many reported some personal or societal benefit from completing the surveys. In addition, we found no evidence of adverse measurement reactivity to EMA in the overall sample.

This general pattern of findings held when separately examining those with low vs. high prolonged grief symptom severity (defined as a score of 30 or higher on the Inventory of Complicated Grief). Adherence was somewhat lower in those with high prolonged grief symptom severity relative to those with low symptom severity (M = 86% and 93%, respectively). Nonetheless, adherence was still in the upper range of that commonly reported in EMA studies with clinical populations (Vachon et al., 2019). There was a small negative correlation between prolonged grief symptom severity and the perceived acceptability of the frequency of survey assessments, suggesting that those with higher prolonged grief symptom severity found the frequency of assessments modestly more challenging. However, prolonged grief symptom severity was not associated with finding the surveys less acceptable in any other way, and acceptability of all aspects of the EMA procedure, including assessment frequency, remained high. Moreover, prolonged grief symptom severity was positively associated with perceived personal relevance and perceived societal benefit from completing the EMA surveys, suggesting that those with high prolonged grief symptom severity are more likely to feel that their responses to the EMA surveys play an important role in advancing our understanding of grief.

These findings extend prior work on EMA in bereaved adults in two important ways. This study is the first to examine and compare EMA feasibility, acceptability, and measurement reactivity among bereaved adults with high and low prolonged grief symptom severity. Our findings suggest a somewhat nuanced picture. Participants with high prolonged grief symptom severity exhibited modestly poorer adherence and reported modestly greater burden from the EMA surveys than did those without high prolonged grief symptom severity. However, acceptability and feasibility remained high among those with high prolonged grief symptom severity, and there was a positive relationship between prolonged grief symptom severity and the extent to which participants felt the surveys were helpful and their responses were important. Similarly, although two participants with elevated symptoms withdrew from the EMA portion of the study and identified their psychological response to the EMA surveys as the reason for withdrawal, these participants showed no evidence of clinical worsening and, on average, those with high prolonged grief symptom severity exhibited a very slight tendency for a decline in the severity of several symptoms during the EMA assessment period. Although this decline was minimal, the possibility that reporting on symptoms in this way may hold some benefit is consistent with the emphasis placed on grief-monitoring in evidence-based treatments for prolonged grief (Shear et al., 2013), with findings from prior EMA (Lenferink et al., 2022), and with findings from prior daily diary research (Eisma et al., 2022, Buyukcan-Tetik et al., 2024). Thus, participating in EMA-based research studies may be both modestly more burdensome but also modestly more beneficial and meaningful for those experiencing high prolonged grief symptom severity. For this reason, our findings suggest that EMA is acceptable, feasible, and safe among those with elevated prolonged grief symptom severity, while also serving as a reminder that care should still be taken to ensure appropriate support is available for participants in those instances in which there is perceived burden or distress.

Second, we found higher adherence in our study (M=90% survey completion and 94% participant retention) than was reported in the one prior study of EMA adherence among bereaved adults (M=60% survey completion and 65% participant retention; Lenferink et al., 2022). In that study, Lenferink and colleagues posited that greater time intervals between measures, fewer assessments per day, and higher incentives may promote higher compliance rates in a bereaved adult population. In the current study, the interval between measures was lower, and the number of assessments per day was higher than in the study by Lenferink, suggesting that frequency of assessments was not the reason for the greater adherence. However, the current study did have greater incentives for EMA survey completion, both in regard to the compensation for participation in the overall study ($200 vs. opportunity to win €50 via raffle) and the inclusion of additional compensation if participants completed a high proportion of the EMA surveys ($50 for completing at least 95% of the phone surveys vs. no opportunity for additional compensation). Notably, although meta-analyses of EMA studies consistently find that providing some compensation improves adherence relative to providing none (Jones et al., 2019; Wrzus and Neubauer et al., 2023), multiple meta-analyses have found no association between the amount of compensation and rates of adherence (Jones et al., 2019; Wen et al., 2017; Ottenstein & Werner, 2021; Wrzus and Neubauer et al., 2023). Accordingly, to promote adherence to EMA surveys in the context of grief research, choosing to provide some direct compensation may be a more impactful study design decision than determining how much to provide.

In addition to the potential impact of financial incentive, differences in study procedure may account for the adherence differences observed across these two EMA studies. Participants in the current study completed screening assessments at two time points prior to enrollment (one via phone and one conducted either in-person or via video conference), relative to one visit prior to enrollment in the study by Lenferink and colleagues. This additional visit may have led to a participant pool more committed to study participation. Overall, the survey completion rate in the current study suggests that high adherence to EMA surveys in bereaved samples is possible and that providing some direct compensation and/or incorporating multiple points of personal contact during the screening phase may be among the factors that help promote EMA adherence in this population.

There are some limitations of the present study. First, in a small number of instances, participants reported to us that they either failed to receive a survey or were not able to submit a completed survey at some point during the study. These instances likely arose due to idiosyncratic technical difficulties (e.g., lack of cellular service at the time the survey was sent or issues with the text message delivery service) and may have artificially lowered adherence rates to a small extent, as the failure to complete the survey in these instances may not have been due to participants’ oversight, but due to technical challenges beyond their control. Second, the sample was relatively homogeneous, including 101 (86%) participants who identified as White and 100 (85%) who identified as female. Accordingly, it is unclear how well the current findings would generalize to other populations with other demographic characteristics. Extending this work to other populations will be critical to future research investigating acceptability and feasibility of EMA in bereaved adults. Third, in the current study, participants were asked to wear a wristband monitoring psychophysiology throughout the period they were completing EMA surveys. This additional study procedure, which sometimes presented technical challenges independent of the EMA surveys, may have adversely affected the retention of participants and led to lower adherence than would have been observed with EMA procedures alone. Fourth, we used the Inventory of Complicated Grief (ICG) to assess prolonged grief symptom severity. Although the ICG has frequently been used to assess prolonged grief symptom severity, including in large clinical trials of prolonged grief disorder treatment (Shear et al., 2005; Shear et al., 2001), it does not precisely reflect symptoms as defined in DSM-5-TR or ICD-11, potentially limiting the generalizability of our findings to those with prolonged grief disorder as defined by these diagnostic manuals.

Conclusion

Our results suggest that EMA is feasible and acceptable in bereaved adults with and without high levels of prolonged grief symptom severity. We found no evidence that participation in the EMA surveys leads to a systematic worsening of symptoms. These findings add to the literature suggesting that EMA is feasible, acceptable, and safe in bereaved adults (Lenferink et al., 2022) and in populations characterized by psychological distress more broadly (Gershon et al., 2019; Porras-Segovia et al., 2022; Vachon et al., 2019). In addition, we found that participants often perceived personal or societal benefit to participating in the study, and that this tendency was somewhat greater among those with high prolonged grief symptom severity. Overall, these findings suggest that EMA can be safely and feasibly administered to bereaved adults with and without high prolonged grief symptom severity. Given the substantial advantages of this methodology for helping us better understand the structure and dynamics of grief as it unfolds over day-to-day life, the findings from this study call for greater use of this method in the study of grief.

Supplementary Material

Supplementary Material

Table 1.

Demographic characteristics of the study sample (n=117)

Demographic Characteristics
Age M (SD)
Age in years 46.39 (13.94)
Gender N (%)
Female 100 (85)
Male 16 (14)
Non-binary 1 (1)
Ethnicity N (%)
Hispanic or Latino 6 (5)
Non-Hispanic or Latino 111 (95)
Race 1 N (%)
White 101 (86)
Black or African American 13 (11)
Asian 5 (4)
Native Hawaiian or Other Pacific Islander 2 (2)
Native American/Alaskan Native 0 (0)
Other 1 (1)
Marital Status N (%)
Single 43 (37)
Living with Partner 4 (3)
Married 23 (20)
Divorced 10 (9)
Widowed 34 (29)
Separated 3 (3)
Highest Education Level N (%)
Graduate School 47 (40)
College Graduate 48 (41)
Partial College 18 (15)
High School Graduate 3 (3)
Partial High School 1 (1)
Present Occupational Status 2 N (%)
Working full-time 73 (62)
Working part-time 18 (15)
Unemployed or laid off 7 (6)
Looking for work 6 (5)
Keeping house or raising children full-time 1 (1)
Retired 11 (9)
Open in a new tab
1

Participants could select more than one category.

2

One participant did not complete this section.

Acknowledgements

This work was supported by a National Institute of Mental Health Career Development Award (K23MH113805). The content is solely the responsibility of the authors and does not necessarily represent the views of this organization.

Footnotes

Declaration of Interest

In the past 3 years Dr. Simon reports receiving grants from the National Institutes of Health (NIH), American Foundation for Suicide Prevention, Patient-Centered Outcomes Research Institute, Ananda Scientific and support from Cohen Veterans Network and MindMed; receiving personal fees from Genomind, Cerevel; receiving fees or royalties from Wiley (Deputy Editor Depression and Anxiety), Wolters Kluwer (UpToDate) and APA Publishing; and having spousal stock from G1 Therapeutics and Zentalis outside the submitted work.

Ethical Statement

All study procedures were reviewed and approved by Mass General Brigham’s Institutional Review Board (Protocol #2017P002775).

References

  1. Bentley KH, Coppersmith DL, Kleiman EM, Nook EC, Mair P, Millner AJ, Reid Russell A, Wang SB, Fortgang RG, Stein MB, et al. (2021). Do patterns and types of negative affect during hospitalization predict short-term post-discharge suicidal thoughts and behaviors? Affective science, 2(4), 484–494. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Borsboom D (2017). A network theory of mental disorders. World psychiatry, 16(1), 5–13. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Bringmann LF, Albers C, Bockting C, Borsboom D, Ceulemans E, Cramer A, Epskamp S, Eronen MI, Hamaker E, Kuppens P, et al. (2022). Psychopathological networks:Theory, methods and practice. Behaviour Research and Therapy, 149, 104011. [DOI] [PubMed] [Google Scholar]
  4. Buyukcan-Tetik A, Topal MA, Deryalar B, Ergun TD, Aydin EN, & Aykutoglu B (2024). Bereaved Parents’ Change in Mental Health during a Dyadic Diary. Journal of Loss and Trauma, 29(4), 359–376. [Google Scholar]
  5. Bui E, Mauro C, Robinaugh DJ, Skritskaya NA, Wang Y, Gribbin C, … & Shear MK (2015). The structured clinical interview for complicated grief: reliability, validity, and exploratory factor analysis. Depression and anxiety, 32(7), 485–492. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Chen T, Niu L, Zhu J, Hou X, Tao H, Ma Y, … & Zhou L (2024). Effects of frequent assessments on the severity of suicidal thoughts: an ecological momentary assessment study. Frontiers in public health, 12, 1358604. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Clayton PJ (1973). The clinical morbidity of the first year of bereavement: A review. Comprehensive Psychiatry. [DOI] [PubMed] [Google Scholar]
  8. Degroote L, DeSmet A, De Bourdeaudhuij I, Van Dyck D, & Crombez G (2020). Content validity and methodological considerations in ecological momentary assessment studies on physical activity and sedentary behaviour: A systematic review. International Journal of Behavioral Nutrition and Physical Activity, 17(1), 1–13. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Djelantik AMJ, Robinaugh DJ, Kleber RJ, Smid GE, & Boelen PA (2020). Symptomatology following loss and trauma: Latent class and network analyses of prolonged grief disorder, posttraumatic stress disorder, and depression in a treatment-seeking trauma exposed sample. Depression and Anxiety, 37(1), 26–34. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Eisma MC, Franzen M, Paauw M, Bleeker A, & aan het Rot M (2022). Rumination, worry and negative and positive affect in prolonged grief: A daily diary study. Clinical psychology & psychotherapy, 29(1), 299–312. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Fisher AJ, & Boswell JF (2016). Enhancing the personalization of psychotherapy with dynamic assessment and modeling. Assessment, 23(4), 496–506. [DOI] [PubMed] [Google Scholar]
  12. Gershon A, Kaufmann CN, Torous J, Depp C, & Ketter TA (2019). Electronic ecological momentary assessment (ema) in youth with bipolar disorder: Demographic and clinical predictors of electronic ema adherence. Journal of Psychiatric Research, 116, 14–18. [DOI] [PubMed] [Google Scholar]
  13. Glenn CR, Kleiman EM, Kearns JC, Boatman AE, Conwell Y, Alpert-Gillis LJ, & Pigeon W (2021). Sleep problems predict next-day suicidal thinking among adolescents: A multimodal real-time monitoring study following discharge from acute psychiatric care. Development and Psychopathology, 33(5), 1701–1721. [Google Scholar]
  14. Glenn CR, Kleiman EM, Kearns JC, Santee AC, Esposito EC, Conwell Y, & Alpert-Gillis LJ (2022). Feasibility and acceptability of ecological momentary assessment with high-risk suicidal adolescents following acute psychiatric care. Journal of Clinical Child & Adolescent Psychology, 51(1), 32–48. [DOI] [PubMed] [Google Scholar]
  15. Jacobs S, & Ostfeld A (1977). An epidemiological review of the mortality of bereavement. Psychosomatic medicine, 39(5), 344–357. [DOI] [PubMed] [Google Scholar]
  16. Jones A, Remmerswaal D, Verveer I, Robinson E, Franken IH, Wen CKF, & Field M (2019). Compliance with ecological momentary assessment protocols in substance users: A meta‐analysis. Addiction, 114(4), 609–619. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Jordan AH, & Litz BT (2014). Prolonged grief disorder: Diagnostic, assessment, and treatment considerations. Professional Psychology: Research and Practice, 45(3), 180. [Google Scholar]
  18. Keyes KM, Pratt C, Galea S, McLaughlin KA, Koenen KC, & Shear MK (2014). The burden of loss: Unexpected death of a loved one and psychiatric disorders across the life course in a national study. American Journal of Psychiatry, 171(8), 864–871. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Kim H, Kim S, Kong SS, Jeong Y-R, Kim H, Kim N, et al. (2020). Possible application of ecological momentary assessment to older adults’ daily depressive mood: Integrative literature review. JMIR mental health, 7(6), e13247. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Lenferink L, van Eersel J, & Franzen M (2022). Is it acceptable and feasible to measure prolonged grief disorder symptoms in daily life using experience sampling methodology? Comprehensive Psychiatry, 152351. [DOI] [PubMed] [Google Scholar]
  21. Lundorff M, Holmgren H, Zachariae R, Farver-Vestergaard I, & O’Connor M (2017). Prevalence of prolonged grief disorder in adult bereavement: A systematic review and meta-analysis. Journal of affective disorders, 212, 138–149. [DOI] [PubMed] [Google Scholar]
  22. Maccallum F, Malgaroli M, & Bonanno GA (2017). Networks of loss: Relationships among symptoms of prolonged grief following spousal and parental loss. Journal of Abnormal Psychology, 126(5), 652. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Moitra E, Gaudiano BA, Davis CH, & Ben-Zeev D (2017). Feasibility and acceptability of post-hospitalization ecological momentary assessment in patients with psychotic-spectrum disorders. Comprehensive psychiatry, 74, 204–213. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Moore RC, Depp CA, Wetherell JL, & Lenze EJ (2016). Ecological momentary assessment versus standard assessment instruments for measuring mindfulness, depressed mood, and anxiety among older adults. Journal of psychiatric research, 75, 116–123. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Ong AD, Bergeman CS, & Bisconti TL (2005). Unique effects of daily perceived control on anxiety symptomatology during conjugal bereavement. Personality and Individual Differences, 38(5), 1057–1067. [Google Scholar]
  26. Ong AD, Bergeman CS, & Bisconti TL (2004). The role of daily positive emotions during conjugal bereavement. The Journals of Gerontology Series B: Psychological Sciences and Social Sciences, 59(4), P168–P176. [DOI] [PubMed] [Google Scholar]
  27. Ottenstein C, & Werner L (2022). Compliance in ambulatory assessment studies: Investigating study and sample characteristics as predictors. Assessment, 29(8), 1765–1776 [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Porras-Segovia A, Dıaz-Oliván I, Barrigón ML, Moreno M, Artés-Rodrıguez A, Pérez-Rodrıguez MM, & Baca-Garcıa E (2022). Real-world feasibility and acceptability of real-time suicide risk monitoring via smartphones: A 6-month follow-up cohort. Journal of Psychiatric Research, 149, 145–154. [DOI] [PubMed] [Google Scholar]
  29. Porras-Segovia A, Molina-Madueño RM, Berrouiguet S, López-Castroman J, Barrigón ML, Pérez-Rodrıguez MS, Marco JH, Dıaz-Oliván I, de León S, Courtet P, et al. (2020). Smartphone-based ecological momentary assessment (ema) in psychiatric patients and student controls: A real-world feasibility study. Journal of Affective Disorders, 274, 733–741. [DOI] [PubMed] [Google Scholar]
  30. Prigerson HG, Boelen PA, Xu J, Smith KV, & Maciejewski PK (2021). Validation of the new DSM‐5‐TR criteria for prolonged grief disorder and the PG‐13‐Revised (PG‐13‐R) scale. World Psychiatry, 20(1), 96–106. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Prigerson HG, Maciejewski PK, Reynolds CF III, Bierhals AJ, Newsom JT, Fasiczka A, Frank E, Doman J, & Miller M (1995). Inventory of complicated grief: A scale to measure maladaptive symptoms of loss. Psychiatry research, 59(1–2), 65–79. [DOI] [PubMed] [Google Scholar]
  32. Robinaugh DJ, Toner ER, & Djelantik AMJ (2022). The causal systems approach to prolonged grief: Recent developments and future directions. Current Opinion in Psychology, 44, 24–30. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Rogers ML (2021). Feasibility and acceptability of ecological momentary assessment in a fully online study of community-based adults at high risk for suicide. Psychological assessment. [DOI] [PubMed] [Google Scholar]
  34. Rosner R, Pfoh G, Kotoučová M, & Hagl M (2014). Efficacy of an outpatient treatment for prolonged grief disorder: A randomized controlled clinical trial. Journal of Affective Disorders, 167, 56–63. [DOI] [PubMed] [Google Scholar]
  35. Ryckebosch-Dayez AS, Zech E, Mac Cord J, & Taverne C (2016). Daily life stressors and coping strategies during widowhood: A diary study after one year of bereavement. Death Studies, 40(8), 461–478. [DOI] [PubMed] [Google Scholar]
  36. Shear MK, Frank E, Foa E, Cherry C, Reynolds CF III, Vander Bilt J, & Masters S (2001). Traumatic grief treatment: A pilot study. American Journal of Psychiatry, 158(9), 1506–1508. [DOI] [PubMed] [Google Scholar]
  37. Shear K, Frank E, Houck PR, & Reynolds CF (2005). Treatment of complicated grief: a randomized controlled trial. Jama, 293(21), 2601–2608. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Shear MK, Ghesquiere A, & Glickman K (2013). Bereavement and complicated grief. Current psychiatry reports, 15, 1–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Shear MK, Reynolds CF, Simon NM, Zisook S, Wang Y, Mauro C, Duan N, Lebowitz B, & Skritskaya N (2016). Optimizing treatment of complicated grief: A randomized clinical trial. JAMA psychiatry, 73(7), 685–694. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Shiffman S, Stone AA, & Hufford MR (2008). Ecological momentary assessment. Annu. Rev. Clin. Psychol, 4, 1–32. [DOI] [PubMed] [Google Scholar]
  41. Stone AA, & Shiffman S (1994). Ecological momentary assessment (ema) in behavorial medicine. Annals of behavioral medicine. [Google Scholar]
  42. Treml J, Brähler E, & Kersting A (2022). Prevalence, factor structure and correlates of dsm-5-tr criteria for prolonged grief disorder. Frontiers in Psychiatry, 13. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Vachon H, Viechtbauer W, Rintala A, & Myin-Germeys I (2019). Compliance and retention with the experience sampling method over the continuum of severe mental disorders: Metaanalysis and recommendations. Journal of medical Internet research, 21(12), e14475. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Walz LC, Nauta MH, & aan het Rot M (2014). Experience sampling and ecological momentary assessment for studying the daily lives of patients with anxiety disorders: A systematic review. Journal of anxiety disorders, 28(8), 925–937. [DOI] [PubMed] [Google Scholar]
  45. Wen CKF, Schneider S, Stone AA, & Spruijt-Metz D (2017). Compliance with mobile ecological momentary assessment protocols in children and adolescents: a systematic review and meta-analysis. Journal of medical Internet research, 19(4), e132. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Wichers M, Riese H, Hodges TM, Snippe E, & Bos FM (2021). A narrative review of network studies in depression: What different methodological approaches tell us about depression. Frontiers in psychiatry, 12, 719490. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Wrzus C, & Neubauer AB (2023). Ecological momentary assessment: A meta-analysis on designs, samples, and compliance across research fields. Assessment, 30(3), 825–846. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Yim SJ, Lui LM, Lee Y, Rosenblat JD, Ragguett R-M, Park C, Subramaniapillai M, Cao B, Zhou A, Rong C, et al. (2020). The utility of smartphone-based, ecological momentary assessment for depressive symptoms. Journal of Affective Disorders, 274, 602–609. [DOI] [PubMed] [Google Scholar]
  49. Zinzow HM, Rheingold AA, Hawkins AO, Saunders BE, & Kilpatrick DG (2009). Losing a loved one to homicide: Prevalence and mental health correlates in a national sample of young adults. Journal of Traumatic Stress: Official Publication of The International Society for Traumatic Stress Studies, 22(1), 20–27. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Zisook S, Iglewicz A, Avanzino J, Maglione J, Glorioso D, Zetumer S, Seay K, Vahia I, Young I, Lebowitz B, et al. (2014). Bereavement: Course, consequences, and care. Current psychiatry reports, 16, 1–10. [DOI] [PubMed] [Google Scholar]
  51. Zisook S, Reynolds CF III, Pies R, Simon N, Lebowitz B, Madowitz J, Tal-Young I, & Shear MK (2010). Bereavement, complicated grief, and dsm, part 1: Depression. The Journal of clinical psychiatry, 71(7), 1871. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Zisook S, Simon NM, Reynolds CF III, Pies R, Lebowitz B, Young IT, Madowitz J, & Shear MK (2010). Bereavement, complicated grief, and dsm, part 2: Complicated grief. The Journal of clinical psychiatry, 71(8), 1869. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary Material
NIHMS2037514-supplement-Supplementary_Material.pdf (368.4KB, pdf)

RESOURCES