Skip to main content
NCBI home page
Wiley Open Access Collection logoLink to Wiley Open Access Collection
. 2025 Oct 8;31(1):33–40. doi: 10.1111/camh.70039

Advancing the measurement of psychotherapy outcomes for youth with irritability using in vivo ecological momentary assessment: an exploratory observational study

Reut Naim 1,2,✉, Urmi Pandya 3, Shannon Shaughnessy 4, Ramaris E German 5, Lauren M Henry 5, Katharina Kircanski 5, Melissa A Brotman 5
PMCID: PMC12832213  PMID: 41059673

Abstract

Background

Ecological momentary assessment (EMA) can reveal naturalistic within‐person changes in symptoms and behaviors, essential for examining psychotherapy's effectiveness. However, no prior study has leveraged EMA to assess real‐time, naturalistic changes of clinical symptoms during psychotherapy in youth with clinically impairing irritability. The present study uses EMA to assess changes in both child‐reported irritability symptoms and caregiver‐reported parental behaviors over the course of treatment, as well as to explore real‐time associations between changes in parental behaviors and youth symptoms.

Method

Here, in an observational analysis, a total of 39 youth (M age = 11.26 years [SD = 1.89 years]; 35.9% female; 79.5% White) undergoing exposure‐based cognitive behavioral therapy, and their parent undergoing parent management skill training (PMT), completed EMA surveys three times a day for 7 days at pre‐, mid‐, and post‐treatment. Given the nested nature of the data, multilevel analyses were conducted to observe change and associations over time.

Results

Results from multilevel modeling showed that both youth and parents reported a statistically significant decrease in real‐time irritability symptoms over the course of treatment (βs ≤ −.27, ps ≤ .02). Parents reported a statistically significant increase in adaptive parental behaviors as taught during the PMT portion of treatment (βs ≥ |.22|, ps < .04). Furthermore, statistically significant associations were found between increased levels of adaptive parental behaviors and decreased levels of youth irritability symptoms.

Conclusion

This study supports the utility of EMA in measuring psychotherapy symptom progress, outcomes, and change in therapy‐targeted behavior. Results suggest that parental behavior could play a substantively important role in the treatment of youth with irritability.

Keywords: Ecological momentary assessment, irritability, cognitive behavioral therapy, parent management training, youth, parent behavior management

Key Practitioner Message.

  • Ecological Momentary Assessment (EMA) has yet to be used in the measurement of treatment outcomes for youth with severe irritability. While clinical changes during psychotherapy are typically assessed based on retrospective self‐reports or clinician‐administered assessments, EMA implementation in the context of treatment adds a unique opportunity to examine the nuances of symptom and behavior change as it transpires in naturalistic environments.

  • Using EMA, we find that over the course of exposure‐based cognitive behavioral treatment with parent management training, parenting behaviors become more adaptive, and that as parents change their behaviors, the youth's level of irritability decreases in their naturalistic environments.

  • Results indicate that using EMA to measure change in behaviors and symptoms during treatment helps identify the treatments' ability to change targeted behaviors and symptoms in real‐time in the patients' natural environment. Psychotherapy research should integrate EMA with efficacy studies and further understand how parenting behaviors influence symptoms and behaviors in youth.

Introduction

The use of ecological momentary assessment (EMA) has increased dramatically over the past decade (Henry et al., 2024; Wrzus & Neubauer, 2023). EMA, a tool for collecting naturalistic, real‐time data, is ideal for identifying targeted features of psychopathology on a momentary basis. EMA has many advantages, including increasing ecological validity and accessibility, minimizing retrospective recall biases, and examining fluctuations within phenomena and individuals. The key features of EMA include in‐the‐moment assessment of an individual's behaviors, experiences, or mood states through repeated observations, delivered at critical times in the individual's natural environment (Stone & Shiffman, 1994), often using mobile devices. While research has applied EMA in youth (Russell & Gajos, 2020) as a useful strategy for capturing day‐to‐day conditions of mood and behaviors, most research has focused on clinical populations such as attention‐deficit/hyperactivity disorder (ADHD; Whalen et al., 2006); autism spectrum disorder (ASD; Chen, Cordier, & Brown, 2015), internalizing disorders (Armey, Schatten, Haradhvala, & Miller, 2015), and substance use disorders (Jones et al., 2019). Critically, EMA has yet to be used in the context of treatment for youth with severe irritability. While clinical changes during psychotherapy are typically assessed based on retrospective self‐reports or clinician‐administered assessments, EMA implementation in this context adds a unique opportunity to examine the nuances of symptom change as it transpires in naturalistic environments.

Several EMA studies have assessed daily mood as a predictor of early response during psychotherapy; however, these were conducted in adults and mainly with depression (e.g., Husen, Rafaeli, Rubel, Bar‐Kalifa, & Lutz, 2016). The current study addresses this gap by reporting on EMA data collected over the course of psychotherapy for youth with severe irritability (Naim et al., 2024; Naim, Kircanski, et al., 2021). Irritability is defined as proneness to anger that may reach a pathological level (Brotman, Kircanski, Stringaris, Pine, & Leibenluft, 2017). Pediatric irritability is common and associated with impaired functioning (Brotman et al., 2017; Leibenluft et al., 2024; Orri et al., 2019), depression and anxiety (Brotman et al., 2006; Vidal‐Ribas, Brotman, Valdivieso, Leibenluft, & Stringaris, 2016), and suicidality (Copeland, Angold, Costello, & Egger, 2013; Orri et al., 2019). Irritability has both mood (i.e., persistent anger and irritable mood) and behavioral (i.e., temper loss and explosive outbursts) components (Copeland et al., 2013). In the present study, youth with irritability received an exposure‐based cognitive behavioral treatment (CBT) that incorporated parent management training (PMT), designed to directly address impairing irritability. Most recently, behavioral parent training and CBT have been the recommended first‐line psychosocial treatments for clinical conditions associated with irritability or externalizing problems such as oppositional defiant disorder (ODD) (Kalvin, Zhong, Rutten, Ibrahim, & Sukhodolsky, 2024; Leibenluft et al., 2024; Waxmonsky, Baweja, Bansal, & Waschbusch, 2021). Support was also demonstrated for dialectical behavioral therapy for children with disruptive mood dysregulation disorder (DMDD) (DBT‐C; Perepletchikova, Nathanson, Axelrod, Merrill, & Walkup, 2017). Our intervention builds on existing research and on a pathophysiological theoretical framework on pediatric irritability (Brotman et al., 2017), emphasizing the behavioral component of in vivo behavioral exposure to anger‐inducing stimuli that trigger patients' symptoms of irritability. The treatment showed feasibility, acceptability, and preliminary efficacy based on clinician ratings, self‐ and parent‐ reports (Naim et al., 2024). The current secondary analyses leverage EMA to explore in vivo symptom change and may unveil novel insights into naturalistic, real‐time therapeutic change, ultimately enhancing our understanding of the treatment process and informing tailored interventions (Thunnissen, aan het Rot, van den Hoofdakker, & Nauta, 2022).

This study is also the first to incorporate EMA from both youth and parents to understand real‐time daily changes in symptomatology during psychotherapy. The parent EMA protocol also included items on parental behaviors associated with the child's irritability, allowing us to investigate real‐time, naturalistic changes in specific parental skills that were targeted during the PMT (Naim et al., 2024; Naim, Kircanski, et al., 2021).

We had three aims. Our first aim was to test EMA‐rated changes in symptoms over the course of the treatment (i.e., from pre‐ to mid‐ and post‐treatment), and we hypothesized that both youth and parents would report decreased irritability. Our second aim was to test EMA‐rated changes in parental behaviors associated with the child's irritability, and we hypothesized that such behaviors would improve. Our third aim was to explore the real‐time, momentary associations between changes in parental behaviors and youth symptoms. We hypothesized that an increase in positive/adaptive parental behaviors would be associated with a decrease in child‐rated irritability.

Methods

Participants

Sample demographic information and characteristics are reported in Table 1. Participants in this study were 39 out of the 40 dyads (youth and their parents) who completed the treatment protocol (Naim et al., 2024) who also completed EMA at three time points (M age = 11.26 years, SD = 1.89 years; 35.9% female; 79.5% white). Participants for the treatment protocol were recruited from the Maryland, District of Columbia, Virginia, USA, metropolitan area between September 2018 and September 2021. Youth participants were diagnosed with severe and impairing irritability, either in the context of DMDD or in the context of significant levels of temper outbursts or irritable mood. Clinical diagnoses were assessed by a licensed clinician using the Kiddie‐Schedule for Affective Disorders and Schizophrenia Present and Lifetime Version (K‐SADS‐PL, Kaufman et al., 1997), including the DMDD module (Wiggins et al., 2016). See Naim et al. (2024) for a detailed description of inclusion/exclusion criteria. Out of the 39 dyads, three children had low prompt response (<5) during either pre‐treatment EMA (N = 1) or mid‐treatment (N = 2), although for two out of these three cases, this was due to technical issues. Three children completed pre‐ and mid‐treatment EMA but did not complete post‐treatment EMA.

Table 1.

Sample characteristics data

N = 39
Sex assigned at birth (n, %) Male (25, 64.1%)
Female (14, 35.9%)
Age (mean, SD) 11.26 (1.89)
Race (n, %) White (31, 79.5%)
Black or African American (4, 10.3%)
Asian (1, 2.55%)
Multiple races (2, 5.1%)
Unknown (1, 2.55%)
Ethnicity (n, %) Not Latino or Hispanic (35, 90%)
Latino or Hispanic (2, 5%)
Unknown (2, 5%)
IQ (mean, SD) 111.38 (12.45)
Primary DSM diagnosis (n, %) DMDD (24, 61.54%)
ODD (10, 25.64%)
ADHD (5, 12.82%)
Any medication (n, %) 30, 77%
Stimulant (n, %) 19, 49%
Non‐stimulant ADHD medication (n, %) 12, 30%
Antidepressant (n, %) 14, 35%
Antipsychotic (n, %) 3, 7.5%
Anticonvulsant (n, %) 1, 2.5%
Open in a new tab

ADHD, attention deficit hyperactive disorder; DMDD, disruptive mood dysregulation disorder; ODD, oppositional defiant disorder.

Procedures

STROBE Statement for this study is included in the Supporting Information. The treatment consisted of 12 exposure‐based CBT sessions with the child and PMT with a parent. The child sessions consist of three phases. The first phase included setting treatment goals, motivational interviewing, and psychoeducation on the concept of exposure to irritability; the second phase included exposures; and the third phase focused on treatment termination and transition back to a community provider. The parent sessions included modules on several topics such as instrumental learning, how to provide praise/positive acknowledgment, how to engage in active ignoring of problematic behaviors, dealing with outbursts, how to give effective commands and set limits for problematic behaviors, increasing pleasant interactions, and providing unexpected and intermittent rewards for positive behaviors. Therapists also worked with parents on how to tolerate their own emotional distress in response to their child's anger and irritability (for a full description of the study protocol, see Naim, Kircanski, et al., 2021).

Both the child and parent completed EMA at pre‐, mid‐, and post‐treatment over the course of 12 sessions. For the EMA, both the child and parent used smartphones to answer questions on in‐the‐moment child mood states three times a day for 1 week before, during, and after treatment. Parents also completed questions on parental behaviors. Pre‐treatment EMA was typically conducted 1–2 weeks before the first session, mid‐treatment EMA was typically conducted between sessions 6 and 7 of the treatment, and post‐treatment EMA was conducted 1–2 weeks after session 12. Each EMA phase included surveys delivered three times a day for 7 days. EMA was administered via ReTAINE technology (http://retaine.org/), where participants received a link to the survey via a text message. Specifically, the EMA schedule was built with participants' preselected 60‐min time periods during standardized time windows within which prompts would be delivered: morning/before school (6:00–9:00 a.m.), afternoon/after school (3:00–6:00 p.m.), and evening/before bedtime (7:00–10:00 p.m.). The actual prompt times were randomized within this period and prompts were delivered at the same time for child and parent. The EMA protocol was previously validated in a pediatric population (Naim, Smith, et al., 2021) separate from the current sample used in this study. For detailed information on the EMA protocol and procedures, see Naim, Kircanski, et al., 2021.

Ethical considerations

Children and parents gave informed assent and consent, respectively. Children and parents were informed that participation was voluntary and that assent/consent could be withdrawn at any time. Participants were monetarily compensated and received $20 for EMA training, $75 for completing EMA, and were offered a monetary bonus of $10 for completing ≥75% of prompts. The study was approved by the NIH IRB (NIH IRB ID numbers: 02‐M‐0021 and 15‐M‐0182). Trial registration numbers: NCT00025935 and NCT02531893, respectively.

EMA items

We had 23 items of interest including questions on children's irritability symptoms and on parental behaviors. The irritability items included in the EMA protocol were previously validated (Naim, Smith, et al., 2021) in a separate sample from the current study and focused on four irritability symptoms and one item related to functional impairment of irritability. Specifically, items probed: (1) temper outburst, (2) irritable mood, (3) frustration, (4) momentary anger, and (5) irritability‐related impairment for different domains. To enable examination of specificity of the treatment in changing irritability symptoms compared to other mood symptoms, we also included EMA items probing momentary anxiety and momentary sadness. Both youth and parents completed EMA based on these items. Parents were asked the same questions worded in a third‐person form, asking the parent to assess their child's momentary symptoms.

The parents' EMA protocol included additional items related to parental behaviors and parent–child dyadic interactions. These items were related to skills targeted during PMT sessions. Specifically, two items were used to measure parental behaviors of praising or acknowledging their child and the frequency of parent and child doing something enjoyable together. As parents of children with irritability often report accommodation behaviors, like changing their plans to prevent temper outbursts (Budagzad‐Jacobson, Musicaro, Marin, Silverman, & Lebowitz, 2024), an additional EMA parental item assessed this parental behavior of plan changing due to the child's irritability. Parents were also asked about their response to their child's temper outburst, including three items: paying attention to the child during the outburst, ignoring, or allowing the child to get what they wanted. For a full description of the EMA items and scales, see Supporting Information.

Of note, while some of the EMA preliminary findings are reported in the supplementary materials of Naim et al. (2024), current analyses investigate EMA‐rated symptoms change as a primary question. We include all parental factors of interest, as well as EMA‐rated irritability to explore real‐time, naturalistic changes in symptoms and in their variation over time. Additionally, we examine the temporal associations between parental factors and youth irritability symptoms.

EMA compliance rate of the current sample was consistent with compliance rates reported in previous EMA youth studies (e.g., Wen, Schneider, Stone, & Spruijt‐Metz, 2017), with an average of 71.80% for youth and 70.59% for parents, although compliance rates decreased throughout the treatment phases (pre‐treatment EMA‐ youth: M = 76.60%, parents: M = 70.20%; mid‐treatment EMA‐ youth: M = 72.90%, parents: M = 68.00%; post‐treatment EMA‐ youth: M = 65.90%, parents: M = 59.50%). At the item level, 7.85% of child's EMA datapoints were missing for each item, aggregated across participants, at pre‐treatment EMA, 9.36% for mid‐EMA, and 12.45% for post‐treatment EMA. Missing data ratios for the parent's EMA items ranged between 30.66% and 36.64%, aggregated across time points and participants, including ones with low compliance rate.

Analytical approach

Data were analyzed in a three‐level multi‐level modeling (MLM) framework (via HLM8 software, Raudenbush, Bryk, Cheong, & Congdon, 2019) to account for the nested nature of the data and to estimate symptom change at multiple levels of analyses. Each EMA prompt at Level 1 is nested within treatment phase (pre‐, mid‐, post‐) at Level 2, which is nested within individual at Level 3. Any missing data were removed via HLM using listwise deletion at Level 1, meaning removed per each analysis.

For the first aim, examining EMA‐rated changes in irritability over the course of the treatment, the irritability item rating for person j at phase i and prompt t (irritability tij ) was predicted by treatment phase (Level 2), coded as a categorical predictor (i.e., pre, mid, post coded as: pre = 0, mid = 0.5, post = 1), and as a function of each person (Level 3). Including treatment phase at Level 2 makes it a predictor by estimating the linear slopes from pre‐ to mid‐ to post‐treatment. The models specified outcome variables as continuous except for the categorical yes or no items of temper outbursts and irritability‐related impairment, which were analyzed using logistic MLM.

Similar models were conducted for our second aim of examining EMA‐rated changes in parental behaviors, with the difference being that the EMA‐parental items were entered as outcome measures in these models. For the third aim of exploring within‐person associations between parental behaviors and child's irritability symptoms, similar MLMs to the models described above observing change in EMA‐rated irritability were conducted, with the addition of each EMA parental item entered as a person‐centered, continuous predictor at Level 1. The Supporting Information includes detailed information on the MLMs conducted and examples for each statistical model.

Given the multiple tests conducted for this study, we adopted the Benjamini–Hochberg procedure for false discovery rate (FDR) correction for each series of analyses across the three main questions we examined. The expected proportion of false positives was set to q = .05. The results reported here represent raw coefficients and standard errors from the multilevel modeling considering FDR.

Supplementary analyses were conducted to test changes in the variability of irritability symptoms (i.e., we used all weekly EMA datapoints to calculate symptom variability) over the course of treatment. To measure the variability of irritability, Root Mean Square of Successive Differences (RMSDD) scores were generated for each weekly EMA ratings of irritability for each of the three phases at pre‐, mid‐, and post‐treatment. A more detailed description of these analyses and the findings is reported in the Supporting Information.

The datasets analyzed during the current study are available in the OSF repository https://osf.io/khmya/. Analyses were not preregistered.

Results

EMA‐rated irritability change

See Table 2 for average EMA‐rated items by parents and youth across the three treatment phases. For all EMA‐rated irritability symptoms, on average across the sample, both parents and youth reported a significant decrease in symptom severity from pre‐ to mid‐ and to post‐treatment. Specifically, betas referring to the treatment phase on the MLMs demonstrated that children and their parents reported improvement in the child's momentary anger (child's rating: β(38) = −.28, SE = 0.10, p = .008; parent's rating: β(38) = −.31, SE = 0.07, p < .001), frustration (child's rating: β(38) = −.37, SE = 0.10, p < .001; parent's rating: β(38) = −.39, SE = 0.08, p < .001), and irritable mood (child's rating: β(38) = −.27, SE = 0.08, p = .003; parent's rating: β(38) = −.33, SE = 0.09, p < .001), from pre‐ to mid‐ to post‐treatment. Additionally, both youth and their parents reported a decrease in the likelihood of the child having a temper outburst (child's rating: β(38) = −.92, SE = 0.36, p = .013; parent's rating: β(38) = −.69, SE = 0.20, p = .002) from pre‐ to mid‐ to post‐treatment. In terms of irritability‐related impairment, both youth and their parents were less likely to endorse child impairment associated with the relationship with the parent (child's rating: β(38) = −1.18, SE = 0.33, p < .001; parent's rating: β(38) = −.92, SE = 0.22, p < .001), and parents (but not children) were less likely to endorse child impairment associated with the relationship with other children (parent's rating: β(38) = −.91, SE = 0.37, p = .02; child's rating: β(38) = −.56, SE = 0.48, p = .252). No significant change was found for impairment at school (parent's rating: β(38) = −.75, SE = 0.47, p = .116; child's rating: β(38) = −.51, SE = 0.60, p = .399).

Table 2.

EMA rating by treatment phase for both youth and parents

Youth‐rated Parent‐rated
Pre‐treatment EMA Mid‐treatment EMA Post‐treatment EMA Pre‐treatment EMA Mid‐treatment EMA Post‐treatment EMA
Youth irritability symptoms
Temper outburst
% (SD) 11.4 (3.18) 7.8 (2.69) 4.3 (2.03) 18.0 (3.88) 14.0 (3.44) 11.0 (3.10)
Irritable mood
Mean (SD) 1.80 (1.04) 1.67 (1.04) 1.56 (0.85) 2.20 (1.00) 2.01 (0.94) 1.83 (0.88)
Frustration
Mean (SD) 2.01 (1.28) 1.79 (1.18) 1.63 (1.05) 2.45 (1.24) 2.25 (1.20) 2.06 (1.12)
Momentary anger
Mean (SD) 1.77 (1.16) 1.68 (1.12) 1.51 (0.99) 1.94 (1.40) 1.7 (1.04) 1.63 (1.01)
Impairment with parents
% (SD) 15 (3.61) 9 (2.91) 10 (3.02) 34 (4.74) 24 (4.29) 18 (3.82)
Impairment with other kids
% (SD) 1 (1.1) 1 (1.2) 1 (1.0) 5 (2.24) 5 (2.08) 3 (1.76)
Impairment at school
% (SD) 3 (1.58) 2 (1.36) 2 (1.39) 0 (0.80) 0 (1.37) 0 (1.17)
Pre‐treatment EMA Mid‐treatment EMA Post‐treatment EMA
Parental behaviors (parent‐rated)
Praising or acknowledging
Mean (SD) 1.20 (1.04) 1.42 (1.09) 1.51 (1.14)
Enjoying together
Mean (SD) 2.13 (1.01) 2.26 (1.02) 2.31 (1.00)
Changing plans
Mean (SD) 1.59 (1.07) 1.42 (0.89) 1.36 (0.82)
Allowing during outbursts
Mean (SD) 1.78 (1.02) 1.43 (0.82) 1.46 (0.63)
Ignoring during outbursts
Mean (SD) 2.01 (0.96) 2.66 (1.23) 2.70 (1.27)
Paying attention during outbursts
Mean (SD) 4.36 (0.86) 4.11 (1.14) 4.00 (1.16)
Open in a new tab

For illustration purposes, temper outbursts are presented with estimated % of prompts on which the item was endorsed.

See Supporting Information for additional analyses and results on changes in irritability variability over the course of treatment. Overall, variability in irritability during the week, measured via root mean successive squared difference (RMSSD) scores, decreased from pre‐ to mid‐ and post‐treatment EMA ratings across irritability‐related items reported by both child and parents.

EMA‐rated other negative mood change

No significant changes were found for EMA‐rated momentary anxiety (parent's rating: β(38) = −.16, SE = 0.09, p = .072; child's rating: β(38) = −.11, SE = 0.08, p = .150). Both youth and their parents reported a decrease in momentary sadness over the course of the treatment (child's rating: β(38) = −.29, SE = 0.09, p = .004; parent's rating: β(38) = −.22, SE = 0.08, p = .008).

EMA‐rated parental behavior change

Parental behaviors of praising (β(38) = .26, SE = 0.08, p = .004) significantly increased from pre‐ to post‐treatment. Parental change in plans due to the child's irritable behavior significantly decreased (β(38) = −.22, SE = 0.09, p = .020). Focusing on parental behavior during an outburst, parents reported increased ability to ignore the child's demands (β(38) = .77, SE = 0.24, p = .002) and pay less attention toward the child (β(38) = −.40, SE = 0.17, p = .026). A significant decrease was found in parents allowing their child to get what they wanted during the outburst (β(38) = −.32, SE = 0.15, p = .038).

Associations between changes in child's symptoms and changes in parental behaviors

Significant associations were found between increased levels of parental adaptive behavior and decreased levels of child irritability symptoms. Specifically, three main significant findings survived FDR correction. First, EMA‐parent report on spending enjoyable time with their child was negatively associated with EMA‐child reported co‐occurring momentary anger: β(38) = −.15, SE = 0.04, p < .001, and this negative association increased in magnitude from pre‐ to mid‐ and post‐treatment (β(38) = .20, SE = 0.06, p = .001). Similarly, EMA‐parent report on praising their child or acknowledging their positive behavior was associated with decreased EMA‐child reported co‐occurring irritable mood (β(38) = −.17, SE = 0.06, p = .012). Third, EMA‐parent report on decrease in change of plans due to the child's irritability was significantly associated with a decrease in EMA child‐rated co‐occurring frustration (β(38) = .14, SE = 0.07, p = .043). The association between change of plans due to the child's irritability and decrease in irritable mood did not reach the level of significance (β(38) = .07, SE = 0.04, p = .060). The magnitude of these associations was similar across treatment phases. The models involving binary variables (i.e., temper outburst and impairment items) failed to converge due to limited data available for the complexity of these models. Overall, these findings suggest improvement in parental behaviors and reduced accommodation during the treatment, which were associated with symptoms improvement (see Table S1 for full report on the effects).

Discussion

The current study provides unique insight into real‐time symptoms and parental behavior change during psychotherapy treatment for irritability. Understanding clinical symptoms and behavioral changes in response to treatment is essential in psychotherapy research. Prior research has typically focused on retrospective self‐ or clinician‐reported symptom measures, which are limited by recall bias (Stoddard et al., 2014). While EMA usage in the context of treatment for clinical populations has increased in the past few decades, this is the first study integrating EMA to measure treatment efficacy in youth with impairing irritability.

In this study, EMA was completed at three phases during exposure‐based CBT and PMT for irritability (i.e., pre‐, mid‐, and post‐treatment) by both children and their parents. This allowed us to observe real‐time, naturalistic changes in irritability symptomatology, as reported by different informants, as well as learn about parental behavioral changes in treatment and how these changes are related to the child's change in irritability symptoms. Specifically, we demonstrated: (a) EMA‐rated irritability significantly decreased during the treatment; (b) parental behaviors taught in PMT significantly changed during treatment, showing that parents were praising/acknowledging their child's positive behavior more, were accommodating less, and had a greater ability to apply active ignore during their child's outburst; and (c) associations between adaptive parental behaviors and decreased irritability symptoms.

Our EMA findings are consistent with our prior work (Naim et al., 2024) using clinician‐rated measures and indicate decreased youth EMA‐rated irritability symptoms, including momentary anger, irritable mood, frustration, and the likelihood of having a temper outburst across treatment. Similar to the published primary outcomes (Naim et al., 2024), the current EMA findings suggest some specificity of the current treatment in improving irritability symptoms, as well as momentary sadness, but no significant change in momentary worry throughout the treatment. This replication speaks to the potential robustness of the findings, with the added value of EMA in assessing real‐time, naturalistic changes in symptomatology while also capturing the multifaceted nature of the irritability construct. Using EMA, we also had an insight into real‐time parental behaviors and responses to their child's irritability as they change over the course of the treatment.

By using EMA, we could assess changes in variability of irritability by examining RMSSD. Lability in irritability decreased from pre‐ to mid‐ and post‐treatment as reported by both child and parent, particularly for irritable mood and frustration (see Supporting Information). As mood lability had been found to be associated with social impairment and risk for psychopathology in general (Høegh et al., 2022; Naim et al., 2022), reduced RMSSD irritability may have short and long‐term implications in terms of lower levels of psychopathology and better well‐being for youth in this study. Future studies should probe irritability lability as a treatment target and test long‐term outcomes to further shed light on this issue.

Our EMA findings also provided a naturalistic insight into changes in impairment. Specifically, improvement in the relationship between the child and parent was consistently reported by both informants. These findings support the rationale of this treatment integrating exposure‐based CBT with PMT and support existing treatment research literature emphasizing the parent–child relationship in the context of child psychopathology (Everett, Martin, & Zalewski, 2021).

Relatedly, another unique and clinically relevant finding is the significant changes observed in parental behaviors as targeted via PMT. Specifically, our PMT protocol incorporated strategies to enhance parental contingency management; parents are coached to praise and acknowledge their child's adaptive behaviors and to actively ignore maladaptive behaviors (Naim, Kircanski, et al., 2021). As expected, our EMA data showed that parents were praising/acknowledging their child more as treatment progressed and reported spending more enjoyable time together with their child. Parents reported reduced accommodations in changing plans due to their child's irritability. Finally, parents were engaging in more active ignoring of unwanted behavior during temper outbursts and reported a greater ability not to allow their child to get what they wanted.

We further explored temporal dynamics between parental behavioral change and co‐occurring child's changes in irritability symptoms over the course of the treatment. Three main findings emerged. First, EMA‐parent report on spending enjoyable time with their kids was associated with decreased momentary anger. Second, parents were praising their child and acknowledging their positive behavior more, which was associated with decreased child's irritable mood. Third, parents less frequently changed their plans due to the child's irritability, which was associated with decreased levels of child's co‐occurring frustration and irritable mood. These findings suggest evidence that parental behaviors and the irritability of the child are temporally associated at the individual level. While these parental behaviors are skills taught in the context of PMT, these findings potentially support the hypothesis that PMT contributes to the change in child's symptoms during treatment. However, this should be interpreted cautiously as children were undergoing exposure‐based CBT while parents were undergoing PMT. Hence, the current study cannot distinguish PMT versus CBT effects nor address the mechanistic question of shared or specific treatment components affecting child's improvement in symptoms. Yet, the emerged data are in line with accumulating research demonstrating that parenting practices and children's disruptive behavior are reciprocally influential (Burke, Pardini, & Loeber, 2008).

While future studies should further explore the specific mechanisms of change, as well as the bidirectionality of these associations among irritable youth via EMA, these findings point out the potential role of parents as agents of change and the importance of continuing to involve parents in treatment for youth. This suggestion is in line with existing literature on the efficacy of parenting interventions for children with disruptive behaviors and externalizing problems (Helander et al., 2024; Zarakoviti, Shafran, Papadimitriou, & Bennett, 2021) as well as the efficacy of integrated intervention models that simultaneously focus on both child and parent to address youth's psychopathology (Pine, Baumann, Modugno, & Compas, 2024). While clinical presentations of chronic irritability appear to be phenomenologically and empirically aligned with externalizing problems and disruptive behaviors, current data suggest that these clinical categories may also share a consistent response to effective treatment approaches.

Alongside these strengths, this study holds several limitations. First, this study reports on secondary analyses of a previously published paper reporting on the preliminary efficacy of the treatment protocol. The EMA data were collected during a treatment study that was not a randomized controlled trial (RCT). Hence, EMA‐reported changes over time within the clinical sample could not be compared to a control group. Second, though EMA provided rich within‐person data (with an overall total number of observations ≈ 1449 per participant), the overall sample size was relatively small, and it was demographically homogeneous, which limits the generalizability of our results. Third, many of the participants were on medication during the study. Although efforts were made to mitigate the potential influence of medication usage on the results of the study, including confirming participants were stable on medications and made no changes to their medication regimen during the treatment, medication effects might still occur. We also did not collect data on the specific days or times participants took their medications on EMA days. Hence, daily effects of medication cannot be controlled and modeled in this study. Fourth, as EMA was delivered only at three timepoints during the treatment, the trajectory of symptom change and the timing in which these changes potentially occurred between these intervals could not be captured in the context of the current design. Future studies aiming to add more intervals would allow a better understanding of the trajectories of symptom change.

At a broader level, current findings support the use of digital, real‐time tools to enhance the understanding of in vivo, naturalistic emotional states and behaviors. The current study also supports EMA feasibility in the context of a treatment study. EMA provides a unique added value of ecological assessments and allows us to explore additional aspects of symptom lability or temporal associations between momentary symptoms and behaviors that cannot be explored using retrospective assessment.

From a clinical implication perspective, this study highlights parental behavior as a potentially important component in the treatment of youth with irritability. Additionally, this study reflects a step forward toward personalization in psychotherapy for youth by allowing the investigation of symptoms change and variability of symptoms change in real time, and at the within‐individual level. There are further steps needed for personalization. For example, the question of which patients might benefit more from which aspect of the intervention was beyond the scope of the current study. Yet, our findings underscore the advantages of EMA strategies providing in‐depth analyses of real‐time patterns, providing a closer examination of the patients' symptoms change during treatment. EMA could be a useful tool for clinicians to monitor and enhance progress and efficacy during psychotherapy (e.g., Lutz, Rubel, Schwartz, Schilling, & Deisenhofer, 2019).

Conclusions

This study supports the utility of EMA as an in vivo, naturalistic method that can capture granular change in symptoms and therapy‐targeted behaviors as they occur in the daily life of irritable youth undergoing psychotherapy. Results suggest that parental behaviors may be a potential treatment target for youth with irritability.

Funding

Funding for this study was provided by the Intramural Research Program at the National Institutes of Health.

Conflict of interest

The authors declare that they have no competing or potential conflicts of interest.

Ethical information

The study was approved by the National Institutes of Health IRB. Trial registration numbers are NCT00025935 and NCT02531893.

Supporting information

Table S1. Within‐prompt associations between child's irritability and parental behaviors across treatment phases.

CAMH-31-33-s001.docx (51.4KB, docx)

Acknowledgments

Conceptualization: Naim, Brotman; Writing – original draft: Naim, Brotman; Writing – final draft: Naim, Brotman, Pandya, Shaughnessy, German, Henry, Kircanski; Revisions: German, Henry, Kircanski; Full data access and analysis: Naim, Pandya, Shaughnessy; Responsibility for the integrity of the data in the study and the accuracy of the data analysis: Naim, Brotman.

Data availability statement

The datasets analyzed during the current study are available in OSF repository https://osf.io/khmya/.

References

  1. Armey, M.F. , Schatten, H.T. , Haradhvala, N. , & Miller, I.W. (2015). Ecological momentary assessment (EMA) of depression‐related phenomena. Current Opinion in Psychology, 4, 21–25. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Brotman, M.A. , Kircanski, K. , Stringaris, A. , Pine, D.S. , & Leibenluft, E. (2017). Irritability in youths: A translational model. American Journal of Psychiatry, 174, 520–532. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Brotman, M.A. , Schmajuk, M. , Rich, B.A. , Dickstein, D.P. , Guyer, A.E. , Costello, E.J. , & Leibenluft, E. (2006). Prevalence, clinical correlates, and longitudinal course of severe mood dysregulation in children. Biological Psychiatry, 60, 991–997. [DOI] [PubMed] [Google Scholar]
  4. Budagzad‐Jacobson, R.S. , Musicaro, R.M. , Marin, C.E. , Silverman, W.K. , & Lebowitz, E.R. (2024). Family accommodation in anxious irritable and anxious non‐irritable youth. The Behavior Therapist, 55, 913–921. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Burke, J.D. , Pardini, D.A. , & Loeber, R. (2008). Reciprocal relationships between parenting behavior and disruptive psychopathology from childhood through adolescence. Journal of Abnormal Child Psychology, 36, 679–692. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Chen, Y.W. , Cordier, R. , & Brown, N. (2015). A preliminary study on the reliability and validity of using experience sampling method in children with autism spectrum disorders. Developmental Neurorehabilitation, 18, 383–389. [DOI] [PubMed] [Google Scholar]
  7. Copeland, W.E. , Angold, A. , Costello, E.J. , & Egger, H. (2013). Prevalence, comorbidity, and correlates of DSM‐5 proposed disruptive mood dysregulation disorder. American Journal of Psychiatry, 170, 173–179. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Everett, Y. , Martin, C.G. , & Zalewski, M. (2021). A systematic review focusing on psychotherapeutic interventions that impact parental psychopathology, child psychopathology and parenting behavior. Clinical Child and Family Psychology Review, 24, 579–598. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Helander, M. , Asperholm, M. , Wetterborg, D. , Öst, L.G. , Hellner, C. , Herlitz, A. , & Enebrink, P. (2024). The efficacy of parent management training with or without involving the child in the treatment among children with clinical levels of disruptive behavior: A meta‐analysis. Child Psychiatry & Human Development, 55, 164–181. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Henry, L.M. , Hansen, E. , Chimoff, J. , Pokstis, K. , Kiderman, M. , Naim, R. , … & Brotman, M.A. (2024). Selecting an ecological momentary assessment platform: Tutorial for researchers. Journal of Medical Internet Research, 26, e51125. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Høegh, M.C. , Melle, I. , Aminoff, S.R. , Olsen, S.H. , Lunding, S.H. , Ueland, T. , & Lagerberg, T.V. (2022). Affective lability and social functioning in severe mental disorders. European Archives of Psychiatry and Clinical Neuroscience, 272, 873–885. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Husen, K. , Rafaeli, E. , Rubel, J.A. , Bar‐Kalifa, E. , & Lutz, W. (2016). Daily affect dynamics predict early response in CBT: Feasibility and predictive validity of EMA for outpatient psychotherapy. Journal of Affective Disorders, 206, 305–314. [DOI] [PubMed] [Google Scholar]
  13. Jones, A. , Remmerswaal, D. , Verveer, I. , Robinson, E. , Franken, I.H. , Wen, C.K.F. , & Field, M. (2019). Compliance with ecological momentary assessment protocols in substance users: A meta‐analysis. Addiction, 114, 609–619. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Kalvin, C. , Zhong, J. , Rutten, M. , Ibrahim, K. , & Sukhodolsky, D.G. (2024). Evidence‐based psychosocial treatments for childhood irritability and aggressive behavior. JAACAP Open, 3, 14–28. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Kaufman, J. , Birmaher, B. , Brent, D. , Rao, U.M.A. , Flynn, C. , Moreci, P. , … & Ryan, N. (1997). Schedule for affective disorders and schizophrenia for school‐age children‐present and lifetime version (K‐SADS‐PL): Initial reliability and validity data. Journal of the American Academy of Child and Adolescent Psychiatry, 36, 980–988. [DOI] [PubMed] [Google Scholar]
  16. Leibenluft, E. , Allen, L.E. , Althoff, R.R. , Brotman, M.A. , Burke, J.D. , Carlson, G.A. , … & Stringaris, A. (2024). Irritability in youths: A critical integrative review. American Journal of Psychiatry, 181, 275–290. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Lutz, W. , Rubel, J.A. , Schwartz, B. , Schilling, V. , & Deisenhofer, A.K. (2019). Towards integrating personalized feedback research into clinical practice: Development of the Trier Treatment Navigator (TTN). Behavior Research and Therapy, 120, 103438. [DOI] [PubMed] [Google Scholar]
  18. Naim, R. , Dombek, K. , German, R.E. , Haller, S.P. , Kircanski, K. , & Brotman, M.A. (2024). An exposure‐based cognitive–behavioral therapy for youth with severe irritability: Feasibility and preliminary efficacy. Journal of Clinical Child and Adolescent Psychology, 53, 260–276. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Naim, R. , Kircanski, K. , Gold, A. , German, R.E. , Davis, M. , Perlstein, S. , & Brotman, M.A. (2021). Across‐subjects multiple baseline trial of exposure‐based cognitive‐behavioral therapy for severe irritability: A study protocol. BMJ Open, 11, e039169. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Naim, R. , Shaughnessy, S. , Smith, A. , Karalunas, S.L. , Kircanski, K. , & Brotman, M.A. (2022). Real‐time assessment of positive and negative affective fluctuations and mood lability in a transdiagnostic sample of youth. Depression and Anxiety, 39, 870–880. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Naim, R. , Smith, A. , Chue, A. , Grassie, H. , Linke, J. , Dombek, K. , … & Kircanski, K. (2021). Using ecological momentary assessment to enhance irritability phenotyping in a transdiagnostic sample of youth. Development and Psychopathology, 33, 1734–1746. [Google Scholar]
  22. Orri, M. , Galera, C. , Turecki, G. , Boivin, M. , Tremblay, R.E. , Geoffroy, M.C. , & Côté, S.M. (2019). Pathways of association between childhood irritability and adolescent suicidality. Journal of the American Academy of Child and Adolescent Psychiatry, 58, 99–107. [DOI] [PubMed] [Google Scholar]
  23. Perepletchikova, F. , Nathanson, D. , Axelrod, S.R. , Merrill, C. , … & Walkup, J. (2017). Randomized clinical trial of dialectical behavior therapy for preadolescent children with disruptive mood dysregulation disorder: Feasibility and outcomes. Journal of the American Academy of Child and Adolescent Psychiatry, 56, 832–840. [DOI] [PubMed] [Google Scholar]
  24. Pine, A.E. , Baumann, M.G. , Modugno, G. , & Compas, B.E. (2024). Parental involvement in adolescent psychological interventions: A meta‐analysis. Clinical Child and Family Psychology Review, 27, 1–20. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Raudenbush, S.W. , Bryk, A.S. , Cheong, Y.F. , & Congdon, R. (2019). HLM 8 for windows. Skokie, IL: Scientific Software International. [Google Scholar]
  26. Russell, M.A. , & Gajos, J.M. (2020). Annual Research Review: Ecological momentary assessment studies in child psychology and psychiatry. Journal of Child Psychology and Psychiatry, 61, 376–394. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Stoddard, J. , Stringaris, A. , Brotman, M.A. , Montville, D. , Pine, D.S. , & Leibenluft, E. (2014). Irritability in child and adolescent anxiety disorders. Depression and Anxiety, 31, 566–573. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Stone, A.A. , & Shiffman, S. (1994). Ecological momentary assessment (EMA) in behavioral medicine. Annals of Behavioral Medicine, 16, 199–202. [Google Scholar]
  29. Thunnissen, M.R. , aan het Rot, M. , van den Hoofdakker, B.J. , & Nauta, M.H. (2022). Youth psychopathology in daily life: Systematically reviewed characteristics and potentials of ecological momentary assessment applications. Child Psychiatry and Human Development, 53, 1129–1147. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Vidal‐Ribas, P. , Brotman, M.A. , Valdivieso, I. , Leibenluft, E. , & Stringaris, A. (2016). The status of irritability in psychiatry: A conceptual and quantitative review. Journal of the American Academy of Child and Adolescent Psychiatry, 55, 556–570. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Waxmonsky, J.G. , Baweja, R. , Bansal, P.S. , & Waschbusch, D.A. (2021). A review of the evidence base for psychosocial interventions for the treatment of emotion dysregulation in children and adolescents. Child and Adolescent Psychiatric Clinics of North America, 30, 573–594. [DOI] [PubMed] [Google Scholar]
  32. Wen, C.K.F. , Schneider, S. , Stone, A.A. , & Spruijt‐Metz, D. (2017). Compliance with mobile ecological momentary assessment protocols in children and adolescents: A systematic review and meta‐analysis. Journal of Medical Internet Research, 19, e132. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Whalen, C.K. , Henker, B. , Jamner, L.D. , Ishikawa, S.S. , Floro, J.N. , Swindle, R. , … & Johnston, J.A. (2006). Toward mapping daily challenges of living with ADHD: Maternal and child perspectives using electronic diaries. Journal of Abnormal Child Psychology, 34, 111–126. [DOI] [PubMed] [Google Scholar]
  34. Wiggins, J.L. , Brotman, M.A. , Adleman, N.E. , Kim, P. , Oakes, A.H. , Reynolds, R.C. , & Leibenluft, E. (2016). Neural correlates of irritability in disruptive mood dysregulation and bipolar disorders. American Journal of Psychiatry, 173, 722–730. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Wrzus, C. , & Neubauer, A.B. (2023). Ecological momentary assessment: A meta‐analysis on designs, samples, and compliance across research fields. Assessment, 30, 825–846. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Zarakoviti, E. , Shafran, R. , Papadimitriou, D. , & Bennett, S.D. (2021). The efficacy of parent training interventions for disruptive behavior disorders in treating untargeted comorbid internalizing symptoms in children and adolescents: A systematic review. Clinical Child and Family Psychology Review, 24, 542–552. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Table S1. Within‐prompt associations between child's irritability and parental behaviors across treatment phases.

CAMH-31-33-s001.docx (51.4KB, docx)

Data Availability Statement

The datasets analyzed during the current study are available in OSF repository https://osf.io/khmya/.

Articles from Child and Adolescent Mental Health are provided here courtesy of Wiley

RESOURCES