Abstract
Objectives
Heightened irritability in adolescence is an impairing symptom that can lead to negative outcomes in adulthood, but effective screening tools are lacking. This study aimed to derive clinically‐optimized cutoff scores using the Multidimensional Assessment Profile Scales–Temper Loss (MAPS‐TL) to pragmatically identify adolescents with impairing irritability.
Methods
A diverse sample of 79 adolescents and their parents completed the MAPS‐TL‐Youth version. Stepwise logistic regression analyses were used to determine the items associated with impairment, and receiver operator characteristic (ROC) analyses were conducted to derive optimal cutoff scores.
Results
Three parent‐report items (become frustrated easily, angry/irritable/grouchy throughout the day, difficulty calming down when angry) and two youth‐report items (hit/shove/kick when lost temper, difficulty calming down when angry) were strongly associated with impairment. Optimal cutoff scores garnered very good sensitivity (91%, 73%) and specificity (77%, 75%) for the parent‐ and youth‐report versions respectively. Scores above these cutoffs were associated with increased internalizing and externalizing problems and lower overall quality of life.
Conclusions
The MAPS‐TL clinically optimized irritability scores show preliminary validity for implementation in practical settings to efficiently identify adolescents who need additional evaluation and/or intervention. Further research is important to validate these cutoff scores with larger population‐based samples and real‐world settings.
Keywords: adolescence, clinically‐optimized cutoff score, developmental psychopathology, irritability
Abbreviations
- ADHD
attention‐deficit/hyperactivity disorder
- ARI
affective reactivity index
- AUC
area under the curve
- BASC‐2
behavioral assessment system for children, second edition
- CBCL
child behavior checklist
- DMDD
disruptive mood dysregulation disorder
- DSM‐IV‐TR
diagnostic and statistical manual of mental disorders, fourth edition, text revision
- DSM‐5
diagnostic and statistical manual of mental disorders, 5th edition
- GAD
generalized anxiety disorder
- IQ
intelligence quotient
- K‐SADS
kiddie schedule for affective disorders and schizophrenia (lifetime version)
- MAP‐DB
multidimensional assessment profile of disruptive behavior
- MAPS‐TL
multidimensional assessment profile scales–temper loss
- MAPS‐TL‐A
multidimensional assessment profile scales–temper loss (adolescent version)
- MDD
major depressive disorder
- MRI
magnetic resonance imaging
- ODD
oppositional defiant disorder
- PDD
persistent depressive disorder
- ROC
receiver operator characteristics
- SD
standard deviation
- YQOL‐R
youth quality of life (research version)
- YQOL‐SF
youth quality of life (short form)
1. INTRODUCTION
Elevated pediatric irritability, expressed as temper outbursts and grumpy/sullen mood relative to peers, has received increasing attention by researchers and clinicians (Brotman, Kircanski, & Leibenluft, 2017; Brotman, Kircanski, Stringaris, et al., 2017; Roy & Comer, 2020) as a transdiagnostic indicator of psychopathology and predictor of future morbidity and functional impairment (Beauchaine & Tackett, 2020; Dougherty et al., 2013; Finlay‐Jones et al., 2023; Klein et al., 2021; Orri et al., 2019; Sorcher et al., 2022; Stringaris et al., 2009; Vidal‐Ribas et al., 2016; Wiggins et al., 2014). Elevated levels of dysregulated irritability are especially prevalent in early childhood, and if this persists and continues to be problematic through adolescence, youth are at a greater risk of experiencing adverse outcomes in adulthood (Evans et al., 2022; Hawes et al., 2020; Stringaris et al., 2009). Despite this, there is currently a lack of developmentally‐specified, pragmatic screening tools to efficiently flag clinically significant irritability in this age group. Such screening tools, to have the greatest uptake in “real‐world” settings (e.g., clinics, primary care), need to be brief, efficient, and implementable with little expertise (e.g., survey rather than observation/interview). In addition, multi‐informant best practices emphasize the need to collect information from both parents and adolescents to obtain a comprehensive understanding of the child's irritability while accounting for the unique contextual differences and perspectives of each reporter (De Los Reyes & Epkins, 2023; Dougherty et al., 2021; Frazier et al., 2016; Stringaris et al., 2009, 2012; Zik et al., 2022). The goal of the present study is to generate parent‐ and youth‐reported developmentally‐specific, clinically optimized screeners of irritability for use in adolescence.
Early measures of irritability were generated post hoc, drawing on DSM symptom clusters (ODD/DMDD irritability symptoms). More recently, scales that specifically assess irritability have been introduced. One such scale, the Affective Reactivity Index (ARI, Stringaris et al., 2012) was designed to briefly examine threshold, frequency, and duration of angry feelings and behaviors. A limitation of this measure is that it is not developmentally specific, using the same items for youth across development, and it only captures more severe irritability symptoms (Dougherty et al., 2021; Wakschlag et al., 2012). Given that adolescence is a sensitive developmental period in which youth are particularly vulnerable to mental health issues and the normal:abnormal distinction is challenging (Casey, 2015; Casey et al., 2010; Pine et al., 1999; Powers & Casey, 2015; Sisk & Gee, 2022), efficient, developmentally‐specified assessment of irritability that can discriminate between normative and atypical behaviors in adolescents is particularly important.
The Multidimensional Assessment Profile Scales—Temper Loss Scale (MAPS‐TL; Wakschlag et al., 2012) was specifically designed to assess irritability within a developmental context. As described in the introduction to this special issue (Wiggins, Roy, et al., 2023), several versions of the MAPS‐TL have been developed, with items that are shared across age groups along with developmentally‐specific items reflecting features of irritability at specific ages. For example, have a temper tantrum, is a core item; have a temper tantrum when asked to stop playing and do something else is an item specific to the infant‐toddler version (Krogh‐Jespersen et al., 2021), and lose temper or have a temper outburst with a friend is an item specific to the MAPS‐TL‐Youth version used in studies of early school age children (Hirsch et al., 2023), preadolescents (Alam et al., 2023) and the present investigation of adolescents. While the comprehensive nature of the MAPS‐TL is beneficial to longitudinal research, it is likely too long and burdensome for use in clinical practice. There has been a recent emphasis on ultra‐brief scales that can be easily implemented and interpreted in clinical settings and epidemiologic studies (Morris et al., 2020; Wakschlag et al., 2023). Such pragmatic tools have been designed to assess various conditions in youth, such as online gaming disorder (Jo et al., 2020), obsessive compulsive disorder (Abramovitch et al., 2022), and general mental health problems (Rivera‐Riquelme et al., 2019). In our own work, we have used the MAPS‐TL to generate brief, clinically optimized screeners for preschoolers, school‐age youth (Hirsch et al., 2023), and preadolescents (Alam et al., 2023). Such an efficient, clinically optimized screening tool has not yet been developed for adolescents.
[Correction added on 19 October 2023, after first online publication: Sentences in the preceding paragraph were modified to clarify information related to the versions of the tool.]
Assessment of adolescents typically involves information obtained from both parents and the adolescent. While such multi‐informant approaches can provide a more comprehensive understanding of children's difficulties, they are often complicated by poor agreement between parent and child, which can hinder the determination of treatment targets (Achenbach, 2006; De Los Reyes et al., 2015; De Los Reyes & Kazdin, 2005; Grills & Ollendick, 2002). Informant correspondence rates tend to be higher for observable, often externalizing symptoms than for internalizing symptoms (De Los Reyes et al., 2015), leading some researchers to suggest that if a multi‐informant approach is not feasible, parent report may be more valuable for overt behaviors, while youth report may be more valuable for internal experiences as well as behaviors that youth may intentionally hide from others (Grills & Ollendick, 2002). Irritability presents a unique challenge as it is typically characterized by both externalizing and internalizing symptoms. As a result, research on agreement between parent‐ and youth‐reported irritability has yielded highly variable results, with concordance rates ranging from no agreement to strong levels of agreement between parents and youth (Evans et al., 2022; Stringaris et al., 2012; Zendarski et al., 2022; Zik et al., 2022). Additionally, multiple studies, both cross‐sectional and longitudinal, have found that parent and youth reports of irritability have differential associations with related constructs, symptoms, and psychopathology (Frazier et al., 2016; Stringaris et al., 2009, 2012; Zik et al., 2022). Evidence suggests that youth self‐reports of irritability may capture a different underlying construct than parent report (Dougherty et al., 2021) or that parents and children may have different interpretations of, and perceptions of, irritability. For example, Zik et al. (2022) found that the term “temper tantrums” in relation to irritability did not resonate with youth, and items about temper tantrums and losing temper cross‐loaded onto anger for adolescents and onto irritability for parents. Notably, current measures of pediatric irritability typically use the same items for parents and youth, but in light of the abovementioned research, different items may be needed for brief screeners to quickly and efficiently capture parent‐versus youth‐reported irritability.
To address these gaps, the present study used the Youth version of the MAPS‐TL‐Youth to generate parent‐ and adolescent‐reported developmentally specific, clinically optimized pragmatic screeners. The first aim was to identify the irritable behaviors most associated with impairment in adolescents based on parent‐ and youth‐reported items and the second aim was to derive cutoff scores, based on these items, that have good sensitivity and specificity in relation to DSM‐5 irritability‐related diagnoses (oppositional defiant disorder [ODD], disruptive mood dysregulation disorder [DMDD], generalized anxiety disorder [GAD], persistent depressive disorders [PDD], and major depressive disorder [MDD]). Our strategy of using both impairment and diagnosis as complementary validation tools for the parsimonious model allowed us to identify the items that are most functionally problematic and also associated with diagnostic status. The third aim further evaluated the validity of the empirical cutoff scores by assessing whether adolescents scoring above the cutoff scores exhibited broader impairments beyond irritability (i.e., greater internalizing and externalizing symptoms, lower quality of life) as would be suggested by studies showing diverse comorbidities and outcomes of irritable youth (Copeland et al., 2014; Stringaris et al., 2009). We refrained from directly comparing parent‐ and youth‐report due to the absence of an objective validation measure that would be unaffected by shared methods variance. Therefore, we evaluate the parent‐ and youth‐report MAPS‐TL‐Youth scales as complementary measures.
2. METHODS
2.1. Overview
Data were obtained as part of a longitudinal follow‐up study of children with varying degrees of irritability as evidenced by the presence or absence of impairing emotional outbursts. In the initial study, families with children ages 5–9 years old were recruited through community posting and school referrals in a large urban center in the United States. Children were enrolled in three groups: youth with impairing emotional outbursts, youth diagnosed with attention‐deficit/hyperactivity disorder (ADHD) but without impairing emotional outbursts, and typically developing controls. Impairing emotional outbursts were defined as verbal and/or physical rages towards people or property occurring, on average, at least three times a week for the past 6 months. Outbursts were required to: (i) be out of proportion in intensity or duration to the situation, (ii) be inconsistent with the child's developmental level, and (iii) cause interference with age‐appropriate function in at least one setting (home or school). Children with impairing emotional outbursts exhibited only in the context of an adjustment disorder (i.e., less than 6 months after a significant stressor such as death, separation, or divorce) were excluded. Children in the ADHD group were required to have a current diagnosis of ADHD based on a clinical interview with the parent and elevated T‐scores on the Inattention and/or Hyperactivity scales of the Teacher Rating Scale of the Behavioral Assessment System for Children (BASC‐2; Reynolds & Kamphaus, 2004). Typically developing controls could not have any current DSM‐IV‐TR Axis I diagnosis (except specific phobias or enuresis). All children were required to have an estimated full‐scale IQ > 75 and be free of magnetic resonance imaging contraindications. Those with a current or lifetime diagnosis of autism spectrum disorder, bipolar disorder, psychotic disorder, or post‐traumatic stress disorder were excluded from all groups. For the current study, data from all the enrolled groups were combined.
2.2. Participants
One hundred seventy‐two children from the initial study were eligible to be recruited for the follow‐up study. The study team reached out to these families via phone, email, and/or postal mail. During this follow‐up period, nationwide COVID‐19 restrictions were put in place, which detrimentally affected retention rates. Eighty adolescents completed the follow‐up study; the other 92 could not be reached or declined participation. Due to missing data, one participant was excluded from both parent and youth‐report analyses and data from another participant was excluded from only the youth‐report analyses. Thus, the total sample for analysis was 79 (M age = 13.4 years, SDage = 1.6 years). Compared to those who did not return for the follow‐up study, participants in the current sample were less likely to be female (17.5% of adolescents who completed the follow‐up study were female, which was significantly lower than the number of females who did not return (32.6%; (χ2 [1, 172] = 5.13, p = 0.024))) and to be from the original typically developing control group (χ 2 [2, 172] = 11.68, p = 0.003). There were no age differences between those who completed the follow up study and those who did not (t [170] = 1.70, p = 0.091; Cohen's d = 0.26).
Participants in the follow‐up study were predominantly male (79.7%), which can be attributed to the enrollment criteria of having impairing emotional outbursts with or without ADHD for the initial study, as boys are more likely to have an ADHD diagnosis (Willcutt, 2012) and temper outbursts (Demmer et al., 2017) than girls. Participants were racially and socioeconomically diverse: over half the participants identified as non‐White, and participants represented a range of income levels. Participant characteristics are presented in Table 1.
TABLE 1.
Participant characteristics.
All participants | Parent‐report | Youth‐report | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
Meet irritability cutoff | Not meet irritability cutoff | t | df | p | Meet irritability cutoff | Not meet irritability cutoff | t | df | p | ||
n | 79 | 34 | 45 | 30 | 48 | ||||||
Age, M (SD) | 13.4 (1.6) | 13.47 (1.00) | 13.42 (2.02) | −0.120 | 77 | 0.905 | 13.65 (1.12) | 13.32 (1.92) | −0.824 | 76 | 0.382 |
χ 2 | df | p | χ 2 | df | p | ||||||
---|---|---|---|---|---|---|---|---|---|---|---|
Boys, % | 79.7 | 73.5 | 84.4 | 1.43 | 1 | 0.232 | 76.7 | 83.3 | 0.528 | 1 | 0.467 |
Child race | 11.43 | 6 | 0.076 | 7.72 | 6 | 0.260 | |||||
White | 45.6 | 50.0 | 42.2 | 53.3 | 39.6 | ||||||
African American | 24.1 | 29.4 | 20.0 | 33.3 | 18.8 | ||||||
Asian | 2.5 | 0.0 | 4.4 | 0.0 | 4.2 | ||||||
American Indian/Alaskan Native | 1.3 | 0.0 | 2.2 | 0.0 | 2.1 | ||||||
Mixed | 8.9 | 2.9 | 13.3 | 3.3 | 12.5 | ||||||
Other | 6.3 | 0.0 | 11.1 | 3.3 | 8.3 | ||||||
Child ethnicity | |||||||||||
Hispanic | 27.8 | 21.2 | 32.6 | 1.31 | 1 | 0.519 | 33.3 | 25.0 | 1.21 | 1 | 0.547 |
Household income, % | 7.00 | 6 | 0.321 | 12.34 | 6 | 0.055 | |||||
Less than $50,000 | 21.5 | 29.3 | 15.5 | 40.0 | 10.5 | ||||||
$50,000–99,999 | 24.1 | 20.6 | 26.7 | 13.3 | 29.1 | ||||||
$100,000 or more | 54.4 | 50.0 | 57.8 | 46.7 | 60.4 | ||||||
Child diagnosis, % | |||||||||||
ODD | 27.5 | 58.8 | 4.4 | 28.50 | 1 | <0.001 | 50.0 | 12.5 | 13.20 | 1 | <0.001 |
DMDD | 7.5 | 17.6 | 0.0 | 8.59 | 1 | 0.003 | 13.3 | 4.2 | 2.19 | 1 | 0.139 |
MDD | 2.5 | 5.9 | 0.0 | 2.72 | 1 | 0.099 | 6.7 | 0.0 | 3.28 | 1 | 0.070 |
PDD | 1.3 | 2.9 | 0.0 | 1.34 | 1 | 0.247 | 3.3 | 0.0 | 1.62 | 1 | 0.203 |
GAD | 11.3 | 24.2 | 2.2 | 9.04 | 1 | 0.003 | 20.7 | 4.2 | 5.30 | 1 | 0.021 |
ADHD | 46.8 | 61.8 | 38.2 | 5.34 | 1 | 0.021 | 46.7 | 53.3 | 0.005 | 1 | 0.943 |
2.3. Measures
2.3.1. Irritability
Parent‐reported irritability was assessed using the Youth version of the Multidimensional Assessment Profile Scales‐Temper Loss (MAPS‐TL‐Youth). The original MAPS‐TL [formerly the Multidimensional Assessment Profile of Disruptive Behavior ‐MAP‐DB] has been previously validated across early childhood (Krogh‐Jespersen et al., 2021; Wakschlag et al., 2014, 2015; White et al., 2016; Wiggins et al., 2018). A youth version was developed to enable developmentally appropriate dimensional measurement as children grew older (Biedzio & Wakschlag, 2019; Hirsch et al., 2023). As described in detail in Hirsch et al. (2023), the parent‐reported MAPS‐TL‐Youth retains the 22 “core” temper loss items from the preschool version of the MAPS‐TL and includes 18 additional developmentally‐specific items to assess behaviors and situations that are more relevant to this age group. Items are rated on frequency within the last month, (0 = Never; 1 = Rarely [less than once per week]; 2 = Some [1–3] days of the week; 3 = Most [4–6] days of the week; 4 = Every day of the week; 5 = Many times each day).
Youth‐reported irritability was measured using a self‐report version of the MAPS‐TL‐Youth Self‐Report (MAPS‐TL‐Youth‐SR) that includes 36 items (32 slightly reworded items from the parent‐reported MAPS‐TL‐Youth and 4 additional items regarding the child’s own feelings/emotions (feel grouchy, feel angry) and losing their temper with others (lose temper with brother/sister, lose temper with other kids). Prior factor analyses were conducted to confirm a single factor model for both parent‐report and self‐report versions of the MAP‐TL‐Youth (Alam et al., 2023). See Supplemental Tables S1 and S2 for lists of items included in parent‐report and youth‐report MAPS‐TL‐Youth.
2.3.2. Impairment
Irritability‐related impairment was measured using the impairment‐of‐functioning item of the ARI (Stringaris et al., 2012). The ARI has seven items; the first six assess irritability over the past 6 months and the seventh item asks the parent [or child] to rate the following statement regarding impairment on a 3‐point scale from never true to certainly true: “Overall, [my] irritability causes him/her [me] problems.” In the present study, impairment was treated as a binary measure using endorsements of somewhat true or certainly true to indicate that impairment was present.
2.3.3. Clinical disorders
Child DSM‐5 diagnoses for the irritability related diagnoses of interest in the current study, ODD, GAD, DMDD, PDD, and MDD, were assessed using the Kiddie Schedule for Affective Disorders and Schizophrenia Lifetime Version (K‐SADS‐PL) (Kauffman et al., 1997), a semi‐structured interview, administered by trained clinical psychology doctoral or master's students to the parent and adolescent individually. Final diagnoses were made by consensus during case conferences led by the study principal investigator, a licensed clinical psychologist.
2.3.4. Internalizing & externalizing problems
The Internalizing and Externalizing Problems subscales of the Child Behavior Checklist (CBCL; Achenbach & Edelbrock, 1979) were used to establish validity of the parent‐report and youth‐report clinically‐optimized irritability cutoff scores. The CBCL is a parent‐reported questionnaire composed of eight DSM‐oriented scales, eight syndrome scales, two combined scale scores (internalizing symptoms, externalizing symptoms) and a total score. Parents rated their child's problem behaviors on a 3‐point Likert scale, where 0 indicates that a behavior is absent, 1 indicates that it occurs sometimes, and 2 that it occurs often.
2.3.5. Quality of life
Child overall quality of life was measured using the Youth Quality of Life Instrument‐Short Form (YQOL‐SF) Version 2.0, derived from the YQOL‐R originally developed by Edwards et al., 2002. The YQOL‐SF is a well validated youth‐report measure of quality of life (Hoang et al., 2021; Patrick et al., 2002; Topolski et al., 2002) that assesses quality of life across two domains: Environment and Relationships and Belief in Self and Family. The YQOL‐SF has 15 questions scored on a 10‐point Likert Scale ranging from Not at all to Very much, with higher scores indicating higher quality of life. Each item's 10‐point score is transposed to a T‐score on a 100‐point scale, and the mean score of the 15 items is used to measure youth quality of life.
2.4. Analytic overview
2.4.1. Aim 1: Derive a pragmatic version of parent‐ and youth‐report MAPS‐TL‐Youth that predicts irritability‐related impairment
Stepwise logistic regression, using forward entry with likelihood ratio comparison of models, in IBM SPSS statistical software (IBM Corp.) was used to identify the MAPS‐TL‐Youth irritability items most strongly related to ARI impairment score for the parent and youth‐reported scales separately. Stepwise logistic regression tests items both individually and in combination to ascertain the items with the strongest relationship to impairment and, compared to other options such as LASSO, is a more robust tool given the collinearity of the items on the MAPS‐TL‐Youth and small sample size, as the purpose is to build a parsimonious model step‐by‐step. Analyses covaried for sociodemographic variables including sex, race, ethnicity, age, and household income (Table 1).
2.4.2. Aim 2: Generate clinically optimized MAPS‐TL‐Youth screeners via empirically derived cutoffs for parent and youth reports in relation to DSM irritability related syndromes
MAPS‐TL‐Youth items identified in Aim 1 as most strongly related to ARI impairment were summed for the parent and youth reports, respectively, to create pragmatic scales for each. For each youth and parent report independently, receiver operating characteristic (ROC) analyses were used to identify cutoff scores that balanced the probability of correctly identifying true positive cases (concurrent diagnosis of an irritability‐related DSM diagnosis) and true negative cases (no diagnosis of an irritability‐related DSM diagnosis). The area under the curve (AUC) statistic was used to determine the accuracy of the ROC analyses; values between 0.7 and 0.8 are considered good, AUC values between 0.8 and 0.9 are considered excellent, and AUC values above 0.9 are outstanding (Hosmer & Lemeshow, 2000). Analyses were implemented in R with package OptimalCutpoints (Lopez‐Raton et al., 2014).
2.4.3. Aim 3: Establish concurrent validity of parent‐ and youth‐reported clinically optimized screeners
To establish concurrent validity of the clinically optimized screeners, participants who scored positive on the screener were compared to youth who did not in relation to internalizing and externalizing problems and quality of life. Three t‐tests were conducted in IBM SPSS statistical software (IBM Corp., Armonk, NY) with scoring positive vs. negative on the parent‐reported screener as the predictor, and youth internalizing and externalizing problems and overall quality of life as outcome variables. These analyses were repeated using the youth‐reported screener dichomotimized scores as the predictor.
[Correction added on 19 October 2023, after first online publication: In the preceding section, ‘cutoff scores’ was revised to ‘screeners’.]
3. RESULTS
3.1. Aim 1: Identify parent‐report and youth‐report clinically significant irritable behaviors
3.1.1. Parent‐report
The final stepwise logistic regression model of the parent‐reported MAPS‐TL‐Youth identified 3 of 40 items associated with parent‐reported ARI impairment: become frustrated easily (p = 0.005), angry/irritable/grouchy throughout the day (p = 0.022), and difficulty calming down when angry (p = 0.049). As the frequencies of these three behaviors increased, the odds of observing impairment, adjusted for sociodemographic characteristics (sex, age, race, ethnicity, and household income), increased by 5.40, 3.63, and 3.26 respectively. These three items together explained 76.1% of the variance in parent‐reported ARI impairment and were a good fit to our data (Hosmer & Lemeshow χ2 = 3.56, df = 7, p = 0.829).
3.1.2. Youth‐report
The confirmatory factor analysis showed acceptable fit for the MAPS‐TL‐Youth‐SR items on one factor statistically (χ 2 [594, N = 199] = 801.393, p = <0.001) and descriptively (CFI = 0.97, TLI = 0.97, RMSEA = 0.067, p = 0.015, WRMR = 0.978). The final stepwise logistic regression model identified 2 of 36 MAPS‐TL‐Youth‐SR items that were associated with youth‐reported ARI impairment, trouble calming down when angry (p = 0.002) and hit/shove/kick when lose temper (p = 0.002). As the frequencies of these two behaviors increased, the odds of observing impairment increased by 4.32 and 4.30 respectively. The two items explained 55.7% of the variance in youth‐reported ARI impairment and showed good model fit (Hosmer & Lemeshow χ2 = 7.44, df = 4, p = 0.114).
See Figures 2a and 2b for distributions of scores on the individual items for those who met versus did not meet the cutoffs.
FIGURE 2.
(a) Comparison of frequencies MAPS‐TL‐Youth items by parent report for participants who did and did not meet the irritability cutoff. (b) Comparison of self‐reported frequencies of the identified MAPS‐TL‐Youth‐SR items for participants who did and did not meet the irritability cutoff. MAPS‐TL‐Youth‐SR rating anchors: 0 = never in the past month; 1 = rarely (less than weekly); 2 = some days (1–3 days per week); 3 = most days (4–6 days of the week); ≥4 = daily or greater. [Correction added on 10 October 2023, after first online publication: Figure 2 has been updated.]
3.2. Aim 2: Generate clinically optimized MAPS‐TL‐Youth screeners via empirically derived cutoffs for parent and youth reports in relation to DSM irritability‐related syndromes
3.2.1. Parent‐report
We summed the scores of the parent‐reported items identified in the previous logistic regression (become frustrated easily, angry/irritable/grouchy throughout the day, and difficulty calming down when angry). In the ROC analysis, area under the curve (AUC = 0.85, p < 0.001) indicated excellent classification accuracy of the summed clinically optimized score for DSM irritability‐related diagnoses (AUCs for individual items ranged from 0.62 to 0.86). A cutoff score of 4 (peak Youden Index = 0.68) optimally balanced sensitivity (91%) and specificity (77%), meaning that 91% of adolescents who were diagnosed with an irritability‐related disorder were correctly classified as meeting the cutoff criteria, and 77% of adolescents who did not meet criteria for an irritability‐related disorder were correctly classified as not meeting the cutoff criteria (Figure 1a).
FIGURE 1.
(a). Receiver operating characteristic (ROC) curve for parent‐reported clinically optimized irritability score (criterion variable) and DSM irritability‐related diagnosis (classification variable). (b). Receiver operating characteristic (ROC) curve for youth‐reported clinically optimized irritability score (criterion variable) and DSM irritability‐related diagnosis (classification variable).
Figure 2a shows the frequencies of scores on each of the MAPS‐TL‐Youth items in the parent‐report clinically optimized cutoff score. For participants who met the parent‐reported cutoff (n = 34, 43.0%), nearly all (94.1%) were rated as frustrated easily 1–3 days per week or more. There was greater variability in item endorsement for the other items that suggest dysregulated/developmentally inappropriate behaviors; that is, difficulty calming down when angry (5.9% never, 23.5% less than once per week 55.9% 1–3 days per week, and 14.7% most days of the week or greater), and angry/irritable/grouchy throughout the day (11.8% never, 32.4% less than once per week, 20.6% 1–3 days per week, and 35.3% every day or more). Of adolescents who did not meet the parent‐reported irritability cutoff, almost two‐thirds (64.5%) were reported to have mild frequencies of becoming frustrated easily (score of 1 or 2), consistent with the more normative nature of this behavior. Additionally, nearly all adolescents below the parent‐report cutoff (97.8%) never or infrequently had difficulty calming down when angry or were angry/irritable/grouchy throughout the day in the past month. Adolescents above and below the parent‐reported cutoff were sociodemographically comparable (Table 1).
3.2.2. Youth‐report
In parallel, we summed the items identified in Aim 1 for the MAPS‐TL‐Youth‐SR, hit/shove/kick when lost temper and trouble calming down when angry (Mean = 1.77, SD = 2.19, Range = 0–10), for use in the ROC. The summed score showed good classification accuracy for irritability‐related diagnoses (AUC = 0.76, p < 0.001; AUCs for individual items were 0.54 and 0.81). The cutoff score of 2 (peak Youden Index = 0.48) optimally balanced sensitivity (73%) and specificity (75%). In other words, using the derived cutoff score, 73% of adolescents who were diagnosed with an irritability‐related disorder were correctly classified as meeting the cutoff criteria, and 75% of adolescents who were not diagnosed with an irritability‐related disorder were correctly classified as not meeting the cutoff criteria (Figure 1b).
For adolescents who met the youth‐reported irritability cutoff (n = 30, 38.5%), nearly all (93.3%) had trouble calming down when angry in the past month, with over half (60.0%) reporting this behavior occurring at least weekly. However, the scores were more evenly distributed for the more dysregulated hit/shove/kick when lost temper item, with 26.7% of adolescents who scored above the cutoff reporting never having this behavior in the past month, 30.0% less than once per week, 13.3% 1–3 days per week, and 30% most days of the week or greater. In contrast, all adolescents who scored below the youth‐reported irritability cutoff never or infrequently had trouble calming down when angry and almost all (97.9%) of these adolescents never hit/shoved/kicked when lost temper in the past month. Adolescents above versus below the youth‐report cutoff did not differ significantly on sociodemographic variables (Table 1).
3.3. Aim 3: Establish concurrent validity of parent and youth‐reported MAPS‐TL‐Youth clinically optimized screeners
3.3.1. Parent report
Participants who scored above the cutoff on the MAPS‐TL‐Youth parent‐report clinically optimized screener had significantly increased internalizing problems (t 72 = 6.06, p < 0.001), externalizing problems (t 72 = 10.21, p < 0.001), and significantly decreased overall quality of life (t 75 = −4.13, p < 0.001) than those who scored below the cutoff (Table 2).
TABLE 2.
Results of t‐test for adolescents scoring above & below each cutoff.
Variable | Above cutoff mean | Below cutoff mean | t | p | 95% CI | |
---|---|---|---|---|---|---|
LL | UL | |||||
Parent‐report | ||||||
CBCL internalizing problems | 60.218 | 46.214 | 6.063 | <0.001 | 9.400 | 18.609 |
CBCL externalizing problems | 63.000 | 45.023 | 10.208 | <0.001 | 14.466 | 21.487 |
YQOL score | 66.791 | 82.385 | −4.514 | <0.001 | −22.474 | −8.713 |
Youth‐report | ||||||
CBCL internalizing problems | 58.206 | 48.444 | 3.696 | <0.001 | 4.497 | 15.028 |
CBCL externalizing problems | 59.551 | 48.444 | 4.496 | <0.001 | 6.183 | 16.032 |
YQOL score | 67.172 | 81.181 | −3.873 | <0.001 | −21.213 | −6.804 |
3.3.2. Youth report
Similarly, participants who scored above the cutoff on the MAPS‐TL‐Youth‐SR clinically optimized screener had significantly increased internalizing problems (t = 0.3.70, p < 0.001), externalizing problems (t = 4.50, p < 0.001), and significantly decreased overall quality of life (t = −3.87, p < 0.001).
4. DISCUSSION
As the science base grows for irritability as a robust transdiagnostic marker of mental health risk (e.g., Evans et al., 2017; Wakschlag et al., 2018; Wiggins et al., 2018), theory‐ and data‐based translation to “real world” clinical practice becomes the next challenge. Here, we draw on theory emphasizing the importance of contrasting normal versus abnormal behavioral patterns within the context of a particular developmental period (Marquand et al., 2019; Wakschlag et al., 2010) to meet the need for a developmentally specified pragmatic screener for adolescent neurodevelopmental vulnerability. Adolescence is a time of increased vulnerability for psychopathology (Merikangas et al., 2010). Moreover, adolescence is a period of heightened neuroplasticity (Nelson et al., 2004), and may serve as the “last chance” to apply interventions before the problems of childhood harden into entrenched psychiatric disorders later in adulthood. Identifying the adolescents in need of intervention must be inexpensive, brief, and pragmatically implemented in order to have “real world” impact, and the screener tools that we generate here are a step toward this reality. Moreover, we increase the impact by allowing the reporter to be either the parent or the adolescent; this flexibility increases the likelihood that clinics can flag risk even as parents and youths become increasingly independent of each other. Findings suggest that parent‐report performs very well as a screener, but in the absence of parent‐report, adolescent‐report is sufficient. The addition of this adolescent pragmatic screener to the complement of MAPS screening tools (preschool age, Wiggins et al., 2018; infant/toddlerhood, Wiggins, Rosario, et al., 2023, and in school age, Hirsch et al., 2023) is a step toward fruition of the promise of developmentally specific measurement in “real world” settings.
The parent report items in the adolescent screener mirror prior work in preschoolers (Wiggins et al., 2018) and infant/toddlers (Wiggins, Rosario, et al., 2023), identifying mild, common irritable behaviors (easily frustrated) at high frequency, in combination with severe, uncommon pathognomonic items at low frequency (angry/irritable/grouchy throughout the day and difficulty calming down when angry) as sensitively and specifically flagging adolescents with neurodevelopmental vulnerability. As with other age‐specific MAPS‐TL screeners that have been derived for preschoolers, school age children, and preadolescents, frustrated easily, a mild, normative item, emerged as one of the key items most closely associated with irritability‐related impairment. Virtually every adolescent (94%) who met the screener cutoff became frustrated easily at least 1–3 days per week. Because frequency clearly differentiates clinical concern from normative misbehavior for low frustration tolerance, this suggests that screening without objective frequency thresholds would be likely to over‐identify. This suggests that low frustration tolerance may be the necessary (but not sufficient) foundation for clinically significant irritability across development, but that less frequent, severe items, are more developmentally specific.
The other items identified for the parent‐report MAPS‐TL‐Youth screener, difficulty calming down when angry and angry/irritable/grouchy throughout the day, were endorsed at various frequencies by parents who scored above the cutoff, but rarely endorsed by parents who scored below the cutoff. Thus, the presence of emotion regulation difficulties along with persistent irritable mood, even if rare (i.e., less than once per week), in combination with a more common item (i.e., low frustration tolerance) at higher frequencies, signifies clinically significant irritability, suggesting that sole reliance on assessment of extreme behaviors would not be discriminative of impairing irritability in adolescence. Further, these three items in the parent report support conceptualizations of clinical irritability as reflecting frustration intolerance, emotion dysregulation, and persistent angry/irritable mood (American Psychiatric Association, 2013; Brotman et al., 2017; Stringaris et al., 2012).
By contrast, the youth self‐report items flagging clinically significant irritability were both severe, uncommon items from the youth's perspective (difficulty calming down when angry and hit/shove/kick when lost temper). That is, low frustration tolerance by youth’s own report was not identified as a highly discriminative item for the screener, which is parallel to findings from an independent sample at pre‐adolescence (Alam et al., 2023). This may be a methodologic artifact in terms of how youth view the meaning of this construct, an area that requires further study, as the papers in this issue are the first to examine validity of the MAPS‐TL via self‐report. On the other hand, it may reflect a true difference in how youth perceive their own regulatory behaviors. [Correction added on 10 October 2023, after first online publication: The preceding three sentences were added and the sentence ‘It may be that adolescents find these severe items more salient than more common irritable behaviors that appeared in the parent‐reported irritability cutoff score like frustrated easily.’ was deleted.] Indeed, Bridgett et al. (2015) found that parents tended to rate their children's negative emotions, such as anger and frustration, as more problematic or abnormal than adolescents did. Moreover, prior research has found that parents may overestimate the severity of their child's difficulties, while children may underreport their own problems (Achenbach & Edelbrock, 1981; Cheah et al., 2013). Thus, more severe items may be necessary from the youth perspective to flag clinically significant irritability.
It is clear from these findings that parents and their children have similar and distinct perceptions of irritability at this developmental stage. This is consistent with recent work that suggests including each reporters' distinct perspectives is essential to garner a holistic view of the child's irritability from multiple contexts (De Los Reyes & Epkins, 2023). Adolescents are better able to accurately report on their own internal experiences and emotions, and self‐reported irritability in adolescence is less likely to be confounded by biases inherent to collateral‐report (Stringaris et al., 2013). Despite parents and children having distinct perspectives, there was a common item that appeared in both reporters' irritability screeners: difficulty calming down when angry. A protracted recovery from irritable mood or behavior was relevant viewed both from the outside (parent‐report) and experienced on the inside (youth‐report)— suggesting that it is particularly salient for clinical differentiation in the preadolescent‐adolescent period (See also Alam et al., 2023).
This study advances application of the MAPS‐TL‐Youth as the only transdiagnostic dimensional and pragmatic scale of its kind. [Correction added on 10 October 2023, after first online publication: The preceding sentence was added.] However, there are several limitations that should be acknowledged. First, our primary validation measures are based on either parent‐report or youth‐report (surveys and semi‐structured interview) which may be susceptible to shared method variance with the parent‐ and youth‐reported MAPS‐TL‐Youth items under evaluation. We mitigated this to an extent by using multiple informant‐reports in analyses where possible. Still, we were unable to compare parent and youth‐reported irritability cutoff scores against one another due to the lack of an objective measurement unaffected by shared‐method variance and reporter biases. Future large population‐based and clinical studies should incorporate cross‐method and cross‐informant approaches, including behavioral and objective measures in conjunction with input from multiple informants. For instance, including direct behavioral observation as well as input from teachers has been shown to improve the sensitivity and specificity of mental health screening (Achenbach et al., 1987; Costello et al., 1996; Goodman et al., 2000). Incorporating these approaches can help researchers control for shared methods variance, analyze informant convergence and discrepancies, and determine the extent to which including youth‐reported measures enhances parent‐reported irritability scores. Second, while our study is a promising first step towards a pragmatic adolescent screener, we are limited by the specific characteristics of our sample. That is, our study was conducted as a follow‐up to a larger study, where retention was detrimentally affected by the COVID‐19 pandemic. Moreover, our sample's higher percentage of boys compared to girls can be attributed to the original study's sex distribution. Replication and extension within both referred and population‐based samples that are more gender balanced and intentionally sample of individuals with varied lived experience will be important for further validation.
The present study empirically identified irritable behaviors that sensitively and specifically characterize clinically impairing irritability in adolescents. These clinically optimized scores consist of a brief 3‐item parent‐report and a 2‐item youth‐report screener, making them practical and efficient tools to screen for clinically salient irritability. While irritability is considered to be normative in adolescence, elevated levels of irritability during this period have been linked to various negative outcomes in adulthood, such as depression, substance use, and poorer social and overall functioning (Copeland et al., 2014; Stringaris et al., 2009). Furthermore, the importance of developmentally‐specific measurement cannot be overstated— the phenotypic expressions of irritability vary greatly from early childhood through adolescence (Copeland et al., 2014; McClellan et al., 2023; Pine et al., 1999; Stringaris & Goodman, 2009; Wakschlag et al., 2010) and measures of irritability as transdiagnostic indicators of mental health risk are optimized when they capture this developmental variation in clinical expression in meaningful ways. Adolescence presents a peak opportunity to identify and address heightened irritability in youth before they transition to adulthood. Therefore, it is crucial for mental health professionals to have pragmatic, developmentally informed tools that can efficiently and effectively identify youth who may need clinical attention.
AUTHOR CONTRIBUTIONS
Nathan Kirk: Data curation; Formal analysis; Writing—original draft; Writing—review & editing. Emily Hirsch: Conceptualization; Investigation; Writing—original draft; Writing—review & editing. Tasmia Alam: Data curation; Formal analysis. Lauren S. Wakschlag: Conceptualization; Supervision; Validation; Writing—review and editing. Jillian Lee Wiggins: Conceptualization; Methodology; Supervision; Validation; Writing—review and editing. Amy Roy: Conceptualization; Funding acquisition; Investigation; Project administration; Supervision; Writing—review and editing.
CONFLICT OF INTEREST STATEMENT
The authors have no conflicts of interest.
Supporting information
Supporting Information S1
ACKNOWLEDGMENTS
Research reported in this publication was supported by funding from R01MH0911401 and R15MH115356. The authors wish to thank Jill Stadterman, Melanie Silverman, Kaley Davis, and Erica Ferrara for assistance in data management and the families who participated.
Kirk, N. , Hirsch, E. , Alam, T. , Wakschlag, L. S. , Wiggins, J. L. , & Roy, A. K. (2023). A pragmatic, clinically optimized approach to characterizing adolescent irritability: Validation of parent‐ and adolescent reports on the Multidimensional Assessment Profile Scales—Temper Loss Scale. International Journal of Methods in Psychiatric Research, 32(S1), e1986. 10.1002/mpr.1986
Jillian Lee Wiggins and Amy K. Roy are equal contribution as senior authors.
[Correction added on 19 October 2023, after first online publication: ‘MAPS‐TL‐Adolescent’ was changed to ‘MAPS‐TL‐Youth’ throughout. The text has been updated to improve readability throughout.]
Contributor Information
Nathan Kirk, Email: nkirk2@sdsu.edu.
Jillian Lee Wiggins, Email: jillian.wiggins@sdsu.edu.
DATA AVAILABILITY STATEMENT
The data that support the findings of this study are available from the corresponding author upon reasonable request.
REFERENCES
- Abramovitch, A. , Abramowitz, J. S. , McKay, D. , Cham, H. , Anderson, K. S. , Farrell, L. J. , Geller, D. A. , Hanna, G. L. , Mathieu, S. , McGuire, J. F. , Rosenberg, D. R. , Evelyn, S. S. , Storch, E. A. , & Wilhelm, S. (2022). An ultra‐brief screening scale for pediatric obsessive‐compulsive disorder: The OCI‐CV‐5. Journal of Affective Disorders, 312, 208–216. 10.1016/j.jad.2022.06.009 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Achenbach, T. M. (2006). As others see us: Clinical and research implications of cross‐informant correlations for psychopathology. Current Directions in Psychological Science, 15(2), 94–98. 10.1111/j.0963-7214.2006.00414.x [DOI] [Google Scholar]
- Achenbach, T. M. , & Edelbrock, C. S. (1979). Child behavior checklist. National Institute of Mental Health. [Google Scholar]
- Achenbach, T. M. , & Edelbrock, C. S. (1981). Behavioral problems and competencies reported by parents of normal and disturbed children aged four through sixteen. Monographs of the Society for Research in Child Development, 46(1), 1–82. 10.2307/3332957 [DOI] [PubMed] [Google Scholar]
- Achenbach, T. M. , McConaughy, S. H. , & Howell, C. T. (1987). Child/adolescent behavioral and emotional problems: Implications of cross‐informant correlations for situational specificity. Psychological Bulletin, 101(2), 213–232. 10.1037/0033-2909.101.2.213 [DOI] [PubMed] [Google Scholar]
- Alam, T. , Hirsch, E. , Nili, A. , Kirk, N. , Zhang, Y. , Wiggins, J. L. , Roy, A. K. , & Wakschlag, L. S. (2023). Characterizing the spectrum of irritability in preadolescence: Dimensional and pragmatic applications. International Journal of Methods in Psychiatric Research, e1988. 10.1002/mpr.1988 [DOI] [PMC free article] [PubMed] [Google Scholar]
- American Psychiatric Association . (2013). Diagnostic and statistical manual of mental disorders (DSM‐5®). American Psychiatric Publication. [Google Scholar]
- Beauchaine, T. P. , & Tackett, J. L. (2020). Irritability as a transdiagnostic vulnerability trait: Current issues and future directions. Behavior Therapy, 51(2), 350–364. 10.1016/j.beth.2019.10.009 [DOI] [PubMed] [Google Scholar]
- Biedzio, D. , & Wakschlag, L. S. (2019). Developmental emergence of disruptive behaviors beginning in infancy: Delineating normal: abnormal boundaries to enhance early identification. In Zeenah C. (Ed.), Handbook of infant mental health (4th ed., pp. 407–425). The Guilford Press. [Google Scholar]
- Bridgett, D. J. , Burt, N. M. , Edwards, E. S. , & Deater‐Deckard, K. (2015). Intergenerational transmission of self‐regulation: A multidisciplinary review and integrative conceptual framework. Psychological Bulletin, 141(3), 602–654. 10.1176/appi.ajp.2016.16070839 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Brotman, M. A. , Kircanski, K. , & Leibenluft, E. (2017a). Irritability in children and adolescents. Annual Review of Clinical Psychology, 13(1), 317–341. 10.1146/annurev-clinpsy-032816-044941 [DOI] [PubMed] [Google Scholar]
- Brotman, M. A. , Kircanski, K. , Stringaris, A. , Pine, D. S. , & Leibenluft, E. (2017b). Irritability in youths: A translational model. American Journal of Psychiatry, 174(6), 520–532. 10.1176/appi.ajp.2016.16070839 [DOI] [PubMed] [Google Scholar]
- Casey, B. (2015). Beyond simple models of self‐control to circuit‐based accounts of adolescent behavior. Annual Review of Psychology, 66(1), 295–319. 10.1146/annurev-psych-010814-015156 [DOI] [PubMed] [Google Scholar]
- Casey, B. J. , Jones, R. M. , Levita, L. , Libby, V. , Pattwell, S. S. , Ruberry, E. J. , Soliman, F. , & Somerville, L. H. (2010). The storm and stress of adolescence: Insights from human imaging and mouse genetics. Developmental Psychobiology: The Journal of the International Society for Developmental Psychobiology, 52(3), 225–235. 10.1002/dev.20447 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cheah, C. S. L. , Van Hook, M. P. , & Ghazarian, S. R. (2013). Understanding discrepancies in parent‐child reporting of emotional and behavioral problems: Effects of relational and socio‐demographic factors. Journal of Child and Family Studies, 22(3), 371–382. 10.1007/s10826-012-9581-2 [DOI] [Google Scholar]
- Copeland, W. E. , Shanahan, L. , Egger, H. , Angold, A. , & Costello, E. J. (2014). Adult diagnostic and functional outcomes of DSM‐5 disruptive mood dysregulation disorder. American Journal of Psychiatry, 171(6), 668–674. 10.1176/appi.ajp.2014.13091213 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Costello, E. J. , Farmer, E. M. Z. , Angold, A. , Burns, B. J. , & Erkanli, A. (1996). Psychiatric disorders among American Indian and white youth in Appalachia: The great smoky mountains study. American Journal of Public Health, 86(2), 230–236. 10.2105/ajph.87.5.827 [DOI] [PMC free article] [PubMed] [Google Scholar]
- De Los Reyes, A. , Augenstein, T. M. , Wang, M. , Thomas, S. A. , Drabick, D. A. , Burgers, D. E. , & Rabinowitz, J. (2015). The validity of the multi‐informant approach to assessing child and adolescent mental health. Psychological Bulletin, 141(4), 858–900. 10.1037/a0038498 [DOI] [PMC free article] [PubMed] [Google Scholar]
- De Los Reyes, A. , & Epkins, C. C. (2023). A dozen years of demonstrating that informant discrepancies are more than measurement error: Toward guidelines for integrating data from multi‐informant assessments of youth mental health. Journal of Clinical Child and Adolescent Psychology, 52(1), 1–18. 10.1080/15374416.2022.2158843 [DOI] [PubMed] [Google Scholar]
- De Los Reyes, A. , & Kazdin, A. E. (2005). Informant discrepancies in the assessment of childhood psychopathology: A critical review, theoretical framework, and recommendations for further study. Psychological Bulletin, 131(4), 483–509. 10.1037/0033-2909.131.4.483 [DOI] [PubMed] [Google Scholar]
- Demmer, D. H. , Hooley, M. , Sheen, J. , McGillivray, J. A. , & Lum, J. A. (2017). Sex differences in the prevalence of oppositional defiant disorder during middle childhood: A meta‐analysis. Journal of Abnormal Child Psychology, 45(2), 313–325. 10.1007/s10802-016-0170-8 [DOI] [PubMed] [Google Scholar]
- Dougherty, L. R. , Galano, M. M. , Chad‐Friedman, E. , Olino, T. M. , Bufferd, S. J. , & Klein, D. N. (2021). Using item response theory to compare irritability measures in early adolescent and childhood samples. Assessment, 28(3), 918–927. 10.1177/1073191120936363 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Dougherty, L. R. , Smith, V. C. , Bufferd, S. J. , Stringaris, A. , Leibenluft, E. , Carlson, G. A. , & Klein, D. N. (2013). Preschool irritability: Longitudinal associations with psychiatric disorders at age 6 and parental psychopathology. Journal of the American Academy of Child & Adolescent Psychiatry, 52(12), 1304–1313. 10.1016/j.jaac.2013.09.007 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Edwards, T. C. , Huebner, C. E. , Connell, F. A. , & Patrick, D. L. (2002). Adolescent quality of life, Part I: Conceptual and measurement model. Journal of Adolescence, 25(3), 275–286. 10.1006/jado.2002.0470 [DOI] [PubMed] [Google Scholar]
- Evans, S. C. , Burke, J. D. , Roberts, M. C. , Fite, P. J. , Lochman, J. E. , Francisco, R. , & Reed, G. M. (2017). Irritability in child and adolescent psychopathology: An integrative review for ICD‐11. Clinical Psychology Review, 53, 29–45. 10.1016/j.cpr.2017.01.004 [DOI] [PubMed] [Google Scholar]
- Evans, S. C. , Corteselli, K. A. , Edelman, A. , Scott, H. , & Weisz, J. R. (2022). Is irritability a top problem in youth mental health care? A multi‐informant, multi‐method investigation. Child Psychiatry and Human Development, 54(4), 1027–1041. 10.1007/s10578-021-01301-8 [DOI] [PubMed] [Google Scholar]
- Finlay‐Jones, A. , Ang, J. , Brook, J. S. , Lucas, J. , Macneill, L. , Mancini, V. , Lim, Y. , Kottampally, K. , Elliott, C. , Smith, J. , & Wakschlag, L. (2023). Systematic review and meta‐analysis: Early irritability as a transdiagnostic neurodevelopmental vulnerability to later mental health problems. Journal of the American Academy of Child & Adolescent Psychiatry. e‐pub ahead of print. 10.1016/j.jaac.2023.01.018 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Frazier, E. A. , Liu, R. T. , Massing‐Schaffer, M. , Hunt, J. , Wolff, J. , & Spirito, A. (2016). Adolescent but not parent report of irritability is related to suicidal ideation in psychiatrically hospitalized adolescents. Archives of Suicide Research, 20(2), 280–289. 10.1080/13811118.2015.1004497 [DOI] [PubMed] [Google Scholar]
- Goodman, R. , Ford, T. , Simmons, H. , Gatward, R. , & Meltzer, H. (2000). Using the Strengths and Difficulties Questionnaire (SDQ) to screen for child psychiatric disorders in a community sample. The British Journal of Psychiatry, 177(6), 534–539. 10.1192/bjp.177.6.534 [DOI] [PubMed] [Google Scholar]
- Grills, A. E. , & Ollendick, T. H. (2002). Issues in parent‐child agreement: The case of structured diagnostic interviews. Clinical Child and Family Psychology Review, 5(1), 57–83. 10.1023/A:1014573708569 [DOI] [PubMed] [Google Scholar]
- Hawes, M. T. , Carlson, G. A. , Finsaas, M. C. , Olino, T. M. , Seely, J. R. , & Klein, D. N. (2020). Dimensions of irritability in adolescents: Longitudinal associations with psychopathology in adulthood. Psychological Medicine, 50(16), 2759–2767. 10.1017/S0033291719002903 [DOI] [PubMed] [Google Scholar]
- Hirsch, E. , Alam, T. , Kirk, N. , Bevans, K. B. , Wakschlag, L. S. , Wiggins, J. L. , & Roy, A. K. (2023). Developmentally specified characterization of the irritability spectrum at early school age: Implications for pragmatic mental health screening. International Journal of Methods in Psychiatric Research, e1985. 10.1002/mpr.1985 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hoang, M. T. , Do, H. N. , Dang, T. Q. , Nguyen, T. T. , Nguyen, L. H. , Nguyen, C. T. , Doan, L. P. , Vu, G. T. , Ngo, T. V. , Latkin, C. A. , & Ho, C. S. H. (2021). Cross‐cultural adaptation and measurement properties of youth quality of life instrument‐short Form (YQOL‐SF) in a developing south‐east Asian country. PLoS One, 16(6), e0253075. 10.1371/journal.pone.0253075 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hosmer, D. W. , & Lemeshow, S. (2000). Applied logistic regression (2nd ed.). Wiley. [Google Scholar]
- Jo, S.‐J. , Jeong, H. , Son, H. J. , Lee, H. K. , Lee, S.‐Y. , Kweon, Y.‐S. , & Yim, H. W. (2020). Diagnostic usefulness of an ultra‐brief screener to identify risk of online gaming disorder for children and adolescents. Psychiatry Investigation, 17(8), 762–768. 10.30773/pi.2019.0279 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kaufman, J. , Birmaher, B. , Brent, D. , Rao, U. , Flynn, C. , Moreci, P. , Williamson, D. , & Ryan, N. (1997). Schedule for affective disorders and Schizophrenia for school‐age children‐present and lifetime version (K‐SADS‐PL): Initial reliability and validity data. Journal of the American Academy of Child & Adolescent Psychiatry, 36(7), 980–988. 10.1097/00004583-199707000-00021 [DOI] [PubMed] [Google Scholar]
- Klein, D. N. , Dougherty, L. R. , Kessel, E. M. , Silver, J. , & Carlson, G. A. (2021). A transdiagnostic perspective on youth irritability. Current Directions in Psychological Science, 30(5), 437–443. 10.1177/09637214211035101 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Krogh‐Jespersen, S. , Kaat, A. J. , Petitclerc, A. , Perlman, S. B. , Briggs‐Gowan, M. J. , Burns, J. L. , Adam, H. , Nili, A. , Gray, L. , & Wakschlag, L. S. (2021). Calibrating temper loss severity in the transition to toddlerhood: Implications for developmental science. Applied Developmental Science, 26(4), 785–798. 10.1080/10888691.2021.1995386 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lopez‐Raton, M. , Cadarso‐Suarez, C. , Rodriguez‐Alvarez, M. X. , & Gude‐Sampedro, F. (2014). OptimalCutpoints: An R Package for selecting optimal cutpoints in diagnostic tests. J StatSoftw, 61(8), 1–36. 10.18637/jss.v061.i08 [DOI] [Google Scholar]
- Marquand, A. F. , Kia, S. M. , Zabihi, M. , Wolfers, T. , Buitelaar, J. K. , & Beckmann, C. F. (2019). Conceptualizing mental disorders as deviations from normative functioning. Molecular Psychiatry, 24(10), 1415–1424. 10.1038/s41380-019-0441-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- McClellan, J. M. , Berliner, L. , & Carlson, G. A. (2023). Strategies for managing impairing emotional outbursts. JAMA Pediatrics, 177(6), 559. 10.1001/jamapediatrics.2023.0787 [DOI] [PubMed] [Google Scholar]
- Merikangas, K. R. , He, J. P. , Burstein, M. , Swanson, S. A. , Avenevoli, S. , Cui, L. , Benjet, C. , Georgiades, K. , & Swendsen, J. (2010). Lifetime prevalence of mental disorders in U.S. adolescents: Results from the national comorbidity survey Replication—Adolescent supplement (NCS‐A). Journal of the American Academy of Child & Adolescent Psychiatry, 49(10), 980–989. 10.1016/j.jaac.2010.05.017 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Morris, A. , Wakschlag, L. , Krogh‐Jespersen, S. , Fox, N. , Planalp, B. , Perlman, S. , Shuffrey, L. , Smith, B. , Lorenzo, N. , Amso, D. , Coles, C. , & Johnson, S. (2020). Principles for guiding the selection of early childhood neurodevelopmental risk and resilience measures: HEALthy brain and child development study as an exemplar. Adversity and Resilience Science, 1(4), 247–267. 10.1007/s42844-020-00025-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Nelson, E. , Leibenluft, E. , McClure, E. , & Pine, D. (2004). The social re‐orientation of adolescence. A neuroscience perspective on the process and its relation to psychopathology Psychological Medicine, 35(2), 163–174. 10.1017/S0033291704003915 [DOI] [PubMed] [Google Scholar]
- Orri, M. , Galera, C. , Turecki, G. , Boivin, M. , Tremblay, R. E. , Geoffroy, M.‐C. , & Côté, S. M. (2019). Pathways of association between childhood irritability and adolescent suicidality. Journal of the American Academy of Child & Adolescent Psychiatry, 58(1), 99–107.e103. 10.1016/j.jaac.2018.06.034 [DOI] [PubMed] [Google Scholar]
- Patrick, D. L. , Edwards, T. C. , & Topolski, T. D. (2002). Adolescent quality of life, Part II: Initial validation of a new instrument. Journal of Adolescence, 25(3), 287–300. 10.1006/jado.2002.0471 [DOI] [PubMed] [Google Scholar]
- Pine, D. , Cohen, E. , Cohen, P. , & Brook, J. (1999). Adolescent depressive symptoms as predictors of adult depression: Moodiness or mood disorder? American Journal of Psychiatry, 156(1), 133–135. 10.1176/ajp.156.1.133 [DOI] [PubMed] [Google Scholar]
- Powers, A. , & Casey, B. (2015). The adolescent brain and the emergence and peak of psychopathology. Journal of Infant, Child, and Adolescent Psychotherapy, 14(1), 3–15. 10.1080/15289168.2015.1004889 [DOI] [Google Scholar]
- Reynolds, C. R. , & Kamphaus, R. W. (2004). Behavior assessment system for children (2nd ed.). Pearson Assessments. [DOI] [PubMed] [Google Scholar]
- Rivera‐Riquelme, M. , Piqueras, J. A. , & Cuijpers, P. (2019). The Revised Mental Health Inventory‐5 (MHI‐5) as an ultra‐brief screening measure of bidimensional mental health in children and adolescents. Psychiatry Research, 274, 247–253. 10.1016/j.psychres.2019.02.045 [DOI] [PubMed] [Google Scholar]
- Roy, A. K. , & Comer, J. S. (2020). Advances in the conceptualization, assessment, and treatment of pediatric irritability. Behavior Therapy, 51(2), 207–210. 10.1016/j.beth.2019.10.010 [DOI] [PubMed] [Google Scholar]
- Sisk, L. M. , & Gee, D. G. (2022). Stress and adolescence: Vulnerability and opportunity during a sensitive window of development. Current Opinion in Psychology, 44, 286–292. 10.1016/j.copsyc.2021.10.005 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sorcher, L. K. , Goldstein, B. L. , Finsaas, M. C. , Carlson, G. A. , Klein, D. N. , & Dougherty, L. R. (2022). Preschool irritability predicts adolescent psychopathology and functional impairment: A 12‐year prospective study. Journal of the American Academy of Child & Adolescent Psychiatry, 61(4), 554–564.e551. 10.1016/j.jaac.2021.08.016 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Stringaris, A. , Cohen, P. , Pine, D. S. , & Leibenluft, E. (2009). Adult outcomes of youth irritability: A 20‐year prospective community‐based study. American Journal of Psychiatry, 166(9), 1048–1054. 10.1176/appi.ajp.2009.08121849 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Stringaris, A. , & Goodman, R. (2009). Longitudinal outcome of youth oppositionality: Irritable, headstrong, and hurtful behaviors have distinctive predictions. Journal of the American Academy of Child & Adolescent Psychiatry, 48(4), 404–412. 10.1097/CHI.0b013e3181984f30 [DOI] [PubMed] [Google Scholar]
- Stringaris, A. , Goodman, R. , Ferdinando, S. , Razdan, V. , Muhrer, E. , Leibenluft, E. , & Brotman, M. A. (2012). The affective reactivity index: A concise irritability scale for clinical and research settings. Journal of Child Psychology and Psychiatry, 53(11), 1109–1117. 10.1111/j.1469-7610.2012.02561 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Stringaris, A. , Maughan, B. , Copeland, W. S. , Costello, E. J. , & Angold, A. (2013). Irritable mood as a symptom of depression in youth: Prevalence, developmental, and clinical correlates in the great smoky mountains study. Journal of the American Academy of Child & Adolescent Psychiatry, 52(8), 831–840. 10.1016/j.jaac.2013.05.017 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Topolski, T. D. , Edwards, T. C. , & Patrick, D. L. (2002). Youth quality of life instruments– U.S. Version: User manual and interpretation guide. University of Washington, Department of Health Services. [Google Scholar]
- Vidal‐Ribas, P. , Brotman, M. A. , Valdivieso, I. , Leibenluft, E. , & Stringaris, A. (2016). The status of irritability in psychiatry: A conceptual and quantitative review. Journal of the American Academy of Child & Adolescent Psychiatry, 55(7), 556–570. 10.1016/j.jaac.2016.04.014 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wakschlag, L. , Carroll, A. , Friedland, S. , Walkup, J. , Wiggins, J. , Mohanty, N. , Papacheck, E. , Bridi, S. , Carroll, R. , Drelicharz, D. , Hasan, Z. , Kotagal, T. , Davis, M. , & Smith, J. (2023). Making it “EASI” for pediatric clinicians to determine when toddler tantrums are “more than the terrible twos”: Proof‐of‐concept for primary care screening with the Multidimensional Assessment Profiles– Early Assessment Scale for Irritability (MAPSEASI). Submitted manuscript. [DOI] [PubMed]
- Wakschlag, L. S. , Briggs‐Gowan, M. J. , Choi, S. W. , Nichols, S. R. , Kestler, J. , Burns, J. L. , & Henry, D. (2014). Advancing a multidimensional, developmental spectrum approach to preschool disruptive behavior. Journal of the American Academy of Child & Adolescent Psychiatry, 53(1), 82–96. 10.1016/j.jaac.2013.10.011 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wakschlag, L. S. , Choi, S. W. , Carter, A. S. , Hullsiek, H. , Burns, J. , McCarthy, K. , Leibenluft, E. , & Briggs‐Gowan, M. J. (2012). Defining the developmental parameters of temper loss in early childhood: Implications for developmental psychopathology. Journal of Child Psychology and Psychiatry, 53(11), 1099–1108. 10.1111/j.1469-7610.2012.02595.x [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wakschlag, L. S. , Estabrook, R. , Petitclerc, A. , Henry, D. , Burns, J. L. , Perlman, S. B. , Briggs‐Gowan, M. L. , Pine, D. S. , & Leibenluft, E. (2015). Clinical implications of a dimensional approach: The normal:abnormal spectrum of early irritability. Journal of the American Academy of Child & Adolescent Psychiatry, 54(8), 626–634. 10.1016/j.jaac.2015.05.016 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wakschlag, L. S. , Perlman, S. B. , Blair, J. , Leibenluft, E. , Briggs‐Gowan, M. J. , & Pine, D. S. (2018). The neurodevelopmental basis of early childhood disruptive behavior: Irritability and callous phenotypes as exemplars. American Journal of Psychiatry, 175(2), 114–130. 10.1176/appi.ajp.2017.17010045 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wakschlag, L. S. , Tolan, P. H. , & Leventhal, B. L. (2010). Research review: ‘Ain’t misbehavin’: Towards a developmentally‐specified nosology for preschool disruptive behavior. Journal of Child Psychology and Psychiatry, 51(1), 3–22. 10.1111/j.1469-7610.2009.02184.x [DOI] [PMC free article] [PubMed] [Google Scholar]
- White, S. F. , Briggs‐Gowan, M. J. , Voss, J. L. , Petitclerc, A. , McCarthy, K. , Blair, R. J. , & Wakschlag, L. S. (2016). Can the fear recognition deficits associated with callous‐unemotional traits be identified in early childhood? Journal of Clinical and Experimental Neuropsychology, 38(6), 672–684. 10.1080/13803395.2016.1149154 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wiggins, J. L. , Briggs‐Gowan, M. J. , Estabrook, R. , Brotman, M. A. , Pine, D. S. , Leibenluft, E. , & Wakschlag, L. S. (2018). Identifying clinically significant irritability in early childhood. Journal of the American Academy of Child & Adolescent Psychiatry, 57(3), 191–199.e2. 10.1016/j.jaac.2017.12.008 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wiggins, J. L. , Mitchell, C. , Stringaris, A. , & Leibenluft, E. (2014). Developmental trajectories of irritability and bidirectional associations with maternal depression. Journal of the American Academy of Child & Adolescent Psychiatry, 53(11), 1191–1205. 10.1016/j.jaac.2014.08.005 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wiggins, J. L. , Rosario, A. U. , Zhang, Y. , MacNeill, L. A. , Yu, Q. , Norton, E. S. , Smith, J. D. , Wakschlag, L. S. (2023). Advancing earlier transdiagnostic identification of mental health risk: A pragmatic approach at the transition to toddlerhood. International Journal of Methods in Psychiatric Research, e1989. 10.1002/mpr.1989 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wiggins, J. L. , Roy, A. K. , & Wakschlag, L. S. (2023). MAPping affective dimensions of behavior: Methodologic and pragmatic advancement of the Multidimensional Assessment Profiles scales. International Journal of Methods in Psychiatric Research, e1990. 10.1002/mpr.1990 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Willcutt, E. G. (2012). The prevalence of DSM‐IV attention‐deficit/hyperactivity disorder: A meta‐analytic review. Neurotherapeutics, 9(3), 490–499. 10.1007/s13311-012-0135-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Zendarski, N. , Galligan, R. , Coghill, D. , Payne, J. M. , De Luca, C. R. , & Mulraney, M. (2022). The associations between child irritability, parental distress, parental irritability and family functioning in children accessing mental health services. Journal of Child and Family Studies, 32(1), 288–300. 10.1007/s10826-022-02390-2 [DOI] [Google Scholar]
- Zik, J. , Deveney, C. M. , Ellingson, J. M. , Haller, S. P. , Kircanski, K. , Cardinale, E. M. , Brotman, M. A. , & Stoddard, J. (2022). Understanding irritability in relation to anger, aggression, and informant in a pediatric clinical population. Journal of the American Academy of Child & Adolescent Psychiatry, 61(5), 711–720. 10.1016/j.jaac.2021.08.012 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supporting Information S1
Data Availability Statement
The data that support the findings of this study are available from the corresponding author upon reasonable request.