Abstract
Objective
Early adolescence is a transition period during which many mental health disorders emerge. The interplay between different internalizing and externalizing mental health problems in adolescence is poorly understood at the within-person level. Executive functioning (EF) in early adolescence has been shown to constitute a transdiagnostic risk factor, but the specificity of the associations between different domains of EF and mental health problems remains unclear.
Method
Network dynamics (ie, temporal effects) of different internalizing and externalizing symptoms were investigated leveraging data from the Tracking Adolescents’ Individual Lives Survey (TRAILS), a large longitudinal panel study of adolescents (>1,641 participants) assessed at ages 11, 13, and 15. Two novel methodological panel network approaches were used: cross-lagged panel network models and graphical vector autoregressive models. Hierarchical regression models were used to investigate prospective associations between different measures of EF and broadband transdiagnostic dimensions.
Results
Depressive problems predicted a range of other internalizing symptoms (ie, panic, somatic problems, separation anxiety, general anxiety, social phobia) over time, particularly during early adolescence. Important feedback loops with reciprocal associations between different anxiety symptoms were identified. Different facets of EF assessed at age 11, particularly sustained attention, showed weak but significant prospective associations with internalizing and externalizing symptoms at age 13.
Conclusion
The present findings emphasize the importance of targeting depressive problems in early adolescence to prevent a spiral of different internalizing symptoms from arising later on.
Key words: executive functioning, externalizing, internalizing, network
Early adolescence is a key transition period during which the prevalence of a range of mental health problems increases and in which top-down executive functions have been proposed to play a role.1 Globally, approximately 25% to 31% of adolescents experience a common mental health disorder,2 making adolescent psychopathology a substantial burden of disease and a growing source of disability.3 The vast majority of mental health disorders diagnosed in adults, including internalizing and externalizing problems, emerge during adolescence and have been associated with a range of negative long-term consequences.4 Thus, studying the emergence of different mental health problems during adolescence is crucial to better identify early risk factors and the deleterious cycles that may contribute to persistence of symptoms.
Symptoms of adolescent mental health problems are highly comorbid and have been shown to predict each other over time.5 This high level of comorbidity and interdependence between symptoms of different disorders motivated the development of novel methodological frameworks, such as the symptom network approach.6 The network approach poses that psychopathology arises from complex interactions between symptoms, both within the same time window and across time. The symptom interactions (edges) between distinct symptom clusters (eg, internalizing–externalizing) are called bridge symptoms and might have an important role in the emergence of comorbidities.7 Therefore, panel network models are uniquely equipped to provide mechanistic insights into the development of psychopathology during key transition periods, such as adolescence.
To date, numerous studies have examined cross-sectional associations between internalizing and externalizing symptoms.8,9 Developmental process models, such as the developmental cascade model,10 have been proposed to explain symptom dynamics according to which externalizing problems primarily predict subsequent internalizing symptoms, rather than the reverse direction. Adolescent externalizing behaviors, such as delinquency and aggression, may give rise to negative responses from the adolescent’s environment, such as social rejection, academic difficulties, or punishment, which, in turn, may precede internalizing symptoms, such as anxiety or depression.11,12 Countless studies found sex differences in the prevalence of adolescent internalizing (ie, higher prevalence among girls)13 and externalizing (ie, higher prevalence among boys) symptoms.14 Moreover, there is evidence suggesting sex differences in the developmental trajectories of externalizing problems across adolescence.15 Little is known about sex differences in developmental cascades (ie, predictive associations between internalizing and externalizing symptoms). Moreover, most studies testing within-person theories have failed to disentangle within- and between-person effects in the symptom associations over time, and thus little is known about the dynamic changes within the network structure and which symptoms bridge the internalizing and externalizing domains at the different stages of adolescence.
Existing studies on the associations between different internalizing and externalizing symptoms show 3 important limitations that we aim to address in the present study. First, most network analysis studies are cross-sectional studies and consequently lack temporal precedence. For instance, a recent network analysis study showed that generalized anxiety disorder symptoms are most central in largely similar symptom network constellations across different age groups (ie, ages 7.5-14).9 However, little is known about how different internalizing and externalizing symptoms predict each other over time—temporal associations that are crucial for informing causal inferences. Second, prior evidence for the co-occurrence of heterogeneous symptoms of different disorders is primarily based on between-person relationships. As widely noted, understanding mechanistic pathways traditionally entails an examination of isolated within-person (change within individuals) effects that form the basis for intervention efforts (within-person change).16 Third, existing studies on the co-occurrence and interplay between internalizing and externalizing symptoms are mostly restricted to the later stages of adolescence and early adulthood. Our study is the first to examine contemporaneous (within the same time window) and temporal associations across a 6-year period in early adolescence (ages 11-16).
A core risk factor for the development of internalizing and externalizing mental health problems may be relatively suboptimal executive functioning (EF) in early adolescence. Impairments of EF, including working memory, response inhibition, cognitive flexibility, and sustained attention,17 have been associated with elevated internalizing and externalizing symptoms. Moreover, a general psychopathology factor (p factor18) has been associated with poorer EF.19 However, existing studies primarily examined associations between single executive functions (eg, working memory) and symptoms in isolation. Little is known about the specificity of associations between executive functions and the development of internalizing and externalizing symptoms at different stages of adolescence.
Using a longitudinal panel dataset of more than 2,100 adolescents, our study aimed to examine the dynamic interplay between a range of internalizing and externalizing mental health problems. Based on developmental cascade models, we predicted that externalizing problems should largely predict internalizing problems over time. We used 2 different panel network analytical approaches in parallel. First, cross-lagged panel network (CLPN) models were used to examine age-specific and wave-by-wave changes in the network structure while conflating within- and between-person effects. Second, panel graphical vector autoregressive (GVAR) models were used to separate within- and between-person effects in the overall network structure. The parallel use of both approaches provided us with a unique perspective for understanding developmental cascades at both the intraindividual level (ie, individuals’ within-person deviations from the mean) and the interindividual level (ie, individuals’ scores on a variable relate to scores of others on another variable). Both statistical approaches serve different goals, namely, the understanding of within-person mechanisms (ie, panel GVAR model) as well as the prediction of future outcomes based on the combined within- and between-person effects (ie, CLPN analysis). Finally, we connected this development of 2 broadband transdiagnostic dimensions (internalizing, externalizing) to different domains of EF assessed at age 11.
Method
Data Source and Procedure
We used data from the Tracking Adolescents’ Individual Lives Survey (TRAILS) study, a longitudinal cohort study of Dutch (pre-)adolescents assessed every 2 to 3 years from ages 10 to 11 (wave 1) to ages 28 to 30 (wave 7). The study recruited a representative general population sample from urban and rural areas across 5 municipalities in the Netherlands. The study design, procedure, and sample characteristics are described in detail elsewhere.20,21 The present study used data from the first 3 assessment waves (ages 10-16), considering that all relevant outcome measures have been assessed consecutively with the same instrument during these waves. At each wave, a range of self-report questionnaires and clinical interviews were administered. Neuropsychological and cognitive tasks were administered during the first wave only.
Measures
Youth Self-Report
The Youth Self-Report (YSR)22 is a commonly used and well-validated measure of behavioral and emotional problems of children 11 to 19 years old. The YSR consists of 112 items that assess internalizing and externalizing mental health problems on a 3-point scale (0 = not true, 1 = somewhat or sometimes true, 2 = very or often true). The YSR includes 2 broadband domains: Internalizing (31 items; Anxious/Depressed, Withdrawn/Depressed, and Somatic Complaints scales) and Externalizing (32 items; Rule-Breaking Behavior and Aggressive Behavior scales) symptoms. Moreover, there are specific DSM-oriented scales for Depressive Problems, Anxious Problems, Somatic Problems, Attention Deficit Hyperactivity Problems, Oppositional Defiant Problems, and Conduct Problems. The respective items were averaged to construct continuous scale scores between 0 (no symptoms present) and 2 (all symptoms always or very present). Our analyses focused on all subscales, excluding the anxiety subscale, as anxiety was assessed in more detail using the Revised Child Anxiety and Depression Scale (RCADS).
Revised Child Anxiety and Depression Scale
The RCADS23 is a self-report measure of 5 anxiety subtypes and depression symptoms. The questionnaire consists of 47 items that are scored on a 4-point Likert scale (0 = never, 1 = sometimes, 2 = often, 3 = always). The RCADS assessment comprises 6 scales (separation anxiety, generalized anxiety, social phobia, panic disorder, obsessive-compulsive disorder [OCD], major depressive disorder) corresponding largely to the DSM-5 dimensions of anxiety and depressive disorders. The depression scale was not assessed at wave 3. Satisfactory psychometric properties of the RCADS have been well documented, including a replication of the factor structure in the TRAILS sample.24
Executive Functioning
EF was assessed at ages 10 to 12 (wave 1) using the Amsterdam Neuropsychological Task battery.25 The following tasks were included: Sustained Attention Dots task, measuring sustained attention; Shifting Attentional Set–Visual task, measuring response inhibition and cognitive flexibility; and Memory Search Letters task, measuring working memory maintenance. Following previous recommendations on these tasks,26 we removed all reaction time (RT) and accuracy measures (outlier) with an absolute z score larger than or equal to 4.
Sustained Attention Dots
The Sustained Attention Dots task assesses the ability to sustain attention over time.25 On each trial, participants are randomly presented with a 3-, 4-, or 5-dot pattern and instructed to press the yes button for a target (4-dot) and the no button for other (3-dot or 5-dot) patterns. An auditory feedback signal is provided for erroneous responses. The task consists of 600 visual dot patterns that are presented across 50 series of 12 trials each. There are 200 trials for each type of stimulus (3-, 4-, or 5-dot pattern), resulting in a 1:2 target-to-nontarget ratio. The primary outcome measure of the task is the within-person standard deviation of the mean RT of 50 series (fluctuation in tempo) as well as the overall percentage of errors. Higher scores on these performance measures indicate worse sustained attention.
Shifting Attentional Set–Visual
The Shifting Attentional Set–Visual task measures 2 components of shifting attention: response inhibition and cognitive flexibility.25 The task comprises 3 parts in which participants mimic the direction of jumping squares by clicking the left or right mouse button. In part 1, one square is green and jumps randomly (fixed-compatible condition); in part 2, the square is red, and participants must mirror the direction (fixed-incompatible condition); in part 3, the square can be green or red and participants must adapt their response based on the color of the square (random condition). Response inhibition is calculated as the difference in mean RT/percentage of errors between part 2 and part 1.27 Cognitive flexibility is computed by subtracting the mean RT/percentage of errors of part 1 from part 3. Higher difference scores indicate weaker cognitive flexibility.
Memory Search Letters
The Memory Search Letters task assesses working memory maintenance in varying memory load and distraction.25 It consists of 3 parts in which participants memorize 1, 2, or 3 target letters. Participants must decide whether 4 letters on the screen contain the target letter. Target and nontarget trials alternate in random order. Working memory maintenance is computed as the difference between part 3 (3 target letters) and part 1 (1 target letter) mean RT/percentage of error.27 Higher difference scores indicate worse working memory maintenance.
Data Analysis
To examine how different internalizing and externalizing symptoms predict each other over time, we used 2 distinct panel network analytical approaches. First, we implemented a CLPN analysis approach following the study by Zainal and Newman28 to examine age-specific developmental changes. Second, we used GVAR models that allowed us to separate within- and between-person effects in the network structure and discern contemporaneous and temporal associations. These 2 approaches offer unique advantages (CLPN: wave-by-wave analysis; panel GVAR: within- and between-person effects separation) in studying the symptom interplay. To examine how EF measures assessed at the first wave relate to the development (waves 2-3) of internalizing and externalizing mental health problems, we used hierarchical regression models.
CLPN Models
CLPN models29 were used to examine the interrelation between all 10 internalizing and externalizing symptoms during the 3 waves of data. CLPN models examine lag-1 cross-lagged relations between all nodes after incorporating autoregressive effects (ie, a node predicting itself over time). Regularized regressions that included the least absolute shrinkage and selection operator with 10-fold cross-validation were used to calculate autoregressive and cross-lagged estimates between 2 consecutive waves (ie, wave 1 to wave 2; wave 2 to wave 3). The least absolute shrinkage and selection operator regularization in the model implements a penalty procedure (using a tuning parameter γ) through which weak coefficients (ie, edges) are set to zero. This estimation method leads to sparser networks with a lower probability of obtaining false-positive edges in the network. Cross-validation was used to select the optimal γ parameter. Directed cross-lagged edges in the temporal networks represent associations between different nodes across time while controlling for all other nodes in the network. Cross-sectional networks were estimated using the EBICglasso algorithm that uses Bayesian information criterion model selection and least absolute shrinkage and selection operator regularization with a hyperparameter set to 0.5 to remove spurious edges from the network structure.
GVAR Model
We used panel GVAR network models30 to discern temporal (predictions over time) and contemporaneous (within the same time window) associations in the network structure. The panel GVAR model is structurally similar to a random-intercept cross-lagged panel model31,32 and separates within-person (changes within individuals) and between-person (relation between means) effects. We detrended the data for linear and quadratic effects of time and standardized across waves to ensure stationarity, as is commonly done in panel GVAR approaches that focus on the correlational structure of interest.33,34 First, we fitted a saturated model (with all edges) that was pruned with a step-up model search (α = .05) to remove false positives.35 Model fit was evaluated according to standard criteria of good fit as indicated by the root mean square error of approximation, comparative fit index, and Tucker-Lewis index. This novel panel GVAR modeling approach yielding average within-person effects ensures that trait-like (between-person) effects are accounted for in the estimation of contemporaneous and temporal networks.
Network Centrality Metrics
For the temporal network, we computed the in-strength (ie, sum incoming edge weights) and out-strength (ie, sum outgoing edge weights) for every node. These commonly used measures capture the degree to which variables exert their influence (out-strength, influence) and are being influenced (in-strength, predictability) by other variables in the network. To examine the extent to which different symptoms act as important bridge symptoms in the contemporaneous networks, we computed measures of bridge centrality using the networktools package.36 Bridge centrality describes the sum of edge weights between a node in one community (eg, internalizing symptoms) and all other nodes from a different community (eg, externalizing symptoms).
Estimation and Model Stability Analysis
There was a substantial dropout of participants throughout the waves (participation rate: wave 2, 96%; wave 3, 76%). We used full information maximum likelihood (FIML) estimation in the panel GVAR and the regression analyses. FIML is a gold-standard approach37 that provides unbiased estimates that are similar to multiple imputation procedures assuming data missing at random. We used a case-dropping bootstrapping analysis to examine the stability of the estimated edge weights and centrality measures. Panel network models were estimated and visualized using the R packages glmnet, psychonetrics, and qgraph.38, 39, 40
Hierarchical Regression Models
We used hierarchical regression models to investigate how executive functions (measured at wave 1) affect the development of internalizing and externalizing symptoms at wave 2 and wave 3. We first included the aggregate measures of symptoms (at the previous wave) and sex as predictors, then added all EF measures. We compared the variance explained by the 2 models. For instance, for the prediction of internalizing symptoms at wave 2, the nested 2-step models follow the specification, and the same was done for wave 3:
Model 1: Internalizing Symptomswave 2 = Sex + Internalizing Symptomswave 1 + Externalizing Symptomswave 1
Model 2: Internalizing Symptomswave 2 = Sex + Internalizing Symptomswave 1 + Externalizing Symptomswave 1 + Working memory RTwave 1 + Working memory Errorswave 1 + Response Inhibition Errorswave 1 + Response Inhibition RTwave 1 + Fluctuation Tempowave 1 + Fluctuation Errorswave 1 + Cognitive Flexibility RTwave 1 + Cognitive Flexibility Errorswave 1
Results
At the group level, we observed an increase in the average level of externalizing symptoms and a decrease in the average level of internalizing symptoms throughout early and middle adolescence (see Supplement 1, available online). In particular, there was a trend of increasing prevalence of attention-deficit/hyperactivity disorder (ADHD) and decreasing prevalence of separation anxiety during adolescence. See Tables S1 and S2 (available online) for further details regarding the significant changes and proportion of missingness across waves. Table 1 provides relevant sample characteristics. According to the cutoff scores for the YSR, 23% of participants showed a borderline (subclinical) or clinical score for any internalizing mental health problems at wave 1; 12% of individuals showed a borderline or clinical score for any externalizing mental health problem at wave 1.
Table 1.
Descriptive Sample Characteristics Information
| Wave 1 | Wave 2 | Wave 3 | ||||
|---|---|---|---|---|---|---|
| n | (%) | n | (%) | n | (%) | |
| Participants (female) | 2,170 | (50.92) | 2,074 | (51.30) | 1,641 | (53.26) |
| Mean | (SD) | Mean | (SD) | Mean | (SD) | |
| Age (y) | 10.61 | (0.65) | 13.07 | (0.61) | 15.78 | (0.77) |
| YSR | ||||||
| Internalizing scale score | 0.36 | (0.24) | 0.33 | (0.24) | 0.31 | (0.25) |
| Externalizing scale score | 0.27 | (0.20) | 0.29 | (0.20) | 0.31 | (0.21) |
| Depressive Problems | 0.29 | (0.25) | 0.27 | (0.26) | 0.30 | (0.27) |
| Somatic Problems | 0.46 | (0.33) | 0.32 | (0.29) | 0.26 | (0.28) |
| Attention Deficit Hyperactivity Problems | 0.59 | (0.36) | 0.67 | (0.38) | 0.68 | (0.38) |
| Oppositional Defiant Problems | 0.45 | (0.35) | 0.46 | (0.35) | 0.46 | (0.35) |
| Conduct Problems | 0.23 | (0.20) | 0.23 | (0.19) | 0.24 | (0.20) |
| RCADS | ||||||
| General anxiety disorder | 0.66 | (0.45) | 0.48 | (0.43) | 0.52 | (0.42) |
| Social phobia | 0.78 | (0.43) | 0.68 | (0.46) | 0.73 | (0.50) |
| Separation anxiety | 0.37 | (0.35) | 0.24 | (0.29) | 0.22 | (0.26) |
| Panic disorder | 0.43 | (0.36) | 0.30 | (0.32) | 0.28 | (0.29) |
| Obsessive-compulsive disorder | 0.60 | (0.44) | 0.34 | (0.35) | 0.29 | (0.36) |
Note: RCADS = Revised Child Anxiety and Depression Scale; YSR = Youth Self-Report.
Contemporaneous Associations Between Internalizing and Externalizing Symptoms
Figure 1 shows cross-sectional associations among internalizing and externalizing symptoms at waves 1, 2, and 3. The overall network structure showed strong similarities across the 3 waves. As expected, internalizing and externalizing symptoms clustered together and showed strong positive associations within their respective cluster. Interestingly, depressive problems emerged as a key bridge symptom connecting a range of internalizing and externalizing symptoms at all 3 waves. Moreover, attention/hyperactivity problems and conduct problems showed moderately strong positive associations with depressive problems and somatic problems. We observed a negative association between social phobia and conduct problems only at wave 2.
Figure 1.
Cross-Sectional Internalizing-Externalizing Networks
Note: (A-C) Waves 1-3. Attention problems abbreviations: AttHyper = attention-deficit/hyperactivity disorder; Externalizing abbreviations: Oppos = oppositional defiant problems; conduct = conduct problems; Internalizing abbreviations: Depr = depressive problems; Somat = somatic problems; GenAnx = Generalized Anxiety Disorder; SocPho = social phobia; SepAnx = separation anxiety disorder; Panic = panic disorder; OCD = obessive-compulsive disorder.
To quantify the importance of different bridge symptoms, connecting internalizing and externalizing symptom clusters, we computed a measure of bridge centrality for the contemporaneous networks at all 3 waves (Figure S1, available online). Consistent with the visual inspection, depressive problems constituted the most important internalizing bridge symptom. It should be noted that the importance of depressive problems as bridge symptoms appeared to be decreasing across the three waves.
Internalizing-Externalizing Symptom Dynamics Over Time
We examined temporal associations (Figure 2) separately between waves (ie, wave 1–wave 2; wave 2–wave 3) in the CLPN model. Overall, we observed a complex interplay between different internalizing and externalizing symptoms that predicted each other over time through numerous pathways. During early adolescence (Figure 2A), depressive problems emerged as a key predictor of other internalizing problems (eg, panic disorder, somatic problems, separation anxiety, general anxiety disorder, social phobia, OCD). This influence of depressive problems on other nodes is also shown in the high out-strength (strength of outgoing edges) of depressive problems. The degree of influence of depressive problems on other symptoms was substantially lower during the later stages of adolescence (waves 2-3). Importantly, social phobia symptoms were predicted by a range of other symptoms (eg, lower conduct problems, depressive problems, separation anxiety symptoms). This high level of predictability of social phobia symptoms during later adolescence can also be seen in the high in-strength (strength of incoming edges) (Figures S2, S3, available online). The temporal networks at both change points (wave 1 to wave 2 and wave 2 to wave 3) indicated various reinforcing feedback loops (eg, attention deficit hyperactivity problems–oppositional defiant problems), in which 2 symptoms predict each over time.
Figure 2.
Temporal Internalizing-Externalizing Networks From Cross-Lagged Panel Model
Note:(A) Wave 1 to 2. (B, C) Wave 2 to 3. AttHyper = attention deficit hyperactivity; Oppos = oppositional defiant problems; Conduct = conduct problems; Depr = depressive problems; Somat = somatic problems; GenAnx = generalized anxiety disorder; SocPhob = social phobia; SepAnx = separation anxiety disorder; Panic = panic disorder; OCD = obsessive-compulsive disorder.
To validate the findings from our CLPN analyses described above, we used a novel panel GVAR approach that can separate within- and between-person effects in the network structure. The pruned panel GVAR models showed a good fit to the data (root mean square error of approximation = 0.043, comparative fit index = 0.95, Tucker-Lewis index = 0.94). The contemporaneous (Figures S4, S5, available online) and temporal networks (Figure S6, available online) from the panel GVAR analyses generally replicated the central findings from our CLPN analysis at a within-person level. Despite the large similarities, we found some differences between the methods as the directionality of a few associations (eg, conduct problems–social phobia) changed in the panel GVAR model (Figure 3).
Figure 3.
Pruned Temporal Network From Panel Graphical Vector Autoregressive Model
Note:AttHyper = attention deficit hyperactivity; Oppos = oppositional defiant problems; Conduct = conduct problems; Depr = depressive problems; Somat = somatic problems; GenAnx = generalized anxiety disorder; SocPhob = social phobia; SepAnx = separation anxiety disorder; Panic = panic disorder; OCD = obsessive-compulsive disorder.
The models presented above included 5 different subscales of anxiety assessed through the RCADS measure. As shared method variance may be an issue that could explain the greater within-measure associations found, we conducted sensitivity analyses in which we replaced the 5 RCADS anxiety subscales with the single YSR anxiety subscales (Supplement 2, available online). The results are shown in Figures S7 to S9 (available online). These models largely replicated some of the most important temporal associations (eg, depressive symptoms predicting most other anxiety symptoms) found above.
Prospective Associations Between EF and Internalizing and Externalizing Symptoms
Hierarchical regression models were used to estimate associations between specific executive functions (Figure S10, available online) assessed at age wave 1 (mean age 10.61) and the development of internalizing and externalizing symptoms at wave 2 (mean age 13.07) and wave 3 (mean age 15.78). The EF measures were assessed only at wave 1, and thus they could not be integrated into the panel network approaches that require assessments at all respective waves. Overall, the regression models showed significant associations between behavioral measures of sustained attention (speed) at the first wave and future externalizing and internalizing symptoms (see Figure 4 for a visualization of regression coefficients). Higher fluctuations in tempo throughout the Sustained Attention Dots task at wave 1 (indicating worse sustained attention) were associated with higher scores of externalizing and internalizing symptoms at wave 2. Moreover, slower response inhibition/shifting was associated with internalizing symptoms at wave 3, but not at wave 2. All the effect sizes were small (all βs < .07) and the total variance explained in the second-step models that included these EF measures was not substantially larger than the first-step models that included only sex and mental health problems at the previous wave as predictors (Tables S3-S10, available online). No other task measure of attentional shifting (response inhibition accuracy, cognitive flexibility speed, cognitive flexibility accuracy) or working memory was significantly associated with prospective internalizing or externalizing symptoms. Tables S1 through S8 (available online) provide an overview of all regression coefficients.
Figure 4.
Regression Estimates for Different Executive Functioning Measures
Note:The regression estimates refer to the regression models (in step 2) that include sex and previous internalizing and externalizing symptoms as predictors. The standard errors for the regression estimates are shown in vertical bars. CogFlex = cognitive flexibility; Fluct = fluctuations in tempo (sustained attention); ResInhib = response inhibition; RT = reaction times.
∗p < .05.
Discussion
The present study is the first to our knowledge to investigate the network dynamics of internalizing and externalizing symptoms in early adolescence at the within-person level. Mental health problems showed dynamic patterns of interactions throughout adolescence, in which depressive problems predicted various other internalizing symptoms in particular. Executive functions assessed in early adolescence were differentially associated with the development of internalizing and externalizing symptoms later in adolescence.
Our investigation revealed mostly stable cross-sectional symptom network constellations throughout different stages of adolescence (ages 11-16). Our bootstrapping analysis confirmed sufficient stability of these networks (Figures S11-S13, available online). In line with previous network analysis studies41 and the recent Hierarchical Taxonomy of Psychopathology (HiTOP) classification system,42 our results support the presence of internalizing and externalizing symptom clusters in which different symptoms of the same domain are strongly co-occurring. Across all 3 waves, we found robust cross-construct associations between ADHD symptoms and anxiety disorder (OCD and social phobia) symptoms (internalizing). This co-occurrence is in line with epidemiological evidence suggesting substantial rates of comorbidity between ADHD and OCD across the life span.43 Moreover, we found a negative association between social phobia and conduct problems that is consistent with observations from clinical practice. Individuals with an overwhelming fear of social situations show low levels of aggressive behavior44 out of fear of negative judgments by others.
A crucial finding from our results concerns the role of depressive problems as a key bridge symptom (cross-sectional networks) and predictive marker (temporal networks). Depressive problems emerged as the most important bridge symptom, connecting various internalizing symptoms to other externalizing symptoms in the contemporaneous network. The importance of depressive problems as a bridge symptom slightly decreased over the course of adolescence. This finding aligns with reports showing that early-onset depression is associated with high rates of comorbidity with other mental disorders, including anxiety disorders in adolescents.45
The temporal networks emphasize the influential role of depressive problems in predicting a range of other internalizing symptoms in early adolescence. This is consistent with the high centrality observed in the contemporaneous network. During the change from age 10 to age 13 (wave 1 to wave 2) in particular, depressive problems predicted higher levels of social phobia, OCD, general anxiety, separation anxiety, panic symptoms, and somatic complaints. Our results are consistent with a recent CLPN analysis indicating depressed mood as an important influencer of other symptoms.8 However, we replicate and extend this finding by separating within- and between-person effects that are conflated in CLPN models.29 Likely, different cognitive-behavioral processes prevalent in depression may explain this catalyst role of depressive symptoms on other symptom states. For instance, rumination as a core symptom of depression has been shown to predict anxiety symptoms and characterize comorbid depression/anxiety.46 Moreover, prior diagnosis of major depressive disorder and negative affect predicted the onset of panic attacks in high school students.47 Particularly in early adolescence, low energy, avoidance, and social isolation triggered by depression may lead to the emergence of various anxiety problems that persist throughout adolescence and adulthood. Thus, our findings suggest that targeting adolescent depressive symptoms in clinical practice may prove viable in preventing various other internalizing symptoms arising later in life. To the best of our knowledge, this is one of the first studies showing this catalyst effect of depression on a range of outcomes at the within-person level in early and middle adolescence.
In addition to the prominent role of depressive problems, we identified important feedback loops (reciprocal associations) between key symptoms. For instance, panic symptoms and somatic problems predicted each other over time in both panel network models. The increased body vigilance common in panic attacks may exacerbate negative or threatening interpretations of physical sensations that manifest in somatic problems. This in turn may lead to increased anxiety sensitivity, which constitutes a major risk factor for panic disorder.48 Interestingly, we also observed reciprocal associations (in both methods) between ADHD symptoms and oppositional defiant problems that are consistent with bidirectional associations reported in prior literature.49 These findings further complement discussions regarding the developmental precursor model of ADHD symptoms50 according to which ADHD symptoms predict argumentative/defiant symptoms.
We complemented our analysis of symptom network dynamics during adolescence by examining how EF assessed at age 10 to 11 plays into the development of broadband transdiagnostic dimensions of internalizing and externalizing symptoms. Our results suggest that primarily sustained attention, and no other executive functions such as working memory or attentional shifting, might represent a transdiagnostic risk factor for the development of internalizing and externalizing mental health problems. Our findings extend prior studies that showed cross-sectional or domain-specific (ie, affective problems) associations with sustained attention.51,52 The ability to sustain attention to relevant stimuli or information despite distractions constitutes a core executive function that plays a major role in daily functioning of individuals. Impairments in sustained attention have been associated with a range of mental disorders, including internalizing conditions such as depression53 and externalizing conditions such as ADHD.54 Our results suggest that sustained attention is an important executive function in early adolescence with negative clinical transdiagnostic repercussions. Shifting attention was associated with more internalizing symptoms only at wave 3. This is consistent with previous links between difficulties shifting attention, response inhibition, and psychopathological processes, such as rumination.55 Importantly, we found that the executive functions, including sustained attention, explained no additional variance over and beyond internalizing and externalizing mental health problems at the previous wave. Likely, associations between sustained attention and internalizing/externalizing mental health problems at the previous waves already accounted for these effects. Other executive functions, such as attentional inhibition and working memory, showed no significant associations with the symptom measures. It is possible that these executive functions are associated with mental health problems at other time lags (ie, later adolescence) or show solely disorder-specific associations. Future studies should also consider integrating a common EF factor at each time point into symptom panel networks—a crucial step toward understanding the dynamic interplay that was not feasible in our study considering that EF was assessed only at the first assessment wave. For instance, integrating cognitive measures of sustained attention in the dynamic network models may bridge our understanding of transdiagnostic risk factors as well as transdiagnostic network dynamics.
Our findings should be interpreted in light of several limitations. First, our analyses relied on self-report of adolescents, which may naturally be biased. Moreover, to foster model estimation and identification, we used scale scores that do not account for measurement error. Future studies should include both parent and teacher reports to validate these findings and model all multiple-indicator constructs as latent variables. Second, the 2 methodological panel network approaches are limited by several methodological constraints. A common concern in all network modeling approaches is the assumption of multivariate normality. Restricted variance in some nodes (eg, OCD at wave 3) may affect the estimation of temporal effects.30 The CLPN model cannot separate within- and between-person effects, which limits the interpretation of directed temporal effects as mechanistic pathways. The panel GVAR approach overcomes this limitation because it separates within- and between-person effects in the network structure. The resulting temporal associations are Granger causal and fulfill the criteria of temporal precedence; however, these associations might not necessarily indicate causal effects.56 Moreover, the panel GVAR modeling approach assumes lag-1 linear dynamics between all variables and an approximately stationary time series, and thus it cannot capture (nonlinear) processes that operate on shorter or broader time scales. Further methodological developments, including methods for formal model comparisons, are needed to better understand the differences in results (eg, the role of oppositional problems) obtained from CLPN and panel GVAR models, which likely concern the separation of within- and between-person effect. Lastly, future studies should adopt confirmatory network modeling strategies to directly replicate and test the predictions emerging from developmental cascade models. We have used FIML estimation to account for missingness in the GVAR and regression models. Although FIML is widely considered an appropriate tool that can produce unbiased parameter estimates,57 it cannot overcome the assumption of missing completely at random, and some evidence suggests that attrition in the TRAILS sample is associated with baseline levels of psychopathology.20,58
Our study is the first to our knowledge to investigate the within-person network dynamics between a range of internalizing and externalizing symptoms throughout early adolescence. We showcase the use of 2 novel methodological panel network approaches in parallel, CLPN and panel GVAR models, that are uniquely equipped to explore dynamic patterns of interaction in longitudinal datasets while controlling for many variables. Adolescence remains a consequential period of sensitivity in which different internalizing and externalizing problems emerge and contribute to symptom persistence in adulthood. Our findings identified sustained attention as a transdiagnostic risk factor, and we pinpointed key catalyst symptoms (eg, depressive problems) in early adolescence. Future empirical investigations of these target points in intervention studies may potentially lead to effective intervention efforts that may prevent a deleterious cycle of symptom enhancement from arising.
Footnotes
This study is part of the project New Science of Mental Disorders (www.nsmd.eu), supported by the Dutch Research Council and the Dutch Ministry of Education, Culture and Science (NWO gravitation grant number 024.004.016). This research is part of the TRacking Adolescents’ Individual Lives Survey (TRAILS). Participating centers of TRAILS include various departments of the University Medical Center and University of Groningen, the University of Utrecht, the Radboud University Medical Center, and the Parnassia Psychiatric Institute, all in the Netherlands. TRAILS has been financially supported by various grants from the Netherlands Organization for Scientific Research (NWO), ZonMW (Netherlands Organisation for Health Research and Development), Social Sciences grants (GB-MaGW), the Dutch Ministry of Justice, the European Science Foundation, the European Research Council, Biobanking and Biomolecular Resources Research Infrastructure (BBMRI-NL), and the participating universities.
This work has been previously posted on a preprint server: https://doi.org/10.31234/osf.io/6suq4.
Sacha Epskamp, PhD, of the National University of Singapore, served as the statistical expert for the panel graphical VAR analysis part of this research.
Author Contributions
Conceptualization: Freichel, Pfirrmann, de Jong, Cousijn, Franken, Oldehinkel, Wiers
Data curation: Freichel, Pfirrmann, Oldehinkel
Formal analysis: Freichel, Pfirrmann
Fundingacquisition: Oldehinkel, Wiers
Investigation: Freichel, Pfirrmann, de Jong, Cousijn, Franken, Oldehinkel, Veer, Wiers
Methodology: Freichel, Pfirrmann, de Jong, Cousijn, Oldehinkel, Veer, Wiers
Resources: Oldehinkel
Supervision: Veer, Wiers
Visualization: Freichel, Veer, Wiers
Writing – original draft: Freichel, Pfirrmann
Writing – review and editing: Freichel, de Jong, Cousijn, Franken, Oldehinkel, Veer, Wiers
Disclosure: Drs. de Jong, Cousijn, Franken, Oldehinkel, Veer, and Wiers, Mr. Freichel, and Ms. Pfirrmann have reported no biomedical financial interests or potential conflicts of interest.
Supplemental Material
References
- 1.Davidson L.L., Grigorenko E.L., Boivin M.J., Rapa E., Stein A. A focus on adolescence to reduce neurological, mental health and substance-use disability. Nature. 2015;527(7578):S161–S166. doi: 10.1038/nature16030. [DOI] [PubMed] [Google Scholar]
- 2.Silva S.A., Silva S.U., Ronca D.B., Gonçalves V.S.S., Dutra E.S., Carvalho K.M.B. Common mental disorders prevalence in adolescents: a systematic review and meta-analyses. PLoS One. 2020;15(4) doi: 10.1371/journal.pone.0232007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Twenge J.M., Cooper A.B., Joiner T.E., Duffy M.E., Binau S.G. Age, period, and cohort trends in mood disorder indicators and suicide-related outcomes in a nationally representative dataset, 2005–2017. J Abnorm Psychol. 2019;128(3):185–199. doi: 10.1037/abn0000410. [DOI] [PubMed] [Google Scholar]
- 4.Kessler R.C., Amminger G.P., Aguilar-Gaxiola S., Alonso J., Lee S., Ustun T.B. Age of onset of mental disorders: a review of recent literature. Curr Opin Psychiatry. 2007;20(4):359–364. doi: 10.1097/YCO.0b013e32816ebc8c. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Kessler R.C., Avenevoli S., McLaughlin K.A., et al. Lifetime co-morbidity of DSM-IV disorders in the US National Comorbidity Survey Replication Adolescent Supplement (NCS-A) Psychol Med. 2012;42(9):1997–2010. doi: 10.1017/S0033291712000025. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Borsboom D., Cramer A.O.J. Network analysis: an integrative approach to the structure of psychopathology. Annu Rev Clin Psychol. 2013;9(1):91–121. doi: 10.1146/annurev-clinpsy-050212-185608. [DOI] [PubMed] [Google Scholar]
- 7.Jones P.J., Ma R., McNally R.J. Bridge Centrality: a network approach to understanding comorbidity. Multivar Behav Res. 2021;56(2):353–367. doi: 10.1080/00273171.2019.1614898. [DOI] [PubMed] [Google Scholar]
- 8.Funkhouser C.J., Chacko A.A., Correa K.A., Kaiser A.J.E., Shankman S.A. Unique longitudinal relationships between symptoms of psychopathology in youth: a cross-lagged panel network analysis in the ABCD study. J Child Psychol Psychiatry. 2021;62(2):184–194. doi: 10.1111/jcpp.13256. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.McElroy E., Shevlin M., Murphy J., McBride O. Co-occurring internalizing and externalizing psychopathology in childhood and adolescence: a network approach. Eur Child Adolesc Psychiatry. 2018;27(11):1449–1457. doi: 10.1007/s00787-018-1128-x. [DOI] [PubMed] [Google Scholar]
- 10.Moilanen K.L., Shaw D.S., Maxwell K.L. Developmental cascades: externalizing, internalizing, and academic competence from middle childhood to early adolescence. Dev Psychopathol. 2010;22(3):635–653. doi: 10.1017/S0954579410000337. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Patterson G.R., Capaldi D.M. In: Risk and Protective Factors in the Development of Psychopathology. Rolf J.E., Masten A.S., Cicchetti D., Nuechterlein K.H., Weintraub S., editors. Cambridge University Press; Cambridge, UK: 1990. A mediational model for boys’ depressed mood; pp. 141–163. [Google Scholar]
- 12.Weeks M., Ploubidis G.B., Cairney J., Wild T.C., Naicker K., Colman I. Developmental pathways linking childhood and adolescent internalizing, externalizing, academic competence, and adolescent depression. J Adolesc. 2016;51:30–40. doi: 10.1016/j.adolescence.2016.05.009. [DOI] [PubMed] [Google Scholar]
- 13.Mojtabai R., Olfson M., Han B. National trends in the prevalence and treatment of depression in adolescents and young adults. Pediatrics. 2016;138(6) doi: 10.1542/peds.2016-1878. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Eme R. In: The Oxford Handbook of Externalizing Spectrum Disorders. (pp. 239–263) Beauchaine T.P., Hinshaw S.P., editors. Oxford University Press; Oxford, UK: 2016. Sex differences in the prevalence and expression of externalizing behavior; pp. 239–263. [Google Scholar]
- 15.Cleverley K., Szatmari P., Vaillancourt T., Boyle M., Lipman E. Developmental trajectories of physical and indirect aggression from late childhood to adolescence: sex differences and outcomes in emerging adulthood. J Am Acad Child Adolesc Psychiatry. 2012;51(10):1037–1051. doi: 10.1016/j.jaac.2012.07.010. [DOI] [PubMed] [Google Scholar]
- 16.Curran P.J., Bauer D.J. The disaggregation of within-person and between-person effects in longitudinal models of change. Annu Rev Psychol. 2011;62:583–619. doi: 10.1146/annurev.psych.093008.100356. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Tillman C., Brocki K.C., Sørensen L., Lundervold A.J. A longitudinal examination of the developmental executive function hierarchy in children with externalizing behavior problems. J Atten Disord. 2015;19(6):496–506. doi: 10.1177/1087054713488439. [DOI] [PubMed] [Google Scholar]
- 18.Caspi A., Houts R.M., Belsky D.W., et al. The p factor: one general psychopathology factor in the structure of psychiatric disorders? Clin Psychol Sci. 2014;2(2):119–137. doi: 10.1177/2167702613497473. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Romer A.L., Pizzagalli D.A. Is executive dysfunction a risk marker or consequence of psychopathology? A test of executive function as a prospective predictor and outcome of general psychopathology in the Adolescent Brain Cognitive Development Study®. Dev Cogn Neurosci. 2021;51 doi: 10.1016/j.dcn.2021.100994. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.de Winter A.F., Oldehinkel A.J., Veenstra R., Brunnekreef J.A., Verhulst F.C., Ormel J. Evaluation of non-response bias in mental health determinants and outcomes in a large sample of pre-adolescents. Eur J Epidemiol. 2005;20(2):173–181. doi: 10.1007/s10654-004-4948-6. [DOI] [PubMed] [Google Scholar]
- 21.Oldehinkel A.J., Rosmalen J.G., Buitelaar J.K., et al. Cohort profile update: the TRacking Adolescents’ Individual Lives Survey (TRAILS) Int J Epidemiol. 2015;44(1):76-76n. doi: 10.1093/ije/dyu225. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Achenbach T.M., Rescorla L.A. University of Vermont Research Center for Children, Youth, & Families; Burlington, VT: 2001. Manual for ASEBA School-Age Forms & Profiles. [Google Scholar]
- 23.Chorpita B.F., Yim L., Moffitt C., Umemoto L.A., Francis S.E. Assessment of symptoms of DSM-IV anxiety and depression in children: a revised child anxiety and depression scale. Behav Res Ther. 2000;38(8):835–855. doi: 10.1016/S0005-7967(99)00130-8. [DOI] [PubMed] [Google Scholar]
- 24.Ferdinand R.F., Lang NDJ van, Ormel J., Verhulst F.C. No distinctions between different types of anxiety symptoms in pre-adolescents from the general population. J Anxiety Disord. 2006;20(2):207–221. doi: 10.1016/j.janxdis.2004.12.003. [DOI] [PubMed] [Google Scholar]
- 25.De Sonneville L.M.J. In: Cognitive Ergonomics, Clinical Assessment and Computer-assisted Learning: Computers in Psychology. Den Brinker B.P.L.M., Beek P.J., Brand A.N., Maarse F.J., Mulder L.J.M., editors. Swets & Zeitlinger; Lisse, The Netherlands: 1999. Amsterdam Neuropsychological Tasks: a computer-aided assessment; pp. 187–203. [Google Scholar]
- 26.Agnes Brunnekreef J., De Sonneville L.M.J., Althaus M., et al. Information processing profiles of internalizing and externalizing behavior problems: evidence from a population-based sample of preadolescents. J Child Psychol Psychiatry. 2007;48(2):185–193. doi: 10.1111/j.1469-7610.2006.01695.x. [DOI] [PubMed] [Google Scholar]
- 27.Bloemen A.J.P., Oldehinkel A.J., Laceulle O.M., Ormel J., Rommelse N.N.J., Hartman C.A. The association between executive functioning and psychopathology: general or specific? Psychol Med. 2018;48(11):1787–1794. doi: 10.1017/S0033291717003269. [DOI] [PubMed] [Google Scholar]
- 28.Zainal N.H., Newman M.G. Elevated anxious and depressed mood relates to future executive dysfunction in older adults: a longitudinal network analysis of psychopathology and cognitive functioning. Clin Psychol Sci. 2023;11:218–238. doi: 10.1177/21677026221114076. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Wysocki A., van Bork R., Cramer A., Rhemtulla M. Cross-lagged network models. PsyArXiv. 2022 doi: 10.31234/osf.io/vjr8z. Accessed December 20, 2023. [DOI] [Google Scholar]
- 30.Epskamp S., Waldorp L.J., Mõttus R., Borsboom D. The gaussian graphical model in cross-sectional and time-series data. Multivariate Behav Res. 2018;53(4):453–480. doi: 10.1080/00273171.2018.1454823. [DOI] [PubMed] [Google Scholar]
- 31.Epskamp S. Psychometric network models from time-series and panel data. Psychometrika. 2020;85(1):206–231. doi: 10.1007/s11336-020-09697-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Hamaker E.L., Kuiper R.M., Grasman R.P. A critique of the cross-lagged panel model. Psychol Methods. 2015;20(1):102. doi: 10.1037/a0038889. [DOI] [PubMed] [Google Scholar]
- 33.Freichel R., Pfirrmann J., Cousjin J., et al. Drinking motives, personality traits, life stressors - identifying pathways to harmful alcohol use in adolescence using a panel network approach. Addiction. 2023;118(10):1908–1919. doi: 10.1111/add.16231. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Speyer L.G., Eisner M., Ribeaud D., Luciano M., Auyeung B., Murray A.L. Developmental relations between internalising problems and ADHD in childhood: a symptom level perspective. Res Child Adolesc Psychopathol. 2021;49(12):1567–1579. doi: 10.1007/s10802-021-00856-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Blanken T.F., Isvoranu A.M., Epskamp S. Network Psychometrics With R. Routledge; Abingdon, UK: 2022. Estimating network structures using model selection. In: pp. 111–132. [Google Scholar]
- 36.P.J. Jones. Networktools: Tools for identifying important nodes in networks, R Package Version, 2018, Accessed December, 20, 2023.
- 37.Enders C.K. The performance of the full information maximum likelihood estimator in multiple regression models with missing data. Educ Psychol Meas. 2001;61:713–740. doi: 10.1177/00131640121971482. [DOI] [Google Scholar]
- 38.Friedman J., Hastie T., Simon N., Tibshirani R., Hastie M.T., Matrix D. Package ‘glmnet. ’ J Stat Softw. 2017;33(1):1–22. [Google Scholar]
- 39.Epskamp S., Cramer A.O.J., Waldorp L.J., Schmittmann V.D., Borsboom D. qgraph: network visualizations of relationships in psychometric data. J Stat Softw. 2012;48(1):1–18. doi: 10.18637/jss.v048.i04. [DOI] [Google Scholar]
- 40.Epskamp S. psychonetrics: Structural Equation Modeling and Confirmatory Network Analysis. https://cran.r-project.org/web/packages/psychonetrics/psychonetrics.pdf Accessed December 20, 2023.
- 41.Blondino C.T., Prom-Wormley E.C. A network approach to substance use, internalizing, and externalizing comorbidity in U.S. adults. Addict Behav. 2022;134 doi: 10.1016/j.addbeh.2022.107421. [DOI] [PubMed] [Google Scholar]
- 42.Conway C.C., Forbes M.K., South S.C. A Hierarchical Taxonomy of Psychopathology (HiTOP) primer for mental health researchers. Clin Psychol Sci. 2022;10(2):236–258. doi: 10.1177/21677026211017834. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Abramovitch A., Dar R., Mittelman A., Wilhelm S. Comorbidity between attention deficit/hyperactivity disorder and obsessive-compulsive disorder across the lifespan: a systematic and critical review. Harv Rev Psychiatry. 2015;23(4):245–262. doi: 10.1097/HRP.0000000000000050. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.DeWall C.N., Buckner J.D., Lambert N.M., Cohen A.S., Fincham F.D. Bracing for the worst, but behaving the best: social anxiety, hostility, and behavioral aggression. J Anxiety Disord. 2010;24(2):260–268. doi: 10.1016/j.janxdis.2009.12.002. [DOI] [PubMed] [Google Scholar]
- 45.Rohde P., Lewinsohn P.M., Seeley J.R. Comorbidity of unipolar depression: II. Comorbidity with other mental disorders in adolescents and adults. J Abnorm Psychol. 1991;100:214–222. doi: 10.1037/0021-843X.100.2.214. [DOI] [PubMed] [Google Scholar]
- 46.Nolen-Hoeksema S. The role of rumination in depressive disorders and mixed anxiety/depressive symptoms. J Abnorm Psychol. 2000;109:504–511. doi: 10.1037/0021-843X.109.3.504. [DOI] [PubMed] [Google Scholar]
- 47.Hayward C., Killen J.D., Kraemer H.C., Taylor C.B. Predictors of panic attacks in adolescents. J Am Acad Child Adolesc Psychiatry. 2000;39(2):207–214. doi: 10.1097/00004583-200002000-00021. [DOI] [PubMed] [Google Scholar]
- 48.McNally R.J. Anxiety sensitivity and panic disorder. Biol Psychiatry. 2002;52(10):938–946. doi: 10.1016/S0006-3223(02)01475-0. [DOI] [PubMed] [Google Scholar]
- 49.Burns G.L., Walsh J.A. The influence of ADHD-hyperactivity/impulsivity symptoms on the development of oppositional defiant disorder symptoms in a 2-year longitudinal study. J Abnorm Child Psychol. 2002;30(3):245–256. doi: 10.1023/A:1015102812958. [DOI] [PubMed] [Google Scholar]
- 50.Harvey E.A., Breaux R.P., Lugo-Candelas C.I. Early development of comorbidity between symptoms of attention-deficit/hyperactivity disorder (ADHD) and oppositional defiant disorder (ODD) J Abnorm Psychol. 2016;125:154–167. doi: 10.1037/abn0000090. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.van Deurzen P.A.M., Buitelaar J.K., Agnes Brunnekreef J., et al. Response time variability and response inhibition predict affective problems in adolescent girls, not in boys: the TRAILS study. Eur Child Adolesc Psychiatry. 2012;21(5):277–287. doi: 10.1007/s00787-012-0260-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52.Bastiaansen J.A., van Roon A.M., Buitelaar J.K., Oldehinkel A.J. Mental health problems are associated with low-frequency fluctuations in reaction time in a large general population sample. The TRAILS study. Eur Psychiatry. 2015;30(2):347–353. doi: 10.1016/j.eurpsy.2014.03.005. [DOI] [PubMed] [Google Scholar]
- 53.Keller A.S., Leikauf J.E., Holt-Gosselin B., Staveland B.R., Williams L.M. Paying attention to attention in depression. Transl Psychiatry. 2019;9(1):1–12. doi: 10.1038/s41398-019-0616-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 54.Avisar A., Shalev L. Sustained attention and behavioral characteristics associated with ADHD in adults. Appl Neuropsychol. 2011;18(2):107–116. doi: 10.1080/09084282.2010.547777. [DOI] [PubMed] [Google Scholar]
- 55.Chuen Yee Lo B., Lau S., Cheung S.H., Allen N.B. The impact of rumination on internal attention switching. Cogn Emot. 2012;26(2):209–223. doi: 10.1080/02699931.2011.574997. [DOI] [PubMed] [Google Scholar]
- 56.Borsboom D., Deserno M.K., Rhemtulla M., et al. Network analysis of multivariate data in psychological science. Nat Rev Methods Primer. 2021;1(1):58. doi: 10.1038/s43586-021-00055-w. [DOI] [Google Scholar]
- 57.Enders C.K., Bandalos D.L. The relative performance of full information maximum likelihood estimation for missing data in structural equation models. Struct Equ Modeling. 2001;8(3):430–457. doi: 10.1207/S15328007SEM0803_5. [DOI] [Google Scholar]
- 58.Huisman M., Oldehinkel A.J., de Winter A., et al. Cohort profile: the Dutch ‘TRacking Adolescents’ Individual Lives’ Survey’; TRAILS. Int J Epidemiol. 2008;37(6):1227–1235. doi: 10.1093/ije/dym273. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.




