An Evaluation of the Specificity of Executive Function Impairment in Developmental Psychopathology

Abstract

Objective

Deficits in executive function (EF) are common in neuropsychiatric disorders, but the specificity of these deficits remains unclear. The aim of the current study was to elucidate the pattern of EF impairment across psychopathologies in children and adolescents. We assessed associations among components of EF with dimensions of psychopathology, including an overall psychopathology factor.

Method

Participants (8–21 years) were from the Philadelphia Neurodevelopmental Cohort (n=9,498). Data from a structured clinical screening interview were reduced into five dimensional domains using factor analyses: overall psychopathology, anxious-misery, fear, externalizing, and psychosis. EF components of attentional vigilance, response inhibition, conceptual flexibility, and working memory were assessed. Associations between the clinical dimensions and both general EF ability and specific EF components were examined.

Results

EF ability showed both common and domain-specific associations with clinical symptoms. General EF was directly associated with general psychopathology, anxious-misery, and psychosis domains, but not with the fear or externalizing domains. For the EF subcomponents, differences emerged in the magnitude and direction of association between the components and clinical domains. Poorer EF was typically associated with increased symptoms across clinical domains; however, in some instances, better EF ability was associated with greater symptom burden, particularly in the fear domain.

Conclusion

EF has widespread associations with psychopathology in youth. Findings reveal some overlap in the type of EF impairment across clinical phenotypes, as indicated by similar patterns of associations between some clinical symptoms and EF. However, findings also revealed domain-specific associations with EF that differed across EF components and clinical domains.

INTRODUCTION

A strong link between executive function (EF) impairment and psychopathology appears across the lifespan and is documented in multiple neuropsychiatric disorders,¹ both at clinical and subclinical manifestations of the disorders. However, questions remain concerning the specificity of these EF deficits.² More research is needed to fully understand if 1) EF impairment is similar or varies across different dimensions of psychopathology and 2) whether the different components of EF have comparable contributions within and across clinical domains. To address these questions, performance on multiple EF components must be directly compared across multiple clinical domains.³ Thus, using the Philadelphia Neurodevelopmental Cohort (PNC), a large community-based youth sample, the current study aimed to elucidate the type and degree of EF dysfunction associated with multiple dimensions of psychiatric symptoms. To do this, the current study used bifactor factor models to examine EF associations with a general psychology factor,⁴ as well as domain-specific dimensions of psychopathology.

Although there are alternative theoretical and empirical approaches to the study of EF,^5,6 it is widely accepted that EF is a hierarchical construct that encompasses component processes. These processes include the ability to orient and sustain attention (i.e., attention vigilance), shift attention and think flexibly (i.e., attentional/conceptual flexibility), inhibit responses, and maintain and update goal-related information in working memory. In both children and adults, these cognitive processes are distinct but interrelated components of EF.^7,8 Therefore, understanding the role of separate EF components, as well as the common EF ability that spans across components (i.e., general EF), is a critical step in better understanding EF deficits in psychopathology.

To date, few studies have investigated the specificity of EF deficits across multiple clinical domains.^9–13 Findings from meta-analyses suggest that the type and level of EF deficits are comparable for many psychiatric disorders.^2,14,15 Specifically, the effect sizes for EF impairment relative to healthy comparisons range from moderate-to-high for attention-deficit/hyperactivity disorder (ADHD), schizophrenia, and bipolar disorder, and from small-to-high for major depression. However, the extant work directly comparing EF performance across neuropsychiatric disorders yields mixed findings.^16–22 For example, some studies document worse EF performance in youth with ADHD compared to youth with anxiety disorders¹⁹; however, other work finds no such EF differences.²⁰

Some recent work used structural equation modeling to form dimensions of psychopathology. This work suggests that general EF dysfunction is a non-specific correlate of developmental psychoapthology.^11–13 For example, Martel et al.¹² found that general EF was negatively associated with a general psychopathology factor, but not with domain-specific externalizing, fear, or distress factors. Shanmugan et al.¹³, using a subset of the PNC cohort, reported robust negative associations between a general psychopathology factor and both behavioral and neuroimaging indices of working memory; however, disorder-specific EF perturbations also emerged for psychosis spectrum, externalizing, and anxious-misery factors. Thus, further work is needed to better understand the common and unique associations between EF and specific domains of psychopathology.

Growing evidence suggests that within a given clinical phenotype, different components of EF have unique patterns of associations with the disorder.^23–26 For ADHD, deficits in certain EF components are larger (i.e., response inhibition) than others (i.e., attention shifting).²⁷ This suggests that specific EF components may be better markers of certain psychopathologies. Although less work has examined EF deficits in anxiety, prior studies suggest that different patterns of EF impairment occur in subtypes of anxiety (e.g., generalized anxiety disorders, social phobia, posttraumatic stress disorders).^18,20,28,29 Notably, some work found links between higher EF ability and anxiety symptomatology in both children and adults.^13,23,30,31 For example, White et al.²³ found that enhanced inhibition predicted higher levels of anxiety in children at temperamental risk for anxiety.

In summary, it is important to understand both the type and degree of EF dysfunction that is common and unique across domains of psychopathology. To address these issues, the current study directly assessed associations among multiple components of EF and multiple domains of psychopathology using a bifactor model with one general psychopathology factor and domain-specific factors (i.e., anxious-misery, fear, externalizing, and psychosis spectrum). We first examined whether impairment in a general EF factor was disorder-specific or a transdiagnostic factor associated with general psychopathology. Next, we examined if different EF components differentially related to the different clinical domains.

METHOD

Participants

The PNC is a community-based sample of youth. Enrollment was based at the Center of Applied Genomics at the Children’s Hospital of Philadelphia (CHOP) and the Brain Behavior Laboratory at the University of Pennsylvania (detailed in Calkins et al. ^32,33 and Satterthwaite et al.³⁴). All study participants previously consented for genomic studies while seeking pediatric services at CHOP. From this pool of about 50,000, a total of 18,344 individuals who had consented for recontact and met inclusion criteria were randomly selected (with stratification for sex, age, and ethnicity) for participation in the PNC study. Exclusionary criteria were: lack of English proficiency and medical conditions that would significantly impact brain function or interfere with the ability to complete study procedures. Of note, participants were not recruited from psychiatric services, thus the sample is not enriched with psychiatric treatment-seeking individuals.

A total of 9,498 youths enrolled in the study, ages 8–21 years at the time of enrollment. Complete and valid assessment across the entire EF battery was obtained for 8,856 participants. For the analyses that only relied on data from one of the EF tasks, more participants were included in the given analyses: Penn Continuous Performance task (PCPT; n=9,145), Penn Conditional Exclusion Test (PCET; n=9,188), and the Penn Letter N-Back Test (N-BACK; n=9,004). In the sample with valid data from at least one EF task, the mean age of participants was 14.23 years (SD=3.64). More information of the sample’s age distribution is provided in Supplement 1, available online. In the sample, 52% were females; 57% were Caucasian; 32% were African-American; and 11% were of mixed or other race. Years of maternal and paternal education were 14.5 (SD= 2.4) and 14.3 (SD=2.7), respectively.

After the study procedures were explained, consent was obtained from participants aged 18 or older; assent and parent/legal guardian permission was obtained for participants aged 17 and younger. All study procedures were approved by CHOP and University of Pennsylvania institutional review boards.

Clinical Assessment

Psychopathology was assessed through a computerized structured clinical screening interview, which was administered by trained clinical coordinators as previously detailed.³² Interviews were conducted in the laboratory or at the participant’s home, depending on participant preference. The clinical assessment was adapted from the National Institute of Mental Health Genetic Epidemiology Research Branch Kiddie-Schedule for Affective Disorders and Schizophrenia³⁵ and included assessment for mood (major depressive episode, manic episode), anxiety (generalized anxiety disorder, social anxiety disorder, specific phobia, separation anxiety disorder, panic disorder, posttraumatic stress disorder, obsessive-compulsive disorder, agoraphobia), behavioral (oppositional defiant disorder, conduct disorder), ADHD, eating (anorexia, bulimia), psychosis spectrum, and suicidal thoughts and behavior. Each section included a screen of several questions assessing disorder-related symptoms. Additional psychosis-related items (n=12) from the Prevention through Risk Identification, Management, and Education (PRIME) Screen-Revised,³⁶ to assess subthreshold positive symptoms, and Scale of Prodromal Symptoms (SOPS; n=6) from the Structured Interview for Prodromal Syndromes (SIPS)³⁷ to assess negative/disorganized symptoms, were also included in the interview.³³ The disorder-related screen items, additional psychosis items, and four general mental health treatment questions were used in the bifactor model of psychopathology (total items=112). Information related to frequency, duration, level of impairment, and distress associated with each disorder were also collected during the interview; however, this information was not included in the bifactor model. The interviews were administered to probands (participants age 11–21) and collaterals (parent or legal guardian of participants age 8–17).

Executive Function Assessment

The EF tasks were administered as part of a larger cognitive assessment battery: the Penn Computerized Neurocognitive Battery.³⁸ Three EF tasks were used to assess the following EF components: response inhibition, attentional vigilance, conceptual flexibility, and working memory performance. A general EF score was also created by averaging the z-scored EF components (attentional vigilance, response inhibition, conceptual flexibility, and working memory). Additional variables were generated using a signal detection approach. More information about EF tasks and scoring are available in Supplement 1, available online. PCPT: The PCPT measured response inhibition and attentional vigilance (aka sustained attention). Stimuli were either numbers or letter (target) or non-letter or non-number (foil). Participants were told to respond to targets by button press (go-trials) and refrain from responding to foil presentations (no-go trials). A response inhibition score was created based on number of true negatives (i.e., accuracy on no-go/foil trials). An attentional vigilance measure was calculated based on the number of true positives (i.e., accuracy on go/target trials).

PCET

The PCET was used as a measure of abstraction and conceptual flexibility (often referred to as cognitive or attentional flexibility). Four stimuli were presented simultaneously, and participants needed to determine which stimulus was the “odd man out” based on various visual features of the stimuli (e.g., shape, size). Feedback (“correct” or “incorrect”) was provided after each trial, from which participants needed to deduce the stimulus exclusion rule. The rule switched throughout the task, requiring abstraction abilities to discover the rule and conceptual flexibility for shifts between rule sets. A performance score was created to reflect overall correct responses and total learning.

N-Back

For this working memory task, a series of letters was presented on the screen. Participants were required to respond to a given letter across three conditions (i.e., working memory loads): 0-back (respond when letter X appears), 1-back (respond when current letter is the same as previous letter), and 2-back (respond when current letter is the same as the letter before the previous letter). A working memory score was created based on overall accuracy.

Signal Detection-Based Variables

Additional variables were generated using a signal detection approach. Prior work suggests that response bias (i.e., tendency to be cautious or impulsive) can influence standard accuracy-based calculations³⁹ and varies across psychopathology.⁴⁰ Thus, indices of discrimination accuracy (P_r) and response bias (B_r) were calculated for PCPT and the N-back task.^39,41,42 Additional information on these variables is provided in Supplement 1, available online.

Data Analysis

Structural equation modeling assessed the relations between EF and psychopathology. There were five main models (i.e., general EF, vigilance, response inhibition, conceptual flexibility, and working memory). Clinical items from the proband report (or collateral report for children younger than 11, where proband data was not available), were modeled as a confirmatory bifactor model,⁴³ which allows each item to load not only on a specific clinical factor (e.g. psychosis spectrum), but also on an overall (“general”) psychopathology factor. The existence of the general factor allows the specific factors to be modeled as orthogonal to each other and to the general factor. We modeled four specific factors, based on previous work with these data¹³ as well as on theory. Krueger et al.⁴⁴ demonstrated that mental disorders (not including psychosis) can be grouped into three categories—externalizing, anxious-misery, and fear—and in our case, we have additional items relating to psychosis, for a total of four specific factors. Preliminary exploratory item-factor analysis of these data strongly supported this decision, with four obliquely rotated factors clearly representing the four specific factors listed above.

The above measurement model was estimated simultaneously with the effects of interest. That is, the observed independent variables of interest (i.e., general EF or one of the subcomponents) was allowed to relate to the latent variables in the measurement model, and these effects were estimated simultaneously with the measurement model itself. The EF variable (general EF or sub-component) was modeled separately, along with the following covariates: sex, age, and race. Interactions among EF, sex, and age were also included. In each model, observed variables (including interactions) were allowed to correlate with each other, as well as relate to all latent variables in the measurement model. All models were estimated in Mplus using the mean- and variance-adjusted weighted least squares (wlsmv) estimator.⁴⁵ Model fit was assessed using cutoff criteria suggested in prior work.^46,47 Factor loadings from the general EF model appear in the supplemental material (see Table S1, available online). Model fits and results for the four models using signal detection variables are presented in the supplemental material (see Table S2, available online). To correct for type 1 error inflation, p was set to ≤ .001.

RESULTS

Model Fit

Figure 1 shows the structural equation model with associations between general EF and the bifactor model of psychopathology. Fit of all models was acceptable (often excellent). Specifically, for the model including overall EF, comparative fit index (CFI) = 0.96; Tucker-Lewis index (TLI) = 0.96; and root mean-square error of approximation (RSMEA) = 0.027±0.001. The models containing the EF sub-components of attentional vigilance, response inhibition, and working memory had the following fit indices: CFI = 0.95, 0.96, and 0.94, respectively; TLI = 0.95, 0.95, and 0.94, respectively; RMSEA = 0.027±0.001, 0.027±0.001, and 0.027±0.001, respectively. Due to convergence problems with the mean- and variance-adjusted weighted least squared estimator in the conceptual flexibility model, this model was estimated with the BAYES estimator, which does not provide conventional fit indices.

Figure 1.

Example of the structure equation model with general executive function (EF) used to assess associations between EF and the bifactor model of psychopathology. Note: Interactions between EF and age and sex are not shown in figure.

Associations Between General EF and Clinical Domains

Table 1 presents specific associations between general EF and the clinical factors from the bifactor model. Total EF was significantly related to symptom levels for the general psychopathology, anxious-misery, and psychosis domains. For these domains, lower general EF scores were associated with increased symptom levels. This pattern of association was strongest for the psychosis domain. General EF was not directly related to fear or externalizing symptoms.

Table 1.

Executive Function (EF) Components (Vigilance, Response Inhibition, Conceptual Flexibility, Working Memory) Predicting Clinical Domains

	Overall Psychopathology		Anxious Misery		Fear		Externalizing		Psychosis
	β	p	β	p	β	p	β	p	β	p
General EF
EF	−0.172	.000	−0.292	.000	−0.075	.032	−0.050	.134	−0.349	.000
Age	0.255	.000	−0.138	.000	−0.114	.000	−0.111	.000	−0.198	.000
Sex	−0.094	.000	0.326	.000	0.315	.000	−0.076	.000	0.067	.000
Race	−0.234	.000	0.222	.000	0.025	.074	0.015	.284	0.000	.989
EF×Age	0.569	.000	−0.418	.000	−0.586	.000	−0.444	.000	−0.219	.000
EF×Sex	−0.425	.000	0.913	.000	0.683	.000	0.391	.000	0.582	.000
Attentional vigilance (ATT)
ATT	−0.08	.085	−1.007a	.000	0.605	.000	−0.086	.152	−0.394	.000
Age	0.294	.000	−0.244	.000	0.033	.146	−0.171	.000	−0.184	.000
Sex	−0.096	.000	0.345	.000	0.279	.000	−0.07	.000	0.084	.000
Race	−0.212	.000	0.17	.000	0.011	.481	−0.02	.143	−0.012	.416
ATT×Age	0.409	.000	0.45	.000	−1.122a	.000	−0.325	.000	−0.094	.153
ATT×Sex	−0.343	.000	0.789	.000	0.427	.000	0.401	.000	0.478	.000
Response inhibition (RI)
RI	−0.128	.000	−0.472	.000	0.093	.019	−0.086	.052	−0.401	.000
Age	0.291	.000	−0.203	.000	−0.118	.000	−0.143	.000	−0.236	.000
Sex	−0.103	.000	0.319	.000	0.311	.000	−0.060	.000	0.062	.000
Race	−0.206	.000	0.190	.000	−0.010	.475	−0.017	.194	−0.024	.106
RI×Age	0.466	.000	−0.121	.008	−0.602	.000	−0.366	.000	−0.080	.097
RI×Sex	−0.407	.000	0.835	.000	0.598	.000	0.375	.000	0.544	.000
Conceptual flexibility (CF)
CF	0.128	.000	−0.228	.000	−0.056	.061	−0.292	.000	0.166	.000
Age	0.223	.000	0.147	.000	−0.019	.000	−0.092	.000	−0.045	.000
Sex	−0.067	.000	0.238	.000	0.220	.000	−0.145	.000	−0.014	.167
Race	−0.185	.000	0.176	.000	−0.058	.000	−0.026	.000	−0.107	.000
CF×Age	−0.146	.000	0.170	.000	−0.006	.435	0.202	.000	−0.194	.000
CF×Sex	−0.007	.305	0.067	.000	0.038	.007	0.017	.124	0.014	.229
Working memory (WM)
WM	−0.282	.000	−1.294a	.000	1.167a	.000	−0.196	.006	−0.547	.000
Age	0.279	.000	−0.251	.000	0.083	.003	−0.184	.000	−0.205	.000
Sex	−0.073	.000	0.340	.000	0.222	.000	−0.072	.000	0.070	.000
Race	−0.228	.000	0.123	.000	0.052	.005	−0.018	.203	−0.035	.032
WM×Age	0.583	.000	0.680	.000	−1.567a	.000	−0.228	.000	0.065	.389
WM×Sex	−0.331	.000	0.873	.000	0.321	.000	0.394	.000	0.531	.000

Note: Boldface indicates significant effects (p ≤ .001).

^aThough model estimation terminated normally, coefficients with absolute value >1.0 were possible due to non-positive-definite residual variance/covariance matrices.

Associations Between EF Components and Clinical Domains

The magnitude of relations among the EF components and symptom levels was different across the five clinical domains (see Table 1). With only a few exceptions, each of the separate EF components significantly predicted symptom levels across the five domains; however, the strength and direction of these associations differed across the domains. None of the four EF components had significant associations with all five clinical domains. Anxious-misery and psychosis were the only domains that had significant associations with all EF components. The externalizing and fear domains had the fewest associations with EF components. Although most EF-symptom associations were negative (i.e., poorer EF was related to higher symptom levels), some positive associations emerged. For example, higher attentional vigilance and working memory abilities were related to higher symptom burden in the fear domain. Surprisingly, conceptual flexibility also showed positive associations with symptoms in the psychosis and general psychopathology domains.

Interactions Between EF and Age and Sex

As reported in Table 1, many significant sex-by-EF and age-by-EF interactions emerged across the five clinical domains. As such, only the largest interactions and those within the general psychopathology model are highlighted in the text. For the significant EF-sex interactions for general psychopathology, high EF was associated with less general psychopathology symptoms for females compared to males. Large interactions emerged for the anxious-misery domain. Across the EF factors, better EF was associated with less anxious-misery symptoms in males than females, and in some cases (i.e., general EF and response inhibition), better EF in females was associated with more anxious-misery symptoms. For the age interactions, the relations between poor general EF, response inhibition, and working memory with higher general psychopathology symptoms tended to be stronger for younger children. Also, the positive association that emerged between conceptual flexibility and general psychopathology was only present in younger children. Large age-by-EF interactions emerged for the fear domain, revealing that higher EF, across the subcomponents, tended to be related to higher fear symptoms in younger children, but in older youth, higher EF was related to lower fear symptoms.

DISCUSSION

Most neuropsychiatric disorders, and even subclinical manifestations of disorders, are associated with EF impairments; however, the specificity of these associations is unclear. To help delineate the role of EF impairment across clinical phenotypes, the current study used a large community sample to assess associations between multiple EF components and different domains of psychopathology, including an overall psychopathology factor. We found overlap in the type of EF impairment across clinical symptoms, as indicated by 1) the significant associations between EF and overall psychopathology and 2) similarities in some of the patterns of associations between EF and symptom levels across domain-specific clinical factors. However, EF also showed significant domain-specific associations that differed across EF components and clinical domains.

The extant work linking EF and latent dimensions of psychopathology finds strong associations with general psychopathology rather than domain-specific factors.^11,12 In the current study, all EF components were significantly associated with overall psychopathology, except for attentional vigilance. This finding supports prior work suggesting that the type of EF impairment linked to overall psychopathology is widespread and involves multiple components of EF. However, in the current study, EF was also associated with several domain-specific symptoms. Indeed, direct effects between EF factors and symptoms were detected for all domain-specific factors. Moreover, the magnitude of these domain-specific associations was often larger than associations between EF and the general psychopathology factor. For example, poor general EF was more strongly related to anxious-misery and psychosis-spectrum symptoms than it was to general psychopathology symptoms.

The current study also found differences in the strength of associations between the specific EF components and symptom levels, both within and across clinical domains. For example, for anxious-misery and fear symptoms, attentional vigilance and working memory had more robust relations to symptoms than response inhibition or conceptual flexibility. Conversely, for the general psychopathology and psychosis domains, the magnitude of effects across EF subcomponents were fairly similar. Taken together, these findings support previous work^23–25,27 that suggests EF components differentially relate to neuropsychiatric symptom clusters. Moreover, these findings suggest that it is important for future work to assess different EF components, as the strength of association between clinical symptoms and EF components varied. Had only response inhibition been used as a proxy for EF in the present study, EF impairment would appear relatively similar across clinical domains. However, had working memory been used as a proxy for EF, more pronounced differences between the domains would have emerged.

Although most EF components had negative associations between EF ability and symptom levels, some positive associations emerged. For example, in the fear domain, better ability in attentional vigilance and working memory was linked to higher fear symptoms. Fear symptoms were also related to a cautious response style on the continuous performance task. Positive, albeit weak, associations also emerged between conceptual flexibility and symptoms in the general psychopathology and psychosis domains. Prior work has linked anxiety-related symptoms to enhanced EF abilities, including better performance monitoring, working memory, and response inhibition.^{13,23,30,48–50} Thus, increased ability of some EF components may serve to cause or maintain certain clinical symptoms related to fear. Alternatively, at least in a community sample, anxiety or other subthreshold symptoms may improve certain EF functions⁵¹ or task motivation. It is worth noting that the positive associations found within the fear domain did not emerge for the anxious-misery domain. Thus, although the anxious-misery domain (reflecting negative mood and intrusive negative thoughts) and the fear domain (reflecting nervousness and fear across multiple contexts) are often considered to be highly related constructs, notable EF differences were detected in the current study. These differences add to a growing literature showing that distress-related and fear-related disorders differ on multiple cognitive and affective processes.^52–54

Understanding how the relations between EF and psychopathology may differ by sex and age is important, as these factors can significantly impact processes underlying psychopathology and EF.^55,56 We found that the EF-symptom relationship differed as a function of both age and sex; however, the direction of these interactions differed across clinical domains and EF components. Of note, when positive associations did emerge between EF and symptoms, interaction effects suggested that the pattern was only present for younger children; in older youth, EF either had little effect on symptom levels or higher EF was related to lower fear symptoms. As for sex differences, within the general psychopathology factor, a pattern emerged in which higher EF tended to be more protective (related to lower symptoms) in females than males. However, for the domain-specific factors, higher EF tended to be more protective in males. For both the age and sex interactions, the causal nature of these interactions is unclear. It will be important for future longitudinal work to tease apart the directionality of these associations.

There are several limitations to the present study. The study used uncorrelated dimensional measures to isolate distinct clinical phenotypes and reduce issues of comorbidity. While this approach enabled better examination of specificity of EF deficits across clinical domains, the results may not hold when comparing clinical groups according to their diagnostic categories or in instances of high comorbidity across clinical domains. A second limitation concerns the cross-sectional nature of the study; we cannot ascertain whether EF dysfunction is a consequence of psychopathology or a developmental risk factor, or both. Prior work suggests that the directionality of such associations may differ among EF components.^57,58 Additionally, the level of variance explained by the domain-specific factors is minimal, which could influence the current pattern of results. Lastly, we assessed EF in neutral contexts; associations between EF and psychopathology may differ when EF is assessed during emotional contexts.⁵⁹

These limitations notwithstanding, our findings highlight important differences in the strength and direction of associations between EF, both general EF and specific components, and psychopathology across development. These findings suggest that future work should consider the differences across EF components, as the strength of association between clinical symptoms and EF components differed. Moreover, the current findings suggest that future treatment and prevention work may benefit by targeting deficits in specific EF components (e.g., working memory) for a given disorder (e.g., depression).