Testing the Stability and Validity of an Executive Dysfunction Classification Using Task-Based Assessment in Children and Adolescents

Abstract

Objective:

It is unclear if pediatric executive dysfunction assessed only with cognitive tasks predicts clinically relevant outcomes independently of psychiatric diagnoses. This study tested the stability and validity of a task-based classification of executive function.

Method:

A total of 2,207 individuals (6–17 years old) from the Brazilian High-Risk Cohort Study participated in this study (1,930 at baseline, 1,532 at follow-up). Executive function was measured using tests of working memory and inhibitory control. Dichotomized age- and sex-standardized performances were used as input in latent class analysis and receiver operating curves to create an executive dysfunction classification (EDC). The study tested EDC’s stability over time, association with symptoms, functional impairment, a polymorphism in the CADM2 gene, polygenic risk scores (PRS), and brain structure. Analyses covaried for age, sex, social class, IQ, and psychiatric diagnoses.

Results:

EDC at baseline predicted itself at follow-up (odds ratio [OR] = 5.11; 95% CI 3.41–7.64). Participants in the EDC reported symptoms spanning several domains of psychopathology and exhibited impairment in multiple settings, including more adverse school events (OR = 2.530; 95% CI 1.838–3.483). Children in the EDC presented higher attention-deficit/hyperactivity disorder and lower educational attainment PRS at baseline; higher schizophrenia PRS at follow-up; and lower chances of presenting a polymorphism in a gene previously linked to high performance in executive function (CADM2 gene). They also exhibited smaller intracranial volumes and smaller bilateral cortical surface areas in several brain regions.

Conclusion:

Task-based executive dysfunction is associated with several validators, independently of psychiatric diagnoses and intelligence. Further refinement of task-based assessments might generate clinically useful tools.

Current definitions of mental disorders rely primarily on behavioral observations and symptom reports.¹ Ongoing initiatives, such as Research Domain Criteria,² seek to integrate task performance measurements into these definitions. With that approach, task-based measurements may connect clinical assessments to neuroscientific understandings in ways that inform pathophysiology³ and increase objectivity in current classification schemes.^4,5 The validity of task-based classification can be evaluated through research of external correlates such as functional impairment, established genetic factors for psychopathology, and brain structure, while adjusting for the effects of current diagnostic categories. The current study extends preliminary work in pediatric psychopathology by examining associations among task-based assessments of executive function, symptom reports, and external validators.

This study used tasks from a well-established construct: executive function (EF). EF encompasses the high-level cognitive skills needed to plan and perform goal-directed behaviors.⁶ Although different models of EF exist, most definitions include domains of working memory and inhibitory control.^7–10 EF deficits relate to overall levels of psychopathology¹¹ and occur in most psychiatric disorders.^12–15 The deficits also predict adverse outcomes.^16,17 Although testing of EF possesses some clinical utility and validity,¹⁸ most work with EF has examined its relationship to specific disorders.^19,20 Much of that research examines pediatric samples, given the relevance of EF for neuro-developmental disorders. However, few comprehensive studies have evaluated the utility of EF in classification. Available work typically combines data from EF tests and symptom reports, despite low correlations between the 2 sets of measurements. Therefore, the utility of stand-alone task-based EF classification in youth remains insufficiently evaluated. Work is needed to evaluate associations between functional outcomes and biological correlates, independent of socioeconomic factors, intelligence, and concurrent psychopathology.

The current study proceeded in 3 stages to define and validate a profile of EF impairment. First, the study used working memory and inhibitory control measurements to identify youth with impaired EF, defined as the executive dysfunction classification (EDC). Next, the study examined associations among the EDC with symptom-based psychopathology measurements and clinically meaningful longitudinal outcomes, aiming to evaluate its validity.²¹ Finally, the study examined relationships between genetic and brain structure variables. All analyses were adjusted for age, sex, IQ, socioeconomic status, and psychiatric diagnoses. Through these stages, this study tested the hypothesis that youth in the EDC will manifest impaired function, associated psychopathology, genetic risk indicators, and differences in brain structure. Using EDC as an example, we aimed to test whether operationalized task-based classifications could add information above and beyond the current symptom-based diagnostic categories.

METHOD

Sample Description

The Brazilian High-Risk Study for Psychiatric Disorders (BHRCS) is a large school-based community cohort from 2 Brazilian cities: Porto Alegre and São Paulo. A total of 9,937 school-enrolled children from 8,012 families participated in the screening process in 2009. The screening process consisted of the Family History Survey, which identified children from families at high-risk for psychiatric disorders. After fulfilling inclusion criteria, high-risk children (n = 1,554) and a random community sample (n = 957) were selected for participation in the cohort. The baseline assessment was performed from 2009 to 2010 and consisted of structured interviews to assess demographic variables, presence of psychiatric symptoms and diagnoses, neuropsychological testing, collection of biological material for genotyping (available for a subsample of 2,185 participants), and neuroimaging procedures (available for a subsample of 741 participants). At their baselines, children were 6–14 years old (mean 9.65, standard deviation ± 7.36 years); 53.8% were males participants; mean parental age was 35.6 years (±7.36 years); and most families were middle-class (67.5%; median monthly income of R$2,900 (US$1,610). The sample was ethnically diverse: 60.5% were White, 28.2% were Multiracial, 10.5% were Black, 0.4% were Brazilian Indigenous, 0.2% were Asian, and 0.2% did not declare race or ethnicity. Follow-up interviews were conducted during 2013 to 2014 (on average 3 years later) when children were 9–17 years old, with retention rates of 80%. The ethical committee of the University of São Paulo approved the study, and parents of all participants provided signed informed consent. Further information for the BHRCS can be found elswhere.²²

Inclusion and Exclusion Criteria

This study’s sample consisted of all participants who had complete information for age, neuropsychological tests, and IQ. We excluded participants with IQ <70 to avoid biases regarding intellectual disability. A total of 1,930 subjects (6–14 years old; 54.7% male participants) were analyzed at baseline, and 1,531 (9–17 years old; 55.8% male participants) were evaluated at follow-up. The analyzed sample was not statistically distinct from the full BHRCS cohort for age, sex, socioeconomic score, presence of any psychiatric diagnosis, or psychopathology (all p > .05). Losses to follow-up were also not statistically distinct regarding age, sex, socioeconomic score, presence of any psychiatric diagnoses, dichotomized test results, and EDC group assignment (all p > .05). Information for the sample after inclusion and exclusion criteria were applied, including comparisons between high-risk and randomly selected groups, is shown in Table 1.

TABLE 1.

Sample Description: Random and High-Risk Sample

	Baseline					Follow-up
	Full sample, %	Random sample, %	High-risk sample, %	X2 (df)	p	Full sample, %	Random sample, %	High-risk sample, %	X2 (df)	p
Demographics
Sex (male participants)	54.7	52.7	55.9	1.9 (1)	.173	55.8	55.2	56.2	0.1 (1)	0.732
Ethnicity (White)	60.4	61.2	59.9	0.3 (1)	.598	60.7	61.5	60.1	0.2 (1)	0.622
Psychiatric disorders
Any	26.7	20.3	30.8	25.6 (1)	<.001*	23.7	18.6	27.0	14.0 (1)	<0.001*
ADHD	10.9	8.4	12.5	7.7 (1)	.006*	5.1	4.0	5.8	2.2 (1)	0.139
Conduct	6.7	5.4	7.6	3.4 (1)	.066	5.2	4.0	6.0	2.7 (1)	0.099
Anxiety	5.3	4.1	6.1	3.4 (1)	.066	9.7	7.1	11.3	6.9 (1)	0.009*
Depression	3.1	1.3	4.3	12.6 (1)	<.001*	7.3	4.6	8.9	9.4 (1)	0.002*
Neuropsychology
EDC status	15.9	14.5	16.8	1.6 (1)	.209	8.8	8.4	9.1	0.1 (1)	0.758
	Full sample (SD)	Random sample (SD)	High-risk sample (SD)	t	p	Full sample (SD)	Random sample (SD)	High-risk sample (SD)	t	p
Demographics
Age	10.2 (1.9)	10.0 (1.9)	10.3 (1.9)	−3.4	<.001*	13.4 (1.9)	13.3 (1.9)	13.5 (1.9)	−2.5	0.014*
Socioeconomic statusa	21.6 (4.7)	21.8 (4.6)	21.5 (4.7)	1.6	.118	21.6 (4.6)	21.9 (4.6)	21.4 (4.5)	2.0	0.040*
Neuropsychology
IQb	100.5 (14.5)	101.7 (14.6)	99.7 (14.3)	2.8	.005*	—	—	—	—	—
Corsi blocks	4.8 (2.1)	4.8 (2.0)	4.8 (2.1)	0.5	.612	5.9 (2.1)	6.0 (2.1)	5.9 (2.1)	0.6	0.518
Digit span	3.6 (1.6)	3.5 (1.5)	3.6 (1.6)	−0.2	.828	4.4 (1.7)	4.5 (1.7)	4.4 (1.7)	0.9	0.372
Go/no-go	24.9 (21.7)	24.6 (22.1)	25.0 (20.0)	−0.4	.709	19.2 (21.0)	17.9 (20.4)	20.1 (21.4)	−2.0	0.045*
Conflict control	59.7 (21.3)	60.0 (22.0)	59.5 (20.8)	0.5	.637	65.4 (25.6)	64.7 (27.1)	65.8 (24.5)	−0.8	0.422

Note: ADHD = attention-deficit/hyperactivity disorder; df = Degrees of Freedom; EDC = Executive Dysfunction Classification.

^aRanges from 3–40, the higher the score the better the socioeconomic status.

^bAssessment available only at baseline.

^*p <.05.

Executive Function Assessment

We assessed EF using cognitive tests conducted by trained mental health professionals. We tested the reliability of the tasks in our cohort using Confirmatory Factor Analyses, Cronbach’s alpha, and omega statistics. All tasks presented adequate reliability (Supplement 1 and Table S1, available online). The 2 constructs of EF assessed in this study were working memory and inhibitory control. The following tasks were used to measure those constructs at baseline and follow-up.

Digit Span Task.²³

Working memory assessment. This subtest of the WISC-III consists of hearing and repeating an increasing number sequences, either as heard (forward) or in reverse order (backward). The level at which the child failed to repeat the numbers on 2 consecutive trials in the backward condition correctly was the outcome variable.

Corsi Blocks Task.²⁴

Working memory assessment. This test involves repeating a spatial sequence of up to 9 identical spatially separated blocks. The sequences are tapped by a researcher and increase in length, either as shown by the examiner (forward) or in reverse order (backward). The level at which the child failed to repeat the sequence of blocks on 2 consecutive trials in the backward condition correctly was the outcome variable.

Conflict Control Task.²⁵

Inhibitory control assessment. In this test, children are instructed to press a button indicating the direction (congruent trial, 75 trials) or the opposite direction (incongruent trials, 25 trails) of the arrow that appears on the screen. A green arrow indicates a congruent trial and a red arrow an incongruent one. The intertrial interval was 1,500 ms, and the stimulus duration was 100 ms. This test’s “conflict effect” is based on suppressing the dominant tendency to indicate the arrow’s direction in the incongruent trials. Both accuracy and speed are equally emphasized in task instructions. The percentage of correct responses in the incongruent trial was the outcome variable.

Go/No-Go.²⁶

Inhibitory control assessment. Analogous to the conflict control task, in this test, children are instructed to indicate the direction of the arrow that appears on the screen (75 trials) or to suppress the stimuli entirely and do not press the button when a double-headed green arrow appears (25 trials). The intertrial interval was 1,500 ms, stimulus duration was 100ms, and accuracy and speed were emphasized in the test instructions. The percentage of failed inhibitions in the no-go trials was the dependent measurement.

Operationalization of the EDC

This study performed a sequential 3-step approach to operationalize the EDC. Multiple tasks were used instead of 1 task because it was assumed that each task was an incomplete indicator of the EF construct. Aggregating information from distinct sources of variance is likely to improve the phenotypic characterization classification and stability. The following 3-step classification procedure was used to operationalize the EDC at both baseline and follow-up assessments.

Test Result Threshold.

First, each test score’s distribution was investigated only in the random sample of the BHRCS (n = 957), thus generating normative performance tables for each age group and sex. Then, the performance of all subjects of the sample (high-risk and random) was dichotomized into low performance (defined as at or below the 10th percentile of the reference population) versus normal or high performance, adjusting for sex and age. The 10th percentile threshold was selected a priori because it has been used to stratify performance in previous studies.¹³ At the end of that step, each subject had a dichotomous result for each of the 4 tests. The study used dichotomization strategies because the intent was to investigate associations with EDC and not characterize a dimensional measurement of EF in this sample. By dichotomizing test results, the study enhanced its clinical applicability, given the ordinary use of age and sex performance norms in neuropsychological assessments.

Classification Threshold.

Using the individual indicators of low versus normal or high test performance in each test as input, a data-driven analysis was performed to find a cluster of subjects with the lowest test performance globally. This analysis was performed using latent class analysis (LCA). At the end of this step, each subject was classified as being a class member of a global low-performance cluster or not. Such data-driven methods are used in modern medical literature to empirically derive clusters, reducing arbitrariness of predetermined group assignment thresholds. Nevertheless, their external validity was problematic, because they are inherently sample-dependent, and so far, no study performed operationalization of these techniques for independent replication or implementation in clinical practice.

Clinical Translation.

As it would be unfeasible for a clinician in a real-world setting to perform an LCA to assign class membership for individual patients, ROC curves were used to determine the number of low-performance test results needed to best identify the low-performance cluster of subjects defined by the LCA (considered the “gold standard”). The optimal cutoff for ROC analyses was estimated using Youden’s J statistic, which maximizes sensitivity and specificity.²⁷ This last step was used to allow the EDC to be determined using simply the number of low-performance tests and therefore be applicable in clinical settings.

Validators

Symptom-Level Analysis and Categorical Diagnoses.

We used the Brazilian version of the Child Behavior Checklist (CBCL)²⁸ to investigate dimensional psychopathology. The CBCL is a 121-item questionnaire that provides information on several dimensional psychopathology domains, including anxiety/depression, withdrawal/depression, aggressive behavior, attention difficulties, rule-breaking behavior, social problems, somatic complaints, thought problems, and others.^29,30 Categorical diagnoses of the main child-adolescent psychiatric higher-order groups (Any Anxiety Disorder, Any Depressive Disorder, Any Attention-Deficit/Hyperactivity Disorder, and Any Disruptive Behavior Disorder) were performed by the Development and Well-Being Assessment (DAWBA) administered by trained lay interviews and answered by the subject’s parents at baseline. At follow-up, the DAWBA was also administered by trained psychologists to children and adolescents for internalizing modules with the final diagnosis being made by a psychiatrist using the best estimate procedure from the 2 separate interviews.³¹

Impact on Functioning.

We evaluated the impact on different settings (education, family life, and friendships). For all these settings, the impact was initially measured using the SDQ impact module, answered by the participant’s parents and teachers.^32,33 In this section of the questionnaire, parents and teachers were asked to what degree the child’s difficulties interfered with the evaluated areas; responses were classified as: “not at all,” “only a little,” “a medium amount,” or “a great deal.” We considered impairment to be present if difficulties interfered at least “a medium amount.” Teacher reports were available for a subsample of 1,189 participants at baseline.

Education.

Subjects’ parents were asked directly about categorical adverse school events (repetition, dropout, suspension, and expulsion). A composite dimensional score containing those items was called “non-attendance”; in addition, a categorical variable denoting the occurrence of “Any Negative School Event” was constructed. School achievement was assessed using school items of the CBCL, where participants were scored regarding their performance in academic subjects (Portuguese or literature, history or social studies, English or Spanish, mathematics, biology, sciences, geography, and computer studies). Reading and writing abilities were evaluated using subtests of the Brazilian School Performance Test.³⁴ A composite score of reading and writing ability (“literacy”) was constructed. The composite scores described above were calculated using the original variables in confirmatory factor analyses. Fit measurements for these composite variables are described in Supplement 2 and Table S2, available online. Individual standardized factor scores were estimated, adjusting for the effects of sex and age.

Polygenic Risk Scores.

DNA was extracted from serum samples, and genotyping was performed using the Global Screening Array (Illumina). The study evaluates associations with specific polymorphisms of the CADM2 gene (rs17518584), previously associated with EF in genome-wide arrays.³⁵ That study investigated the additive, dominant, and recessive models of this single nucleotide polymorphism, which was imputed based on a highly linked polymorphism, rs10865610 (r² = 0.96). Also, the study examined associations between EDC and PRS for specific constructs, including ADHD,³⁶ education attainment,³⁷ major depression,³⁸ and schizophrenia.³⁹ The Cross-Disorder PRS,⁴⁰ which includes shared genetic variance for autism spectrum disorder, ADHD, bipolar disorder, major depressive disorder, and schizophrenia was also investigated. PRS were calculated using PRSice version 2 software (King’s College, London, UK).⁴¹ All associated single nucleotide polymorphisms were included in the analysis without setting any threshold. PRS were transformed into z scores to facilitate interpretation, and analyses were adjusted for the first 10 principal components of ancestry. Genetic analyses were available for 1,821 participants at baseline and 1,416 participants at follow-up.

Neuroimaging.

Magnetic resonance imaging scans were performed at 2 sites, using 1.5-T scanners (model Signa HDX or model Signa HD; GE Healthcare, Chicago, Illinois) running identical imaging protocols. Structural neuroimage variables included total intracranial volumes, total cortical thickness, and total cortical surface area bilaterally. Images from the structural sequences were processed using FreeSurfer, version 6.0 (Martinos Center, Boston, Massachusetts),⁴² and a visual inspection quality control led to the exclusion of 82 scans. We performed a stepwise analysis. First, we investigated global measurements of area, thickness, and volume. If global measurements were significant, we further explored specific parcellations provided by the Desikan-Killany cortical atlas. Neuroimage analyses were also controlled for the site. Magnetic resonance scans were available for a random subsample of 547 participants at baseline and 359 participants at follow-up.

Covariates

Categorical psychiatric diagnoses were assessed as previously described. The study measured IQ at baseline using vocabulary and block design subtests of the WISC-III.²³ Socioeconomic status was measured using the Brazilian Economic Classification Criteria, which considers the family’s possessions and educational status.⁴³

Statistical Analysis

Temporal Stability.

We tested the EDC’s temporal stability by assessing longitudinal patterns of incidence, remission, and persistence of the classification at baseline and follow-up assessments. We also calculated the odds of the individual having EDC at follow-up based on its status at the baseline.

Validity.

All analyses were performed using generalized additive mixed models to account for nonlinearities between age and the measured outcomes, using the site (Porto Alegre or São Paulo) as random intercepts, and adjusting for sex, socioeconomic status, IQ, any anxiety disorder, any depressive disorder, any ADHD, and any conduct disorder. For neuroimaging analysis, we fitted separate age splines for sex, given well-known distinctions in the brain volumes’ trajectories between boys and girls. Symptomatic and neuroimaging analyses were corrected using the false discovery rate due to a high number of statistical tests.⁴⁴ Longitudinal analyses were also repeated using imputation methods to account for loss to follow-up. Using the tests of normality and homoscedasticity, we evaluated missingness and did not reject the missing completely at random pattern. Therefore, we completed the data in 1 dataset using an imputation method of chain equations.⁴⁵ As a supplemental analysis, an alternative method to define group assignment was tested to evaluate its associations with impairment outcomes. This alternative method consisted of a confirmatory factor analysis technique, performed at the baseline with 6 tasks (assessing working memory, inhibitory control, and planning domains). All analyses were conducted using R version 3.6.1 software (R Project, Vienna, Austria),⁴⁶ using the following applications: poLCA version 1.4.1⁴⁷ for performing LCAs; pROC version 1.15.3⁴⁸ for assessing the ROC curves; gamm4 version 0.2.5⁴⁹ for performing generalized addictive mixed-models; and MICE version 3.0.8⁵⁰ for imputation.

RESULTS

As a descriptive assessment, we evaluated the correlation matrices between both task-based performance and symptom-based performance. EF and CBCL-based variables segregated into 2 clusters correlated in minor magnitude (Figure S1 and Table S3, available online).

Operationalization

Data were examined using a 10th-percentile threshold for each test, adjusted for age and sex. The LCA found the 2-class distribution (low vs normal/high performance) as the best solution, with ROC analysis suggesting an optimal threshold for identifying EF dysfunction as ≥2 low-performance tests. Using this cutoff, at both baseline and follow-up, AUC was > 0.98, sensitivity was = 1.00, and specificity was >0.95. Full information for the operationalization, sample description, and normative test results are available in Figure S2, Table S4, and Table S5, available online.

Stability Over Time

From the 1,364 individuals with full EDC information at baseline and follow-up, longitudinal trajectories consisted of 1,088 controls (79.8%), 159 remittent cases (11.7%), 67 incident cases (4.9%), and 50 persistent cases (3.7%). From those 117 participants with EDC at follow-up, 50 (43.7%) were already classified as EDC at baseline. From those 1,155 not classified as EDC at baseline, incidence occurred in 67 (5.8%). Being classified in the EDC at baseline increased the odds of follow-up EDC assignment 5 times (odds ratio [OR] = 5.11; 95% CI 3.41 to 7.64; area under the curve [AUC] = 0.750). Baseline subthreshold executive dysfunction (defined as having 1 impaired test result) was more common in the incident group than in the control group (55% vs 31%, respectively), which probably reflects EF’s dimensional nature.

Validity

Symptom-Level Regressions and Categorical Diagnoses.

At baseline, 7 CBCL items were statistically associated with EDC after correction for multiple comparisons: “poor schoolwork,” “easily embarrassed,” “gets teased a lot,” “too shy or timid,” “physical problems without known medical cause,” “daydreams or gets lost in his/her thoughts,” and “complains of loneliness.” At follow-up, 3 items were statistically significant: “acts too young for his/her age,” “poor schoolwork,” and “has strange ideas.” Associations spanned almost all domains of psychopathology (Figure 1). EDC was not associated with categorical diagnoses of mental disorders at baseline or follow-up, when analyses were conducted correcting for comorbidity, including all but the tested disorder (Table S6, available online). Among those participants with EDC, single-domain impairment (exclusive working memory deficit, exclusive inhibitory control deficit, and mixed deficits) was also generally not associated with psychiatric diagnoses, except for the exclusive inhibitory control deficit and conduct disorders at the follow-up (Supplement 3 and Table S7, available online).

FIGURE 1.

Bottom-up Symptomatology of the Executive Dysfunction Classification Graphs

Note: These show the-log(p) for multiple logistical regressions. The reference lines mark p = .05 and the FDR threshold. Variables below the .05 threshold are signaled in bold and those over FDR threshold are marked in red. FDR = false discovery rate. Please note color figures are available online.

Functional Impairment.

Participants with EDC presented worse scores on nonattendance, school achievement, and literacy, as well as a higher frequency of adverse school events on both baseline and follow-up. Worse school attendance and achievement on follow-up were predicted by baseline EDC status, even adjusting for the presence of those impairments at baseline. Impairment of family life and friendships was seen with less consistency (Table 2). No longitudinal results were modified when analyzed using imputation techniques (Table S8, available online).

TABLE 2.

Assessment of Functional Impairment–School/Education, Family Life and Friendships

	Cross-sectional baseline associations			Cross-sectional follow-up associations			Longitudinal predictions
School impairment	OR	CI 95%	p	OR	CI 95%	p	OR	CI 95%	p
SDQ school impairment	1.751	1.303; 2.354	<.001*	1.248	0.786; 1.983	.347	1.212	0.849; 1.731	.290
SDQ school impairment (Teacher-rated)a	1.618	1.012; 2.588	.044*	–	–	–	–	–	–
Any adverse school event	2.530	1.838; 3.483	<.001*	1.624	1.093; 2.414	.016*	1.373	0.997; 1.891	.052
	SMD	CI 95%	p	SMD	CI 95%	p	SMD	CI 95%	p
Non-Attendenceb	0.178	0.129; 0.227	<.001*	0.097	0.011; 0.183	.026*	0.071	0.003; 0.140	.041*
Achievementb	−0.245	−0.353; −0.137	<.001*	−0.196	−0.358; −0.034	.017*	−0.244	−0.367; −0.121	<.001*
Literacyb	−0.410	−0.499; −0.321	<.001*	−0.395	−0.525; −0.265	<.001*	−0.064	−0.158; 0.023	.147
Family life impairment	OR	CI 95%	p	OR	CI 95%	p	OR	CI 95%	p
SDQ family life impairment	1.571	1.083; 2.280	.017*	0.827	0.439; 1.559	.557	0.939	0.599; 1.473	.784
SDQ family life impairment (Teacher-rated)a	1.534	0.973; 2.417	.065	–	–	–	–	–	–
Friendship impairment	OR	CI 95%	p	OR	CI 95%	p	OR	CI 95%	p
SDQ friendship impairment	1.281	0.861; 1.904	.220	1.301	0.662; 2.557	.445	1.176	0.701; 1.971	.539
SDQ friendship impairment (Teacher-rated)a	1.715	1.179; 2.493	.004*	–	–	–	–	–	–

Note: OR = Odds Ratio; SDQ = Strengths and Difficulties Questionnaire; SMD = Standardized Mean Difference.

^aData only available for a subsample of the baseline assessment.

^bMean standardized factor score.

^*p <.05.

Genetic Analysis.

Under a dominant model, the TT genotype at rs17518584 was nominally associated with EDC at follow-up (OR = 0.632; 95% CI 0.397–0.991; p = .046). We found no associations for the additive or recessive models. Participants with EDC had higher ADHD PRS and lower educational attainment PRS at baseline and higher schizophrenia PRS at follow-up (Table 3).

TABLE 3.

Assessment of Biological Validators—Neuroimage and Polygenic Risk Scores

	Cross-sectional baseline associations			Cross-sectional follow-up associations
	SMD	t	p	SMD	t	p
Neuroimage
Intracranial volume	−0.206	−1.898	.058	−0.309	−2.008	.045*
Left cortical thickness	0.169	1.445	.149	0.183	1.203	.230
Right cortical thickness	0.201	1.711	.087	0.116	0.75	.454
Left surface area	−0.280	−2.518	.012*	−0.436	−2.997	.003*
Right surface area	−0.259	−2.347	.019*	−0.447	−3.11	.002*
Polygenic risk scores
ADHD	0.153	2.534	.011*	0.014	0.159	.874
MDD	0.046	0.921	.357	−0.046	−0.640	.522
SCZ	0.008	0.303	.761	0.102	2.736	.006*
Education attainment	−0.145	−2.328	.020*	−0.145	−1.592	.111
Cross-disorder	−0.094	−1.761	.078	0.001	0.018	.986

Note: ADHD = attention-deficit/hyperactivity disorder; MDD = major depression disorder; SCZ = schizophrenia; SMD = standardized mean difference.

^*p <.05.

Neuroimaging.

Children in the EDC presented with lower cortical surface areas bilaterally, with no significant associations observed for cortical thickness or volume of subcortical structures (Table 2). Given significant associations with global cortical areas, we further explored the 68-area parcellations correcting for multiple comparisons (Figure 2). At baseline, lower cuneus area was observed in right hemisphere (SMD = 0.371; p_adj = .047), and lower superior parietal areas were seen bilaterally (right SMD = −0.375; p_adj = .047; left SMD = −0.439; p_adj = .033). At follow-up, superior temporal (SMD = −0.462; p_adj = .036), banks superior temporal (SMD = −0.504; p_adj = .032), cuneus (SMD = −0.477; p_adj = .032), and pars triangularis (SMD = −0.486; p_adj = 0.32) were observed in the right hemisphere; again, lower superior parietal areas were seen bilaterally (right SMD = −0.529; p_adj = .036; left SMD = −0.492; p_adj = .032). Information on each cortical surface area and its association with the EDC is described in Table S9, available online.

FIGURE 2.

Cortical Surface Areas Associated With Executive Dysfunction Classification at Baseline and Follow-up Assessments

Note: FDR = false discovery rate; SMD = standardized mean difference; W0 = baseline assessment; W1 = follow-up assessment. Please note color figures are available

Supplementary Analysis

The EDC operationalized by the 4 EF tests and LCA-based solution was more stable and more consistently associated with external correlates than each task taken individually (Table S10, available online). Impairment associated with EDC maintained significance after adjustment for other confounding variables, such as maltreatment, exposure to violence, and family environment (Supplement 4 and Table S11, available online). EDC was not associated with the shared variance of psychiatric symptomatology, the p factor, but was associated with its externalizing domain at the baseline (Supplement 5 and Table S12, available online). Alternative group assignment methods (6-task confirmatory factor analysis-based division) also demonstrated baseline cross-sectional impairment at multiple life domains and predicted future educational impairment with overlapping confidence intervals with the original strategy. Information on the operationalization and alternative group membership analyses results is available in Supplement 6, Figure S3, Table S13, and Table S14, available online.

DISCUSSION

This study evaluated the stability and validity of task-based classification approaches, using deficits in EF (EDC) as an example of a clinically useful group. The results suggest that a threshold of ≥2 low-performance scores among 4 objective neuropsychological tests identifies a meaningful group of lowperforming youth. EDC caseness predicted itself over time and predicted symptoms related to several psychopathology domains and adverse impacts on learning both concurrently and over time. Educational impairments included lower academic performance and a substantially higher frequency of adverse school events. Interestingly, associations with functional impairment appear to be more consistent than those with psychopathology, reinforcing the hypothesis that the association of EF deficits and psychiatric symptoms may not always be present.⁵¹ Moreover, caseness also predicted profiles of genetic and neuroimaging external correlates, with all such findings emerging independent of existing psychiatric disorder classification and IQ. Thus, further refinement of task-based assessments for use in children and adolescents may generate clinically useful information.

The potential utility of data-driven classification^52,53 has been shown through several studies,^54–59 including several investigations examining EF. The current study builds upon previous data-driven medical literature that showed EF deficits to be transdiagnostic.^60,61 For example, Ing et al.⁵⁸ found specific functional neuroimaging correlates of EF deficits. Such findings extend other work linking EF deficits to brain function modifications that manifest across current psychiatric classifications.^62,63 However, whereas past research provides a framework for continued studies of EF deficits, previously used cluster-based methods do not easily extend previous pediatric psychopathology research. This occurs given the lack of comparability and standardization across studies, which is particularly important among youth, given age-related changes in EF. Ultimately, classification serves functions beyond informing studies of pathophysiology; it also predictively informs patients and clinicians on the likely occurrence of functional impairment and the patient prognosis. The present study addresses each aspect of classification. The data suggest that a) EF deficits can be recognized in community samples using simple tests; b) operationalization of such classification is feasible, in a way that c) possesses utility and validity independent of current symptom-based classification.

By showing that such classifications bring objectivity to psychiatric assessment without losing the capacity to detect children with unfavorable outcomes, we open the possibility to further advance this approach to other phenotypes and investigate finely tune interventions to more specific domains. For example, instead of focusing on current diagnostic groups, EF interventions^64–68 could focus more specifically on children likely to have EF impairments, which already carry significant risks of an atypical development such as the ones captured by the EDC. Further studies, designed to capture and intervene in a specific cluster of participants with executive dysfunction are required. However, it is unclear whether such a strategy would result in better outcomes for subjects with EF deficits, given the current evidence.⁶⁹ Nevertheless, we emphasize that the objective of developing classification methods is to better identify children at risk so that interventions can be tailored to avoid further negative outcomes, and should not be used for discriminatory purposes.

Limitations of the study should be noted. First, the EDC construction was based on working memory and inhibitory control but not in cognitive flexibility, planning, or other components of EF. Even though the EF domains often converge, it could be argued that the absence of cognitive flexibility or planning measurements yields an incomplete assessment of EF. Working memory and inhibitory control are basic EF domains that support high-order EF, which are known to be reliably testable¹⁸ and have the potential for intervention.^{64,68,70–72} Furthermore, those were the 2 domains that were assessed on the present cohort’s 2 time points, therefore allowing the study to investigate longitudinal patterns of the EDC. We tested alternative models as a supplemental analysis which included the planning domain, finding similar results. However, the lack of cognitive flexibility measurements in both main and supplemental analyses should be considered a significant limitation. Second, the analysis was limited to 1 cohort, and as such, replication in independent samples is essential. Third, by categorizing our EF outcomes, we may have lost some information. We understand that dichotomization represents an artificial, rather than a natural boundary for EF. Nonetheless, our intention with this study was to use data-driven cluster analyses to find a clinical group with low performance in several EF tasks, instead of characterizing the dimensionality of EF in the population. Children with this profile of multiple deficits are more likely to manifest impairment and may require clinical attention. Furthermore, this strategy enables the EDC to be used in clinical settings, where dichotomization is often required, given clinical assessments are often categorical (eg, to treat or not to treat). The study has important strengths. First, we used simple and well-validated tests to build our operationalization. This testing is possible in real-world settings, such as primary care and clinical offices. Second, we validated our phenotype on matters of symptomatology, impairment, and biological variables. Third, by controlling for psychiatric comorbidities and IQ, we were able to validate EDC independently of our current classificatory system. Finally, we demonstrated that task-based measurements could predict clinically relevant outcomes over a 3-year period, over and above symptomatology rating measurements. These results were also seen when performing alternative approaches to define executive dysfunction, demonstrating the findings’ robustness.

This study demonstrates the potential value of operationalized criteria using a task-based classification. It is, to our knowledge, the first study to propose and test a method to operationalize task-based and data-driven classifications into clinical practice. In this way, our proposed and evaluated strategy should not be considered as a definitive approach. Future studies should aim not only to replicate the findings but also to propose, test, and compare different approaches to operationalization. Still, this strategy, based on objective evaluations, identifies neurobiological underpinnings and associated impairments that may be currently diluted throughout several psychiatric disorders. Thus, it may a) facilitate communication in research and clinical practice; b) augment existing symptom-based assessment; and c) inform research on therapeutics. The operationalization of task-based classifications may play a role in translating research efforts into clinical practice.