Associations between task performance and self-report measures of cognitive control: Shared vs. distinct abilities

Abstract

Despite overlapping terminology and assumptions that they tap the same constructs, executive function (EF) task performance and EF/effortful control (EC) questionnaires have been reported to be only weakly correlated. It is unclear if this reflects true lack of association or methodological limitations. The current study addresses past methodological limitations using a pre-registered latent variable approach in a community youth sample (n = 291, age 13-22). EF task performance was assessed with a well-validated battery inhibition, shifting, and updating tasks. Self-reported EF/EC was assessed using the predominant temperament measure (EATQ-R), and a self-report assessment more closely aligned with EF constructs (BRIEF-SR). Bifactor models fit the BRIEF-SR, EATQ-R and EF task measures well. Self-reported EF/EC and EF task factors were only weakly correlated on average in youth, although there were some stronger associations in older youth. These results suggest that task-based measures of EF and self-report measures of EF/EC may be best viewed as complementary, but largely distinct, windows on cognitive control.

Cognitive control is essential for successfully navigating most daily activities, enabling self-directed behavior toward goals and flexible adaptation to changing demands (e.g., Friedman & Miyake, 2017). Individual differences in cognitive control are associated with many important outcomes, including academic and social functioning (Best et al., 2009), and physical (Reimann et al., 2018) and mental (Snyder, Miyake, et al., 2015) health. How best to conceptualize and assess cognitive control is thus an important question and a topic of much debate. A key aspect of this debate concerns the extent to which two distinct ways of assessing cognitive control, executive function (EF) task performance and questionnaire measures, converge (Toplak et al., 2012; Zhou et al., 2012). Self- or informant-reported cognitive control are referred to with different terms depending on the subfield and questionnaire measure, including effortful control (EC), attentional control, self-control, and executive function. For consistency and clarity, we use the term self- (or informant)- reported EF/EC as a general term for questionnaire measures throughout, with distinctions made between more specific constructs in particular studies.

EF tasks and self-reported EF/EC measures use overlapping terminology and are often assumed in the literature to be measuring the same constructs. But correlations between EF tasks and EF/EC questionnaires are generally weak, suggesting they may be tapping relatively independent constructs (Duckworth & Kern, 2011). However, it remains unclear if these weak correlations reflect a true lack of association between task and questionnaire measures or methodological limitations in each type of measurement. This question is of both theoretical and practical importance for optimizing assessment of cognitive control. The current study addresses methodological limitations of the previous literature, using a latent variable approach to investigate how EF task performance and self-report measures of EF/EC are related in youth. We assess these links in adolescence and emerging adulthood, a key period for continuing cognitive control development (e.g., Friedman et al., 2016). Cognitive control during this period is related to academic and social functioning as well as physical and mental health, all of which undergo considerable change during adolescence and young adulthood (Moffitt et al., 2011).

Task Performance Measures of Executive Function

Task measures of EF come from the cognitive psychology/neuroscience and clinical neuropsychology traditions, and aim to capture specific EF processes. EF is best characterized as consisting of separable but related cognitive processes, with both unique and shared individual differences, genetic influences, and neural substrates. This view is shared across multiple models of EF (e.g., Baddeley & Repovs, 2006; Diamond, 2013; Duncan & Owen, 2000; Engle & Kane, 2003; Shallice, 2002). Here we focus on one such model, the unity/diversity model (Friedman & Miyake, 2017; Friedman et al., 2008; Miyake et al., 2000), because it captures several key components of EF and has the potential to shed light on commonalities and differences in task and questionnaire measures by differentiating common and specific components of EF.

This model focuses on three aspects of EF: (1) updating working memory (WM), (2) shifting, and (3) response inhibition. Updating involves adding task-relevant information to WM and replacing no longer relevant information (e.g., as assessed with n-back tasks). Shifting involves switching between task sets or response rules (e.g., as assessed by task switching tasks). Inhibition involves suppressing a prepotent (automatic) response in order to make a task-relevant response (e.g., as assessed with the stop signal task). EF tasks in these three domains can be modeled with latent factors, which are moderately strongly, but not perfectly, correlated, suggesting that there is a common EF ability involved in all three (Friedman et al., 2008; Friedman & Miyake, 2017; Miyake et al., 2000). Common EF is posited to reflect the ability to monitor for goal-relevant cues, actively maintain task goals and goal-related information in WM, and use this information to bias lower-level processing (Friedman & Miyake, 2017).

Using latent variable bifactor modeling, each EF ability (e.g., updating) can be decomposed into what is common across all three EFs, or unity (common EF), and what is unique to that particular ability, or diversity. In two large, independent samples, after accounting for common EF in the model, there was remaining covariance specific to updating and shifting tasks, which is captured with specific factors (Friedman et al., 2008; Ito et al., 2015). Covariance among inhibition tasks, however, was fully accounted for by the common EF factor; that is, after accounting for covariance among all tasks with the common EF factor, there is no inhibition-specific factor (Friedman et al., 2008; Ito et al., 2015). EF bifactor models using the same tasks tested here have been tested in large twin studies, which found the factors to be highly heritable (Friedman et al., 2016; Gustavson et al., 2018), rank-order stable across 6 years from adolescence to early adulthood (Friedman et al., 2016), and related to psychopathology and functioning (Friedman et al., 2020; 2018; Gustavson et al., 2015; 2017). Of course, even with the same tasks, factors and their nomological nets may differ across studies. However, data from the same study reported here showed that the common EF factor was associated with stress and psychopathology symptoms in older adolescents and emerging adults (Snyder et al., 2019).

Task-based measures of EF, when optimally assessed (i.e., by multiple specific and sensitive tasks), capture objective cognitive control capacity and efficiency in each of these components (e.g., Snyder, Miyake, et al., 2015). However, many studies lack optimal measurement, relying instead on individual neuropsychological tasks that lack reliability, specificity, and sensitivity in assessing individual differences in EF (e.g., Snyder, Miyake, et al., 2015). In addition, because they are given in a highly controlled, structured environment (e.g., low distraction), EF task performance likely reflects individuals’ highest possible level of performance, rather than typical use of EF in the less controlled environments typical of daily life (e.g., Toplak et al., 2012). Thus, some have argued that questionnaire measuresmay better capture impairments in daily life linked to outcomes such as psychopathology (e.g., Barkley & Fischer, 2011).

EF/EC Questionnaires

Questionnaire measures of cognitive control are frequently employed in the temperament and clinical literatures. These questionnaires seek to capture constructs that are defined similarly to task-based EF constructs (e.g., EC has been defined as “the efficiency of executive attention” (Rothbart & Bates, 2006)), but are not necessarily aligned with the core components of EF identified in the task-based cognitive and neuroscience literatures.

The most widely used model of EC includes three key components, assessed by self- and informant-report in adolescence (Early Adolescent Temperament Questionnaire-Revised [EATQ-R]): (i) attentional control (capacity to focus and shift attention), (ii) inhibitory control (capacity to suppress inappropriate responses and plan future action), and (iii) activation control (ability to perform an action when there is a strong tendency to avoid it) (Ellis & Rothbart, 2001; Muris & Meesters, 2009; Putnam et al., 2001). A bifactor model of these EATQ-R EC subscales in a large combined adolescent sample found common EC and an activation-control specific factors (the common EC factor fully accounted for the attentional control and inhibitory control items; Snyder et al., 2015). This structure replicated in a hold-out sample (Snyder et al., 2015) and an independent study (Tiego et al., 2020), and the factors were related to multiple life outcomes (common EC related negative emotionality, depression and anxiety symptoms, antisocial behavior, and grades, activation control-specific related to negative emotionality, fear, and harm avoidance; Snyder et al., 2015).

Other questionnaires are designed to more closely align with task-based models of EF. For example, the Behavior Rating Inventory of Executive Function (BRIEF; PAR Inc.) is a widely used measure that includes subscales putatively assessing self or informant reported inhibition, shifting, and working memory. The BRIEF has been found to be best represented by models including both common and specific factors (Pérez-Salas et al., 2016; Roth et al., 2013) and has been validated in relation brain injury, developmental disabilities and ADHD (Gioia et al., 2002). Data from the same study reported here found that total (GEC) BRIEF scores were correlated with depression, anxiety, ADHD and conduct disorder symptoms (Mullin et al., 2020).

Although the EATQ-R and BRIEF use different terminology (EC vs. EF), they have overlapping content, asking about many of the same behaviors (e.g., BRIEF-SR “I have trouble changing from one activity to another” and EATQ-R “I find it hard to shift gears when I go from one class to another at school”, BRIEF-SR “I forget what I’m doing in the middle of things” and EATQ-R “I tend to get in the middle of one thing, then go off and do something else”, etc.). Of note however, correlations between these questionnaires have not been reported in the literature, with the exception of one study finding a very strong correlation between parent-report BRIEF and EATQ-R EC factors in children, such that they appeared to measure the same construct (Tiego et al., 2019). Thus, there is some preliminary evidence that questionnaire measures of cognitive control, whether conceptualized as EF or EC, measure the same construct. However, this has not been examined in older youth or using self-report, a gap the current study addresses.

Such self-reported EF/EC measures ask about behavior in complex real-world situations (e.g., completing tasks on time, staying organized). Thus, they may have advantages in terms of ecological validity, leading some to argue in favor of using questionnaires rather than tasks (e.g., Barkley & Fischer, 2011). However, questionnaire measures pose interpretational problems in that these real-world behaviors involve multiple EF and non-EF processes, and can also be heavily influenced by contextual factors (e.g., having the motivation and opportunity to complete homework on time). In addition, questionnaire responses can also be influenced by reporter biases (e.g., negativity or positivity bias). Thus, despite putatively assessing the same cognitive control constructs, task and questionnaire measures are likely to diverge for several reasons.

Associations Between Task and Questionnaire Measures of Cognitive Control

Indeed, there is evidence that correlations between EF tasks and EF/EC questionnaires are generally low. A meta-analysis found that EF task performance was only weakly associated with self-reported (r = .10) and informant-reported (r = .14) self-control measures (Duckworth & Kern, 2011). However, this meta-analysis included a wide variety of self-control measures, some of which (e.g., sensation seeking) do not assess behaviors directly related to EF constructs. In addition, EF tasks in the included studies correlated with one another to about the same extent (r = .15), suggesting that the lack of strong correlation between EF tasks and self-control questionnaires measures may result from methodological problems with the EF measures used.

Specifically, most studies used only a single EF task, or, at best, one task designed to tap each of several EF abilities. This approach severely hampers the ability to detect EF task-questionnaire relations because these individual EF tasks : (a) are contaminated by non-EF variance that can mask underlying commonalities (task impurity problem), and (b) generally have relatively low reliability, and thus necessarily have low correlations with other measures (for review see Snyder, Miyake, et al., 2015). Specifically, the traditional neuropsychological tests (e.g., Trail Making Test, WCST) used in many studies are relatively non-specific, tapping multiple EF and non-EF processes. In addition, they were originally designed to assess EF in patients with frontal lobe damage or dementia, and thus may lack sensitivity to normal-range individual differences in community samples (e.g., Snyder, Miyake, et al., 2015).

These measurement problems can be alleviated by using multiple sensitive and specific measures of each EF component and employing latent variable models. In this approach, multiple exemplar tasks are chosen that capture the target ability (e.g., three tasks that require shifting) but differ on non-EF requirements. If exemplar tasks are chosen such that they share little systematic non-EF variance, one can statistically extract what is common across those tasks and use the resulting “purer” latent variable as the measure of EF. Task-specific and error variance is left behind in the residual term of each indicator, resulting in a more reliable measure of the construct of interest. Likewise, latent self-reported EF/EC factor models can be used to capture covariance across EF/EC questionnaire items, removing item-specific and error variance. However, it should be noted that even latent variables are limited by their indicator variables, and no measure provides a perfectly reliable and valid estimate of these broad and complex constructs. Nonetheless, this latent variable approach enables a more conclusive test of associations between EF task performance and self-report EF/EC questionnaires than can be achieved with manifest variables (i.e., individual tasks, questionnaire sum scores).

A few recent studies have used such latent variable approaches. In two adult samples, an EC factor correlated moderately with a WM and verbal fluency factor (r =.31/.36) but not with a factor defined by three scores from the Stroop task (r = .25/ r =.11) (Bridgett et al., 2013). However, because these inhibition factor indicators were all from the same task, this factor likely included substantial systematic non-EF (task-specific) variance. Another study in adults found no association between a self-control factor and a single EF factor spanning measures of shifting, WM, and inhibition (r = .01) (Nęcka et al., 2018). Finally, in a large sample of children and adolescents (ages 8-15), a hierarchical common EF factor predicting inhibition, shifting, updating and WM lower level factors was weakly but significantly correlated with questionnaire-based factors of impulse control (r = .25) and conscientiousness (r = .27); associations with the lower level EF factors were not reported (Malanchini et al., 2018). Thus, there is some evidence that even when latent variable models of EF are used, associations with EF/EC questionnaires vary from non-existent to moderate, suggesting they are tapping largely independent constructs.

However, these previous latent variable studies have a number of methodological limitations that preclude strongly drawing this conclusion. First, they have not looked at specific EF factors (Malanchini et al., 2018; Nęcka et al., 2018), or have defined factors based on a limited set of neuropsychological tests which don’t adequately address the task-impurity problem (Bridgett et al., 2013). In contrast, as described earlier, the best current evidence indicates that individual differences in EFs reflect both common and specific components, which may be differentially related to different aspects of self-reported EF/EC. Using latent variable bifactor modeling, each EF ability (e.g., updating) can be decomposed into what is common across all three EFs, or unity (common EF), and what is unique to that particular ability, or diversity. However, this approach has not been applied to better understanding associations with questionnaire measures.

Second, studies have examined associations between EF tasks and a wide range of questionnaires (e.g., conscientiousness, impulse control) that, while putatively broadly related to cognitive control, are not generally well aligned with the constructs assessed by EF tasks (e.g., inhibition, shifting, working memory), or with the most widely used model of EC temperament (e.g., Putnam et al., 2001). Moreover, most studies have used a single composite questionnaire measures or factors, despite evidence that different components of EC are related but separable (Snyder, Gulley, et al., 2015; Verstraeten et al., 2010). Specifically, EC temperament, as assessed with the EATQ-R, is best modeled using a bifactor model with both a common EC and an activation-control specific factor, which are differentially related to other temperament dimensions and psychopathology (Snyder, Gulley, et al., 2015). Likewise, models including both common and specific factors have been found to best represent the structure of self-reported EF as assessed by the BRIEF (Pérez-Salas et al., 2016; Roth et al., 2013). Thus, studies using empirically supported models that capture the common and specific dimensions of both task-based EF and self-reported EF/EC measurements are needed to determine how these different components of each type of cognitive control measure may be related.

Finally, there has been little consideration of potential moderators of links between EF tasks and EF/EC questionnaires. Most studies (both those with individual measures and latent variables) have tested these links either earlier in childhood (e.g., Malanchini et al., 2018) or adulthood (e.g., Bridgett et al., 2013; Nęcka et al., 2018), missing the adolescent and emerging adult period. This is an important gap because adolescence is a critical period for the development of EF, with performance continuing to increase until early adulthood (Friedman et al., 2016; e.g., Huizinga et al., 2006; Shulman et al., 2015). EF/EC in daily life, as assessed by questionnaires, may also be increasingly important as adolescents take on more decision-making responsibilities, and older youth may provide more accurate reports of their own EF/EC, potentially leading to stronger correlations between EF task performance and self-reported EF/EC than in younger youth. In addition, although there is little evidence for gender differences in EF task performance (Grissom & Reyes, 2018), a meta-analysis found substantially higher primarily parent-rated EF/EC in girls than boys in childhood and early adolescence (Else-Quest et al., 2006). This disconnect could indicate that EF abilities as assessed with tasks are translated differently into real world behaviors in girls and boys, that boys have stronger bottom-up processes competing with EF processes to drive behavior, or that there are reporter biases in EF/EC questionnaire ratings related to youth gender. In any case, this could lead to gender differences in the association between EF tasks and EF/EC questionnaires. However, it is unclear if such gender differences extend to self-reported EF/EC in adolescence and emerging adulthood.

In sum, despite the frequent treatment in the literature of EF tasks and EF/EC questionnaires as alternative measures of the same constructs, most evidence suggests they are only weakly correlated. However, most studies have been limited by task impurity problems and low reliability in assessing EF task performance, and broad self- or informant-reported EF/EC measures that are often not well aligned with task-based EF constructs. A few recent studies attempted to address these problems using a latent variable approach, but did not use current, empirically supported models that differentiate common and specific components of cognitive control. In addition, variation in links between EF task performance and EF/EC questionnaires across age (especially during the key adolescent and emerging adult period) and gender have not been systematically investigated. We address these limitations in the current study.

Current Study

To address these limitations, this study uses a latent variable approach to investigate how EF task performance and self-reported EF/EC questionnaires are related in a community sample of youth (age 13-22 years), with the goal of ascertaining to what degree task and self-report measures of cognitive control converge when using methods that maximize the potential to detect such associations. We chose EF tasks based on the well-validated unity/diversity model of EF (Friedman et al., 2016), enabling differentiation of the associations between self-reported EF/EC and common EF, shifting-specific, and updating-specific abilities. We examine associations between EF tasks and self-reported EF/EC in two ways: (1) Using the self-reported EC measure aligned with the most common temperament model (EATQ-R), and (2) using a self-reported EF measure putatively more closely aligned with task-based EF constructs (inhibit, shift and WM subscales of the BRIEF-SR). This allowed us to test the robustness of findings across two different self-report measures which are frequently used in different subfields but infrequently compared to one another. We hypothesized that there are both shared processes (e.g. maintaining goals) and unique processes (e.g. motivational processes involved in self-reported daily behaviors on the EF/EC questionnaires), leading to correlated but separable task-based EF and self-reported EF/EC factors (i.e., moderate correlations). Overall, the relations among individual aspects of self-reported EF/EC and task-based EF have received little attention and have not been examined at a latent variable level. Thus, the we did not have strong hypotheses about the degree to which different task-based EF and self-reported EF/EC factors are related. This project was pre-registered prior to hypothesis testing.¹ In addition, we include exploratory analyses testing for age and gender differences in associations between factors.

Method

Participants

Participants were 291 13-22-year-old youth (Mean age = 16.20, SD = 2.35; 56% female) recruited from the greater metro Denver area through direct mail and from an ongoing longitudinal study (GEM Study, (Hankin et al., 2015)). ZIP codes were selected based on US Census data to maximize racial and economic diversity. Interested families contacted the lab and were screened for eligibility criteria: 13-22 years old, fluent in English, and, for minors, have a parent/guardian who was fluent in English to enable consent. Sample size was determined by a priori power analyses in the grant proposal (see Supplemental Materials). Participants identified their race as 70% white, 11% more than one, 9% African American, 4% American Indian/Native Alaskan, 2% Asian, and 4% other or declined to answer; 19% identified as Hispanic/Latino.

Procedure

Participants gave written informed consent (18-22) or assent with parent/guardian consent (13-17). Participants completed one 5-hour or two 2.5-hour laboratory visits, with breaks to reduce fatigue. They completed (1) three EF tasks each assessing updating, shifting and inhibition, (2) the BRIEF-SR at the end of their first lab visit and (3) the EATQ-R either online before their visit or at the end of their first lab visit if they had not finished beforehand². All study procedures were approved by the University of Denver Institutional Review Board.

Measures

EF Tasks.

All tasks were adapted from Friedman et al. (2016), except stop signal, which was from Chatham et al. (Chatham et al., 2012). See Supplemental Materials for additional task details. For all reaction time (RTs) measures, within-subject outlier RTs were removed with Wilcox–Keselman trims (Wilcox & Keselman, 2003) before computing final measures.

Updating.

For all updating tasks, the performance measure is the proportion of correct responses across all trials.

Keep Track.

On each trial, participants were shown the names of 2-5 target categories (e.g., animals, colors), which remained on the bottom of the screen throughout the trial, in which 15-25 words were serially presented. Participants were instructed to recall the last exemplar seen in the target categories at the end of each trial. Since multiple exemplars from each category were presented in each trial, this requires updating which exemplars to remember.

Letter Memory.

Letters were presented serially in the center of the screen, with 9-13 letters in each trial, and participants said out loud the last three letters, adding the most recent letter and dropping the fourth letter back. Each letter triad was scored as correct if participants reported all three letters correctly in order.

Spatial 2-Back.

Twelve squares scattered across the screen became dark one at a time, and participants pressed a button to indicate if the dark square is the same as the one two trials earlier; 30% of “no” trials were “lures”: flashes that matched the square from three flashes back.

Shifting.

For all tasks, participants first practiced a block of each sub-task separately, followed by mixed-task blocks. For all tasks, the cue-to-stimulus and response-to-stimulus intervals were 350 ms, and 50% of trials required a task switch. The performance measure for all shifting tasks is the switch cost: the difference in mean RT between correct task switch trials and task repeat trials in the mixed blocks. Responses were recorded using a ms accurate button box, using the same two buttons for both subtasks within each task. Participants were instructed to respond as quickly as possible without making mistakes, which were indicated by an error beep.

Number-Letter.

A number-letter pair (e.g. 7G) was presented on each trial in the top or bottom squares of a four-square grid. Before the pair appeared, the border of one square turned dark, cueing the task. When it was at the top, participants indicated whether the number is odd or even. When it was at the bottom, they indicated if the letter is a vowel or consonant.

Color-Shape.

On each trial a cue (C for color or S for shape) was presented above a green or red circle or triangle, and participants indicated the color or shape.

Category Switch.

On each trial a cue symbol above a word indicated whether the word (from a list of 16 pre-familiarized words), should be categorized as living vs. non-living or as smaller vs. larger than a soccer ball.

Inhibition.

Antisaccade.

On each trial, a cue flashed on one side of the screen (200-250 ms), followed by a target (a box containing a number) on the other side of the screen that was masked after 150 ms. Thus, to identify the number, participants must inhibit the automatic tendency to saccade to the cue and instead immediately look in the opposite direction. The performance measure is accuracy.

Stop signal.

Participants pressed a button to indicate if an arrow was pointing left or right as quickly as possible. A square signaling participants to not respond was presented on 25% of trials after a stop signal delay (100-300 ms). The performance measure is the stop-signal RT (SSRT, the average time needed to stop a response), calculated using the integration method (Logan & Cowan, 1984).

Stroop.

Participants named the color of each stimulus for blocks of neutral trials (asterisks in color ink) and incongruent trials (color words in a different color ink), with RT measured by ms-accurate voice-onset microphone. The performance measure is interference (mean incongruent RT–mean neutral RT).

Early Adolescent Temperament Questionnaire Revised (EATQ-R).

The EATQ-R (Ellis & Rothbart, 2001) is a commonly used temperament questionnaire in children and adolescents. Only the EC scale was analyzed in this study; it includes (a) Attentional Control (capacity to focus and shift attention appropriately, e.g., “It is easy for me to really concentrate on homework problems”, 5 items), (b) Inhibitory Control (capacity to suppress inappropriate responses and plan future action, e.g., “When someone tells me to stop doing something, it is easy for me to stop”, 5 items), and (c) Activation Control (ability to perform an action when there is a strong tendency to avoid it, e.g., “If I have a hard assignment to do I get started right away”, 6 items). Higher scores indicate better effortful control. The EATQ-R EC subscales have been shown to have good test-retest reliability (Muris & Meesters, 2009) and construct validity in relation to measures of personality and psychopathology (e.g., Muris & Meesters, 2009; Snyder, Gulley, et al., 2015). Internal consistency in the current sample was good for the total EC scale (α = .85) and Activation Control subscale (α = .83), but lower for the Attentional Control subscale (α = .70) and Inhibitory Control (α = .56) subscales; these values are similar to those found previously in large samples (e.g., Muris & Meesters, 2009) (see Results for factor reliabilities).

Behavioral Rating Inventory of Executive Function-Self Report Version (BRIEF-SR).

The BRIEF-SR (Psychological Assessment Resources; (Gioia, Isquith, Retzlaff, & Espy, 2002) is a widely used measure of self-reported EF in adolescents. It has 80 items in eight sub-scales: Inhibit, Shift, Emotional Control, Monitor, WM, Plan/Organize, Organization of Materials and Task Completion. We analyzed three subscales aligned with the components of EF assessed in the current study: (i) Inhibit (11 items), (ii) Shift (10 items), and (iii) Working Memory (WM; 12 items). Higher scores indicate worse self-reported EF, but for ease of interpretation we have reversed the sign in analyses so that the BRIEF goes in the same direction as the EF task and EATQ-R measures, where higher scores indicate better EF/EC. The BRIEF-SR has been shown to have good test-retest and inter-rater reliability, and convergent validity with other measures of behavior (Guy et al., 2004). Internal consistency was good for the three subscales used in the current study: Inhibit (α = .85), Shift (α = .83), and WM (α = .87), similar to that in the norming sample (Guy et al., 2004) (see Results for factor reliabilities).

Analysis Approach

Confirmatory factors analysis (CFA), and structural equation modeling (SEM) was conducted with Mplus, using full information maximum likelihood (FIML) to handle missing data (L. K. Muthén & Muthén, 2017). Individual tasks and items were used as indicators for all factors; signs were reversed for RT-based EF task measures so that for all EF tasks, more positive values indicated better performance. For all models, because the χ² is sensitive to sample size, fit criteria were as follows: root mean square error of approximation (RMSEA) < .06 good, <.08 acceptable; comparative fit index (CFI) >.95 good, >.90 acceptable; and standardized root mean square residual (SRMR) < .08 good, acceptable .10 (Hu & Bentler, 1999)³. Significance of parameters was assessed with z-tests. Given the large number of correlations, FDR correction was applied to control for multiple comparisons (Benjamini et al., 2006), and only those correlations with an FDR corrected p < .05 are reported in text (for full results see tables).

We analyze results using two complementary methods: correlated factors and bifactor models. Rather than being competing models, these are best thought of as two ways of modeling the same covariance pattern that have different strengths and limitations: in correlated factor models, the common variance is modeled in the correlations between factors, whereas in a bifactor model it is captured by the common factor. The advantage of the latter is that it separates this common variance from variance that is specific to each sub-component, allowing each to be examined in relation to other variables (e.g., Bornovalova et al., 2020). Bifactor models are thus useful for parsing the different components of multidimensional constructs. On the other hand, correlated factor models may be more appropriate for questions regarding the total association of one construct with another. We emphasize that we test bifactor models alongside the correlated factors models as a complementary means of answering questions regarding the relations between common vs. specific components of these constructs, and not based on fit alone. Together, they provide a more complete picture of the associations between task-based measures of EF and self-reported EF/EC.

Results

Descriptive statistics are reported in Table 1 and bivariate correlations in Supplemental Materials (Table S1). Participants were screened for validity of questionnaire responding using the BRIEF-SR inconsistency index, which checks for consistency between pairs of similarly-worded items; 11 participants were excluded from analysis because this index fell in the inconsistent range (BRIEF-SR manual). Two additional participants each did not complete the BRIEF-SR or EATQ-R. EF data were screened for scores that were not above chance or > 3 SD from the sample mean, as this indicated probable failure to understand or follow task directions; this led to exclusion of no more than six scores on any measure. Missing and excluded data on individual EF tasks is described in detail in Supplemental Materials and was handled with the FIML estimator. For ease of interpretation across measures we have reversed the signs when reporting relations with the BRIEF-SR (where higher scores normally indicate worse functioning) to be consistent with the EF and EATQ-R measures, where higher scores indicate better functioning. Thus, throughout the results, positive signed effects indicate that better scores on one factor are associated with better scores on another factor.

Table 1:

Descriptive Statistics

Measure	Mean	SD	n	Skewness	Kurtosis
BRIEF-SR GEC	124.05	26.85	278	0.80	0.64
BRIEF-SR Inhibit	17.13	4.29	278	1.36	2.58
BRIEF-SR Shift	15.74	3.80	278	0.78	0.76
BRIEF-SR WM	19.15	4.91	278	0.48	−0.24
EATQ-R EC	57.10	9.89	278	−0.14	−0.25
EATQ-R Inhibition	20.03	3.11	278	−0.57	0.57
EATQ-R Attentional Control	21.22	3.99	278	−0.17	−0.30
EATQ-R Activation Control	15.85	4.65	278	−0.02	−0.53
Stroop blocked interference (ms)	154.22	86.16	274	−0.59	0.38
Antisaccade proportion correct	.68	.16	278	−0.10	−0.29
Stop Signal SSRT (ms)	272.35	136.63	267	−0.44	−0.51
Category switch cost (ms)	272.63	159.53	255	0.05	0.38
Color-Shape switch cost (ms)	216.17	159.55	274	−0.34	0.00
Number-Letter switch cost (ms)	388.13	211.55	276	−0.58	0.57
Keep track proportion correct	.70	.11	278	−0.08	0.04
Letter Memory proportion correct	.86	.13	276	−0.63	0.69
Spatial 2-back proportion correct	.79	.10	277	0.15	−0.16

Note. Variable means and SD given in raw metric for ease of interpretation. Skewness and kurtosis are reported for transformed variables; to reduce skewness, accuracy measures (antisaccade, keep track, letter memory, spatial 2-back) were arcsine transformed, switch costs were square-root transformed, and stop signal SSRT was natural log transformed. Note that BRIEF-SR and EATQ-R manifest scale and subscale scores were not used in the CFA analyses (all indicators were individual items), and are reported here only for comparison with other samples. BRIEF-SR = Behavior Rating Inventory of Executive Function Self-Report; GEC = Global Executive Composite (total BRIEF-SR score); WM = working memory; EATQ-R = Early Adolescent Temperament Questionnaire Revised; EC = effortful control (EATQ-R total score).

Correlated Factors Models

We first examined correlations among separate factors for each subcomponent of EF task performance (inhibition, shifting, and updating), each of the three EATQ-R subscales (Inhibitory Control, Attentional Control and Activation Control) and each of the three selected BRIEF subscales (Inhibit, Shift and Working Memory; Table 2).

Table 2:

Correlated Factor Model Correlations

	EF Inhibition tasks	EF Updating tasks	EF Shifting tasks	EATQ-R Inhibition/Attentional Control	EATQ-R Activation Control	BRIEF-SR Inhibit	BRIEF-SR Shift
EF Inhibition tasks	-
EF Updating tasks	.841**	-
EF Shifting tasks	.375**	.109	-
EATQ-R Inhibition/Attentional Control	.226**	.134	.010	-
EATQ-R Activation Control	.010	−.132	−.061	.793*	-
BRIEF-SR Inhibit	.218**	.149**	.014	.575**	.372**	-
BRIEF-SR Shift	.200**	.260**	.169**	.632**	.343**	.597**	-
BRIEF-SR Working Memory	.234**	.167**	.050	.801**	.593**	.712**	.764**

Note. For BRIEF, signs are reversed so that higher scores indicate higher EC for consistency with EATQ-R and EF measures. BRIEF-SR = Behavior Rating Inventory of Executive Function Self-Report; EATQ-R = Early Adolescent Temperament Questionnaire Revised

^*p < .05 uncorrected

^**p < .05 FDR corrected

Measurement Models.

We first tested correlated factor CFAs for each measure type (EF tasks, BREIF-SR and EATQ-R) to ensure acceptable model fit before proceeding to test SEMs.

EF tasks.

The model fit well, χ²(24) = 33.10, p= .102; CFI=.98, RMSEA=.036, SRMR=.041. All indicators loaded significantly on their respective factors (Table S2). Metrics of factor reliability (ω) and replicability (H) (e.g., Rodriguez et al., 2016) were highest for the updating factor (ω = .73, H = .75), followed by the shifting factor (ω = .66, H = .72), and lowest for the inhibition factor (ω = .47, H = .52). It is important to note that these indices, like alpha, are affected by the number of indicators/items, and thus will nearly always be higher for questionnaires, which can include many items, then for task batteries, which due to time and participant fatigue constraints must be more limited in number. These metrics were comparable or better than those in a large twin sample at two waves (Friedman et al. (2016) wave 1/wave: Updating ω = .61/.69, H = .64/.75, Shifting ω = .70/.71, H = .70/.70, Inhibition ω = .40/.49, H = .40/.55), that demonstrated high heritability and stability of these factors, and associations with multiple external validators (see introduction). Thus, although the ω values for the task-based factors are not as high as would be ideal, they likely represent close to a best-case scenario for task-based measures, and reliable enough to detect associations with other factors.

BRIEF-SR.

Residual correlations were included for three pairs of BRIEF-SR items (one in each subscale) that were designed to be very similar for use in the inconsistency index. The model fit well (χ² (521) = 1018.40, p< .001; RMSEA=.057, SRMR=.060). All indicators loaded significantly on their respective factors (Table S2). Metrics of factor reliability and replicability were good for the Inhibit (ω = .86, H = .88), Shift (ω = .82, H = .83) and WM (ω = .87, H = .88) factors.

EATQ-R.

An initial test found that the EATQ-R Inhibitory control and Attentional Control latent factors were correlated r = .95 and so not separable; a combined Inhibitory Control/Attentional Control factor was thus used in all further analyses. This model fit acceptably (χ² (103) = 284.68, p< .001; RMSEA=.080, SRMR=.060. Metrics of factor reliability and replicability were acceptable to good for both the Inhibitory Control/Attentional Control (ω = .77, H = .80), and Activation Control (ω = .84, H = .85) factors.

Overall correlated factors model.

The overall model fit well (χ² (1621) = 2765.88, p< .001; RMSEA=.050, SRMR=.063). Item loadings and additional metrics are reported in Supplemental Materials (Table S2). All correlations are reported in Table 2.

Correlations Among Factors within Measurement Types.

Within EF task performance, the inhibition factor was strongly correlated with the updating factor (r = .841) and moderately correlated with the shifting factor (r = .375), whereas the shifting and updating factors were not significantly correlated (r = .109). Within the EATQ-R, the Inhibitory Control/Attentional Control and Activation Control factors were strongly correlated (r = .793). Within the BRIEF-SR, the Inhibit factor was strongly correlated with the Shift (r = .765) and WM (r = .712) factors, which were moderately strongly correlated with one another (r = .597). Looking across the two self-report measures, EATQ-R Inhibitory Control/Attentional Control factor also correlated strongly with the BRIEF-SR WM factor (r = .801), and moderately strongly with the BRIEF-SR Inhibit (r = .575) and Shift (r = .632) factors. The EATQ-R Activation Control factor correlated moderately with the BRIEF-SR WM (r = .593), Inhibit (r = .372) and Shift (r = .343) factors.

Correlations of EF task factors with BRIEF-SR and EATQ-R Factors.

The EF inhibition task factor had small but significant correlations with the EATQ-R Inhibitory Control/Attentional Control factor (r = .23) and all three BRIEF-SR factors (Inhibit r = .22, Shift r = .20, WM r = .23). The EF updating task factor also had small but significant correlations with all three BRIEF-SR factors (Inhibit r = .15, Shift r = .26, WM r = .17). The EF shifting task factor had a small but significant correlation with the BRIEF-SR Shift factor (r = .17). Exploratory multi-group models by gender and age are provided in Supplemental Materials (Table S3). Results with manifest EF task performance and BRIEF-SR and EATQ-R subscale variables were similar (Supplemental Materials Tables S4–S5).

Bifactor Models

Results from correlated factor models, although informative, have multiple possible interpretations: because the subfactors within each construct (EF tasks, EATQ-R and BRIEF-SR) were correlated (i.e., they have both shared and specific variance), cross-construct correlations could be driven by correlations between the common components of each, multiple correlations between specific components of each, or some combination of these. Thus, we next modeled each construct with a bifactor model, which enable separating common and specific factors within each construct in order to clarify the pattern of associations across constructs.

Measurement models.

We first bifactor CFAs for each measure type (EF tasks, BREIF-SR and EATQ-R) to ensure acceptable model fit before proceeding to test SEMs.

EF tasks.

An EF bifactor (unity/diversity, Fig. 1a) model identical to that previously used with these tasks (Friedman et al., 2016) fit well (χ² (21) = 23.24, p= .332, CFI=.99, RMSEA=.019, SRMR=.039). All tasks loaded significantly on their factors, with the exception of the category switch task, which loaded strongly on the shifting-specific factor but only marginally on the common EF factor (Table S6). Replicating previous research with this unity/diversity model (Friedman et al., 2016; Ito et al., 2015), an alternative model including an inhibition-specific factor found no significant variance for the specific factor, indicating that covariance among the inhibition tasks was fully accounted for by the common EF factor.

Figure 1. Bifactor Measurement Models.

Bifactor measurement models for (A) Executive function (EF) tasks, (B) the Behavior Rating Inventory of Executive Funciton Self-Reprot (BRIEF-SR) and (C) the Early Adolescent Temperament Questionnaire Revised (EATQ-R). Numbers on arrows are standardized factor loadings. Numbers in boxes are item numbers. Antisac = antisaccade; Stop = stop signal; Letter = letter memory; SnBack = Spatal n-back; Number = number/letter switch; Color = color/shape switch; Category = category switch; EC = effortful control; WM = working memory; (R) = reverse coded item.

Total reliability (ω) was .78 for common EF, .69 for shifting and .74 for updating. The percentage of reliable variance in total scores attributed to individual differences in the common factor (ωH) was 57%, and the percentage of reliable variance in subscale scores due to specific factors after controlling for the common factor (ωHS) was 62% for shifting-specific and 29% for updating-specific. The common-EF factor accounted for 50% of the explained common variance across all tasks (ECV), the updating-specific factor accounted for 39% of the explained common variance in the updating tasks, and the shifting-specific factor accounted for 90% of the explained common variance in the shifting tasks (ECVs, (Dueber, 2017)). Factor replicability (H) was .74 for common EF, .84 for shifting-specific and .44 for updating-specific. Thus, the tasks cannot be interpreted as unidimensional, and are best represented as multidimensional as in the bifactor and correlated factors models. However, associations with the updating-specific factor should be interpreted with some caution as it was less replicable and accounted for a smaller percentage of variance after accounting for the common EF factor. These metrics were generally comparable to those in a large twin sample at two waves (Friedman et al., 2016), with the exception that updating task covariances were more strongly accounted for by the common EF vs. updating-specific factor in the current study (Friedman et al. (2016) wave 1/wave: Common EF ω = .78/.78., ωH= .60/.58, H = .68/.72; Shifting-specific ω = .71/.72, ωHS = .39/.62 H = .49/.60; Updating-specific ω = .72/.69., ωHS = .47/.42, H = .57/.52)

BRIEF-SR (Figure 1b).

Residual correlations were included for three pairs of items (one in each subscale) that were designed to be very similar for use in the inconsistency index. A bifactor model parallel to the EF task model, with common-BRIEF, Shift-specific and WM-specific factors fit acceptably (χ² (502) = 1016.61, p< .001; RMSEA=.060, SRMR=.069). As for the EF task model, an alternative model including an Inhibit-specific factor found no significant variance for this factor, indicating that covariance among the Inhibit items was fully accounted for by the common-BRIEF factor. All indicators loaded significantly on their respective factors (Table S7).

Total reliability (ω) was .93 for the common- BRIEF, .83 for the Shift and .88 for the WM factors. The percentage of reliable variance in total scores attributed to individual differences in the common factor (ωH) was 80%, and the percentage of reliable variance in subscale scores due to specific factors after controlling for the common factor (ωHS) was 48% for Shift-specific and 38% for WM-specific. The common-BRIEF factor accounted for 67% of the explained common variance across all items (ECV), the Shift-specific factor accounted for 59% of the explained common variance in the Shift subscale items, and the WM-specific factor accounted for 43% of the explained common variance in the WM subscale items (ECVs, (Dueber, 2017)). Factor replicability (H) was .92 for common-BRIEF, .72 for Shift-specific and .72 for WM-specific. Thus, although the common-BRIEF factor accounts for a large amount of variance in total scores, it falls below the ECV cut-off of >.7-.8 for considering a construct unidimensional in applied settings (Rodriguez et al., 2016)), and the specific factors account for substantial additional covariance and have acceptable replicability for use in analyses.

EATQ-R.

An EATQ-R EC model identical to the best fitting model in our previous work in a different sample (Snyder et al., 2015), was tested, with common EC and Activation Control-specific factors (Figure 1c). The model fit acceptably (χ² (96) = 207.32, p< .001; RMSEA=.064, SRMR=.052). All items loaded significantly on their factors. Replicating the previous study (Snyder et al., 2015), an alternative model including Attentional Control-specific and Inhibitory Control-specific factors found no significant variance for the specific factors, indicating that covariance among these items was fully accounted for by the common-EATQ-R EC factor. All items loaded significantly on their respective factors (Table S8).

Total reliability (ω) was .87 for common- EATQ-R EC and .85 for Activation Control. The percentage of reliable variance in total scores attributed to individual differences in the common- EATQ-R EC factor (ωH) was 82%, and the percentage of reliable variance in the Activation Control subscale score due to the Activation Control-specific factor after controlling for the common factor (ωHS) was 27%. The common-EATQ EC factor accounted for 81% of the explained common variance across all items (ECV), and the Activation Control-specific factor accounted for 36% of the explained common variance in the Activation Control subscale items (ECVs, (Dueber, 2017)). Factor replicability (H) was .87 for common-EATQ EC factor, and .65 for the Activation Control-specific factor. Thus, although the common-EATQ EC factor accounts for a large amount of variance in total scores (potentially allowing the EATQ-R EC subscales to be analyzed with a total sum score in future studies and applied settings, (Rodriguez et al., 2016)), the specific Activation Control factor still account for additional covariance, though associations should be treated with some caution do to its somewhat lower H value.

Associations between bifactor models (Table 3).

Table 3:

Associations among EF task, EATQ-R and BRIEF-SR bifactor models

Factor 1	Factor 2	Correlation			Partial correlation controlling for age & gender			Age Interactions			Gender Interactions
		r	SE	p	r	SE	p	b	SE	p	b	SE	p
BRIEF-SR with EF Task Model:
Common EF tasks	BRIEF Common EC	.110	.080	.168	.030	.088	.738	−.073*	.036	.044	−.006	.086	.945
Common EF tasks	BRIEF WM-spec.	.150	.098	.127	.175	.106	.097	.204**	.057	<.001	−.184	.115	.111
Common EF tasks	BRIEF Shift-spec.	.034	.103	.743	.024	.106	.820	.179*	.089	.043	−.208	.198	.292
Updating-spec. EF tasks	BRIEF Common EC	.089	.084	.286	.095	.099	.341	.040	.048	.406	−.014	.137	.920
Updating-spec. EF tasks	BRIEF WM-spec.	.096	.102	.350	.118	.120	.327	−.057	.055	.308	.018	.138	.895
Updating-spec. EF tasks	BRIEF Shift-specific	.325**	.116	.005	.376**	.115	.001	.004	.072	.953	−.102	.175	.562
Shifting-spec. EF tasks	BRIEF Common EC	.012	.067	.856	−.003	.068	.962	.065	.037	.076	.054	.086	.531
Shifting-spec. EF tasks	BRIEF WM-spec.	−.021	.081	.793	−.020	.082	.807	.018	.054	.739	−.125	.118	.289
Shifting-spec. EF tasks	BRIEF Shift-spec.	.182*	.078	.020	.173*	.079	.029	.001	.075	.984	−.674*	.303	.026
EATQ-R with EF Task Model:
Common EF tasks	EATQ Common EC	.214**	.083	.010	.201*	.092	.025	.109**	.040	.007	−.162	.092	.079
Common EF tasks	EATQ Activation-Spec	−.275**	.093	.003	−.231*	.103	.025	.003	.049	.944	−.040	.109	.714
Updating-spec. EF tasks	EATQ Common EC	−.067	.106	.525	−.075	.121	.533	−.025	.047	.600	.063	.117	.591
Updating-spec. EF tasks	EATQ Activation-Spec	−.214	.118	.070	−.240	.128	.060	.044	.054	.424	−.146	.128	.252
Shifting-spec. EF tasks	EATQ Common EC	−.018	.068	.795	−.024	.070	.731	.016	.032	.622	.011	.072	.880
Shifting-spec. EF tasks	EATQ Activation-Spec	.000	.075	.999	−.005	.077	.951	−.016	.035	.646	−.029	.083	.739
BRIEF-SR with EATQ-R Model:
EATQ Common EC	BRIEF Common EC	.589**	.052	<.001	.575**	.053	<.001	−.073	.042	.084	−.103	.087	.240
EATQ Common EC	BRIEF WM-spec.	.515**	.060	<.001	.521**	.060	<.001	.030	.057	.600	−.233*	.102	.022
EATQ Common EC	BRIEF Shift-spec.	.082	.063	.196	.088	.064	.169	.000	.090	.999	−.105	.093	.260
EATQ Activation-Spec	BRIEF Common EC	.082	.076	.280	−.056	.077	.468	.063	.098	.523	−.051	.086	.553
EATQ Activation-Spec	BRIEF WM-spec.	.066	.088	.454	.058	.088	.511	−.015	.063	.808	−.104	.119	.381
EATQ Activation-Spec	BRIEF Shift-spec.	−.256**	.089	.004	−.253**	.089	.004	−.098	.096	.304	−.034	.100	.734

Note. For BRIEF, signs are reversed so that higher scores indicate higher EC for consistency with EATQ-R and EF measures.

Gender is coded -1 = female, 1 = male; negative gender interaction terms indicate stronger effects in female participants

^*p < .05 uncorrected

^**p < .05 FDR corrected

The common EF task factor significantly increased with age (β = .54, p<.001), and shifting-specific EF task (β = .16, p=.010) and Shift-specific BRIEF-SR (β = .14, p=.010) factors were higher in male participants; thus, we included age and gender as covariates and tested for age and gender moderation.

EF task performance with BRIEF-SR.

The BRIEF-SR Shift-specific factor was correlated with the EF task updating-specific factor (r = .33, p = .005; controlling for age and gender r = .38, p = .001). There were no other significant correlations with FDR correction.

We next tested for EF task factor x age and x gender interactions predicting BRIEF-SR factors, in separate SEMs. There was a significant common EF x age interaction on the BRIEF-SR WM-specific factor such that the association was stronger in older youth (Table 3, p < .001). A loop plot revealed that this association was significant for youth 17 and older (Supplemental Materials Figure 1); a follow-up multiple group analysis showed that in the significant age range (17-22), the factors were correlated r=.562, p < .001, whereas they were uncorrelated for younger youth (r=.019, p =.876). No gender interactions were significant with FDR correction.

EF task performance with EATQ-R.

EATQ-R common EC and common EF task factors were significantly correlated (r = .214, p = .010; controlling for age and gender r = .201, p = .029). The EATQ-R Activation Control-specific factor was significantly negatively correlated with common EF (r = −.275, p = .003; controlling for age and gender r = −.231, p = .025). There were no other significant correlations.

We next tested for EF task factor x age and x gender interactions predicting EATQ-R factors, in separate SEMs. There was a significant common EF x age interaction on the EATQ-R common EC factor such that the association was stronger in older youth (Table 3, p = .007). A loop plot revealed that the association between these factors was significant for youth 16 and older (Supplemental Materials Figure 2); a follow-up multiple group analysis showed that in the significant age range (16-22), the factors were correlated r =.433, p < .001, whereas they were uncorrelated for younger youth (r= .063, p =.577). There were no significant gender interactions.

Relation between EATQ-R – BRIEF-SR bifactor models.

The EATQ-R common EC factor was significantly correlated with the common-BRIEF factor (r = .589, p < .001; controlling for age and gender r = .575, p < .001), and BRIEF-SR WM-specific factor (r = .515, p < .001; controlling for age and gender r = .521, p < .001). The EATQ-R Activation Control-specific factor was negatively correlated with the BRIEF-SR Shift-specific factor (r = −.256, p = .004; controlling for age and gender r = −.253, p = .004). We next tested for EATQ-R factor x age and x gender interactions predicting BRIEF-SR factors, in separate SEMs (Table 3). No age or gender interactions were significant with FDR correction.

Discussion

Despite overlapping terminology and assumptions that they are tapping the same constructs, task-based EF and questionnaire-based EF/EC factors were only weakly associated in the total sample, even using empirically supported latent variable models that ameliorate the measurement impurity and reliability problems present in the previous literature. In correlated factors models, there were weak but significant correlations between many of the EF task and EF/EC self-report factors, with more significant correlations between EF task factors and the BRIEF-SR than EATQ-R factors. Using bifactor models that further differentiate common and specific aspects of each construct clarified and simplified the pattern of associations, but likewise revealed few significant associations in the full sample, with small to moderate correlations between the common EF task factor and EATQ-R common EC factor, and the EF task updating-specific factor and the BRIEF-SR Shift-specific factor. Thus, results were consistent across two alternative ways of conceptualizing and modeling the latent constructs, and results with manifest variables (see Supplemental Materials) likewise found only some weak correlations between tasks and questionnaire measures.

Notably, the magnitude of the EF task-EF/EC questionnaire factor associations in the current study (r =.15-.33) are only slightly larger than those in a meta-analysis of correlations between manifest EF/EC questionnaire and EF tasks (Duckworth & Kern, 2011). This could not be attributed solely to unreliability of the factors, as demonstrated by the strong correlations between the EF inhibition and updating task factors and among most of the questionnaire factors, demonstrating high convergence within measurement types, but low convergence across measurement types. These findings suggest that weak associations between EF tasks and EF/EC questionnaires are not solely a result of measurement error, but rather reflect that they measure largely independent constructs, at least on average across adolescents and emerging adults.

Interestingly, the common EF task factor was negatively correlated with the EATQ-R Activation Control-specific factor, consistent with previous evidence that the Activation Control-specific factor may represent maladaptive aspects of EC related to risk-avoidance and negative emotionality (Snyder, Gulley, et al., 2015). Indeed, in the current study it was negatively correlated with the BRIEF-SR Shift-specific factor, suggesting it may be related to rigidity (low shifting), and uncorrelated with the other BRIEF-SR factors. This is in contrast to the moderately strong positive correlations between the EATQ-R common EC factor and both the common and WM-specific factors of the BRIEF, showing convergence of across these self-report measures.

There are several practical implications from the latent variable models for research using manifest variables. First, although bifactor metrics indicated the EATQ-R EC items could be treated as unidimensional in applied settings, the above results suggests that researchers using the EATQ-R as a manifest variable should consider analyzing the Activation Control subscale separately from the Attentional Control and Inhibitory Control subscales (which can be combined). Second, bifactor metrics for the three BRIEF-SR subscales analyzed here and the EF tasks indicate that neither should be considered unidimensional, and thus caution is warranted in using a total composite score for these measures. Third, when participant time is limited, results from such factor analyses can be used to select a shorter set of tasks or questionnaire items to assess the constructs of interest. For example, instead of a very long questionnaire like the BRIEF, one could administer a much shorter set of items that load heavily on the common factor if a general, unidimensional score is desired, or a subsets of items that load most strongly on specific factors and least on the common factor (i.e., least “contaminated” by the general factor) if a measure of those specific facets is needed (see IECV, Tables S6-S; Stucky & Edelen, 2015).

Despite overall weak associations between task-based EF and self-reported EF/EC factors in the total sample, some stronger associations were found for older youth. Notably, there were significant age interactions whereby the common EF task factor was more strongly associated with the EATQ-R common EC factor, though interestingly not with the common BRIEF factor, in older youth. Common EF was also more associated with the BRIEF WM-specific factor in older youth. This latter correlation is consistent with the idea that common EF captures the ability to actively maintain task goals and goal-related information, and thus should be strongly related to WM (Friedman & Miyake, 2017). The BRIEF WM-specific factor is likely associated with the common EF rather than updating-specific task factor because the BRIEF WM items are more related to goal maintenance than to updating WM (e.g., “I forget what I am doing in the middle of things”). In sum, in older youth there is evidence that at the latent level task-based common EF is moderately associated with some aspects of self-reported EF/EC, but not always in the ways implied by shared construct labels across domains.

There are several potential reasons for these age differences. First, younger youth may be systematically biased in their self-reported EF/EC (such that self-report measures correlate well with one another but not with EF tasks) compared to older youth, a possibility that could be tested by future research comparing results across reporters. Of note, common EF task performance increased with age, whereas the self-report measures did not; thus, it is possible younger youth are either over-estimating their EF/EC, or that youth are comparing themselves to same-age peers. Alternatively, EF may play a greater role in daily cognitive control behaviors in older youth, whereas these behaviors in younger youth may be more driven by situational factors. This would be consistent with findings that successful use of cognitive control in early to mid-adolescence is more sensitive to factors such as social rewards and the presence of peers than it is either earlier or later in development (e.g., Shulman et al., 2015). Thus, younger adolescents’ EF ability under ideal conditions (low distraction, no peers present) may be less representative of their real-world behavior than that of older adolescents and emerging adults.

With FDR correction, we found no gender moderation of any associations. Notably, unlike in studies using parent- and teacher-report for children and early adolescents (Else-Quest et al., 2006), female participants did not report higher levels of EF/EC in the current study. Indeed, male youth reported slightly higher levels on the BRIEF Shift-specific factor, consistent with their slightly better shifting-specific EF task performance. Thus, adult informants may show gender biases in their perception of youth cognitive control that youth themselves do not share.

Future research is needed to probe the reasons for the lack of strong correspondence between most EF task performance and EF/EC self-report measures. First, it is possible that self- (and informant) -reported EF/EC may be inaccurate. For example, youths may perceive or wish to present themselves as having better EF/EC than they actually do. This would be consistent with the modest correlations between youth- and parent-reported EF/EC found in most studies (e.g., Muris et al., 2007), which of course may also reflect biased parent reports (e.g., perceiving teens negatively). Future research could test these possibilities with a multi-reporter approach and measures of such potential reporter biases. Alternatively, EF/EC questionnaire measures may accurately reflect real-world behaviors, but these behaviors may not strongly correspond to EF performance in low distraction, structured laboratory conditions. That is, individuals may not always optimally employ their EF abilities in daily life because the environmental context makes doing so difficult (e.g., distractions present when trying to study), the task demands are more complex than for laboratory tasks (e.g., control must be more self-directed), and/or because they lack motivation. In addition, self-report EF/EC questionnaires ask about behaviors over an extended period of time (six months for the BRIEF, in general for the EATQ-R), whereas tasks assess performance at only one time point. However, it should be noted that the EF task latent variables assessed here have been found to be highly rank-order stable even over six years from adolescence to early adulthood (Friedman et al., 2016). Thus, state or short-term influences are unlikely to account for the low correlations between EF tasks and self-report measures.

Future research could test these possibilities by assessing environmental and motivational factors, and measuring EF performance in more ecologically valid environments (e.g., with distractions), and with tasks requiring more self-direction (e.g., organization and prioritization of multiple sub-tasks over an extended period (Lamberts et al., 2010). Importantly however, increased ecological validity comes at the cost of process specificity. Thus, using a combination of more specific and more ecologically valid EF tasks, alongside EF/EC questionnaires, may provide the clearest picture of how different EF processes translate to daily behaviors.

In addition, EF tasks and EF/EC questionnaires can be assessed in relation to outcomes of interest (e.g., academic/occupational achievement, interpersonal functioning, psychopathology) to evaluate whether they predict independent variance in these outcomes. For example, a study in younger youth found that a common EF task factor significantly predicted reading and math performance over and above self-reported impulse control and conscientiousness; in contrast, impulse control did not predict academic ability beyond EF, and conscientiousness only weakly predicted math ability (Malanchini et al., 2018). The relative predictive validity of EF tasks and EF/EC questionnaire measures is likely to differ across outcomes and populations, further reinforcing the importance of including both types of measures whenever possible.

In sum, even with the enhanced power, reliability, and measurement purity provided by a latent variable approach, the current study suggests that self-reported EF/EC and EF task performance are at best moderately correlated on average in youth, although these associations can vary by age and by questionnaire (EATQ-R vs. BRIEF-SR). Moreover, the shared labels applied across EF task and EF/EC questionnaire constructs (e.g., shifting) do not necessarily presage correlations across these measures. Future research is needed to better understand how EF abilities manifest in daily behaviors as assessed by questionnaires, and the reasons they may not be strongly associated. In the meantime, EF/EC questionnaire and EF task measures should not be assumed to assess the same constructs, and extreme caution should be taken when drawing conclusions based on evidence spanning these measure types. Rather, they may be best viewed as complementary, but largely distinct, windows on cognitive control.