An Evaluation of the Structure of Attention in Adolescence

Abstract

This study evaluated the factor structure of attention, considering internal and external attention (Chun et al., 2011), and juxtaposed alongside processing speed (PS) and working memory (WM). We expected the hypothesized model to fit better than unitary or methods factors. We included 27 measures with 212 Hispanic middle schoolers from Spanish-speaking backgrounds, where a substantial proportion were at risk for learning difficulties. Confirmatory factor analytic models separated factors of PS and WM, but the final model did not align with theoretical predictions; rather only measurement factors emerged. Findings extend and refine our understanding of the structure of attention in adolescents.

There are numerous models, frameworks, and empirical studies about the structure of attention, including those of Chun et al. (2011), Cooley and Morris (1990), Dennis et al. (2008), Mirsky et al. (1999), and Posner (Posner & Peterson, 1990; Posner & Rothbart, 2007). However, direct testing of the structure of these is relatively rare. Chun et al. (2011) note that the construct of attention is very large and the term is used in myriad ways. In addition, as attention is a key component to most other cognitive operations, it is difficult to study, and therefore an organizing framework is useful. The Chun et al. (2011) taxonomy distinguishes external from internal attention, which is particularly relevant as this distinction can be overlaid against other common attention models. The purpose of the present study was to extend prior theoretical and empirical work on attentional classification through testing the validity of the Chun et al. (2011) taxonomy using a confirmatory factor analytic approach. To our knowledge, such work has not been attempted, particularly at younger ages (the context of the present study). Although the Chun et al. (2011) taxonomy is developmentally agnostic, attention is expected to increase with development, though by adolescence components are expected to be present and differentiable (Hoyer, Elshafei, Hemmerlin, Bouet, & Bidet-Caulet, 2021; Mullane, Lawrence, Corkum, Klein, & McLaghlin, 2016). To the extent that subdomains are identifiable, it is possible to evaluate their relations to functional outcomes (e.g., achievement) and inform intervention efforts.

Attention Targets and Related Constructs

Chun et al. (2011) note that the principles of limited capacity, selectivity (amongst other things that are competing for attention), modulation (attention that varies by situation), and vigilance (attention that is extended over time), all apply to attention. But because these principles apply to other cognitive and perceptual processes as well, the Chun et al. (2011) taxonomy instead focuses on the target of attention. At root, when attention is acting upon perceptual stimuli, it is external; when it is acting upon existing mental contents, it is internal.

In the Chun et al. (2011) taxonomy, external attention varies according to the modality of the stimuli, to its location in space, or to different stimuli that occupy the same location (though at different times). External attention also varies according to whether it is directed at a whole stimulus, or only to certain features/characteristics (e.g., color, pitch). In the present study, we focused on a particular set of external attention characteristics. Specifically, we operationalize external attention as visually-based, where instructions focus examinees on some location or object in the visual field, in a time and capacity-bound manner.

Internal attention in the Chun et al. (2011) taxonomy can vary according to task rules, responses, and long-term memory. That is, choosing among thoughts, or selecting a response to a stimulus in a decision-dependent manner, or selecting amongst the contents of long-term memory, all have in common that an external stimulus is not automatically driving attention. In the present study, we operationalize internal attention with mind-wandering, which occurs when there is “a shift away from a primary task toward internal information” (Smallwood & Schooler, 2006). In this manner, mind-wandering is clearly selecting internal thought contents as the focus of attention, and therefore, are good candidates for measuring internal attention. We chose this focus over others (e.g., response-selection, task-switching) to address internal attention targets that more separable from other functions such as processing speed and executive function, as discussed below.

Not all aspects of attention appear to be well captured by the internal-external distinction. For example, when there is a selection process (e.g., for an object), but the emphasis is on maintaining this process over an extended period, one could argue that this is an example of external attention. Alternatively, it could be argued that the maintenance of the target, and of the goal-direction, must include internal attention as well. Continuous performance tasks (CPTs) assess consistency and maintenance of responding, and so include both external and internal elements, and also happen to be widely used in the literature. Other aspects of attention, such as those emphasizing speed of decision making or behavioral responses, also fit less well onto an external versus internal distinction. One example is evaluating the tempo with which thoughts or behaviors across a variety of conditions take place, as in the phenomenon of sluggish cognitive tempo (Barkley, 2014, 2015; Becker, 2013; Becker et al., 2014). The present study seeks to ascertain whether such constructs are separable from internal and external attention, though this question has not yet been empirically addressed.

In the Chun et al. (2011) taxonomy, as in most models of attention, attention is measured with cognitive or performance measures. In contrast, studies of children with disorders such as attention deficit hyperactivity disorder (ADHD), or of children with learning difficulties, often assess attention behaviorally – that is, through asking others such as teachers and parents to rate attention-like behaviors of the individuals being studied. We therefore further sought to evaluate behavioral attention as a further distinguishable factor. Behavioral attention is not described explicitly in the Chun et al. (2011) taxonomy, so it is unclear whether it should more closely align with internal versus external attention, although the present study has the potential to evaluate this directly.

Given the many ways of construing attention, it is important to consider the extent to which attention components are separable from closely related (or potentially confusable) constructs, although this is rare in the literature. Two constructs of particular relevance are processing speed and executive function. First, most processing speed tasks require the speeded completion of a relatively easy task, often requiring simple (external) attentional selection, where individual differences in performances would be reduced or eliminated with unlimited time. Processing speed itself can vary in complexity, with some tasks requiring very little “processing” (such as reaction time), others requiring a moderate amount, and still others that are complex and could be considered executive tasks (see Gerst et al., 2021, for a test of these aspects). Therefore, the present study includes processing speed as a potentially discernable factor.

Second, executive function is confusable with attention, and these terms are often mixed (e.g., attentional control, executive or controlled attention, cognitive control). There are numerous executive-focused factor analytic studies (e.g., Cirino et al., 2018; Miyake et al., 2000; see meta-analysis of Karr et al., 2018). However, such studies do not set out to evaluate the structure of attention along theoretical lines (there is typically no mention of the common theories noted above), and the tasks they employ have different and more complex demands (e.g., those requiring inhibition of a prepotent response; those requiring manipulation or simultaneous processing/storage; those requiring shifting between alternating demands). Working memory is the most widely studied “executive” process; in fact, it is discussed within the Chun et al. (2011) model as being situated at the juncture of internal and external attention. Because working memory necessarily deals with active processing, and because this processing can be internally or externally directed, one’s capacity for working memory can be used in the service of either internal or external attention. Therefore, the present study also includes working memory as a potentially discernable factor.

Prior Factor Analytic Studies of Attention

A selective review evaluated 21 prior factor analytic studies of attention. While this is not a small number, it is quite low given how much is assumed about the structure of attention and how much is written about its organization. An overview of these studies is presented in Table 1. Other studies are available, but this particular overview is focused on representative examples of each type of study in the literature (e.g., where only rating scales, a single rating scale, or a given attention “battery” was factor-analyzed within different populations). Clinically, and theoretically, types of attention are often discussed, but the lack of factor analytic studies leaves open the question of the strength of the similarities and differences among these attention subdomains. More evidence in this regard is critical for being able to understand the way these subdomains might then relate to relevant outcomes, or else be utilized as mechanisms for explaining other phenomena. Studies that do exist vary widely with regard to age, population, approach (how broadly attention is construed), and sample size.

Table 1.

Overview of Prior Factor Analytic Studies of Attention

Citation	Age Range	N	Model	Analyses	Type	Test	Attention Differentiated from Other Non-Attention Factors?
Mirsky et al., 1991, 1999	Child & Adult	435/203	(Mirsky, 1987)	PCA	Performance	Multiple	No
Park et al., 2009	Child	201	(Mirsky et al., 1991)	Confirmatory	Performance	Multiple	No
Pogge et al., 1994	Child	278	(Mirsky et al., 1991)	Confirmatory	Performance	Multiple	No
Kelly, 2000	Child	100	(Mirsky et al., 1991)	Exploratory/PCA	Performance	Multiple	No
Shapiro et al., 1998	Child	107	(Cooley & Morris, 1990)	Confirmatory	Performance	Multiple	No
Tao et al., 2007	Child	365	(Sturm & Zimmermann, 2000)	Exploratory	Performance	Multiple	No
Shannon et al., 2021	Child	99	No	Exploratory	Performance	Multiple	No
Leopold et al., 2016	Child	489	No	Confirmatory	Rating	Multiple	No
Willcutt et al., 2014	Child	731	No	Exploratory	Rating	Multiple	No
Breckenridge et al., 2013	Child	154	No	Exploratory/PCA	Both	Single Battery (ECAB)	No
Malegiannaki et al., 2015	Child	174	No	Confirmatory	Performance	Single Battery (TEA-Ch)	No
Strauss et al., 2000	Adult	160	(Mirsky et al., 1991)	Confirmatory	Performance	Multiple	No
Combs & Gouvier, 2004	Adult	65	(Mirsky et al., 1991)	PCA	Performance	Multiple	No
Goldhammer et al., 2007	Adult	232	(Posner & Boies, 1971; Posner & Rafal, 1987)	Confirmatory	Performance	Multiple	No
Chan et al., 2006	Adult	133	(Posner & Peterson, 1999)	Confirmatory	Performance	Single Battery (TEA)	No
Williams et al., 2017	Adult	315/78	No	Exploratory	Both	Multiple	Yes
Carlozzi et al., 2008	Adult	175	No	Exploratory/Confirmatory	Performance	Single Battery (RBANS)	Yes
Schmitt et al., 2010	Adult	636	No	Exploratory	Performance	Single Battery (RBANS)	Yes
Jones et al., 2015	Adult	103	No	Exploratory	Performance	Single Battery (DalCAB)	No
Van Calster et al., 2018	Adult	206/294/111	No	Exploratory/Confirmatory	Rating	Single Measure (Attentional Style Quest.)	No

Many factor analytic studies of attention focus on adults rather than children or adolescents, often in a general population (e.g., Goldhammer et al., 2007; VanCalster et al., 2018; Williams et al., 2017). Other adult studies focus exclusively on a narrow group such as veterans referred through a memory disorders clinic (Carlozzi et al., 2008), chronic schizophrenia (Combs & Gouvier, 2004), or dementia (Schmitt et al., 2010). Some adult-focused studies have also been conducted assessing a particular battery, which sometimes does (Chan et al., 2006) but often does not (Carlozzi et al., 2008; Jones et al., 2015; VanCalster et al., 2018) correspond as well to a particular model of attention.

Other studies do focus on children like the present study. Some of these include specific populations such as adolescents on a psychiatric inpatient unit (Pogge et al., 1994), or children with brain injury (Park et al., 2009). A number of child studies also include small sample sizes (e.g., Kelly, 2000; Shannon et al., 2021; Shapiro et al., 1998), and only one study was identified that utilized both rating scales and performance measures (Breckenridge et al., 2013). Some studies feature large samples but focus specifically on rating scales in their factor analytic models, such as those distinguishing ADHD and sluggish cognitive tempo, rather than on a more general structure of attention (Leopold et al., 2016; Willcutt et al., 2014). Some child-focused studies evaluate a battery of measures, rather than a specific model of attention. For example, Malegiannaki et al. (2015) evaluated the structure of the Test of Everyday Attention for Children in Greek school age children, whereas Breckenridge et al. (2013) evaluated their own measure of attention for use in children between 3 and 6 years. Child-focused studies have evaluated attentional models (e.g., Mirsky et al., 1991; Kelly, 2000; Park et al., 2009; Pogge et al., 1994; Shapiro et al., 1998; Tao et al., 2017). However, even these studies do not always use a confirmatory approach (e.g., Kelly, 2000; Mirsky et al., 1991; Tao et al., 2017), and all include only performance-based measures.

Across child and adult samples, the model of Mirsky et al. (1991), which separates focus/executing, sustain/stabilize, shifting, and encoding, is the most commonly evaluated (Combs & Gouvier, 2004; Kelly, 2000; Park et al., 2009; Pogge et al., 1994; Strauss et al., 2000). For example, Kelly (2000), in a sample of 100 children ages 7–13 years, replicated Mirsky’s adult four-factor model. However, the Mirsky model is difficult to evaluate, because a number of measures used are often considered to be aligned with constructs outside of attention. For example, the Trail Making Test (Army Individual Test Battery, 1944; Reitan & Wolfson, 1985) or Wisconsin Card Sorting Test (Berg, 1948; Grant & Berg, 1948) are typically considered measures of executive function rather than attention per se. Further, the IQ subtests used in this model seem more aligned with either executive function or working memory (Digit Span, Arithmetic) or processing speed (Coding, Symbol Search), than attention per se. It is notable that across a range of studies (e.g., Breckenridge et al., 2013; Shannon et al., 2021; Tao et al., 2017), when complex measures that overlap with executive skills are included, they form factors separate from more elemental attention components, which is the focus of the current study.

Only studies that include measures from a variety of tests can be evaluated for method bias. For example, Tao et al. (2017) evaluated 365 children ages 7–12; across 7 measures of attention from 3 separate tests, a unitary factor was found for 7–8 year-olds, whereas two factors emerged in children ages 9 and older – perceptual attention (alertness, focused attention, divided attention, and sustained attention) and executive attention (consisting of attentional switching, spatial attention, and supervisory attention). Of note, the two factors identified in the older groups did not fall out among test lines; instead, multiple tests loaded on each factor, supporting the idea that there were more than methodological factors. However, in part because many factor analytic studies are exploratory or use principal components analyses, it is rare to see explicit model comparisons. Such an approach has been successfully used in other domains, including executive function (Cirino et al., 2018) and processing speed (Gerst et al., 2021).

Among performance-based measures, analyzed variables also differ across studies. For example, CPTs are frequently used, but the actual metric used might include hit rate, and/or omission and/or commission error rate, and/or reaction time, and/or variability (Combs & Gouvier, 2004; Kelly, 2000; Mirsky et al., 1991; Pogge et al., 1994; Shapiro et al., 1998; Strauss et al., 2000; Tao et al., 2017; Willcutt et al., 2014; Williams et al., 2017). The d-prime (d’) index, denoting sensitivity/discriminability, has also been utilized (Combs & Gouvier, 2004), and can function as a summary of overall performance. In most studies, CPT variables load on a factor of/measure sustained attention or vigilance (Combs & Gouvier, 2004; Mirsky et al., 1991; Pogge et al., 1994; Shapiro et al., 1998; Strauss et al., 2000; Tao et al., 2017), although in at least one study, reaction time measures loaded on a perceptual motor speed factor or an information processing factor related to speed of response (Kelly, 2000). As also noted above, CPTs are tasks that may be construed as either an index of external (driven by stimuli) or internal (requiring resistance to distraction over time) attention in the Chun et al. (2011) taxonomy.

In summary, the number of factor analytic studies is not large, and available studies vary widely on key dimensions. It is rare to see studies in children that evaluate a broad range of attention measures aligned with a theoretical model, and test this in a confirmatory way, and particularly against attentional components not well-covered in the Chun et al. (2011) taxonomy, against other related cognitive constructs, and against methodological factors. The present study addresses each of these issues. The context of this study is with a sample of middle school students selected from a larger project focused on language and reading; the impact of these characteristics is evaluated directly (see below).

The Present Study

Based on the above literature, our main hypothesis was that we could empirically separate internal and external aspects of attention in a confirmatory factor analytic manner, and that this model would evidence better fit than a unitary model. We also hypothesized that these two resulting factors would also be separable from measures (a) with characteristics of both external and internal attention, (b) of behavioral attention, and (c) of processing speed and (d) working memory; we also expected these factors to be related to one another. We expected this content focused model to fit better than those focused on methods (by tests or by type – ratings versus objective tests; or further by specific tests or by rater – teacher versus student). Table 2 displays a schematic of the tests and variables utilized, and the factor on which they were expected to load, along with the methodological factors they also assess.

Table 2.

Indicator Variables by Hypothesized and Final Model

		Theoretically Expected Factor						Method Type
Task Name	Analysis Part	Internal	External	Behavioral	Combined	Working Memory	Processing Speed	Objective	Self-Report	Other Report	Final Factor Designation
MW Trait1	I	X							X		SR_TRAIT
MW Trait2		X							X		SR_TRAIT
MW TBRR1		X							X		SR_STATE
MW TBRR2		X							X		SR_STATE
MW Post		X							X		SR_STATE
VAS L	I		X					X			VAS
VAS N			X					X			VAS
VAS H			X					X			VAS
External Trait			X						X		SR_TRAIT
VSEARCH L			X					X			VSEARCH
VSEARCH N			X					X			VSEARCH
VSEARCH H			X					X			VSEARCH
Self Inattention	II			X					X		SR_BEH
Self Hyp/Imp				X					X		SR_BEH
Teach Inattention				X						X	T_BEH
Teach Hyp/Imp				X						X	T_BEH
CPT d’ B1	III				X			X			CPT
CPT d’ B2					X			X			CPT
CPT d’ B3					X			X			CPT
Self SCT					X				X		SR_TRAIT
Teach SCT					X					X	T_BEH
Digit Forward	IV					X		X			WM
Digit Backward						X		X			WM
Digit Sequence						X		X			WM
Picture Span						X		X			WM
Coding	IV						X	X			PS
Symbol Search							X	X			PS

Note: Indicator variables appear in the first column, whereas the hypothesized model loadings are represented in the second group of columns; the names of the final factors appear in the far right column. SR_TRAIT = self trait reports; MW Trait1 = 5-item self-report mind wandering questionnaire; MW Trait2 = revised 6-item self-report mind wandering questionnaire; SR_STATE = self state reports; MW TBRR1 = retrospective report of mind wandering followed VAS task; MW TBRR2 = retrospective report of mind wandering followed CPT task; MW Post = after-task mind wandering questions; VAS = visual attention span; VAS L = VAS with letters; VAS N = VAS with numbers; VAS H = VAS with Hiragana; External Trait = self-report external attention questionnaire; VSEARCH = visual search task; VSEARCH L = visual search task with letters; VSEARCH N = visual search task with numbers; VSEARCH H = visual search task with Hiragana; SR_BEH = self behavioral reports; Self Inattention = SWAN self-report inattention items; Self Hyp/Imp = SWAN self-report hyperactivity/impulsivity items; T_BEH = teacher behavioral reports; Teach Inattention = SWAN teacher-report inattention items; Teach Hyp/Imp = SWAN teacher-report hyperactivity/impulsivity items; CPT = continuous performance task; CPT d’ B1 = CPT discriminability at block 1; CPT d’ B2 = CPT discriminability at block 2; CPT d’ B3 = CPT discriminability at block 3; Self SCT = self-report SCT items; Teach SCT = teacher-report SCT items; WM = working memory; Digit Forward = WISC-V Digit Span Forward subtest; Digit Backward = WISC-V Digit Span Backward subtest; Digit Sequence = WISC-V Digit Span Sequencing subtest; Picture Span = WISC-V Picture Span subtest; PS = processing speed; Coding = WISC-V Coding subtest; Symbol Search = WISC-V Symbol Search subtest

Methods

Participants

The present study included 218 students in grades 6 and 7 from a larger parent study. Given that context, all students were Hispanic, and half were currently or formerly classified as limited English proficient; half were also struggling readers. Although some participants were randomized to a reading intervention or control conditions, the measures here were assessed in Fall and so not affected by the intervention (and this was addressed directly in preliminary analyses). The sample was divided over two sites in a Southwestern state. Six students were not included as they had nonverbal reasoning and vocabulary scores below −2 standard deviations. Table 3 displays demographics for the analyzed sample of 212.

Table 3.

Demographic Characteristics (N = 212)

		N (%)
Gender	Female	102 (48.11)
Site	Houston	97 (45.75)
	Austin	115 (54.25)
Ethnicity	Hispanic	212 (100.00)
Special Education	No	193 (91.04)
Economic Disadvantage	Yes	149 (70.28)
	No	10 (4.72)
Limited English Proficiency	Yes	106 (50.00)
Grade	6	126 (59.43)
	7	86 (40.57)
Treatment Condition	BAU	46 (21.70)
	INT	64 (30.19)
	Not Randomized	102 (48.11)
	Mean	Standard Deviation
WASI-2 Matrix Reasoning	45.53	8.01
Age (years)	12.37	0.73

Note. Within each bolded section, when only one line is presented, the remainder are the other designation (e.g., if “no” is 91.04%, then “yes” is 8.96%). For economic disadvantage, there were missing data for 53 students (25%). WASI-2: Wechsler Abbreviated Scales of Intelligence, 2^nd Edition. Treatment refers to randomized condition for struggling readers according to the larger parent study (BAU = Business as Usual control; INT = Intervention).

Measures

There were 27 measures of attention, working memory, and processing speed. Table 4 provides psychometric data, distributional characteristics, and means and standard deviations.

Table 4.

Descriptive Statistics for Measures (Means, SD, Reliabilities)

Variable	N	Mean	SD	Reliability	Skewness	Kurtosis
MW Trait1	197	2.946	1.104	0.710	0.365	−0.608
MW Trait2	212	3.083	1.293	0.856	0.281	−0.925
MW TBRR1	211	1.892	0.916	0.875	1.052	0.142
MW TBRR2	212	2.060	0.945	0.871	0.692	−0.587
MW Post	212	2.126	0.612	0.710	0.381	−0.276
VAS H	207	17.314	5.095	0.490	−0.292	−0.107
VAS L	205	42.405	11.765	0.856	0.535	0.407
VAS N	205	49.839	12.495	0.858	0.176	1.175
External Trait	212	3.077	1.157	0.843	0.076	−0.851
VSEARCH H	211	11.854	2.612	0.703*	0.573	0.799
VSEARCH L	211	15.562	3.083	0.703*	0.102	−0.151
VSEARCH N	211	13.719	2.509	0.703*	−0.140	0.226
Self Inattention	212	0.594	10.326	0.828	−0.015	−0.059
Self Hyp/Imp	212	−0.986	12.067	0.862	0.069	−0.329
Teach Inattention	208	0.063	12.014	0.982	−0.252	0.113
Teach Hyp/Imp	208	−2.538	10.495	0.970	−0.437	0.840
CPT d’ B1	208	3.052	1.056	0.892*	−0.747	0.196
CPT d’ B2	209	3.150	1.120	0.892*	−0.746	−0.247
CPT d’ B3	210	2.977	1.123	0.892*	−0.714	−0.024
Self SCT	212	1.670	1.054	0.918	0.753	0.155
Teach SCT	208	0.755	1.009	0.975	1.842	3.646
Digit Forward	212	7.557	1.747	0.674	0.222	0.802
Digit Backward	212	8.000	1.947	0.656	0.031	−0.028
Digit Sequence	212	7.099	2.100	0.703	−0.017	−0.382
Picture Span	212	26.675	6.171	0.780	−0.002	0.100
Coding	212	46.038	9.832	0.830**	0.185	−0.410
Symbol Search	211	25.142	6.114	0.810**	−0.377	0.858

Note: See Table 2 for variable definitions. Reliability is determined by Cronbach’s alpha except:

^*= alternate forms reliability

^**= test-retest coefficients taken from test manual.

External Attention

There were 7 measures. Three each were varieties of a visual attention span (VAS) task, and of a visual search task. The VAS task involves 20 trials of the very brief (200ms) presentation of 5 stimuli, followed by recall. The task was adapted from Lallier et al. (2013) and was used in a non-overlapping study (Cirino, Barnes, Roberts, Miciak, & Gioia, 2022). Three types of stimuli (letters, numbers, and Hiragana) were used (each 20 trials), with stimuli order counterbalanced. The analyzed variable for each of the three conditions was recall in its proper order (range 0 to 100). Visual search was a standard speeded scanning task, with the same stimuli as the VAS task (also used in Cirino et al., 2022). The analyzed variables were the mean number of correctly identified targets (minus commissions) within a 30 second time limit, with each task given twice. The final (seventh) measure of external attention was a researcher-generated student 8-item questionnaire that directly asked students about external distraction in the context of specific situations (including reading and math). Total score was the analyzed variable.

Internal Attention

There were 5 measures. First, the Mind Wandering Questionnaire (Mrazek et al., 2013) is a student-report questionnaire assessing loss of internal control of attention as a trait. Second, a revised 6-item version (with some wording changes, and specific questions pertaining to reading and math) was also administered. Two additional measures were self-report indices of task-based retrospective reporting, using the task-irrelevant cognitive interference items of the Dundee Stress State Questionnaire (DSSQ; Matthews et al., 1999, 2002). This measure has 8 items about task-unrelated internal distraction during a specific task (e.g., “I thought about something that happened earlier today”). The same questionnaire was administered twice, once following the VAS (see above) and continuous performance task (see below) measures. Finally, an additional measure of task-based retrospective reporting was utilized, consisting of a single item (ratings of the extent to which “My mind was wandering…”) administered following four tasks. The score across these items was combined into a total and used as the analyzed variable.

Behavioral Attention

The SWAN (Strengths and Weaknesses of Attention-Deficit/Hyperactivity Disorder Symptoms and Normal Behavior Scale, Swanson et al., 2012) was administered. It has scales of both inattention and hyperactivity/impulsivity and was completed by both teachers and students. This measure is extensively used to assess “attention” in many predictive studies (e.g., Crosbie et al., 2013; Lui & Tannock, 2007; Roberts et al., 2015; Sims & Lonigan, 2013). It has excellent psychometric properties, such as high internal consistency and moderate test–retest reliability, expected skewness and kurtosis, and adequate convergent and discriminant validity (Lakes et al., 2012); comparable validity, reliability, and heritability to the Disruptive Behavior Rating Scale (Arnett et al., 2013); and high convergent validity with the Conners’ scales (Burton et al., 2019). The total score was the analyzed variable.

Combined Attention Measures

A continuous performance task (CPT) was administered, using the same stimuli as the VAS and Visual Search tasks. Stimuli appear after variable interstimulus intervals (500 ms, 1500 ms, 3000 ms), and students press the spacebar after (and only after) a target stimuli. Targets appear on 20% of the 312 trials, dispersed across 3 blocks. Within each block, a discriminability index (d’) is computed, representing a balance of hits and false alarms. The CPT represents both external (presented stimuli drive the response) and internal (attention must be sustained over ~12 minutes) aspects of attention; the d’ from each block were the indicators for a Combined latent attention factor. The remaining two measures were of Sluggish Cognitive Tempo (SCT) from the Child and Adolescent Behavior Inventory (CABI; Burns et al., 2018). This questionnaire (completed by teachers and students) represents both internal and external distraction, and so these were also included in the Combined latent factor; the total score was the analyzed variable.

Working Memory

Two measures were included, both from the Wechsler Intelligence Scales for Children-5 (WISC-5; Wechsler, 2014). Picture Span assesses visual working memory by requiring examinees to select a set of pictures that is briefly presented. Digit Span assesses auditory rehearsal and mental manipulation of information by requiring examinees to recite a sequence of numbers read aloud in the same order, in reverse order, and in ascending order. Both subtests are widely used and psychometrically sound measures of working memory.

Processing Speed

Two measures were included, both from the WISC-5 (Wechsler, 2014). Coding requires examinees to quickly copy symbols corresponding to geometric shapes or numbers within a time limit. Also within a time limit, Symbol Search requires examinees to scan a group of symbols and indicate whether or not target symbols are present. Both subtests are widely used and considered psychometrically sound measures of processing speed.

Procedure

All procedures were reviewed and approved by the Institutional Review Board at (blinded), as well as by the school district and schools. Students were tested at their schools over ~2 hours (in one or two sessions), with half receiving the battery in reverse order to combat order effects. All examiners were highly trained on the measures and their administration, and were closely supervised during training, passed formal “check out” procedures, and also were provided extensive onsite (school) supervision. Iterative quality checks were conducted onsite and at the project lab; data forms were optically scanned, with further data checks employed.

Analysis

Preliminary Analyses.

We first interrogated characteristics of the data in two ways prior to evaluating hypotheses. The first set of these evaluated distributional assumptions, examiner notes, and other indicators of data quality. Across the 27 variables for the 212 participants (5724 data points), we found 1 error for the Symbol Search task (examiner error), 5 errors across the three VAS tasks (computer error), and 16 outliers across the 6 Visual Search tasks (0.3% of all data points). Errors were set to missing, and the outliers were winsorized. Of note, analyses that included the 6 excluded participants, and the 23 individual data-point errors and outliers, were substantively nearly identical to those displayed below.

The second set of preliminary analyses addressed the role of sample characteristics on attentional measures. First, we considered demographic variables including age, grade, district, gender, randomization status, and limited English proficiency status. For 19 of the 21 attention variables, these six sample characteristic variables together accounted for only between R² = 5% and 16%; age was significant only for 1 of the 21 attention variables, and other characteristics were significant for between 4 and 7 attention variables, with no consistent pattern. Also, 14 of the 29 significant results were .01 < p < .05 (without any correction). Sample characteristics were most related to teacher ratings (R² = 25%), with grade (p = .047 and .021 for hyperactivity and inattention, respectively), site (p = .004 and .011), gender (p < .001 for both), limited English proficiency status (p = .049 and .019), and randomization status (p = .005 and < .001) relevant. We also correlated the 21 attention variables with six reading and language variables (126 correlations), and the range r was = |.00 to .43|, with a median of r = |.12| (in this sample, correlations of r = ~.13 would be statistically significant at p = .050). Thus, many correlations were significant, but generally small in size. The specific relations of word-reading and reading comprehension variables with self-report measures showed a median relation of r = .05 and r = .10, respectively. Overall, the relations of attention to student characteristics and reading and language variables is consistent with the larger literature in that attention relates to such variables, though not in a deterministic manner. Overall, the pattern indicates that the results are unlikely to be unduly influenced by the characteristics of this particular sample.

Analyses for Hypotheses.

The key approach used were confirmatory factor-analytic models, conducted in MPLUS (Muthén & Muthén, 1998–2017). We followed steps as suggested by key sources from initial specification, to identification, estimation, fit, and modification (e.g., Bollen & Noble, 2011). Models were expected to converge, and overall fit indices included AIC, BIC, CFI, RMSEA, SRMR (e.g., Gunzler & Morris, 2015; Kline, 2015; Wang & Wang, 2019). Nested models were compared by tests of chi-square comparability (with degrees of freedom equal to the difference between models). Non-nested models were compared on the basis of the additional overall fit indices.

The original study design called for a sample twice as large, collected over two cohorts. However, data on the second cohort were uncollected due to Covid-19, which meant relatively lean statistical power. Although there are numerous rules of thumbs for sample size for factor analysis, these are not well supported definitively, and sample size restrictions can be mitigated with model characteristics that increase convergence and model stability, including fewer latent variables, more indicators per latent variables, and higher factor loadings and communalities (Kyriazos, 2018; MacCullum, Widaman, Zhang, & Hong, 1999; Mundfrom, Shaw, & Ke, 2005). In the present case, we approached our analyses in a way that allowed us to check our conclusions by dividing analyses into four Parts (I-IV), with the best-fitting model from each Part serving as the base for the next Part. We first tested our primary hypothesis (regarding the internal/external distinction) with a restricted set of indicator and latent variables (so that sample size was not an issue). We then expanded and contextualized these results by including other variables. What is key is that the pattern of results (factor loadings and standard errors, and relations among indicator and latent variables) found in Part I were substantively unchanged in subsequent Parts; in additional, all final models converged and showed good overall model fit.

Part I considered only the 12 variables aligning with the internal/external distinction, and compared the preferred/hypothesized model against a unitary attention model and against a methods-specific model. All hypothesized models were designed to reflect simple structure – i.e., multiple indicators for each factor, each indicator assigned to only one factor, with uncorrelated residuals, but with correlated factors.

Proceeding with the best fitting Part I model, we added the four measures of behavioral attention (Part II, 16 variables total), hypothesizing this would separate from the latent factors of Part I, and also comparing it to a methods model where self- and teacher- ratings loaded separately. Part III added to the best fitting Part II model, the five combined variables that cross the internal/external dimension (21 variables total); these were expected to load together, and separate from the factors of each of Part I and Part II. This model was then compared to a methods model that disaggregated according to type of measure (objective or direct child assessment versus self-report versus other-report). Finally, to the best fitting Part III model we added the six indicators of working memory and processing speed (Part IV; 27 variables total), expecting these latent variables to separate from the latent factors of all prior Parts.

Results

Table 5 contains indicator correlations. Table 6 contains model fit statistics and comparisons; in that table, accepted models appear in bold. Table 7 documents the specific factors identified for each model of Table 6, and also includes factor loadings of the final accepted model.

Table 5.

Indicator Measure Correlations

	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19	20	21	22	23	24	25	26
1. MW Trait1	--
2. MW Trait2	.53**	--
3. MW TBRR1	.26**	.37**	--
4. MW TBRR2	.27**	.43**	.81**	--
5. MW Post	.21*	.24**	.33**	.37**	--
6. VAS H	−0.08	−0.04	0.02	0.11	−0.01	--
7. VAS L	<.01	0.02	0.09	0.09	−0.12	.29**	--
8. VAS N	−0.05	−0.07	0.06	0.05	−.14*	.41**	.68**	--
9. External Trait	.43**	.72**	.31**	.38**	0.13	0.07	0.03	<.01	--
10. VSEARCH H	−0.1	−0.11	−0.13	−0.11	−.15*	0.06	.19*	.18*	−0.1	--
11. VSEARCH L	−0.07	−0.08	0.02	<.01	−0.11	0.04	.17*	.18*	−0.11	.68**	--
12. VSEARCH N	−0.11	−0.09	−0.04	−0.05	−0.13	−0.05	0.1	.15*	−0.09	.64**	.75**	--
13. Self Inattention	0.04	<.01	<−.01	0.03	0.03	−0.07	−0.08	−0.08	−0.05	−0.13	−0.01	−0.07	--
14. Self Hyp/Imp	0.04	0.06	−0.04	−0.04	0.05	−.15*	−0.13	−0.12	−0.03	−0.09	−0.05	−0.06	.72**	--
15. Teach Inattention	.20*	.19*	0.05	0.03	0.1	−.15*	−.21*	−.22*	.18*	−0.12	−.18*	−0.11	0.1	.17*	--
16. Teach Hyp/Imp	.12*	.14*	0.06	0.02	0.03	−.15*	−.15*	−0.14	.16*	−0.05	−0.09	−0.04	0.06	0.09	.85**	--
17. CPT d’ B1	−0.04	0.02	−0.09	−0.06	−0.06	.20*	.21*	.18*	0.01	.20*	0.09	0.09	−0.07	−0.08	−0.11	−0.06	--
18. CPT d’ B2	−0.13	−0.05	−0.11	−0.09	−.16*	.25**	.24**	.30**	−0.02	.22*	.18*	0.11	−0.05	−0.11	−0.1	−0.05	.73**	--
19. CPT d’ B3	−0.12	−0.04	−0.09	−0.08	−0.09	.34**	.27**	.27**	0.02	.19*	.17*	0.13	−0.04	−0.11	−.15*	−0.1	.74**	.82**	--
20. Self SCT	.52**	.74**	.42**	.44**	.26**	−0.05	<−.01	−0.09	.58**	−0.11	<−.01	−0.05	0.06	0.1	.19*	.15*	−0.07	−0.1	−0.11	--
21. Teach SCT	0.13	.14*	0.04	0.02	0.1	<.01	−.15*	−.15*	.16*	−0.13	−.14*	−0.09	0.07	0.13	.55**	.32**	−.17*	−.19*	−.19*	0.1	--
22. Digit Forward	−0.04	−0.02	−0.11	−0.1	−0.09	−0.11	.14*	0.03	−0.03	.18*	.18*	.14*	−0.07	<−.01	0.02	<−.01	0.08	0.03	−0.02	−0.01	<.01	--
23. Digit Backward	−0.09	−0.06	0.09	0.02	−0.1	0.05	.27**	.22*	−0.03	.20*	.14*	.14*	−0.07	−0.13	−.17*	−.16*	.18*	.20*	0.13	−0.06	−0.1	.28**	--
24. Digit Sequence	−0.08	−0.1	0.06	−<.01	−0.07	−0.03	.24**	.17*	0.01	0.13	0.08	0.04	−0.06	−0.05	−0.12	−0.1	0.07	0.1	0.05	−0.07	−0.09	.34**	.48**	--
25. Picture Span	−0.04	−0.06	<−.01	0.07	−0.02	0.13	0.13	.15*	−0.02	0.08	0.04	0.04	0.03	−0.02	−.20*	−.16*	.30**	.24**	.29**	−0.04	−.28**	.20*	.29**	.33**	--
26. Coding	−0.01	−0.04	<−.01	−0.04	−0.07	0.12	0.12	.21*	−0.08	.35**	.43**	.34**	0.07	0.09	−.23**	−.21*	.18*	.21*	.18*	−0.06	−.15*	0.04	.16*	0.13	0.11	--
27. Symbol Search	−0.05	−0.1	−0.07	−0.12	−.15*	0.04	0.06	0.07	−0.06	.37**	.40**	.33**	0.05	0.09	−0.13	−0.1	−0.03	0.1	0.08	−0.02	−0.08	0.03	0.05	0.09	−0.01	0.49**

Note. See Table 2 for variable definitions.

^*p < .05

^**p < .001

Table 6.

Measurement Model Fit Statistics (n = 212)

Model	AIC	BIC	ABIC	χ2	df	RMSEA (90% C. I.)	CFI	TLI	SRMR	χ2 diff	p
Part I (12 vars)
1a. Unitary (1)	10726.0	10846.8	10732.7	707.34	54	0.239 0.223 to 0.255	0.357	0.214	0.181	--	--
1b. Internal/External (2)	10433.1	10557.3	10440.1	412.48	53	0.179 0.163 to 0.195	0.646	0.559	0.148	294.86	<0.001
1c. Methods (3)	10246.9	10377.8	10254.3	222.30	51	0.126 0.109 to 0.143	0.831	0.782	0.088	190.18	<0.001
1d. State/Trait (4)	10083.1	10224.1	10091.0	52.51	48	0.021 0.000 to 0.051	0.996	0.994	0.047	169.79	<0.001
Part II (16 vars)
2a. Behavioral (5)	16256.6	16448.0	16267.3	247.31	95	0.087 0.074 to 0.100	0.895	0.867	0.074	--	--
2b. State/Trait (5)	16264.4	16455.7	16275.1	255.11	95	0.089 0.076 to 0.102	0.890	0.861	0.079	--	--
2c. Methods (6)	16113.6	16318.3	16125.0	96.25	91	0.016 0.000 to 0.041	0.996	0.995	0.046	151.06	<0.001
Part III (21 vars)
3a. Combination (7)	18810.7	19085.9	18826.1	454.52	170	0.089 0.079 to 0.099	0.869	0.838	0.102	--	--
3b. Methods (8)	18750.2	19045.5	18766.7	382.02	164	0.079 0.069 to 0.090	0.900	0.872	0.093	--	--
3c. Methods (7)	18745.5	19020.8	18761.0	389.40	170	0.078 0.068 to 0.088	0.899	0.875	0.096	--	--
3d. Methods (7)	18563.7	18838.9	18579.1	207.53	170	0.032 0.012 to 0.047	0.983	0.979	0.050	--	--
3e. Methods (3)	19340.7	19558.8	19352.9	1018.5	187	0.145 0.136 to 0.154	0.618	0.571	0.121	--	--
Part IV (27 vars)
4. Add WM/PS (9)	25300.5	25686.5	25322.2	360.28	290	0.034 0.021 to 0.045	0.971	0.965	0.054	--	--

Note: Under the Model column, numbers in parentheses refer to the number of latent variables posited. AIC = Akaike Information Criteria; BIC = Bayesian Information Criteria; ABIC = Sample-Size Adjusted Bayesian Information Criteria; df = degrees of freedom; RMSEA = Root Mean Squared Error of Approximation; CFI = Comparative Fit Index; TLI = Tucker-Lewis Fit Index; SRMR = Squared Root Mean Residual; SB χ2 diff = t value for model comparison using the chi-square difference test, relative to the previous model, within each part. Models within Part III are not comparable via chi-square difference, given same degrees of freedom, but are comparable on other metrics.

Table 7.

Factor Placement of Each Variable for Accepted Models from each Part

	Part I: Model 1d Factors				Part II: Model 2c Factors						Part III: Model 3d Factors							Part IV: Model 4 Factors									Factor Loadings for Final Model 4
	VAS	VSEARCH	SR_STATE	SR_TRAIT	VAS	VSEARCH	SR_STATE	SR_TRAIT	T_BEH	SR_BEH	VAS	VSEARCH	SR_STATE	SR_TRAIT	T_BEH	SR_BEH	CPT	VAS	VSEARCH	SR_STATE	SR_TRAIT	T_BEH	SR_BEH	CPT	WM	PS
MW Trait1				X				X						X							X						0.596
MW Trait2				X				X						X							X						0.920
MW TBRR1			X				X						X							X							0.944
MW TBRR2			X				X						X							X							0.867
MW Post			X										X							X							0.396
VAS L	X				X						X							X									0.780
VAS N	X				X						X							X									0.865
VAS H	X				X						X							X									0.449
External Trait				X				X						X							X						0.768
VSEARCH L		X				X						X							X								0.901
VSEARCH N		X				X						X							X								0.829
VSEARCH H		X				X						X							X								0.774
Self Inattention										X						X							X				1.000
Self Hyp/Imp										X						X							X				0.721
Teach Inattention									X						X							X					1.000
Teach Hyp/Imp									X						X							X					0.847
CPT d’ B1																	X							X			0.821
CPT d’ B2																	X							X			0.912
CPT d’ B3																	X							X			0.907
Self SCT														X							X						0.808
Teach SCT															X							X					0.550
Digit Forward																									X		0.422
Digit Backward																									X		0.684
Digit Sequence																									X		0.707
Picture Span																									X		0.469
Coding																										X	0.757
Symbol Search																										X	0.653

Note: See Table 2 for variable definitions. Shaded areas indicate that these variables were not included in the models of this column.

Internal/External (12 variables)

The unitary model fit the data poorly (see Table 6, Model 1a; e.g., CFI = .357; RMSEA = 0.239). The proposed model, distinguishing between Internal and External (Model 1b), was a marked improvement (χ2 diff (1) = 294.86, p < .001). Nonetheless, the resulting model fit remained poor (e.g., CFI = .646; RMSEA = .179). A methods-focused 3 factor model (Model 1c) that separated VAS measures, from Visual Search measures, from self-report measures, showed further improvement in model fit (χ2 diff (2) = 190.18, p < .001; CFI = .831; RMSEA = .126), although it remained unacceptable. When self-report measures were further separated according to general tendencies (i.e., traits) or specific behaviors that occurred in the moment (i.e., states), model fit became excellent (see Tables 6 and 7 Model 1d; CFI = .996; RMSEA = .021), with no suggested covariances, reflecting simple structure. As an exploratory analysis, we additionally evaluated bifactor analogues to these Part 1 models, but they did not yield any clearer differentiation of internal and external attention, or alter conclusions, and so are not detailed further.

Adding Behavioral Attention (16 variables total)

When ratings of behavioral attention were added as a single factor to the internal/external Model 1d (4 factors total), the fit was not adequate (e.g., CFI = 0.895; RMSEA = 0.087; see Model 2a in Tables 6 and 7), with widely disparate factor loadings for student versus teacher reports. Model 2b separated teacher ratings (5 factors) and placed student ratings with other self-report trait scales, but this model also was inadequate (e.g., CFI = 0.890; RMSEA = 0.089). However, when student and teacher ratings of behavioral attention were placed on separate factors (Model 2c, 6 factors total), there was substantial improvement (χ2 diff (4) = 151.06, p < .001) and the model fit was excellent (e.g., CFI = .996; RMSEA = .016; see Tables 6 and 7), and highly similar to that of Model 1d. All behavioral models 2a-2c contained negative residuals for one or more behavioral attention ratings (i.e., Heywood cases), and the model was constrained so that the negative residual be set to zero at the cost of a degree of freedom (the model converged whether constrained and unconstrained, and local parameters were substantively similar).

Adding Combined Attention Measures (21 variables total)

The combined measures (of the CPT, and of SCT) were then added to Model 2c, resulting in 7 factors (Model 3a, see Tables 6 and 7). This model did not fit the data well (e.g., CFI = .869; RMSEA = .089). When the CPT and SCT were relegated to their own, separate factors (Model 3b, 8 factors), fit improved, though it was still not acceptable (e.g., CFI = .900; RMSEA = .079; see Tables 6 and 7). This model also required an additional residual set to zero (for the SCT). Therefore, the next Model (3c, 7 factors) placed the teacher-rated SCT with the other teacher ratings (of behavioral attention), and analogously placed the student-rated SCT with student ratings of behavioral attention; the fit here was highly similar to that of Model 3b (e.g., CFI = .899; RMSEA = .078; see Tables 6 and 7). The next model kept all the teacher ratings together, and placed the student ratings of SCT with student ratings of other traits other than behavioral attention, which was kept separate (Model 3d, 7 factors); this resulted in substantial improvement in model fit (e.g., CFI = .983; RMSEA = .032; see Tables 6 and 7); these indices were similar to those of Model 2c and Model 1d.

At this point, the overall model appeared to cluster around individual measures; however, we had not tested the extent to which this model fit would compare to a more parsimonious methods-specific model, where factors were defined based on measure type (Objective, Self-Report, Other-Report; see Table 1). This model (3e, 3 factors) fit the data poorly (e.g., CFI = .618; RMSEA = .145; see Tables 6 and 7). This model suggested several (six) large modifications, which all reflected within-measure relations (e.g., covariances between the two TBRR measures; covariances of visual search tasks with one another). With these changes, the result (not shown in Table 6) showed improvement relative to Model 3e, and fit was acceptable (e.g., χ² = 263.56; CFI = .962; RMSEA = .046). Nonetheless, this was a weaker fit than that of Model 3d, despite having 11 more degrees of freedom. Therefore, Model 3d was chosen as the penultimate model, and was the final attention-only model.

Addition of Other Constructs (27 variables total)

As a final step, we added (separate) factors for working memory and for processing speed to Model 3d (yielding Model 4, which has 9 factors). This final model fit the data well (e.g., CFI = .971; RMSEA = .034; see Tables 6 and 7), with each of these new factors identified and separable from the prior attentional factors; the overall fit was similar to those of Model 3d, 2c, and 1d as well. This model fit similarly whether or not the “Digits Forward” measure was included with the other WM indicators. Table 7 also displays factor loadings for the final 9 factors, which attained simple structure. Table 8 displays factor intercorrelations for Model 4. Most were weak to moderate in size (e.g., of the 36 correlations, 32 were below r = .28).

Table 8.

Factor Correlations for Latent Variables of Final Model 4

	1	2	3	4	5	6	7	8
1. VAS	--
2. VSEARCH	.237	--
3. SR_STATE	.085	−.055	--
4. SR_TRAIT	−.054	−.117	.517	--
5. T_BEH	−.271	−.178	.049	.222	--
6. SR_BEH	−.108	−.060	.016	.011	.096	--
7. CPT	.364	.206	−.109	−.057	−.128	−.053	--
8. WM	.359	.214	.012	−.111	−.202	−.085	.225	--
9. PS	.241	.627	−.088	−.096	−.260	.089	.217	.219

Note: See Table 2 for variable definitions.

Post-hoc Models (15 variables)

We ran additional models not represented in Tables 6 or 7. The first (Model 5a) included only one indicator per test to determine whether prior models may have been overfit due to the inclusion of multiple variables from similar performance measures. This model, with 6 hypothesized factors (Internal, External, Behavioral, Combination, WM, PS), did not converge. A subsequent model (Model 5b) eliminated the Combination factor, and distributed the CPT to the External factor, and collected the (non-mind wandering) self-reports and teacher reports together in addition to the WM and PS factors (5 factors). This model showed adequate fit (CFI = .912; TLI = .877; RMSEA = .053; SRMR = .057), with reasonable factor loadings (median = .59), but with high factor intercorrelations (Internal with Self-Report, r = .96; External with PS, r = .82). Combining all student self-reports together (Model 5c) solved some issues, and evidenced similar model fit as Model 5b. However, this methods-focused model was less adequate than Model 4 (and Model 5c was in essence not substantively different from Model 4).

Discussion

The goal of the present study was to evaluate the structure of attention, based on the Chun et al. (2011) taxonomy, which distinguishes between “internal” and “external” attention. We further sought to determine whether and where behavioral attention fits within this taxonomy, as well as to test how constructs closely related to attention (processing speed and working memory) relate to attention factors. In all, little support was found for the anticipated taxonomic structure, at least with these indicators, in this sample. Instead, the best fitting models were clearly those that segregated along methodological and specific measurement characteristics, and resulting factors showed weak correlations with one another.

Our results do not align well with prior factor analytic studies of attention, and we were unable to support the external/internal distinction of Chun et al. (2011). There are three specific issues relevant to the current findings, which are discussed in turn, before turning to implications; these include: (a) sample characteristics; (b) selection and/or features of the indicator measures used; and (c) the nature of prior studies.

Sample

There is little question that the present sample differs from samples of most prior factor analytic studies, given the age of the sample, the fact that all were Hispanic, that many were struggling readers, and the fact that approximately half were designated by their school as limited English proficient. However, we know of no theoretical reason to expect differences in the structure of attention (i.e., covariances among model parameters) based on these characteristics. The objective measures involved quite simple manipulations, and the general instructions were straightforward. Questionnaires were completed with examiners, such that questions could be read to students, and they did not contain long or difficult phrases. Finally, differences in levels of performance should not be confused with differences in covariance structure. Preliminary analyses did not suggest that sample characteristics, or reading or language levels, drove attentional performances. Of course, it would be important to test similar models in samples with other characteristics, but the present study provides a key roadmap for doing so.

Indicator Measures

The relevance of the specific measures chosen to represent internal versus external attention was based on Chun et al. (2011), although their taxonomy was developed from the cognitive neuroscience of vision. The measures chosen here, albeit some being from experimental tasks, were generally applied in nature (e.g., from the neurodevelopmental literature, and also with overlap in terms of several prior factor analytic studies of attention). In addition, different and/or additional performance and/or rating measures could have been chosen to more discretely isolate Chun et al.’s components, particularly those of internal attention. While tasks such as switching and response selection are mentioned by Chun et al. (2011), doing so would generate additional issues about the separability of internal attention from executive function, although future studies that do this would likely be beneficial. Nonetheless, the measures chosen do clearly align with the key distinction of having the target of attention be internally versus externally focused. In doing so, the present study provides useful validity data. Further, in addition to the primary goal of choosing tasks because they align with the Chun et al. (2011) theoretical distinctions, all of the measures evaluated here have been shown to have some relation to academic achievement (particularly reading), in line with the broader goals of the overall project (and the nature of the sample), and so tasks chosen based only on cognitive neuroscience or utilizing only performance-based measures would likely have less relevance to this context.

Relation to Prior Factor Analytic Studies

It is not surprising that results do not align with prior work on the structure of attention, for at least three reasons. First, fewer of those studies focused on children, fewer use confirmatory techniques, and most evaluated one particular model (that of Mirsky) using the same indicator measures across studies, or else evaluate an intact battery, or else include only objective measures, or only ratings measures. The present study purposefully did none of these. Thus, in some ways, the present results are not comparable to prior work. The most closely aligned studies are those of Breckenridge et al. (2013); of Kelly (2000); of Malegiannaki et al. (2015); and of Tao et al. (2017). However, Kelly (2000), Malegiannaki et al. (2015), and Tao et al. (2017) used only performance-based measures, whereas the present study uses both performance as well as rating-based measures. In fact, Breckenridge et al. (2013) was the only previous study to use both types of measures. Breckenridge et al.’s (2013) study, however, only evaluated a single battery of attention tests, and Malegiannaki et al.’s (2015) evaluated a single test, meaning that method bias could not be evaluated in these studies. As the current study employs multiple tests from multiple batteries, we were able to evaluate method bias directly. Further, Kelly (2000) had a small sample size and employed measures considered to assess non-attention constructs, similar to other studies evaluating Mirky et al.’s (1991) model of attention. Only the Malegiannaki et al. (2015) study used the confirmatory approach to factor analysis used in the present study. Finally, in some studies (e.g., Goldhammer et al., 2007), attentional factors were each composed of two indicators from five separate tasks, making it unclear whether these were conceptual or methodological factors.

Second, the present explicitly separated attentional factors from closely related constructs (in our case, processing speed and working memory). Several prior studies include multiple complex measures, whose focus could be considered to be more closely aligned with executive function as opposed to attention. In many such studies (e.g., Breckenridge et al., 2013; Shannon et al., 2021; Tao et al., 2017), these measures tended to cluster separately from more elemental attention components. It is also relevant that most studies of the Mirsky model predate key executive structural studies, and Mirsky et al. (1991) noted that considering tasks of switching as measures of attention rather than executive function was debatable and perhaps subject to later revision; the results of the present study would support this separable but related relationship among these constructs. There is a need for future work to continue to delineate where one domain general process “ends” and another “begins”.

Third, and in corollary with the comparison with prior factor analytic work, no prior study has evaluated the Chun et al. (2011) taxonomy of external and internal attention in factor analytic fashion. In this sense, it is not possible to compare our results to prior work. The issues raised above regarding the particular measures chosen are relevant here. Obviously, more work is needed, but the present study clearly raises interesting questions, attempts to validate a theoretical structure, sheds light on extant studies, and provides a template for issues that need to be considered in evaluations of the structure of attention.

Implications

The present study has both theoretical and practical implications. From a theoretical perspective, and taken to their logical extreme, the results might question the construct of attention per se. In fact, this has been proposed previously. For example, others have argued that the concept of attention is not useful or viable (Anderson, 2011; DiLollo et al., 2018; Hommel et al., 2019). A common point in such works is that attention should not be considered a unitary phenomenon, and the words we use to describe attention are either/both too broad or too incomplete, even though the word itself may be useful for enhancing communication, similar to the way the term memory is used (DiLollo et al., 2018). Fernandez-Duque and Johnson (2002) for example, juxtapose the oft-cited William James quote, “everyone knows what attention is” (1890/1950, p. 261), with an excerpt from the Pashler attention book (1998, p. 1), “no one knows what attention is…”, which is the title of the Hommel et al. (2019) article.

Another common theme from such works is that theories of attention focus too much on forcing phenomena into expected dichotomies (e.g., exogenous vs. endogenous; pre-attentive vs. attentive; attention vs. intention; in this case, external vs. internal) rather than considering them as continuua. As noted, Chun et al. (2011) might agree with this view despite their taxonomy. In fact, this is one of the reasons why we expected the factors derived to be related to one another. Finally, most works of this type view the construction of attention theory to be more productive when attention is viewed less as a cause of other phenomena, which can lead to circularity (Anderson, 2011; DiLollo et al., 2018), but rather as an effect, or epiphenomenon of more basic processes. Hommel et al. (2019) refer to this as a synthetic (as opposed to analytic approach), but simultaneously note that this approach is not meant to be solely reductionistic, because any neural or biological model must remain consistent with functional/cognitive models. Fernandez-Duque and Johnson (2002) contrast attention-as-cause theories that invoke metaphors (e.g., attention as a spotlight, or as a limited resource) with attention-as-effect theories (competition models) with hybrid theories (biased competition models). However, they note that although hybrid models do not remove the attention “homunculus”, metaphors are still necessary because they help to organize data and assist in sense making, and likely would continue to do so even if there were working neurocomputational models.

Most of the above works generally limit their perspective to the cognitive neuroscience of visual attention. The present study makes many of the same assumptions but uses the relatively unique sample and broader measures to address a broader range of questions involving neurodevelopmental disorders and common functional outcomes for children. We agree with the general ideas that attention as a word is most useful for organizing communication, that dichotomies are much more likely to be continuua, and that there is need to focus on the specific stimulus parameters and cognitive and biological mechanisms that give rise to attentional phenomenon. More bluntly, our results underscore the need for a more careful consideration of the relationship of constructs discussed and the measures used in their operationalization. If distinctions among types of attention are to be used and be useful, particularly in the latent context, core constructs will need to share measurement features (as well as be differentiated from other constructs). We do not argue from hegemony, but consensus at a fundamental level will be necessary to a more nuanced understanding of attention as a meaningful construct. This is particularly important for constructs such as attention that can be considered a “hub” or one involved in many other cognitive processes, therefore making it potentially confusable with most other domains of cognition.

One positive aspect of the present work was the fact that the working memory and processing measures chosen here were separable from one another, as well as from the “attention” latent variables, maintaining the simple structure of the overall model. Granted, for these additional constructs we used indicators that have been well-established and already known to be differentiated from other factors (e.g., they are subtests from a very widely used IQ test, the WISC-5) – we did so because they were not the focus of this work. But it is recognized that both working memory (e.g., see Baddeley, 2012; Barak & Tsodyks, 2014; Wingfield, 2016) and processing speed (Gerst et al., 2021; Papadopoulos et al., 2018) can themselves be conceptualized in myriad ways. For example, the PS measures here would align with “complex” PS as noted by Gerst et al. (2021). It is also interesting that the Mirsky model features numerous “attention” measures that the present study has labeled as WM and PS. Thus, it may be less relevant how the measures here cluster together than evaluating what factors produce individual differences in these measures, and which measures relate to more functional outcomes and why. For example, if mind-wandering measures are reliable, separable from other measures, and uniquely predictive, they can be used to inform our understanding of important functional outcomes (e.g., reading comprehension or reasoning, as they often are – Schooler et al., 2004; Smallwood et al., 2008), then that focus may be more important than in evaluating where “mind-wandering” fits in a taxonomy of attention.

Taken further, from a practical standpoint, the present work also highlights a key need to integrate different types of information. It is well known that cognitive and rating measures do not always align, and this has been demonstrated in the domains of executive function (Gerst et al., 2017; Toplak et al., 2013) as well as in individual studies of attention (Macdonald et al., 2020; Sims & Lonigan, 2013). However, it is as yet unclear what level of (weak) relation is necessary to consider these subdomains separate constructs. This key distinction can be made both empirically and experimentally. For example, what makes a group of measures one construct versus two (regardless of their name, or rationale behind their development) is not only the weakness of their correlation, but also how they relate to other factors. In other words, a key issue is whether they have different nomological networks (Cronbach & Meehl, 1955). Experimentally, a practical goal, once correlational relations are established, is to then be able to manipulate these in the service of some larger goal. For example, in the context of struggling readers, such as those included in the current sample, what would be key is evaluating and demonstrating that one or more of the latent variables established here has effects on whether or not a student responds to a reading intervention. Such studies have not yet been conducted. Any such effects would likely need to be expressed by adding these to existing content-focused interventions, as transfer effects of cognitive training are known to be weak (Cortese et al., 2015; Luis-Ruiz et al., 2020; Rapport et al., 2013 Sonuga-Barke et al., 2013).

Conclusions

The present study contributes to the literature by addressing the structure of attention from a theoretical perspective, by addressing weaknesses in prior factor analytic studies of attention, and by contextualizing results against other related constructs. The results demonstrated that the structure of attention measures segregate clearly and consistently along the lines of their task parameters, rather than along the broader theoretical lines of Chun et al. (2011) – or we expect, along the theoretical lines of any other broad cognitive theory of attention. The focus moving forward should be two-fold. First, the present work is a single study, and novel in its attempt to model a taxonomy, and therefore it is necessary to consider a broader and different range of measures and constructs to better understand the extent to which measures that address attention aggregate and disaggregate from one another, as well as from closely related non-attention constructs, and to evaluate the developmental trajectory and stability of individual differences, with varying populations. It is possible that some more elemental (and perhaps, more concrete) individual differences could give rise to the individual differences on these various measures of attention. Second, from amongst the specific measures and factors evaluated here, it is necessary to focus on those specific factors that in turn have strong practical value in predicting or influencing functional outcomes relevant to children and adolescents.