Examining the Dimensionality of Effortful Control in Preschool Children and its Relation to Academic and Socio-emotional Indicators

Abstract

Effortful control (EC) is an important developmental construct, associated with socio-emotional growth, academic performance, and psychopathology. EC is defined as the ability to execute goal directed behavior to inhibit or delay a prepotent response in favor of a subdominant response. Extant research indicates that EC might be multidimensional. Confirmatory factor analysis with a sample of 234 preschoolers was used to determine if tasks designed to measure EC were best described by hot (affectively salient) and cool (affectively neutral) dimensions or by a single factor. Analyses revealed that EC is best described by a single factor, even when variance associated with children’s language skills was removed. This EC factor was strongly related to measures of academic performance and significantly less related to measures of socio-emotional development.

There is renewed interest in discovering the nature of temperament due to its potential developmental implications. Temperament is defined as constitutionally based individual differences in emotional, motor, and attentional reactivity, and the accompanying self-regulatory processes that modulate this reactivity (Rothbart, 2007). Temperament emerges in infancy and stabilizes throughout preadolescence (Kochanska, Murray, & Harlan, 2000). Individual differences in temperament account for cognitive, social, and academic development differences (Blair & Razza, 2007; Rothbart, Ahadi, & Hershey, 1994), later individual differences in personality (Evans & Rothbart, 2007), and development of psychopathology (see Lonigan, Phillips, Wilson, & Allan, in press; Nigg, 2006).

Temperament is conceptualized in a hierarchical structure, comprised of three broad dimensions, most commonly referred to as extraversion, negative affectivity, and effortful control (EC) (Rothbart, 2007). This higher order structure has been consistently recovered across time and methods (Lonigan & Phillips, 2001; Rothbart et al., 1994). Extraversion and negative affectivity represent individual sensitivity to reward and punishment, respectively, and are considered the reactive aspects of temperament (Posner & Rothbart, 2000). EC represents the ability to delay or inhibit a prepotent response, often in favor of a subdominant response, and is considered the regulatory dimension of temperament (Rothbart & Bates, 2006). Reactive temperament appears early in infancy and stabilizes into childhood (Kagan & Fox, 2006). Regulatory temperament develops over the preschool period, but it is generally measurable by 2-1/2 years of age (Rothbart, Posner, & Kieras, 2006). The hierarchical structure and stability of temperament is important in that it links temperament to hierarchical models of personality and psychopathology. However, according to Rothbart (2007) the dynamic nature of the relation between reactive and regulatory factors of temperament is even more important. According to Rothbart (2004), “Using effortful control we can flexibly approach situations we fear, inhibit actions we desire, and activate activities we would prefer not to engage in” (p. 497). Evidence for this conceptualization of EC has been produced from research using behavioral tasks designed to tap the construct. For example, Kochanska et al. (2000) found that children with higher levels of EC were better able to regulate their anger. Other studies have shown that EC allows children to modulate elements of reactive control (e.g., Kochanska, Murray, Jacques, Koenig, & Vandergeest, 1996).

Whereas there is a growing consensus that EC is an important temperamental factor, there is no agreement regarding the exact nature of this construct. Often, the confusion lies in the overlapping definitions for EC and the cognitive construct of executive functioning (EF). Researchers define EF as a regulatory cognitive process that is an integral component of effortful thought and action (Carlson, 2005) and, synonymous to EC (e.g., Rothbart & Bates, 2006), consider it to be “the ability to overcome automatic, prepotent behavior despite the pull of previous experience” (Garon, Bryson, & Smith, 2008, p. 33). Although several dimensional models of EF have been proposed (e.g., Barkley, 2001; Duncan & Owen, 2000), factor analytic methods have uncovered at least three common dimensions in adolescents and adults: working memory, response inhibition, and shifting (Lehto, Juujärvi, Kooistra, & Pulkkinen, 2003; Miyake et al., 2000).

The similarities between EC and EF are most apparent when considering the response inhibition dimension of EF, defined as the flexible, cognitive suppression of thought processes or actions that are not goal relevant (Blair, Zelazo, & Greenberg, 2005). Tasks developed to measure response inhibition correlate with questionnaires designed to measure EC (Blair & Razza, 2007). Laboratory tasks designed to measure response inhibition are often the same as those used to measure EC and studies often utilize findings from one research tradition in support of the other (e.g., Garon et al., 2008; Kochanska et al., 2000). Similar neural substrates, commonly involving the anterior cingulate cortex, have been identified in response inhibition and EC tasks (Fan, Flombaum, McCandliss, Thomas, & Posner, 2003; Garavan, Ross, Murphy, Roche, & Stein, 2002). Given these similarities, EC and EF are highly related, if not identical, constructs that come from two different research traditions, linked to the same neural substrates, and require the same response capacities (i.e., inhibition and delay).

Attempts to bridge the domains of temperament and cognitive research offer a novel and intriguing way to consider EC composition. The concept of hot and cool dimensions has been proposed in EF literature (Zelazo & Müller, 2002). Hot EF is considered necessary to perform tasks in situations involving affect or emotion regulation, such as when there is a reward or punishment associated with the task (Kerr & Zelazo, 2004). In contrast, cool EF is needed in more decontextualized, abstract situations such as when a task requires action from a participant but has no proximal or specific reward or punishment attached to task performance (Metcalfe & Mischel, 1999; Zelazo & Müller, 2002). It is easy to extrapolate hot/cool dimensionality in EC from the EF delineation. The distinction is in whether a task is considered to require a high level of affective regulation or not. Typically, tasks requiring high levels of affective regulation provide a reward for performance. The child performs with the potential rewards in mind, presumably making the task more affectively salient. In line with the extant research that makes this hot versus cool distinction, hot tasks are operationally distinguished from cool tasks based on whether there is an extrinsic and proximal reward associated with task performance (e.g., Kerr & Zelazo, 2004; Hongwanishkul, Happaney, Lee, & Zelazo, 2005).

Results of a number of EC and EF studies provide initial support for at least two dimensions that are defined by the presence or absence of a salient affective component of the tasks. Carlson and Wang (2007) examined EF in preschoolers, using several tasks that included Simon Says (a child is told only to respond when prompted by “Simon says”), Gift Delay (a child is told to ignore a present being wrapped by the researcher), and Forbidden Toy (a child is told not to play with a desirable toy). Simon Says, because it has no extrinsic rewards for performance, can be classified as a cool task; because Gift Delay and Forbidden Toy both have extrinsic reward components they can be classified as hot tasks. As would be expected if there were hot and cool dimensions of EC, Simon Says performance did not correlate significantly with performance on either Forbidden Toy (r = .09, p > .09), or Gift Delay (r = .19, p > .05); in contrast, Forbidden Toy and Gift Delay correlated significantly (r = .43, p <. 01). Murray and Kochanska (2002) did not identify hot and cool tasks in a principal components analysis (PCA) of EC task performance of preschool children; however, examination of the items reported to form their factors reveals that four of the six tasks (e.g., tongue, pinball, snack delay, and gift bag) that comprised their first factor could be considered hot tasks because rewards were associated with task performance, and six of the seven tasks (e.g., walk a line slowly, dragon, telephone poles, circle, tower, and whisper) that loaded on the other factors could be considered cool tasks because there was no reward associated with task performance. Using confirmatory factor analysis (CFA), Brock et al. (2009) found a two-factor hot/cool model fit the data better than a single factor model in kindergarten children. EC and overlapping regulatory constructs are associated with positive and negative developmental outcomes. EC is positively related to socio-emotional development such as conscience development (Kochanska et al., 2000) and social adjustment (Derryberry & Rothbart, 1997). EC and related constructs (i.e., EF, self-regulation) have been positively associated with academic performance. Regulatory attributes measured in kindergarten significantly contributed to individual differences in reading skills in the 1st grade (Spira, Storch Bracken, & Fischel, 2005). A behavioral measure of EC predicted preschoolers’ later academic skills and their preparedness for kindergarten (McClelland et al., 2007). Similarly, EC in preschoolers covaried with both math and emergent literacy skills (Blair & Razza, 2007).

EC is also associated with problematic development. Low EC has been linked to Attention Deficit Hyperactivity Disorder (ADHD) cross-sectionally and longitudinally (Goldsmith, Lemery, & Essex, 2004; Willcutt, Doyle, Nigg, Faraone, & Pennington, 2005). Muris, Meesters, and Blijlevens (2007) reported that low levels of EC were associated with youth self-reports of internalizing and externalizing symptoms. Others have found similar links using parent and teacher reports of externalizing and internalizing behavior problems (Eisenberg et al., 2004; Murray & Kochanska, 2002).

Emerging evidence highlights the value of distinguishing dimensions of EC for understanding its developmental correlates. A number of studies have reported data suggesting that different dimensions of EC or EF have different correlates and developmental implications. Researchers exploring the relation between EC and externalizing behavior of 9-year-olds who were prenatally exposed to tobacco found that these children had higher rates of parent reported conduct problems and ADHD symptoms than did controls. They also performed worse than controls on a hot EC task but did not differ from controls in a cool EC task (Huijbregts et al., 2008). These results suggest that there is a stronger relation between performance on tasks designed to measure hot EC and externalizing behavior than between tasks designed to measure cool EC and externalizing behavior.

Results from a number of studies suggest that hot EC is associated with socio-emotional related outcomes and that cool EC is associated with academic-related outcomes. Performance on cool self-regulatory tasks is correlated with and predictive of growth in emergent literacy and math skills in preschoolers (Blair & Razza, 2007; McClelland et al., 2007); however, these correlations have not been found between hot EC and academic performance (Duncan et al., 2007). Performance on a cool EC task is associated with general intelligence (Hongwanishkul et al., 2005), whereas performances on hot EC tasks are significantly related to scores on measure of emotional and social intelligence but not general intelligence (Bar-On, Tranel, Denburg, & Bechara, 2003; Hongwanishkul et al.).

Researchers have theorized that EC or EC tasks themselves might have dimensional properties other than hot/cool. Nigg (2000) suggested that EC tasks might vary as a function of their cognitive versus motor task demands. That is, tasks primarily involving inhibition of only internal behaviors such as thought (cognitive) to perform a task would be considered different compared to tasks primarily involving inhibition or delay of motor or vocal responses. Carlson, Moses, and Breton (2002) argued that EC could be distinguished by whether the response involves delaying an action (delay) versus withholding a prepotent response in favor of a subdominant response (conflict). Conflict tasks were viewed both as more complicated and as having greater working memory demands than delay tasks (Carlson et al., 2002). It has also been suggested that tasks may load on a single factor (Wiebe, Espy, & Charak, 2008).

The lack of an adequately defined EC construct compromises research on its relations to other developmental factors like socio-emotional and academic development. Little work has been done to refine the EC construct. Using (PCA) to analyze children’s performance on a battery of EC tasks, Murray and Kochanska (2002) reported that EC was a complex, multidimensional construct in preschool children. However, because PCA was used, it is difficult to extrapolate a definitive model of EC. PCA does not distinguish between common and unique variance, therefore conflating common variance with potential task-related error (Fabrigar, MacCallum, Wegener, & Strahan, 1999). Moreover, PCA models are under-identified and can have many potential solutions that fit the data equally (Maruyama, 1998). CFA is preferred in studies such as this, as there is a specific hypothesis to test and multiple fit indices can be applied to verify model fit (Hurley et al., 1997). Another benefit of CFA is the ability to examine the influence of potential moderators that affect model parameters. Age differences in exhibited levels of EC have been detected (Kochanska et al., 2000). These differences are likely to manifest themselves in the preschool years, during which EC is rapidly developing. Gender differences have also been detected in exhibited levels of EC, with girls showing higher levels of EC than do boys (see Else-Quest, Hyde, Goldsmith, & Van Hulle, 2006). Testing for measurement invariance using CFA can determine if developmental and gender-specific factors in EC affect dimensionality in addition to absolute performance levels.

Current Study

The purpose of this study was to examine the dimensionality of EC in preschool children and to explore the potential differential relations between these dimensions of EC and both socio-emotional and academic outcomes. Eight tasks previously used in the literature were selected to represent hot EC and cool EC. These tasks were additionally selected to represent the two other proposed dimensions of EC (i.e., cognitive/motor, delay/conflict). CFA was used to compare the adequacy of models representing these different dimensions. We hypothesized that the eight EC tasks would be best described by a two-factor hot/cool model. We further hypothesized that tasks representing the cool dimension of EC would correlate more highly than tasks representing the hot dimension of EC with measures of academic performance administered to these same preschool children and that the opposite pattern of correlations would be obtained for measures of socio-emotional development collected for these same preschool children.

Method

Participants

Children were recruited from 16 preschools in Northern Florida. Facility directors signed agreements allowing for data collection at their facilities, and parents of 234 preschool children (118 boys, 116 girls) signed consent/permission forms allowing their children to participate in this study. There was no attrition in this sample. Children ranged in age from 36 to 71 months (M = 51.08 months, SD = 7.59 months; 86 3-year-olds, 121 4-year-olds, 26 5-year-olds). The sample was racially diverse, with 65% Caucasian, 27.4 % African American, 2.6% Asian, 1.3% Hispanic, and 3.8% failing to report race or indicating multiracial. Average family income ranged from $5,000 to $187,500 per year (M = $63,329, SD = $40,436), with 29 families not reporting income.

Measures

As part of a larger study, all children completed a standardized measure of early language and literacy skills and eight tasks designed to measure EC. Additionally, teachers completed a measure of children’s socio-emotional functioning. The eight EC tasks were selected to represent the three dimensional variations, hot-cool, motoric-cognitive, and delay-conflict. A listing of the EC tasks, the battery in which they were administered, and their classification on the three dimensions are shown in Table 1.

Table 1.

Classification of Effortful Control Tasks Used in Study and Order of Administration

Tasks	Battery and Order	Hot/Cool	Motor/Cognitive	Delay/Conflict
Box Search	B-1	Hot	Motor	Conflict
Delay of Gratification	B-4	Hot	Cognitive	Delay
Less is More	A-1	Hot	Cognitive	Conflict
Gift Delay	A-4	Hot	Motor	Delay
Grass/Snow	B-2	Cool	Cognitive	Conflict
Head to Toes	B-3	Cool	Motor	Conflict
KRISP	A-2	Cool	Cognitive	Delay
Walk a Line Slowly	A-3	Cool	Motor	Delay

Hot EC Tasks

Box Search task

This task followed the protocol of Simpson and Riggs (2007). Twenty different boxes, half with squares on the lid and half with triangles on the lid, were presented to children (4 practice trials and 16 test trials). Children were told to open boxes with squares on the lid and to leave shut boxes with triangles on the lid. Children were given three seconds to respond to each box (Simpson & Riggs concluded that if there were no response within 3-seconds, a child was unlikely to respond). Each box was scored on a pass/fail basis, with 1 point awarded if the child opened the correct box or left closed the incorrect box.

Delay of Gratification “Shift” task

This task was modeled after Hongwanishkul and colleagues’ (2005) version. Eleven different immediate versus delayed choices were presented to children (2 practice trials and 9 test trials). Children were given the opportunity to win immediate or delayed rewards of three types (stickers, plastic tops, eraser toppers). The experimenter presented nine test trials in random order, and asked the child if they would like to have the smaller immediate prize or the delayed but greater in number prize (i.e., one eraser now or six erasers later). Upon selection, the reward was either given immediately to the child or placed in a brown bag for later if the child selected the larger, delayed prize. Children received 1 point for each delayed choice made.

Less is More task

This task was based on the work of Carlson, Davis, & Leach (2005). Stickers were placed in two piles to children’s left and right with five stickers in one pile and two stickers in the other pile. Children were asked which pile they preferred (all children preferred the pile with five stickers). After explaining that the purpose of the game was to get as many stickers as possible, children were introduced to a stuffed bear and told that the bear likes to get all the stickers for himself (Carlson et al.). The experimenter explained to children that each time they are asked to make a choice between piles, the group of stickers they pointed to went to the bear, and the ones they did not point to went to them. Following one practice trial, 16 test trials were administered (half of all trials had the arrangement of five stickers on the left, determined randomly). Children received 1 point for each time they pointed to the correct (smaller) sticker pile.

Gift Delay task

This task was based on the design and methods of Kochanska et al. (1996). Children were told that the experimenter wanted to give them a present, but had to wrap it first. Children were asked to sit in a chair and remain facing away from the experimenter so the prize could be wrapped. The experimenter then went through a standardized 60 second “wrapping” procedure and then gave children their present. If a child turned his or her head more than 90 degrees, ostensibly to peek, the time that this first occurred was recorded. If the child turned his or her entire body around before 60 seconds elapsed, this time was recorded as well. Children were given up to three warnings for this behavior with a “no peeking” prompt. Latency to peek over shoulder and latency to turn around and peek scores were aggregated to create a composite score (Kochanska et al., 1996).

Cool EC Tasks

Grass/Snow task

The Grass/Snow task was developed by Carlson and Moses (2001). Children were told to point to a green block when the experimenter said “snow” and to a white block when the experimenter said “grass.” Children were given 2 practice trials and 16 test trials. No points were given if the child failed to point, 1 point was given if the child incorrectly pointed, 2 points were given if the child self-corrected by first pointing to the wrong block and then without experimenter correction, immediately pointed to the correct block without any further switching, and 3 points were given if the child pointed only to the correct block..

Head to Toes task

This task was based on the task developed by Cameron Ponitz et al., (2008). Children were told that they were supposed to do the opposite of what the experimenter told them to do. When told to touch their toes, children were told that they were supposed to touch their heads instead (and vice versa). For each trial, 2 points were given if the child responded correctly; 1 point was given for a self-correct trial; 0 points were given for an incorrect response.

Kansas Reflection-Impulsivity Scale for Preschoolers (KRISP)

The procedure for this task was based on the protocol of Carlson and Moses (2001). Children were asked to find the match of a picture from an array of 4 to 6 other pictures. All but one of the pictures differed from the target picture in minor ways. Children were given 5 practice trials and then 15 test trials. If children pointed to an incorrect picture they were told that it was incorrect and to choose again. Children were given up to three opportunities to identify the correct picture. Scoring was done by calculating the reversed total of number of errors (Murray & Kochanska, 2002).

Walk a Line Slowly

The design and procedure of Kochanska et al. (1996) for this task were followed. A 2.5-inch by 12-foot piece of blue painter’s tape was laid in a straight line across the floor. Children were instructed to first walk normally from one side of the tape to the other. Two test trials followed, during which children were asked to walk across the line as slowly as possible. Scoring for this task was the mean time to walk the line on the slow trials.

Test of Preschool Early Literacy (TOPEL)

The TOPEL (Lonigan, Wagner, Torgesen, & Rashotte, 2007) was used as a measure of academic skills in preschoolers. It consists of three subtests designed to measure three primary emergent literacy skills. The Print Knowledge (PK) subtests measures print concepts, letter discrimination, word discrimination, and letter-name and sound discrimination. The Definitional Vocabulary (DV) subtest measures the ability to define words and use single word spoken vocabulary. The Phonological Awareness (PA) subtest measures the developmental continuum of phonological awareness skills. Together, these skills provide a good measure of academic abilities. The TOPEL provides standard scores that were developed based on a large normative sample. Internal consistency reliability for these subtests ranges from .86 to .96 for 3- to 5-year-old children.

Social Competence and Behavior Evaluation--Short Form (SCBE-30)

The SCBE-30 (LaFreniere & Dumas, 1996) is a 30-item teacher-report short form designed to measure behaviors in 30- to 78-month old children (LaFreniere & Dumas, 1996). It consists of three scales that have inter-rater reliability estimates ranging from .78 to .91. Internal consistency reliability ranges from .80 to .92. Test-retest reliability over a two-week period ranges from .78 to .86. Answers to the SCBE-30 are given on a 6-point Likert scale from strongly disagree (1) to strongly agree (6). The SCBE-30 yields three subscales, Prosocial, Externalizing, and Internalizing Behaviors.

Procedure

Each child was tested in a quiet location during school hours at his or her preschool. All testers were trained individuals who had either completed a bachelor’s degree or were in the process of completing a degree. Testing took place over three sessions, with each session lasting roughly 30 to 45 minutes. Sessions were separated by at least several days, and the average duration between the first and third session was 40 days (SD = 29 days). The TOPEL was administered to each child in the first session, followed by one of two EC batteries in the second session and the other EC battery in the third session. Tests were administered in the same order within each battery and were scored during testing. Children’s preschool teachers completed a SCBE-30 on each child in the study that was in her classroom concurrently with assessment of the children.

Results

Descriptive Statistics and Preliminary Analyses

Descriptive statistics, reported on the raw data, for children’s performance on the eight EC tasks are presented in Table 2. Although outliers were found in the data, CFA models with outliers corrected did not differ from CFA models in which outliers were not corrected; therefore, outliers were not corrected in the final CFA models. All tasks except for Delay of Gratification and Head to Toes had significant skew (p < .05; see Table 2). Logarithmic transformations were used to correct for skew in the Walk a Line Slowly and Grass/Snow tasks. Inverse transformations were used to correct skew for Box Search and Gift Delay. Square root transformations were used to correct for skew in Less is More and KRISP. These transformations resulted in distributions of variables without significant skew.

Table 2.

Descriptive Statistics for Effortful Control Tasks, Emergent Literacy Measures, and Behavior Ratings

Tasks, Measures, and Ratings	Mean	SD	Range		Skew	Kurtosis	Reliability
EC Tasks			Minimum	Maximum
Box Search	13.70	3.23	7	16	−1.00	−.77	.94a
Delay of Gratification	4.32	3.17	0	9	.03	−1.37	.88b
Less is More	10.60	4.37	0	16	−.44	−.76	.83a
Gift Delay	34.09	10.38	3.67	41.67	−1.21	.36	.82b
Grass/Snow	24.79	6.06	10	30	−1.11	−.01	.77a
Head to Toes	11.35	7.57	0	20	−.35	−1.51	.79a
KRISP	32.09	6.32	9	44	−.78	.54	.72b
Walk a Line Slowly	13.88#	7.23	2.50	41.95	1.51	2.63	.77 b
TOPEL
Phonological Awareness	99.78	17.13	54	142	−.17	−.37	.87 b
Print Knowledge	102.39	16.15	69	145	.45	−.55	.95 b
Definitional Vocabulary	103.95	12.99	58	128	−1.10	.84	.94 b
SCBE-(30)+
Externalizing	22.40	10.96	10	60	.93	.22	.90b
Internalizing	17.86	7.97	10	60	1.43	2.43	.84b
Prosocial	40.22	10.27	11	60	−.12	−.51	.90b

Note. N = 234.

⁺N = 231.

KRISP = Kansas Reflection-Impulsivity Scale for Preschoolers. TOPEL = Test of Preschool Early Literacy. SCBE = Social Competence and Behavior Evaluation: Short Form.

^#Time in seconds.

^aSplit-half reliability.

^bCronbach’s alpha.

ANOVAs were conducted to examine gender differences on the EC tasks. Differences emerged for the KRISP task, with girls (M = 33.06, SD = 5.75; reported as reverse-scored for errors) making fewer errors than did boys (M = 31.14, SD = 6.72), F(1, 232) = 5.46, p = .02, and for Gift Delay task, with boys (M = 33.07, SD = 10.84) having lower peeking scores (indicating more peeking) than did girls (M = 35.13, SD = 9.83), F(1, 232) = 4.78, p = .03. Correlations were conducted to examine the relation between age and performance on EC tasks. With the exception of Delay of Gratification, all EC tasks were significantly correlated with age (see Table 3).

Table 3.

Correlations among Effortful Control Tasks and Age and Factor Loadings from One-Factor Model

Tasks	Box	Delay	Less	Gift	Grass	Head	KRISP	Walk
Box Search	--
Delay of Gratification	−.08	--
Less is More	.43***	−.07	--
Gift Delay	.35***	−.07	.27***	--
Grass/Snow	.42***	−.07	.33***	.31***	--
Head to Toes	.57***	−.01	.43***	.40***	.57***	--
KRISP	.48***	−.03	.48***	.35***	.45***	.52***	--
Walk a Line Slowly	.30***	−.02	.29***	.20***	.26***	.32***	.28***	--
Standardized Loadings	.69***	--	.60***	.53***	.65***	.78***	.68***	.43***
Standardized Residuals	.53	--	.64	.72	.58	.39	.54	.82
Child Age	.55***	.07	.43***	.35***	.48***	.57***	.56***	.30***

Note. KRISP = Kansas Reflection-Impulsivity Scale for Preschoolers.

^***p ≤ .001.

Descriptive statistics for the TOPEL and SCBE-30 are shown in Table 2. TOPEL scores were standardized. Scores on the TOPEL and ratings on the SCBE-30 were analyzed for missing values, outliers, and non-normality. There were no missing values on the TOPEL. Because scores on the Definitional Vocabulary subtest departed significantly from normal due to skew, an inverse transformation was conducted on this variable. There were 19 items missing at random in the SCBE-30 across the entire dataset and ratings of child behavior were not returned for three of the children. Children who did not receive SCBE-30 ratings were not used in analyses involving the measure. Multiple imputation was used for missing data of children that had one or two missing values across the 30 SCBE items. Scores on the Externalizing and Internalizing Behavior subscales of the SCBE-30 departed significantly from normal due to skew. A logarithmic transformation was used to correct for skew in Externalizing Behavior and an inverse transformation was used to correct for skew in Internalizing Behavior. ANOVAs were conducted to examine gender differences on the three TOPEL and three SCBE-30 subscales. There were no significant gender differences on the TOPEL subscales (all ps > .05). On the SCBE-30, reported internalizing problems were higher for boys (M = 19.44, SD = 9.16) than they were for girls (M = 16.24, SD =6.18), F(1, 229) = 9.70, p < .01.

Confirmatory Factor Analyses: Models Examined

Four a priori models for the EC tasks were examined using CFA. Model 1 was a single-factor model in which all EC tasks were explained by a single factor. Model 2 was a two-factor model in which the Hot factor consisted of the Box Search, Delay of Gratification, Less is More, and Gift Delay tasks, and the Cool factor consisted of the Grass/Snow, Head to Toes, KRISP, and Walk a Line Slowly tasks. Model 3 was a two-factor model in which the Motoric factor consisted of the Box Search, Gift Delay, Head to Toes, and Walk a Line Slowly tasks, and the Cognitive factor consisted of the Delay of Gratification, Less is More, Grass/Snow, and KRISP tasks. Model 4 was a two-factor model in which the Delay factor consisted of the Delay of Gratification, Gift Delay, KRISP, and Walk a Line Slowly tasks, and the Conflict factor consisted of the Box Search, Less is More, Grass/Snow, and Head to Toes tasks.

CFAs were conducted with MPlus version 5.1 (Muthén & Muthén, 2008) using Full Information Maximum Likelihood to address missing values and the Satorra-Bentler Scaled Chi-square (S-B χ²) for adjustments to correct standard errors for non-normality. The raw data file was analyzed. The S-B χ² was used to test overall model fit. A nonsignificant S-B χ² value indicated that the overall test of model fit was acceptable. The S-B χ² statistic significance test and several fit indices, described below, were used to determine if the different two-factor models (Models 2 – 4) provided significantly better fit than the one-factor model. The model that was the most parsimonious and was the best fit within the indicated parameters was chosen when more than one model fit the data well based on the fit indices. Following model comparison, model parameters were examined to determine whether poor indicators affected the model, as indicated by nonsignificant path coefficients or other substantive issues that might have affected model fit.

In these models, a comparative fit index (CFI) greater than or equal to .95 and a root-mean-square error of approximation (RMSEA) less than or equal to .06 (Hu & Bentler, 1999) indicate a well fitting model. Lower Akaike’s information criterion (AIC) and Bayesian Information Criterion values indicate better model fit (Miyake et al., 2000; Raftery, 1993). AIC and BIC can be used to compare non-nested models (i.e., to compare Models 2 – 4 to each other).

Correlations between measures of EC (see Table 3) indicated that most measures were significantly related to each other. An exception was that the Delay of Gratification task was not related to any of the other measures of EC (rs = −.07 to −.02). As the hypothesis was based on using tasks that measured EC and the Delay of Gratification task did not appear to measure the same construct as the other EC tasks or measured the construct poorly, it was dropped from further analysis. Removing Delay of Gratification from the model did not significantly affect global model fit.

Fit indices for the four a priori models revealed that the one-factor model fit the data well. The S-B χ² difference test indicated that none of the two-factor models resulted in a significant improvement in model fit over the one-factor model, and the overall fits of the different two-factor models were similar to each other (see upper panel of Table 4). Standardized factor loadings for the one-factor model as well as residuals are presented in Table 3. All factor loadings were significant (p < .001) in the one-factor model.

Table 4.

Goodness of Fit Indices and Model Comparisons for Confirmatory Factor Analyses Models

Model	S-B χ2	df	CFI	RMSEA	AIC	BIC	S-B χ2 dif.
	Models Using Raw Scores on Effortful Control Measures
One-Factor EC	22.68	14	.98	.05	3186	3258	--
Hot/Cool EC	20.44	13	.98	.05	3185	3261	1.92
Motoric/Cognitive EC	20.29	13	.98	.05	3185	3261	2.38
Delay/Conflict EC	22.42*	13	.98	.06	3188	3264	.04
	Models Using Language-Adjusted Scores on Effortful Control Measures
One-Factor EC	21.57	14	.98	.05	4274	4347	--
Hot/Cool EC	19.37	13	.98	.05	4273	4349	2.06
Motoric/Cognitive EC	19.25	13	.98	.05	4274	4350	2.32
Delay/Conflict EC	21.35	13	.98	.05	4276	4352	.01
	Exploratory Models Suggested by Modification Indices
Exploratory Two-Factor EC	15.65	13	.99	.03	3181	3257	7.68*
Exploratory Three-Factor EC	11.39	11	1.00	.01	3180	3263	3.89

Note. EC = Effortful Control. S-B χ² = Satorra-Bentler χ². CFI = Comparative Fit Index. RMSEA = Root Mean Square Error of Approximation. AIC = Akaike’s Information Criterion. BIC = Bayesian Information Criterion. S-B χ² dif. = χ² difference test between nested models.

Two-factor models were compared to the one-factor model. The Three-Factor model was compared to the Two-Factor model which was compared to the One-Factor model.

^*p ≤ .05

To evaluate the possibility that the EC tasks might have been represented as a single factor primarily due to children’s ability to understand the verbal instructions associated with each task, a second set of models were examined in which scores on the DV subtest of the TOPEL, as a index of children’s language skills, were regressed out of all the EC tasks. Results from the models using language-adjusted EC tasks are shown in the middle panel of Table 4. As can be seen in the table, results of these analyses were nearly identical to results using the raw scores on the EC tasks. The one-factor model fit the data well, and none of the two-factor models resulted in a significant improvement in model fit over the one-factor model.

Multiple-group analyses were conducted to determine if the model fit differently depending on either gender or age. Age was treated as a dichotomous variable by comparing the youngest half of the children to the oldest half of the children. Models were analyzed in a “step-up” approach as suggested by Brown (2006) in which models were first examined with equal factor structures to determine an equal forms baseline model and then constraints were added to test for equal factor loadings and indicator intercepts. As testing for equality of residual variance is considered overly restrictive (Byrne, 1998), it was not done.

For age, the equality constraints on factor loadings and indicator intercepts did not result in a significantly worse fit of the model, with all fit indices indicating good model fit (S-B χ² = 48.52, p > .05, CFI = .96, RMSEA = .04, AIC = 3053, BIC = 3157, S-B χ² difference = 3.38, p > .05). For gender, the equality constraints on factor loadings and indicators also did not result in a significantly worse fit for of the model with all fit indices indicating good model fit (S-B χ² = 44.06, p > .05, CFI = .99, RMSEA = .03, AIC = 3191, BIC = 3295, S-B χ² difference = 6.70, p > .05). These results indicate that the relations of the indicators of EC to the EC factor were comparable across age and gender.

Exploratory Analyses

Although none of the hypothesized alternate models of EC provided a better fit than the one-factor model, modification indices were examined to determine if any post hoc alternate models would improve model fit. Modification indices provide values for model alterations to improve the χ² statistic. Modification indices revealed one potential model to test with two different levels of changes: first, a two-factor model comprised of Head to Toes and Grass/Snow (EC Factor One) on one factor and the other tasks loading on a separate factor (EC Factor Two; S-B χ² value of 5.81) and, second, a model comprised of KRISP and Less is More as a third factor (S-B χ² value of 4.54). Whereas both exploratory models fit better than the one-factor model, the three-factor model did not fit significantly better than the two-factor model (see lower panel of Table 4).

Association of EC with Academic and Socio-emotional Measures

One-factor model

Correlations between the latent EC variable from the one-factor model and the emergent literacy and socio-emotional measures are shown in Table 5. To examine whether there were significant differences in correlations of EC with TOPEL or SCBE-30, models in which pairs of correlations were constrained to equality were compared to models without the constraint using the S-B χ² difference test (a significant S-B χ² difference indicates that the correlation differs significantly across the variable pair tested). The correlation between EC and the PA subtest of the TOPEL was significantly greater than those between EC and the DV subtest of the TOPEL (Δ S-B χ² = 23.46, p < .05), and the Externalizing Behavior, (Δ S-B χ² = 28.94, p < .05), Internalizing Behavior (Δ S-B χ² = 19.12, p < .05), and Social Competence subscales of the SCBE-30 (Δ S-B χ² = 8.36, p < .05). The correlation between EC and PK subtest of the TOPEL was significantly greater than those between EC and the DV subtest of the TOPEL (Δ S-B χ² = 24.40, p < .05), and the Externalizing Behavior (Δ S-B χ² = 20.09, p < .05), Internalizing Behavior (Δ S-B χ² = 19.45, p < .05), and Social Competence subscales of the SCBE-30 (Δ S-B χ² = 10.67, p < .05). The correlation between EC and the DV subtest of the TOPEL was greater than that between EC and the Externalizing Behavior subscale of the SCBE-30 (Δ S-B χ² = 8.35, p < .05). No other significant differences in correlations emerged.

Table 5.

Correlations between One-Factor and Two-Factor Exploratory Factors and Emergent Literacy Measures and Behavior Ratings

Model	Phonological Awareness	Print Knowledge	Definitional Vocabulary	External Behavior	Internal Behavior	Prosocial Behavior
One-Factor	.35***	.37***	.27***	−.21**	−.14*	.03
Two-Factor: Factor One	.38***	.36***	.18*	−.24***	−.12	.05
Two-Factor: Factor Two	.29***	.35***	.33***	−.17*	−.13	−.01

Note. EC Factor One is the factor with Head to Toes and Grass/Snow loadings. EC Factor Two is the factor with the remaining task loadings.

^*p ≤ .05,

^**p ≤ .01,

^***p ≤ .001.

Two-factor model

Correlations between the two EC factors from the exploratory two-factor model and the emergent literacy and socio-emotional measures are shown Table 5. Across and within factors, models in which pairs of correlations were constrained to equality were compared to models without the constraint. The correlation between Factor One and the PA subtest of the TOPEL was significantly greater than those between Factor One and the DV subtest of the TOPEL (Δ S-B χ² = 31.30, p < .05), and the Externalizing Behavior (Δ S-B χ² = 14.99, p < .05), Internalizing Behavior (Δ S-B χ² = 16.51, p < .05), and Prosocial Behavior subscales of the SCBE-30 (Δ S-B χ² = 6.12, p < .05). The correlation between Factor One and the PK subtest of the TOPEL was significantly greater than those between Factor One and the DV subtest of the TOPEL (Δ S-B χ² = 21.18, p < .05), and the Externalizing Behavior (Δ S-B χ² = 11.40, p < .05), Internalizing Behavior (Δ S-B χ² = 12.26, p < .05), and Prosocial Behavior subscales of the SCBE-30 (Δ S-B χ² = 5.82, p < .05). The correlation between Factor One and the DV subtest of the TOPEL was greater than that between Factor One and the Externalizing Behavior subscale of the SCBE-30 (Δ S-B χ² = 4.31, p < .05).

The correlation between Factor Two and the PA subtest of the TOPEL was significantly greater than those between Factor Two and the DV subtest of the TOPEL (Δ S-B χ² = 16.06, p < .05), and the Externalizing Behavior (Δ S-B χ² = 8.55, p < .05), Internalizing Behavior (Δ S-B χ² = 8.72, p < .05), and Prosocial Behavior subscales of the SCBE-30 (Δ S-B χ² = 5.08, p < .05). The correlation between Factor Two and the PK subtest of the TOPEL were significantly greater than those between Factor Two and the DV subtest of the TOPEL (Δ S-B χ² = 21.50, p < .05), and the Externalizing Behavior (Δ S-B χ² = 12.87, p < .05), Internalizing Behavior (Δ S-B χ² = 11.79, p < .05), and Prosocial Behavior subscales of the SCBE-30 (Δ S-B χ² = 7.50, p < .05). Across factors, the PA subtest of the TOPEL correlated more strongly with EC Factor One than with EC Factor Two (Δ S-B χ² = 4.91, p < .05), and the DV subtest of the TOPEL correlated more strongly with Factor Two than with Factor One (Δ S-B χ² = 6.94, p < .05). No other differences emerged.

Discussion

The results of this study indicate that tasks used to index EC with preschool children are best represented as a single dimension and that this unidimensionality is not the result of the language demands of the tasks. Although we had hypothesized that these EC measures would reflect hot and cool EC dimensions, this study did not support this two-factor model or any of the alternative a priori models, and this single-factor EC model was invariant across age and gender. Although exploratory analyses suggested a multidimensional structure to the EC tasks, there were only two significant differences in the external correlates of these two EC factors, calling into question the significance of this distinction between tasks. The tasks used to measure EC in preschool children in this study were drawn both from the literature primarily concerned with EC and from the literature primarily concerned with EF. That a single dimension best described these measures strongly suggests that the EC and EF are a single construct, emerging from different research domains. Results of this study also demonstrated that although EC was related to both academic and socio-emotional measures, EC was more strongly related to academic skills than to socio-emotional indices.

The results of this study do not support a multi-dimensional EC construct in preschoolers. Of the a priori models analyzed, the unitary construct model fit the data best. Other studies have provided support for a unitary construct model (e.g., Hughes, Ensor, Wilson, and Graham, 2010; Sulik et al., 2010). Rothbart et al. (2001) extracted a single EC factor in a sample of 3- to 7-year-old children on a caregiver report of temperament. Exploring different EF dimensions, Wiebe et al. (2008) found a one-factor model to fit the data best in a preschool population. Factor analytic studies that have found multidimensionality in EC used few tasks, were conducted with older children, and used analyses less stringent than CFA (e.g., Brock et al., 2009; Lehto et al., 2003, Murray & Kochanska, 2000). The evidence supports a one-factor model of EC over any alternate models in preschool children; however, because EC ability continues to develop in children into primary school (Rueda et al., 2004), it is possible that distinct dimensions of EC emerge later in development. Taken together, these findings suggest caution in arguments that hot and cool EC tasks measure distinct dimensions that are differentially important for different developmental outcomes for preschool children.

The model was invariant across age and gender in this study. Others have reported similar results for young children (e.g., Wiebe et al., 2008). For example, gender did not affect model fit in a sample of low-income preschool children (Sulik et al., 2010). As evidenced by a meta-analysis of temperament, girls appear to have greater EC than do boys (Else-Quest et al., 2006). Our results reflect this, as boys performed worse than girls on two of the EC tasks. Researchers have not directly examined EC invariance across age in young children. Kochanska et al. (2000) reported that the correlations between EC measures increase across the preschool period. Their battery of EC measures correlated .44 from 22 to 33 months, .59 from 32 to 46 months, and .65 from 46 to 66 months. A one-factor model fit the data in younger (2 years, 4 months to 3 years, 11 months) and older (4 to 6 years) children (Wiebe et al., 2008). This gender and age invariance provides further evidence for the stability of this construct during this developmental timeframe.

Whereas the one-factor model of EC fit the data better than the a priori two-factor models, modification indices suggested, and post hoc testing demonstrated, that an alternate two-factor model fit the data better. The first factor consisted of tasks that were labeled cool and conflict tasks (Grass/Snow and Head to Toes) and the second factor consisted of tasks that were alternately labeled hot (Box Search, Less is More, and Gift Delay) or delay tasks (KRISP and Walk a Line Slowly). At this time, there is limited evidence for an empirical distinction along these lines. In this study, the only differences related to external correlates of the factors were the relative strength of the correlations with vocabulary versus PA. The structural equation modeling literature cautions against using modification indices as anything but exploratory because this method takes advantage of chance variance present in the data. Further, modification indices can implicate problems with a model that are not the true source of model misfit (Brown, 2006). Despite these modification indices, there is only valid evidence of a single EC factor.

Our study increased precision of the EC construct using a battery of tasks that measure this construct. These tasks have been used in EC and EF research (e.g., Carlson et al., 2002; Kochanska et al., 2000; Wiebe et al., 2008). EC and EF definitions overlap as well, both concerned with the inhibition of some response (Derryberry & Rothbart, 1997, Garon et al., 2008). There is some confusion in the literature about the definition of these and related terms. Blair and Razza (2007) believed EC and EF differed in whether the response was automatic (EC) or cognitive and volitional (EF). In contrast, Eisenberg (2002) argued that EC was volitional and reactive control was automatic but also that these terms shared overlapping elements (Eisenberg et al., 2003). Block (1996) claimed that EC, self-regulation, action control, executive systems, and response modulation among others were identically defined. Based on our findings that EC and EF behavior tasks form a single factor in preschool children and the overlapping definitions used in cognitive and temperament research, a compelling argument can be made that EC and EF, especially response inhibition in EF, are identical constructs in preschool children that have emerged within distinct research domains (i.e., EC from research with a developmental focus and EF from research with a cognitive focus).

Increased precision has important implications beyond the developmental significance of EC. Examining the utility of EC related to other important aspects of development indicated that EC was more strongly related to emergent literacy skills than to ratings of behavior. In this study, EC and emergent literacy both directly measured children’s performance, whereas children’s behavior was indirectly rated by teachers. Teacher ratings are subjective, and teacher reports of emergent literacy skills correlate only moderately with direct measures of these skills (e.g., Cabell, Justice, Zucker, & Kilday, 2009). Teacher reports of EC correlate low to moderate with EC tasks (Blair & Razza, 2007). Given these findings and findings that ratings of preschool children’s behavior by different teachers in the same classroom correlate only moderately (e.g., Achenbach, McConaughy, & Howell, 1987), the higher correlations between EC and emergent literacy compared to behavior ratings were not unexpected and may reflect measurement factors rather than true differences in the relation between EC and other developmental outcomes.

Similar to our findings regarding emergent literacy, other researchers have reported concurrent and predictive relations between EC and emergent literacy skills in preschool and kindergarten children (Blair & Razza, 2007; McClelland et al., 2007). In this study, EC was significantly more related to phonological awareness and print knowledge, which are precursors to decoding, than it was to vocabulary, which is related to general cognitive ability. This finding provides support that the inhibition of a prepotent response uniquely contributes to emerging academic performance over and above cognitive abilities (Blair & Razza, 2007) and is not merely part of the development of general cognitive abilities (Zelazo, & Müller, 2002). The global association between EC and all emergent literacy skills might be explained by the role of EC in motivation and classroom engagement (Rothbart & Jones, 1998).

Our study provided further support for EC as an important correlate for identifying behavior problems, as we found EC was moderately related to measures of externalizing and internalizing behavior. Others have reported similar findings. EC was related to disruptive conduct in preschool children (Kochansksa et al., 2009). It has also implicated in the development of psychopathology (i.e., Martel & Nigg, 2006). Identifying behavior problems in preschool children is difficult because of the reliance on other report measures which are subjective and correlate only modestly across raters (Achenbach et al., 1987). EC has demonstrated stability in preschool children (Kochanska et al., 1996) and therefore could be useful for identifying preschool children at risk for behavior problems.

The Delay of Gratification task did not relate to the other tasks. It is possible that the task was not administered properly, although the task protocol followed that of Hongwanishkul et al. (2005) and the task demonstrated adequate reliability. There was some evidence of a bimodal distribution, with roughly 20% of the children performing correctly on all trials and 20% of the children performing incorrectly on all trials, suggesting that intelligence or age might affect task performance; however, there were no significant correlations between the task and age or IQ. It has been suggested that tasks like this are unreliable because of the difficulty of the child in comprehending the directions (Hughes et al., 2010). It appears most likely that the Delay of Gratification task was not appropriate for measuring EC in preschool children, stressing the importance of using tasks that appropriately measure this construct.

There were several elements that our study could not address. Although the sample in this study was relatively diverse it is possible that EC might vary as a function of SES. A larger sample would be needed to test this. Although no differences were found across age and gender, the study had just adequate sample size to test these questions of invariance. Examining factor loadings revealed that EC accounted for much of task performance; however, examining residuals revealed that there was still much to be explained about what these tasks were measuring. Including measures of working memory and intelligence would allow for extraction of these factors from these tasks and possibly result in a more pure EC construct. Finally, although our results demonstrated significant associations between EC and both academic and socio-emotional measures, because all measures in this study were collected concurrently the direction of influence could not be established. However, findings that the model of these EC tasks was not substantially affected when variance associated with language skills was removed, and findings from other studies indicating that EC-like tasks predict growth in important developmental outcomes (e.g., Blair & Razza, 2007; McClelland et al., 2007) suggests that EC influences the development of these skills. Ultimately, additional longitudinal and experimental studies are needed to clarify the nature and extent of influence between these constructs.

In summary, results of this study indicate EC is best represented as a unidimensional construct that is invariant across gender and the age range included in this study (i.e., 3- to 6-years of age). Although there was some post hoc evidence that there could be two distinct but correlated dimensions of EC, the absence of clear discriminant validity with external measures suggests these dimensions may not reflect important distinctions. In this study, EC was significantly related to both academic measures and behavior problems. Significantly, this construct was more related to certain emergent literacy measures than to others, providing support that EC is uniquely important for development.