Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2020 Jan 1.
Published in final edited form as: Early Child Res Q. 2018 Oct 22;47:1–8. doi: 10.1016/j.ecresq.2018.10.004

Measurement of self-regulation in early childhood: Relations between laboratory and performance-based measures of effortful control and executive functioning

Brenna Lin a,*, Jeffrey Liew a, Marisol Perez b
PMCID: PMC6585984  NIHMSID: NIHMS1509567  PMID: 31223199

Abstract

Effortful control (EC) and executive functioning (EF) are two focal constructs in the study of self-regulation in early childhood.Given a number of conceptual and empirical overlaps between EC and EF, this study examined the associations between commonly used laboratory and performance-based measures of EC and EF in early childhood. Children (N =247; age 4–6 years) completed the Shape Stroop, Snack Delay and Toy Delay tasks, as well as the Conner’s Kiddie-Continuous erformance Task (KCPT).Partial correlations and confirmatory factor analysis (CFA) were conducted to assess the relations between performance on the EC and F tasks and the factor structure of self-regulation. Convergent and divergent validity were found amongst the performance-based measures. Inaddition, results from CFA support a one-factor model of self-regulation with “hot” EC and “cool” EF loading onto a general self-regulation factor. Study results highlight the similarities that exist between EC and EF during early childhood and the need for integrative, whole-child approaches in order to understand the neurophysiological and behavioral underpinnings of self-regulation and its development.

Keywords: Executive functioning, effortful control, self-regulation, early childhood

Introduction

Children’s capacities for self-regulation have important developmental implications for their ability to successfully achieve short and long-term goals (Griffin, Freund, McCardke, & DelCarmen-Wiggens, & Haydon 2016; Mischel et al., 2010). Self-regulation has been defined as the ability to inhibit or to activate responses, through neurocognitive and behavioral processes, in the service of goal attainment and is a core component underlying children’s school readiness (Blair & Diamond, 2008; Liew, 2012; Liew, Cameron, & Lockman, 2018). School readiness, usually assessed between 3 to 5 years, refers to children’s ability to successfully enter or make a smooth transition into formal schooling. Research has identified self-regulation as one of the core developmental capacities or assets for school readiness as well as for future learning and achievement (Blair & Raver, 2012; Mazzocco & Kover, 2007). Thus, there is a need for valid and reliable methods of assessing self-regulation skills in young children to identify and offer assistance to those who exhibit difficulties managing their attention, emotions, or behaviors (Winsler et al., 2008). The primary purpose of this study was to examine relations between performance-based measures of effortful control and executive functioning, which are two major aspects of self-regulation in young children.

The literature on self-regulation in early childhood has primarily focused on effortful control (EC) and on executive functioning (EF) as the focal constructs that underlie self-regulation (Karoly, 1993; Nigg, 2017). While EC and EF are distinct constructs and come from different research traditions, they are overlapping constructs. Such overlap between EC and EF makes measurement of these constructs challenging. Thus, it is important to integrate the constructs and measurement of EC and EF in research to allow for a deeper and nuanced understanding of their unique and shared contributions to the broader construct of self-regulation.

Effortful Control (EC)

Temperament-based reactivity and self-regulation can be observed as early as in infancy and these early temperament traits are typically carried forward into adulthood (Bornstein et al., 2015; Chen & Schmidt, 2015). EC is one of the core temperament factors that is most directly relevant for self-regulation. EC is defined as the ability to inhibit a dominant (motor, vocal, emotional, or cognitive) response and to activate a subdominant response (Rueda, 2012; Rothbart, Ellis, Rueda, & Posner, 2003). EC includes inhibitory control, effortful attention, conflict resolution, and the ability to identify and correct errors and plan actions (Kochanska, Murray, & Harlan, 2000). Researchers who study self-regulation in very young children such as infants and toddlers tend to focus on the construct of EC given that individual differences in EC can be observed early in life. For example, EC has been successfully assessed using behavioral batteries and parents’ ratings with children at 22 months of age (Kochanska et al., 2000).

Executive Functioning (EF)

Researchers who study self-regulation from a neurocognitive perspective tend to focus on EF as the primary construct of study. EF originated from the field of clinical neuropsychology, and refers to the more deliberate or goal-directed, top-down neurocognitive processes involved in self-regulation. uch processes include inhibitory control, attention shifting or cognitive flexibility, and working memory processes (Diamond, 2013; Zelazo & Carlson, 2012; Zelazo, Craik, & Booth, 2004). EF skills are believed to develop shortly after birth and show significant improvements between 3 to 5 years, and continue to improve throughout adolescence and into early adulthood (Carlson, 2010). Different models have been used to conceptualize EF, with some researchers conceptualizing EF as a unitary construct with multiple components (e.g., Garon, Bryson, & Smith, 2008; Wiebe, Epsy, & Charak, 2008) and some researchers conceptualizing EF as a multi-faceted construct (e.g., Miyake, Friedman, Emerson, Witzki, and Howerter, 2000). Some researchers have also called for the need to integrate these models because there may be more similarities than differences between models of EF as a unitary and as a multi-faceted construct (e.g., Zhou, Chen, & Main, 2012).

Relations between EC and EF

While EC and EF come from different historical disciplines, there are many conceptual and empirical overlaps between EC and EF (see Liew, 2012). For example, neurologically, both EC and EF involve the frontal lobe, specifically the prefrontal cortex (Hrabok & Kerns, 2010). Both constructs refer to the ability to inhibit a salient or dominant response in favor of a less salient or subdominant response. Additionally, both EF and EC focus on top-down goal-oriented processes, such as attention planning (Eisenberg & Zhou, 2016). The use of the term EC and EF often depends on the researchers’ discipline or the measures used to assess self-regulation. Importantly, it is often difficult to differentiate between EC and EF empirically because researchers often are not careful in explicitly selecting and using measurements or assessment tasks that are designed to tap primarily EC or EF (Spinrad, Gaertner, Eisenberg, 2007; Zhou, Chen & Main, 2012).

EC and EF as “Hot” or “Cool” Self-regulation

Self-regulation has been conceptualized as consisting of “hot” and “cool” components or processes (e.g., Zelazo & Mueller, 2002) and such a framework of self-regulation has been endorsed by a number of researchers (e.g., Kim et al., 2013; Willoughby, Kupersmidt, Voegler-Lee, & Bryan, 2011; Zelazo & Carlson, 2012). “Hot” self-regulation involves situations that are emotionally arousing while “cool” self-regulation involves situations that are emotionally neutral. Although EC and EF both have “hot” and “cool” aspects, EC is typically considered part of the “hot” system while EF is typically considered part of the “cool” system (Hongwanishkul, Happaney, Lee, & Zelazo, 2005; Kim et al., 2013; Willoughby et al., 2011). However, empirical support is somewhat mixed f or the “hot” and “cool” framework for self-regulation and for viewing EC as primarily “hot” and F as primarily “cool” (e.g., Cunningham & Zelazo, 2007; Prencipe et al., 2011). Furthermore, in light of the mixed evidence and a lack of agreement amongst researchers on the factor structure of self-regulation in early childhood, the relations between EC and EF remain an important and unresolved empirical question (Allan & Lonigan, 2011; Denham et al., 2012).

Performance-Based Measurement of EC and EF

Laboratory and performance-based measures of EC and EF have been developed and validated for use with young children. Among the methods that have been used in published studies, Kochanska’s Battery for Assessing Effortful Control, Mischel’s Delay of Gratification Task, EF or Stroop Tasks, including computerized performance-based measurement of EF such as the Conners’ Kiddie Continuous Performance Test (K-CPT; Conners, 2006), are some of the most widely used performance-based methods of child self-regulation (for a review, see Spinrad, Eisenberg, & Gaertner, 2007; also see Barnard et al., 2015).

Kochanska’s Battery for Assessing Effortful Control (Kochanska, Coy, & Murray, 2001; Kochanska, Murray, & Harlan, 2000) consists of multiple behavioral tasks that were designed for use with toddlers (as early as 22 months), preschoolers, and young school-age children. Kochanska’s Battery has tasks that assess abilities for delaying or waiting, slowing down of gross and fine motor activity, suppressing or activating behavior, effortful attention, and lowering of voice or whispering. Several of the tasks that are included in Kochanska’s Battery include the Shape Stroop and the Snack Delay tasks, with the former task tapping more “cool” aspects of self-regulation and the latter task tapping more “hot” aspects of self-regulation. There are multiple advantages to using Kochanska’s tasks to assess young children’s self-regulation, including the fact that reliabilities on the tasks tend to be relatively high and the tasks have been widely used in child research on self-regulation and have shown good criterion, concurrent, and predictive validity.

Since the 21st century, computerized performance-based tasks have been increasingly used and have become a common method to assess young children’s EC or EF. One such task is the Conners’ Kiddie Continuous Performance Test (K-CPT; Conners, 2006), which was designed to assess young children’s EF or executive control and attention and is often used as a neuropsychological and diagnostic tool for Attention-Deficit/Hyperactivity Disorder (ADHD). The K-CPT is often viewed as tapping “cool” aspects of self-regulation. Researchers have found that the K-CPT useful in detecting early onset of problems with EF in preschool-aged children. However, Barnard et al. (2015) found that performance on the K-CPT does not correlate with parents’ self-reports of children’s EF.

Present study

Given conceptual and empirical overlaps between EC and EF, it is important to examine whether and how performance-based EC and EF tasks relate to one another. Although EC and EF tasks have been developed and validated, some researchers use these tasks interchangeably. Thus, it is important to evaluate the degree to which children’s performance on EC and EF tasks are similar or different from one another. The present study aims to explicitly select and use tasks that were designed to primarily tap young children’s EC or EF in order to examine the interrelations amongst the performance-based EC and EF tasks. Convergent and discriminant validity amongst the EC and EF tasks will be examined using confirmatory factor analysis (CFA) to determine whether a one-factor (general self-regulation factor) or a two-factor (“hot” EC vs. “cool” EF) model fit the data best. In addition, potential socio-demographic (age, gender, ethnicity, and SES) differences on the performance-based EC and EF tasks will be examined given that prior studies have found such differences on performance on self-regulation tasks (Allan & Lonigan, 2014, Mezzacappa, 2004). We hypothesize that significant relations will exist between performance on the EC and EF tasks and that a one-factor (general self-regulation factor) model will fit the data best as has been found by Raffaelli, Crockett, and Shen (2005) in early childhood.

Methods

Participants

Participants were 243 children aged 4 –6 years old (126 males and 116 females; M = 4.80 years, SD = 0.84) and their caregivers (89% biological mothers, 8% biological fathers, and 3% step- or grand-parent). The majority of parents (49.8%) reported a monthly household income of $3000 or below, and 11.3% reported a monthly household income above $9000. Children’s ethnicity was coded as White Non-Hispanic (44.6%), White Hispanic (26.3%), Black, (21.3%), Asian (5.8%), and American Indian (2.1%).

Procedures

This study was part of a larger study that examined children’s emotion and self-regulation approved by the university Institutional Review Board (IRB). Participants were recruited using flyers posted at local preschools, centers, and businesses that were frequented by families. In addition, recruitment information was sent via electronic mail to university-affiliated personnel who worked with families. To participate, children must have been 4–6 years old. Exclusion criteria were if children or parents (1) were unable to use English fluently, (2) had a history of traumatic brain injury, (3) had a significant disability that would prevent them from completing the tasks in this study, such as blindness, or (4) had food allergies related to the food groups (chocolate or grapes) that were provided in the larger study. For this study, children and their parents visited the laboratory for one session that lasted approximately 1.5 h that included a series of observational tasks, including the performance-based EC and EF tasks. For the session, parents received $50 and children received a toy as a token of appreciation and compensation for their time and participation.

Measures

Children completed the Shape Stroop task, the Snack Delay task, Toy Delay task from Kochanska, Murray, & Harlan’s (2000) battery of EC tasks along with the Conners’ Kiddie Continuous Performance Test (Conners’ K-CPT; Conners, 2001). These performance-based tasks of EC and EF were selected because they have been successfully used with young children in prior research, and have shown good criterion, concurrent, and predictive validity as individual tests of EC or EF (but rarely used simultaneously in a single study). The Snack and Toy Delay Tasks are often considered tasks that primarily tap EC, while the Shape Stroop task and the K-CPT are often considered as tasks that primarily tap EF.

Shape Stroop Task (Kochanska et al., 2000).

The Shape Stroop Task is part of Kochanska et al.’s (2000) battery of EC tasks, but Stroop tasks are traditionally viewed as tapping EF including cognitive flexibility and attention (Carlson, 2005). Under the “hot/cool” framework, the Shape Stroop would be considered a primarily “cool” self-regulation task as it does not directly tap into motivational or emotional elements nor is it tied to any immediate reward. For this task, the experimenter first asked children to name the visual features (e.g., shapes, animals, numbers, and letters) displayed on a series of large (8.5 by 11.0 inches) cards. For each trial, children were shown two side-by-side visual features with one side showing a dominant (big and solid) visual feature and the other side showing the dominant visual feature but embedded or filled with subdominant visual features. Children were asked to call out the subdominant visual features for each trial. For example, if a diamond was shown on one side of a page and a diamond embedded with small triangles was shown on the other side then the correct response would be triangle. Children were instructed to call out the subdominant or small figures in every picture shown and to do so as quickly as possible. Children received 2 points for a correct response, 1 point for an initial wrong response followed by a self-corrected response, and 0 points for a wrong response. There were 24 trials and scores were computed by summing points across all trials. Inter-rater reliability was computed using correlations between ratings across two coders who scored all children’s performance on the Shape Stroop Task, r=.72.

Snack Delay Task (Kochanska et al., 2000).

The Snack Delay (MMDelay) is another task that is part of Kochanska et al’s (2000) battery of EC assessment, and taps delay of gratification and inhibitory control. This task is categorized as a primarily “hot” self-regulation task and requires children to delay or inhibit from touching or eating a snack. In the present study, we used M&M chocolate candies as the snack. Children were instructed to place their hands on a mat on a table. Then, the experimenter placed one M&M chocolate candy under an inverted transparent cup in front of the children on the table. The experimenter told the children to wait until she rang a bell before reaching for and eating the snack or candy. After a demonstration or practice trial, children participated in four trials with delays of 10, 20, 30, and 15 seconds before the experimenter rang the bell. On every trial, children were reminded of the rule (to wait until the bell rang before reaching for and eating the snack). Children’s behaviors on the four trials were coded by the experimenter from 0 (eats snack before the experimenter lifts the bell) to 4 (waits until the bell is rung). Scores were averaged across the four trials, with higher scores representing better inhibitory control. Inter-rater reliability was computed using correlations between ratings across two coders who scored all children’s performance on the Snack Delay Task, r=.93.

Toy Delay Task.

The Toy Delay Task is a modification of the Snack Delay Task (see Kochanska et al., 1996), using an inedible but desirable stimulus. We included the Toy Delay Task as a comparison to the Snack Delay Task to assess whether there were differences in EC performance based on the type of attractive or desirable (i.e., inedible vs. edible) stimulus. The Toy Delay Task taps delay of gratification and inhibitory control and requires children to wait before touching and playing with an attractive stimulus (“silly bandz”). Similar to the Snack Delay Task, the Toy Delay Task is categorized as a primarily “hot” self-regulation task. Children were shown the “silly bandz”, a very popular toy/game at the time. Children were instructed to wait for the experimenter to ring a bell before reaching for and playing with the “silly bandz.” All children underwent a demonstration or practice trial, which required them to wait 10 seconds before the experimenter rang the bell. After ensuring children understood the instructions, children completed 4 trials with intervals of 10, 20, 30, and 15 seconds before the experimenter rang the bell. On every trial, children were reminded of the rule (to wait until the bell rang before reaching for and playing with the “silly bandz”). Children’s behaviors on the four trials were coded by the experimenter from 0 (touches toy before the experimenter lifts the bell) to 4 (waits until the bell is rung), with higher scores denoting better delaying or inhibitory capacity, and scores were averaged across the four trials. Inter-rater reliability was computed using correlations between ratings across two coders who scored all children’s performance on the Toy Delay Task, r=.92.

Conners’ Kiddie Continuous Performance Test (K-CPT; Conners, 2006).

The Conners’ Kiddie Continuous Performance Test (Conners’ K-CPT; Conners, 2001) is a standardized and computerized assessment of executive attention and attention-related problems normed for children 4–6 years of age. The K-CPT was designed as a measure of EF, measuring response inhibition and sustained attention (Willcutt, Doyle, Nigg, Faraone, & Pennington, 2005; Pasini, Paloscia, Allesandrelli, Porfirio & Curatolo, 2007). The K-CPT captures a range of regulation constructs including inattentiveness, impulsivity, attention, and vigilance. It has shown to be an ecologically valid measure of EF (Weis & Totten, 2004). It is historically used by neuropsychologists under the EF framework. Under the “hot/cool” framework, the K-CPT would be categorized as a primarily “cool” self-regulation task as the task does not directly tap into a motivational or emotional element nor is it tied to any immediate reward. In this study, developmentally appropriate versions of the task were administered based on the child’s age (i.e., 4–5 years of age or 6 years). Children were asked to press a button every time they saw a picture other than a ball on the computer screen. Each picture was presented for 500 ms, and in different blocks the inter-stimulus interval (ISI) was either 1.5 or 3 s. There were a total of five blocks, with two sub-blocks of 20 trials each for each ISI. The total testing time was 7.5 min. For this study, we used data from the CPT non-clinical profiles that indicate the degree to which participants’ responses matched a non-clinical (typically developing) profile. Psychometric studies have established acceptable reliability results for the CPT (Smith & Corkum, 2007) with test-retest reliability ranging from .65 to .74 (Halperin, Sharma, & Greenblatt, 1991).

Analytic approach

Descriptive statistics were conducted and potential sociodemographic differences on major study variables were examined using multivariate analyses of variance (MANOVA) while the interrelations among the EC and EF tasks were examined with correlational analyses. Convergent validity amongst the EC and EF tasks was tested through examining the strength of the relation (Pearson’s r) between performance scores on the EC and EF tasks.

To examine the discriminant validity and construct validity of a latent factor structure of self-regulation, CFA was conducted using Mplus version 7.4 (Muthen & Muthen, 2007). CFA was applied because it allows for the direct representation of a hypothesized factor model, leading to a test of model fit. For estimation method, maximum likelihood (ML) estimator was applied. To evaluate the goodness-of-fit of the model, the following statistics were examined: Root Mean Square Error of Approximation (RMSEA), Confirmatory Fit Index (CFI), and standardized root mean square residual (SRMR).

Results

Demographic characteristics of the sample are presented in Table 1. Means and standard deviations for all major variables are presented in Table 2. Correlations among major variables were conducted to examine how performance on EC and EF tasks relate to one another by age and income (see Table 3). single-factor (gender & ethnicity) MANOVA was conducted to test for differences on major variables by child gender and by child ethnicity (see Table 4). Partial correlations controlling for demographic variables were conducted to examine relations among the scores on the EC and EF tasks (see Table 5). A CFA was conducted to examine to assess the factor structure of the model fit (see Table 6).

Table 1.

Demographic Characteristics of Study Sample

N %
Age
4 116 47%
5 61 25%
6 67 27%
Missing 3 1%
Gender
Male 126 51%
Female 121 49%
Missing
Ethnicity
White Not Hispanic 107 43%
White Hispanic 63 26%
Black 51 21%
Asian 14 6%
American Indian 5 2%
Missing 7 3%
Income (monthly)
0–2,000 87 35%
2,001–5,000 73 30%
5,001 – 9,000 54 22%
9,001 – 13,000 12 5%
13,001 - or more 16 6%
Missing 5 2%
Open in a new tab

Note. Total N = 247

Table 2.

Descriptive Statistics for Performance-based Self-regulation Measures

N Mean Std Dev Min Max
MMDelay 229 3.61 0.95 0.00 4.00
Toy Delay 228 3.61 0.92 0.00 4.00
Stroop 230 40.73 9.05 2.00 48.00
K-CPT 66 52.60 20.44 16.89 95.13
Open in a new tab

Table 3.

Correlations Between Age or Income and Self-regulation Measures

Variables MMdelay Toydelay Stroop K-CPT
Age 0.08 .20** .45*** −0.04
Income 0.09 0.038 .26*** 0.05
Open in a new tab

Note.

+

p<.10

*

p<.05

**

p<.01

***

p<.001

Table 4.

Multivariate Effects for Ethnicity and Gender

Variables Wilks’ λ DF Error DF F
Ethnicity 0.71 12 154 1.74+
Gender 0.82 4 60 3.30*
Open in a new tab

Note.

+

p<.10

*

p<.05

**

p<.01,

***

p<.001

Table 5.

Relations Amongst Self-regulation Measures Controlling for Age,Ethnicity, Gender, and Income

Tasks 1 2 3 4
MMdelay -
Toydelay 0.76*** -
Stroop 0.26*** 0.28*** -
K-CPT 0.24+ 0.37** 0.25+ -
Open in a new tab

Note.

+

p<.10

*

p<.05

**

p<.01

***

p<.001

Table 6.

Confirmatory Factor Analysis Model Fit Indices

Variable X2 (df) RMSEA CFI SRMR Overall
One-factor Model 3.15 (2) 0.05 0.99 0.07 Good Fit
Two-factor Model 1.03 (1) 0.01 1.00 0.02 Good Fit
Open in a new tab

Note. RMSEA = root-mean-square roor of approximation (good fit <.05); CFI =comparative fit index (good fit > .90); SRMR = standardized root mean square residual, (good fit <.08)

Descriptive Statistics

To determine if demographic variables need to be included as covariates in subsequent analyses, MANOVAs (to examine potential gender and ethnicity differences) and correlations (to examine potential age and family income differences) were conducted with the major study variables. Results indicate that performance on the Toy Delay and the Stroop Tasks were associated with age, rs(228 and 230) = .20, .45, ps < .01, respectively (see Table 3). In addition, results from correlations show that income levels were associated with performance on the Shape Stroop Task, r(228) = .26, p < .001. Regarding results from MANOVAs, gender differences as well as ethnic differences were found on the EC and EF tasks, Wilks’s Fs(4, 60) and (12,154) produced 3.30 and 1.74, p < .01 and p < .10, respectively (see Table 4).

Interrelations Amongst EC and EF Tasks

Due to differences on major variables that were found for child age, gender, ethnicity and family income, partial correlations controlling for these demographic variables were conducted to examine relations among the scores on the EC and EF tasks (see Table 5). Results from partial correlations showed that the three measures from Kochanska, Murray, & Harlan’s (2000) battery (the Snack Delay, Toy Delay, and Shape Stroop Tasks) were all related to one another. Scores on the Snack and Toy Delay Tasks were highly correlated, r(217) = .76, p <.001. And scores on the Shape Stroop Task were associated with scores on the Snack and Toy Delay Tasks, rs(218 and 217) = .26 and .28, ps< .001, respectively. Furthermore, children’s K-CPT non-clinical profile scores were positively related to their scores on the Toy Delay Task, r(58) = .37, p <.01, while marginally significant associations were found between the K-CPT and the Snack Delay Task, r(58) = .24, p < .10 and between K-CPT and the Shape Stroop Task, r(58) = .25, p < .05.

Confirmatory Factor Analysis

A CFA was performed to assess the factor structure of a one-factor model. The determination of model fit was based on RMSEA (values <.05), the CFI (values >.90), and SRMR (values <.08) fit indices and cutoff scores as typically cited in the literature on psychometrics (e.g., Bentler & Stein,1992; Hu & Bentler, 1999; MacCallum, Browne & Sugawara, 1996). The CFA model loaded all tasks: MMDelay, ToyDelay, Stroop, and K-CPT as first-order factors onto one second-order factor. The results of the fit statistics (RMSEA =.05, CFI = 0.99, and SRMR = 0.07) revealed a “good” fit (Table 6) based on a one-factor model. See Figure 1 for parameter estimates.

Figure 1.

Figure 1.

Open in a new tab

Parameter Estimates for a one factor figure

A second CFA model was then conducted to test model fit of “hot” and “cool” factors based on the EC and EF tasks. The Snack Delay and Toy Delay asks were loaded as first-order factors onto one second order factor (“hot” self-regulation). The Shape Stroop Task and K-CPT were loaded as first-order factors onto another second order factor (“cool” self-regulation). The two second order factors were allowed to correlate. The results of the fit statistics (RMSEA =.01, CFI = 1.00, and SRMR = 0.02) revealed a “good” fit (Table 6) as well. See Figure 2 for parameter estimates.

Figure 2.

Figure 2.

Open in a new tab

Parameter Estimates for a two factor figure

Because both models produced good fitting models, and are nested, a likelihood ratio test was used to test if these two models were statistically significant. The hypothesis that the data is equally likely under the two models was not rejected [X2 = 2.12 (1), p = .14]. This indicates that the two models are not statistically significantly different from each other. The insignificant p value indicates that both models fit equally well statistically, so the model with the more parameters should be eliminated and the most parsimonious model can be accepted as the model that best fits the data (Bollen, 1989; Kline, 1998). Thus, we accept the one-factor model as the most parsimonious and best fitting-model to the data on performance-based measures of self-regulation in early childhood.

Conclusions

The primary aims of the present study were to establish the convergent or divergent validity of four commonly used performance-based measures of young children’s self-regulation (specifically EC and EF) and to examine the factor structure of children’s performance on the EC and EF tasks. To our knowledge, few studies have examined the relations amongst laboratory and performance-based behavioral assessments of EC and EF.

Convergent and Divergent Validity of Performance-based EC and EF Assessments

Results showed a relatively high correlation (.76) between children’s performance on the Snack and Toy Delay tasks. This finding suggests that, regardless of socio-demographic backgrounds, young children find the M&M chocolate candy and the “silly bandz” toy similarly attractive or desirable. The Shape Stroop Task was the assessment that produced the greatest variability across different sociodemographic factors, because performance on the Shape Stroop differed by age, ethnic, and family household income. Interestingly, this task did not vary by gender, as noted in previous research (Kochanska et al., 2000). While Kochanska categorized this task as primarily tapping effortful attention or EF, there are also elements of inhibition and delaying. And though EC is typically said to be a measure of both “hot” and “cool” self-regulation, the Shape Stroop task actually does not provide any salient reward or immediate incentive for the child. Yet, the Shape Stroop correlated with both the Snack and Toy Delay tasks (rs(219 and 218 ) =.26 and .28, ps<.001, respectively). As these three tasks were behavioral measures administered in similar formats, interrelations or convergence amongst these three tasks should not be overly surprising. However, from the “hot” vs. “cool” self-regulation framework, we would also expect to find some divergence between the Shape Stroop or “cool” (primarily tapping EF) task from the Snack and Toy Delay or “hot” (primarily tapping EC) tasks. Indeed, results from correlations support this “hot” vs. “cool” self-regulation perspective, because the strength of relation between performance on the Shape Stroop and on the Snack and Toy Delay tasks was much lower (rs = .26 and .28, respectively) than that of the strength of relation between the two delay (“hot”) tasks (r = .76).

Relations between Neuropsychological and Behavioral Assessments

As a neuropsychological test, the K-CPT differed from the three behavioral assessments in terms of assessment format and length. he behavioral tasks were administered in a gamelike fashion while the K-CPT may appear more test-like than gamelike for young children. In addition, the behavioral tasks typically took just a couple of minutes to complete while the K-CPT took 7.5 minutes to complete. The format and length of the K-CPT may make it difficult for some young children to persist and complete the test in its entirety. Furthermore, while the behavioral tasks involved social interaction with an experimenter who administered the tasks, the K-CPT’s computerized format did not involve social interaction in the completion of the task. Due to the “cool” nature of the K-CPT and the difference in format or length between the K-CPT and the EC tasks, there were reasons to expect little to no relation between the Toy and Snack Delay tasks with the K-CPT. Instead, results showed that performance on the K-CPT was associated with performance on the EC tasks, notably most strongly with the Toy Delay Task. The interrelations amongst all EC and EF tasks demonstrate that, despite different administration methods, there is convergent validity amongst the neuropsychological and behavioral measures and they tap a common or overarching (self-regulation) construct.

“Hot” vs “Cool” Self-regulation Framework

Recall that results from correlations amongst the EC and EF tasks support that these tasks show both convergent and divergent validity, which appears consistent with the “hot” vs. “cool” framework of self-regulation. Nonetheless, results from confirmatory factor analysis support that in young children, performance on “hot” EC and “cool” EF tasks load onto one general self-regulation factor. This finding highlights the similarities that exist between EC and EF during early childhood and the need to view these constructs as complementary and core aspects of self-regulation. Further, this finding suggests that the one-factor model of self-regulation may represent how self-regulation manifests itself in early childhood. This is consistent with work by other researchers who call for an integrated or unitary model of self-regulation (e.g., Allan & Lonigan, 2011; Sulik et al., 2010; Zhou et al., 2012). The distinctions of “hot” and “cool” self-regulation may be less differentiated in early childhood and also in everyday life (Zelazo & Carlson, 2012), with “hot” and “cool” stimuli or situations often blurred as most situations do not involve purely cool/neutral environments.

Limitations and Future Directions

The use of multiple performance-based measures of self-regulation is a strength of the present study, but there are also limitations to the study that warrant attention. We included several tasks from Kochanska et al.’s (2000) EC battery, but inclusion of other developmentally appropriate behavioral assessments of self-regulation such as the Head-Toes-Knees-Shoulders task (Cameron Ponitz, McClelland, Matthews, & Morrison, 2009) would be important so that tasks come from more than one battery. Further, inclusion of additional behavioral and neuropsychological assessments will also allow for including three or more indicators for each latent factor in the testing of the 2-factor model of self-regulation. Another limitation is the fact that a substantial number of children did not complete the K-CPT in its entirety (n=59). The administration time for the K-CPT is set at 7.5 minutes, which was designed to be long enough to challenge young children and their attention capacity but not to the point that they cannot finish the task. However, a number of young children in our sample were either bored or frustrated to the point where they did not complete the test in its entirety. Additionally, while the K-CPT is a well-validated measure for young children, it is noted that for preschoolers, it may be especially difficult to balance out validity with test difficulty due to high error rates (Hagelthorn, Hiemenz, Pillion, Mahone, 2003).

Of note is that a significant portion of our sample had children living in families that could be considered relatively low income (49.8% of parents reported a monthly household income of $3000 or below). To our knowledge, no published studies have documented the performance of low-income or at-risk young children on the K-CPT. We found that family income was associated with performance on the Shape Stroop Task, a measure that primarily taps EF. Thus, it is plausible that children from low-income families, particularly those living at or below the poverty line, may exhibit poor self-regulation skills and have great difficulty completing the K-CPT which required them to persist and be attentive for at least 7.5 minutes. There is a need for future studies to explore the appropriateness of different types of self-regulation assessments for children from low-income and at-risk backgrounds.

Nonetheless, our results from confirmatory factor analysis are consistent with Wiebe, Epsy, and Charak’s (2008) work on EF using confirmatory factor analysis that showed that the single-factor model might be specific to the pre-school years (also see Raffaelli et al., 2005). Thus, there are likely changes in the factor structure of self-regulation across development (Prencipe et al., 2011). Early childhood is a critical period for EC and EF development that may be associated with brain development that includes changes in grey and white matter, anatomical, and physiological changes to the prefrontal cortex (Gogtay et al., 2004; Matsuzawa et al., 2004). These age-related changes in the brain, individual differences, as well as the lack of clarity on motivational factors involved in a task or assessment, call for a need to carefully consider developmentally appropriate measures of “hot” and “cool” self-regulation in early childhood (Carlson, 2010). The present study results contribute to the literature on performance-based assessments of self-regulation and also highlight the conceptual and empirical overlaps between EC and EF in early childhood. In conclusion, although EC and EF are distinct constructs with different theoretical frameworks, there are conceptual and empirical linkages between these constructs. Integrative, whole-child approaches are much needed in order to understand and appreciate the neurophysiological and behavioral underpinnings of self-regulation and its development (see Liew et al., 2018). Such approaches could foster the development of cohesive language to communicate with parents, educators, and policymakers about children’s self-regulation, as well as identifying developmentally appropriate, valid, and reliable measures or assessments of self-regulation for screening, diagnostic, research, and intervention purposes.

Highlights.

  • Convergent and divergent validity were found amongst four commonly used performance-based measures of children’s self-regulation.

  • Effortful control and executive functioning tasks loaded onto a one-factor model.

  • Empirical overlaps exist between effortful control and executive functioning in early childhood.

Acknowledgments

Funding Statement

The project described was supported by Award Number R03HD058734 from the Eunice Kennedy Shriver National Institute of Child Health & Human Development. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Eunice Kennedy Shriver National Institute of Child Health & Human Development or the National Institutes of Health.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Disclosure Statement

The authors declare that they have no relevant or material financial interests that relate to the research described in this paper.

References

  1. Allan NP, & Lonigan CJ (2011). Examining the dimensionality of effortful control in preschool children and its relation to academic and socioemotional indicators. Developmental Psychology, 47, 905–915. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Allan NP, & Lonigan CJ (2014). Exploring dimensionality of effortful control using hot and cool tasks in a sample of preschool children. Journal of experimental child psychology, 122, 33–47. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Barkley RA (2012). Executive functions: What they are, how they work, and why they evolved Guilford Press. [Google Scholar]
  4. Barnard H, Rao R, Xu Y, Froehlich T, Epstein J, Lanphear BP, & Yolton K (2015). Association of the Conners’ Kiddie Continuous Performance Test (K-CPT) performance and parent-report measures of behavior and executive functioning. Journal of attention disorders, 1087054715578271. [DOI] [PubMed]
  5. Bentler PM, & Stein JA (1992). Structural equation models in medical research. Statistical Methods in Medical Research, 1(2), 159–181. [DOI] [PubMed] [Google Scholar]
  6. Blair C, & Diamond A (2008). Biological processes in prevention and intervention: The promotion of self-regulation as a means of preventing school failure. Development and Psychopathology, 20(03), 899–911. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Blair C, & Razza RP (2007). Relating effortful control, executive function, and false belief understanding to emerging math and literacy ability in kindergarten. Child Development, 78(2), 647–663. [DOI] [PubMed] [Google Scholar]
  8. Blair C, & Raver CC (2015). School readiness and self-regulation: A developmental psychobiological approach. Annual review of psychology, 66, 711–731. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Bollen KA (1989). A new incremental fit index for general structural equation models. Sociological Methods & Research, 17(3), 303–316. [Google Scholar]
  10. Bornstein MH, Putnick DL, Gartstein MA, Hahn CS, Auestad N, & O’Connor DL (2015). Infant temperament: Stability by age, gender, birth order, term status, and socioeconomic status. Child Development, 86, 844–863. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Cameron Ponitz CE, McClelland MM, Matthews JS, & Morrison FJ (2009). A structured observation of behavioral self-regulation and its contribution to kindergarten outcomes. Developmental Psychology, 45, 605–619. [DOI] [PubMed] [Google Scholar]
  12. Carlson SM (2005) Developmentally Sensitive Measures of Executive Function in Preschool Children, Developmental Neuropsychology, 28:2, 595–616. [DOI] [PubMed] [Google Scholar]
  13. Chen X, & Schmidt L (2015) Temperament and personality. In Lerner R (Series Ed.) & Lamb M (Vol. Ed.), Handbook of child psychology and developmental science. Socioemotional processes (7th ed., Vol. 3, pp. 152–200). New York: Wiley. [Google Scholar]
  14. Conners CK (2006). Conners’ Kiddie Continuous Performance Test. Multi-Health Systems Incorporated
  15. Cunningham WA, & Zelazo PD (2007). Attitudes and evaluations: A social cognitive neuroscience perspective. Trends in Cognitive Sciences, 11(3), 97–104. [DOI] [PubMed] [Google Scholar]
  16. Denham SA, Warren-Khot HK, Bassett HH, Wyatt T, & Perna A (2012). Factor structure of self-regulation in preschoolers: Testing models of field-based assessment for predicting early school readiness. Journal of Experimental Child Psychology, 111, 386–404. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Diamond A (2013). Executive functions. Annual Review of Psychology, 64, 135–168. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Eisenberg N, Spinrad TL, & Eggum ND (2010). Emotion-related self-regulation and its relation to children’s maladjustment. Annual Review of Clinical Psychology, 6, 495–525. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Eisenberg N, & Zhou Q (2016). Conceptions of executive function and regulation: When and to what degree do theyoverlap? In Griffin JA, McCardle P & Freund LS (Eds.), Executive function in preschool-age children: Integrating measurement, neurodevelopment, and translational research (pp. 115–136). Washington, DC: American Psychological Association. [Google Scholar]
  20. Garon N, Bryson SE, & Smith IM (2008). Executive function in preschoolers: a review using an integrative framework. Psychological Bulletin, 134(1), 31. [DOI] [PubMed] [Google Scholar]
  21. Griffin JA, Freund LS, McCardle P, DelCarmen-Wiggins R, & Haydon A (2016). Introduction to Executive Function in Preschool-Age Children
  22. Gogtay N, Giedd JN, Lusk L, Hayashi KM, Greenstein D, Vaituzis AC, Nugent TF, Herman DH, Clasen L. Toga, & Rapoport JL, Thompson PM (2004). Dynamic mapping of human cortical development during childhood through early adulthood. Proceedings of the National academy of Sciences of the United States of America, 101(21), 8174–8179. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Hagelthorn KM, Hiemenz JR, Pillion JP, & Mahone EM (2003). Age and task parameters in continuous performance tests for preschoolers. Perceptual and Motor Skills, 96(3), 975–989. [DOI] [PubMed] [Google Scholar]
  24. Halperin JM, Sharma V, & Greenblatt E (1991). Assessment of the Continuous Performance Test: Reliability and validity in a non-referred sample. Psychological Assessment, 3(4), 603–608. [Google Scholar]
  25. Hongwanishkul D, Happaney KR, Lee WS, & Zelazo PD (2005). Assessment of hot and cool executive function in young children: Age-related changes and individual differences. Developmental Neuropsychology, 28(2), 617–644. [DOI] [PubMed] [Google Scholar]
  26. Hrabok M, & Kerns KA (2010). The development of self-regulation: A neuropsychological perspective. Self and social regulation: Social interaction and the development of social understanding and executive functions, 129–154.
  27. Hu LT, & Bentler PM (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: a multidisciplinary journal, 6(1), 1–55. [Google Scholar]
  28. Karoly P (1993). Mechanisms of self-regulation: systems view. Annual review of psychology, 44(1), 23–52. [Google Scholar]
  29. Kim S, Nordling JK, Yoon JE, Boldt LJ, & Kochanska G (2013). Effortful control in “hot” and “cool” tasks differentially predicts children’s behavior problems and academic performance. Journal of Abnormal Child Psychology, 41(1), 43–56. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Kline RB (1998). Software review: Software programs for structural equation modeling: Amos, EQS, and LISREL. Journal of psychoeducational assessment, 16(4), 343–364 [Google Scholar]
  31. Kochanska G, Coy KC, & Murray KT (2001). The development of self-regulation in the first four years of life. Child Development, 72, 1092–1111. [DOI] [PubMed] [Google Scholar]
  32. Kochanska G, Murray KT, & Harlan ET (2000). Effortful control in early childhood: continuity and change, antecedents, and implications for social development. Developmental Psychology, 36(2), 220. [PubMed] [Google Scholar]
  33. Kochanska G, Murray K, Jacques TY, Koenig AL, & Vandegeest KA (1996). Inhibitory control in young children and its role in emerging internalization. Child Development, 490–507. [PubMed]
  34. Liew J (2012). Effortful control, executive functions, and education: Bringing self‐regulatory and social‐emotional competencies to the table. Child Development Perspectives, 6(2), 105–111 [Google Scholar]
  35. Liew J, Cameron CE, & Lockman JJ (2018). Parts of the whole: Motor and behavioral skills in self-regulation and schooling outcomes. Early Education and Development, 29, 909–913. [Google Scholar]
  36. MacCallum RC, Browne MW, & Sugawara HM (1996). Power analysis and determination of sample size for covariance structure modeling. Psychological Methods, 1(2), 130. [Google Scholar]
  37. Matsuzawa J, Matsui M, Konishi T, Noguchi K, Gur RC, Bilker W, & Miyawaki T (2001). Age-related volumetric changes of brain gray and white matter in healthy infants and children. Cerebral Cortex, 11(4), 335–342. [DOI] [PubMed] [Google Scholar]
  38. Mazzocco MM, & Kover ST (2007). longitudinal assessment of executive function skills and their association with math performance. Child neuropsychology, 13(1), 18–45. [DOI] [PubMed] [Google Scholar]
  39. Mezzacappa E (2004). Alerting, orienting, and executive attention: Developmental properties and sociodemographic correlates in an epidemiological sample of young, urban children. Child development, 75(5), 1373–1386. [DOI] [PubMed] [Google Scholar]
  40. Mischel W, Ayduk O, Berman MG, Casey BJ, Gotlib IH, Jonides J, Kross E, Teslovich T, Wilson N, Zayas V, & Shoda Y (2010). ‘Willpower’ over the life span: decomposing self-regulation. Social Cognitive and Affective Neuroscience, 6, 252–256. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Miyake A, Friedman NP, Emerson MJ, Witzki AH, Howerter A, & Wager TD (2000). The unity and diversity of executive functions and their contributions to complex “frontal lobe” tasks: A latent variable analysis. Cognitive Psychology, 41(1), 49–100.. [DOI] [PubMed] [Google Scholar]
  42. Moffitt TE, Arseneault L, Belsky D, Dickson N, Hancox RJ, Harrington H, Houts R, Poulton R, Roberts BW, Ross S, Sears MR, Thompson WM, Caspi A (2011). A gradient of childhood self-control predicts health, wealth, and public safety. Proceedings of the National Academy of Sciences, 108(7), 2693–2698. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Muthén LK, & Muthén BO (2010). Mplus: Statistical analysis with latent variables: User’s guide (pp. 1998–2007). Los Angeles: Muthén & Muthén. [Google Scholar]
  44. Nigg JT (2017). Annual research review: On the relations among self-regulation, self-control, executive functioning, effortful control, cognitive control, impulsivity, risk-taking, and inhibition for developmental psychopathology. The Journal of Child Psychology and Psychiatry, 58, 361–383. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Pasini A, Paloscia C, Alessandrelli R, Porfirio MC, & Curatolo P (2007). Attention and executive functions profile in drug naive DHD subtypes. Brain and Development, 29(7), 400–408. [DOI] [PubMed] [Google Scholar]
  46. Prencipe A, Kesek A, Cohen J, Lamm C, Lewis MD, & Zelazo PD (2011). Development of hot and cool executive function during the transition to adolescence. Journal of Experimental Child Psychology, 108(3), 621–637. [DOI] [PubMed] [Google Scholar]
  47. Raffaelli M, Crockett LJ, & Shen YL (2005). Developmental stability and change in self-regulation from childhood to adolescence. The Journal of Genetic Psychology, 166, 54–75. [DOI] [PubMed] [Google Scholar]
  48. Rothbart MK, Ellis LK, & Posner MI (2004). Temperament and self-regulation. Handbook of self-regulation: Research, theory, and applications, 2, 441–460. [Google Scholar]
  49. Robthbart MK, Ellis LK, Rueda MR, & Posner MI (2003). Developing mechanisms of temperamental effortful control. Journal of Personality, 71, 1113–1144. [DOI] [PubMed] [Google Scholar]
  50. Rueda MR (2012). Effortful control. In Zentner M & Shiner RL (Eds.), Handbook of temperament (pp. 145–167). New York, NY: Guilford Press. [Google Scholar]
  51. Smith KG, & Corkum P (2007). Systematic review of measures used to diagnose attention-deficit/hyperactivity disorder in research on preschool children. Topics in Early Childhood Special Education, 27(3), 164–173. [Google Scholar]
  52. Spinrad TL, Eisenberg N, & Gaertner BM (2007). Measures of effortful regulation for young children. Infant Mental Health Journal, 28, 606–626. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Spinrad TL, Eisenberg N, Gaertner B, Popp T, Smith CL, Kupfer A, Greving K, Liew J, & Hofer C (2007). Relations of maternal socialization and toddlers’ effortful control to children’s adjustment and social competence. Developmental Psychology, 43(5), 1170. [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Sulik MJ, Huerta S, Zerr AA, Eisenberg N, Spinrad TL, Valiente C, et al. (2010). The factor structure of effortful control and measurement invariance across ethnicity and sex in a high-risk sample. Journal of Psychopathology and Behavioral Assessment, 32, 8–22. [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Weis R, & Totten SJ (2004). Ecological validity of the Conners’ Continuous Performance Test II in a school-based sample. Journal of Psychoeducational Assessment, 22(1), 47–61. [Google Scholar]
  56. Wiebe SA, Espy KA, & Charak D (2008). Using confirmatory factor analysis to understand executive control in preschool children: I. Latent structure. Developmental Psychology, 44(2), 575. [DOI] [PubMed] [Google Scholar]
  57. Willcutt EG, Doyle AE, Nigg JT, Faraone SV, & Pennington BF (2005). Validity of the executive function theory of attention-deficit/hyperactivity disorder: a meta-analytic review. Biological Psychiatry, 57(11), 1336–1346. [DOI] [PubMed] [Google Scholar]
  58. Willoughby M, Kupersmidt J, Voegler-Lee M, & Bryan D (2011). Contributions of hot and cool self-regulation to preschool disruptive behavior and academic achievement. Developmental Neuropsychology, 36, 162–180. [DOI] [PMC free article] [PubMed] [Google Scholar]
  59. Winsler A, Tran H, Hartman SC, Madigan AL, Manfra L, & Bleiker C (2008). School readiness gains made by ethnically diverse children in poverty attending center-based childcare and public school pre-kindergarten programs. Early Childhood Research Quarterly, 23(3), 314–329. [Google Scholar]
  60. Zelazo PD, & Carlson SM (2012). Hot and cool executive function in childhood and adolescence: Development and plasticity. Child Development Perspectives, 6, 354–360. [Google Scholar]
  61. Zelazo PD, Craik FIM, & Booth L (2004). Executive function across the life span. Acta Psychologica, 115, 167–183. [DOI] [PubMed] [Google Scholar]
  62. Zelazo PD, & Mueller U (2002). Executive function in typical and atypical development. In Goswami U (Ed.), Blackwell Handbook of Childhood Cognitive Development (pp.445–460). Malden, MA: Blackwell Publishers. [Google Scholar]
  63. Zhou Q, Chen SH, & Main A (2012). Commonalities and differences in the research on children’s effortful control and executive function: A call for an integrated model of self‐ regulation. Child Development Perspectives, 6(2), 112–121. [Google Scholar]

RESOURCES