Relations between Response Trajectories on the Continuous Performance Test and Teacher-Rated Problem Behaviors in Preschoolers

Abstract

Although both the Continuous Performance Test (CPT) and behavior rating scales are used in both practice and research to assess inattentive and hyperactive/impulsive behaviors, the correlations between performance on the CPT and teachers' ratings are typically only small-to-moderate. This study examined trajectories of performance on a low target-frequency visual CPT in a sample of preschool children and how these trajectories were associated with teacher-ratings of problem behaviors (i.e., inattention, hyperactivity/impulsivity [H/I], and oppositional/defiant behavior). Participants included 399 preschool children (Mean age = 56 months; 49.4% female; 73.7% White/Caucasian). An ADHD-rating scale was completed by teachers, and the CPT was completed by the preschoolers. Results showed that children's performance across four temporal blocks on the CPT was not stable across the duration of the task, with error rates generally increasing from initial to later blocks. The predictive relations of teacher-rated problem behaviors to performance trajectories on the CPT were examined using growth curve models. Higher rates of teacher-reported inattention and H/I were uniquely associated with higher rates of initial omission errors and initial commission errors, respectively. Higher rates of teacher-reported overall problem behaviors were associated with increasing rates of omission but not commission errors during the CPT; however, the relation was not specific to one type of problem behavior. The results of this study indicate that the pattern of errors on the CPT in preschool samples is complex and may be determined by multiple behavioral factors. These findings have implications for the interpretation of CPT performance in young children.

Behavioral rating forms and cognitive tasks, such as continuous performance tests (CPTs), are two methods that are used in both research and applied settings for assessing children's inattention and hyperactivity/impulsivity (H/I). Although these methods presumably measure the same underlying constructs, the correlations between behavior ratings and CPT performance are typically small (Egeland, Johansen, & Ueland, 2009; McGee, Clark, & Symons, 2000). The reasons for these weak relations are unclear. Research explicating the specific ways in which direct measures and behavior-ratings of problem behaviors do and do not converge is needed to provide a stronger foundation for the use of these measures to operationalize inattention and H/I. Such research may be used to guide the valid use of the CPT in both clinical practice and research.

To date, most research focusing on the assessment of attention in childhood has been conducted with school-age populations. However, the increasing focus on the preschool years as a period in which children learn important early academic skills has resulted in a growing interest in studying factors that impact learning, such as attention, in preschool-age children. Preschool is a time-period during which self-regulatory behaviors undergo rapid development (Espy, 2004). It is a time when many children first encounter structured social and academic activities outside the home and when self-regulatory behaviors first begin to show reliable long-term associations with social and academic outcomes (for review, see Spira & Fischel, 2005). Failure to identify attention problems early and accurately may lead to missed opportunities to provide early intervention that could ameliorate problem behaviors before they have long-term impact on academic and social development. A better understanding of how to best measure problem behaviors in young children may result in improved early identification of children who are experiencing problem behaviors that may lead to long-term difficulties. The purpose of this study was to examine preschoolers' trajectories of performance during a CPT and to determine how teachers' ratings of inattention, H/I, and oppositional/defiant behaviors (ODB) were jointly and uniquely associated with these trajectories.

The CPT is a computer-administered test that was designed to assess cognitive deficits related to a variety of organic brain injuries and mental disorders (Rosvold, Mirsky, Sarason, Bransome, & Beck, 1956) and has commonly been characterized as measuring different facets of attention (e.g., sustained attention, selective attention) as well as H/I. The CPT requires individuals to view a stimulus sequence, respond to certain stimuli, and withhold responses to other stimuli. Several versions of CPTs exist that vary in stimulus presentation (e.g., stimulus' time interval on the screen, the time interval between displayed stimuli, types of stimuli) and complexity. Versions of the CPT with simple directions (Valiente, Lemery-Chalfant, & Swanson, 2010) and minimal language demands (Greenberg & Waldman, 1993; McGee et al., 2000) have been created for use with preschool children.

There are also differences in the response requirements related to target stimuli across variations of the CPT. The most commonly-used commercially available version of the CPT, the Conners' CPT (Conners, 1993), requires participants to respond to all stimuli except the target stimulus (i.e., NotX-CPT). This contrasts with the parameters of the traditional CPT created by Rosvold et al. (1956) during which participants are required to respond only to the target stimulus (i.e., X-CPT). Although both versions of the CPT belong to the same family of cognitive tasks, research has demonstrated differences in the tasks in terms of performance patterns and susceptibility to environmental influences (Ballard, 2001). Researchers have suggested that the traditional CPT assesses sustained attention whereas CPTs like the Conners' CPT assess response inhibition (e.g., Ballard, 2001; Egeland & Kovalik-Gran, 2010). However, there is a dearth of research directly comparing different versions of the CPT. Therefore, it is not clear whether these different tasks are in fact measuring different constructs. It is likely that both attention and response inhibition are measured to an extent by each version of the CPT. Nevertheless, given the numerous variations of the CPT and the debate regarding the ways in which they differ, it is important to note that a low target-frequency X-CPT was used in the present study.

Many of the studies that examine the link between observable problem behaviors and CPT performance do so in the context of investigating performance characteristics that differentiate children with certain diagnoses (e.g., Attention Deficit/Hyperactivity Disorder [ADHD]; Corkum, Schachar, & Siegel, 1996; Collings, 2003). These studies inform the use of CPT measures in applied settings; however, they do not necessarily provide information about the underlying behavioral and cognitive constructs that are reflected in CPT performance parameters. Despite the common practice of characterizing behavior problems (i.e., inattention, H/I, ODB) as categorical constructs for diagnostic purposes, these behaviors are generally recognized as dimensional constructs (Levy, Hay, McStephen, Wood, & Waldman, 1997). Research is needed that examines the continuous associations between problem behaviors and performance trajectories on the CPT in preschoolers to provide a better understanding of the observable behaviors that may influence CPT performance.

CPTs are used both in research and in practice to assess attention and H/I; however, the correlations between performance on these tasks and behavior ratings of inattention and H/I are typically small (e.g., McGee et al., 2000; Sims & Lonigan, 2012). Teachers are considered valuable informants for behavior ratings because they interact with a relatively large number of same-age children in different settings that require varying levels of attention and restraint (e.g., Evans, Allen, Moore, & Strauss, 2005). In the assessment of ADHD, information provided by teachers helps determine that symptoms exist in multiple settings and cause impairment in daily functioning, as is required for diagnosis (American Psychiatric Association, 2013). However, the weak relation between teacher-ratings and the CPT challenges the assumption that these measures assess the same types of behaviors and underlying attributes. Studies that ascribe a particular performance characteristic to specific behavioral attributes also have been inconsistent and sometimes contradictory. An omission error, which occurs when the child fails to respond to a target stimulus, is presumed to indicate inattention. A commission error, which occurs when the child responds to a non-target stimulus, is presumed to measure H/I (e.g., Barkley, 1991). However, many studies report that certain parameters are not associated with the ostensibly related behavior (e.g., Chhabildas, Pennington, & Willcutt, 2001) or relate indiscriminately to multiple types of problem behaviors (e.g., Epstein et al., 2003). Further, the clinical utility of the CPT has been challenged because scores on the task do not adequately differentiate between children with and without ADHD (McGee et al., 2000).

The relatively weak link between performance on the CPT and behavior ratings may be attributed to the multifaceted nature of attention that makes this construct difficult to operationalize using a single measure. In fact, researchers have suggested that cognitive-attention difficulties may be distinct from real-world behavioral-attention difficulties (e.g., Biederman et al., 2004). Furthermore, although tasks typically are labeled as primarily assessing a specific type of attention (e.g., dichotic listening task measures divided attention), the differentiation of these types of attention is not completely clear, and most tasks likely measure multiple aspects of attention simultaneously. For example, it has been argued that sustained attention is not distinct from selective attention but, rather, the result of multiple trials of selective attention over time (Ballard, 1996). Different parameters (e.g., reaction times, omission errors in the first/last portion of the task) may capture different aspects of attention capacity (Miranda et al., 2012). Thus, the relations between behavior ratings and CPT performance may depend on how performance on the task is operationalized and when during the task measures are taken.

The relation between behavior ratings of inattention and CPT performance may also be impacted by other cognitive and psychological factors (e.g., ODB, intelligence) that potentially influence CPT performance. Children who exhibit ODB perform worse on both low target-frequency (e.g., Hobson, Scott, & Rubia, 2011) and high target-frequency (Youngwirth, Harvey, Gates, Hashim, & Friedman-Weieneth, 2007) versions of the CPT than do their typically developing peers. Furthermore, it is unlikely that children's performance across the CPT task is constant. Distinct behavioral attributes may have different impacts not only on overall performance but also on performance decrements during the task. Although overall numbers of errors on the CPT are used to assess children's attention and H/I, changes in performance on the CPT in terms of omission and commission errors may provide valuable information concerning the source of children's difficulties on the task. For example, hyperactive children may produce numerous commission errors toward the beginning of the task, but they may eventually become so fidgety that they turn their attention away from the task and commit fewer commission errors, lowering the overall number of commission errors recorded. Consequently, distinct problem behaviors may jointly impact observed performance parameters on the CPT.

The majority of studies on the CPT have been conducted with school-age children and adolescents (Brocki, Tillman, & Bohlin, 2010; Corkum et al., 1996; Epstein et al., 2003). Findings associated with the link between behavior-ratings and CPT performance in school-age samples may not generalize to preschool-age samples due to the developmental differences between these age groups. Further, many studies with older children that have examined changes in performance over the course of the task have not included an examination of changes in omission and commission errors (Erdodi, Lajiness-O' Neill, & Saules, 2010; Miranda et al., 2012). This may be because older children typically are able to complete the tasks with high levels of accuracy, making variables associated with reaction time more relevant. Research examining the associations between observable problem behaviors and different aspects of performance on the CPT with preschoolers is needed to provide a stronger basis for the valid use of this task in young children.

In summary, both the CPT and teacher-ratings are commonly used to measure problem behaviors such as inattention and H/I. Recently, there has been increased interest in measuring these constructs effectively in young children. Understanding the extent to which the CPT measures the problem behaviors reported by individuals, such as teachers, who observe preschool children's everyday functioning is important to understanding the validity of performance on this task for identifying inattentive and hyperactive/impulsive behaviors in young children. Although the relations between these two methods of measurement are generally modest (e.g., McGee et al., 2000), researchers typically examine these relationships without accounting for the interrelatedness of informants' ratings of different problem behaviors and often characterize CPT performance based only on overall scores, which may not adequately capture the complexity of performance on the task. To date, no study has examined how different types of problem behaviors, as rated by teachers, are jointly and uniquely associated both with initial performance on the CPT and change in performance over the course of the task in a sample of preschoolers.

Overview of Study

The purpose of this study was to examine the linkages between teacher-rated problem behaviors and different aspects of performance on the CPT using a latent growth curve framework with a sample of preschool-age children. To our knowledge, this is the first study to use a latent growth curve analytic approach to examine the relations between these measures. This analytic technique allows for precise modeling of several aspects of individual differences in performance including initial performance, rate of change in performance during the task, and changes in the rate of change in performance during the task. Understanding the relative associations of different problem behaviors to different components of performance on the CPT will provide a more refined understanding of how the CPT can be used to measure developmentally important constructs that may have different associations with outcomes.

The predictive relation of three forms of teacher-rated problem behaviors (i.e., inattention, H/I, ODB) to performance trajectories across four temporal blocks of the CPT were examined while controlling for several factors, including age (Lin, Hsiao, & Chen, 1999), gender (Gershon & Gershon, 2002), and nonverbal intelligence (Weyandt, Mitzlaff, & Thomas, 2002) that are associated with inattentive and hyperactive/impulsive behaviors. Although evidence regarding the convergent and discriminant relations among these measures is mixed, we based our hypotheses on the conceptualizations of each measure as used both in research and in practice. Given the presumed association between omission errors and inattention (McGee et al., 2000), it was hypothesized that teacher-rated inattention would uniquely predict both the initial number of omission errors and the change in the number of omission errors during the task, but that neither teacher-rated H/I nor teacher-rated ODB would uniquely predict initial number of omission errors or change over time. Given the presumed association between commission errors and H/I (McGee et al., 2000), it was hypothesized that teacher-rated H/I would uniquely predict both the initial number of commission errors and change in the number of commission errors across performance of the task, but that neither teacher-rated inattention nor teacher-rated ODB would uniquely predict initial number of commission errors or change over time.

Method

Participants

Participants included 399 preschool children recruited from 31 preschools in Northeast Florida. Informed consent was obtained from parents prior to their children's participation. Of the 388 children for whom ethnicity was reported, 73.7% were identified as White/Caucasian, 11.9% Black/African American, and 14.4% other. The sample comprised 49.4% females. Participants ranged in age from 34 to 70 months of age (M = 56.16 months, SD = 6.38). Parent report of household income was available for 90% of the sample and had a wide range, with 10% reporting income of $25,000 or less and 30% reporting income greater than $75,000 (M = $ 64,231, SD = 46,930).

Measures

Conners' Teacher-ratings Scale-Revised (CTRS-R)

The Conners' rating scales have been widely used for several decades to assess the presence of problem behaviors in children (Conners, 1997). The CTRS-R has good internal consistency (alphas ranging from .73 - .95) across subscales (Conners, Sitarenios, Parker, & Epstein, 1998), and has good sensitivity (78%) and specificity (91%) for distinguishing children with and without ADHD. This scale contains items assessing three problem-behavior constructs: inattention, H/I, and conduct problems. The 44-item version of the CTRS based on the factor structure outlined by Gerhardstein, Lonigan, Cukrowicz, and McGuffey (2003) was utilized for this study. All three subscales (inattention, H/I, ODB; αs = .90 - .96) had high internal consistency in this sample.

Continuous Performance Test (CPT)

The CPT used in this study was previously created by an affiliated research group using Visual Studio software (e.g., Valiente et al., 2010). It is based on the original design by Rosvold et al. (1956) and variations of the task designed to be developmentally appropriate for preschool children. During the CPT, pictures of common objects (e.g., blocks, a rocking chair, a beach ball) were displayed on a screen and the child was required to press the space bar “ as fast as you can” when the target image (a fish) appeared on the screen. The CPT was administered to children via laptop computers. A 20-trial practice CPT was administered prior to task administration to ensure children understood and were able to complete the task. Examiners provided feedback during the practice trials and were allowed to repeat the practice once if needed. The task lasted approximately 7.3 minutes and consisted of 220 trials (500-ms stimulus presentation followed by a 1500-ms inter-stimuli interval). During the task, the target stimulus appeared 44 times and non-target stimuli appeared 176 times. Examiners remained in the room throughout the duration of the task. Responses were recorded in terms of omission errors and commission errors. In this study, the split-half reliabilities (i.e., rs of omission errors and commission errors from the first and last quarter blocks of performance to the middle two quarter blocks of performance on the task) were adequate (rs > .77).

Nonverbal cognitive ability

The Copying subtest of the Stanford-Binet Intelligence Scale 4th Edition (S-B IV; Thorndike, Hagen, & Sattler, 1986) was used as a control measure of nonverbal cognitive ability. This test comprises 28 items that require the child to duplicate designs made from blocks and drawings. The Copying subtest has adequate internal consistency (coefficients ≥ .81) and test-retest reliability (r = .71) for preschool-age children.

Procedure

The protocol was approved by the local Institutional Review Board and Informed Consent/Permission was obtained from the parents of each participant. Children were recruited from local preschools. Directors and teachers at participating preschools distributed consent forms containing a description of the project to the parents of all 3- to 5-year-old children as an invitation to participate in the study. Only children for whom parents provided consent were included in the study. The CPT and Copying subtest were administered in one session, lasting approximately 20 minutes, in a quiet area of the children's preschools by a trained research assistant. All research assistants had either completed a bachelor's degree in the social sciences or were working toward the completion of a degree and were trained to administer the CPT task and Copying subtest in a standardized fashion. For each participant, one preschool teacher completed the CTRS. Teachers were informed that they would be participating in a study designed to learn more about classroom behavior and the ability to sustain attention in preschool children, but they were blind to the specific purposes and hypotheses of the study.

Results

Descriptive Statistic and Preliminary Data Analyses

Descriptive statistics for raw scores on all tasks are presented in Table 1. Mean imputations were used for the 31 children who were missing data for one or two items on the CTRS at random for the purpose of computing subscale scores. Two children were not administered the CPT because they reportedly refused to participate in testing, seven children were not administered the CPT because they reportedly did not understand the task and failed to complete practice trials successfully, and five children were excluded because data were not collected for all trials of the CPT because of computer error. These children were not included in the sample of 399. Following the correction of outliers (scores +/- 2 interquartile ranges from the median) to their respective fences, CTRS and CPT scores evidenced significant positive skew. Given that this was most likely attributable to the true distribution of these variables in a community-based sample of children, this skew was not corrected. To allow for the examination of change in performance across time, CPT block scores were created by segmenting the 220 trials of the CPT into four blocks of 55 trials, each lasting the same length of time (approximately 1 minute and 50 seconds). For each block, scores were created based on the number of errors in the block. The number of possible omission errors and commission errors was the same for blocks 3 and 4 (i.e., 11 omission errors and 44 commission errors). Block 1 contained one more target stimulus and block 2 one fewer target stimulus, slightly altering the proportion of possible errors. Scores on these blocks were converted so that each of the four blocks would be on the same scale (i.e., 11 and 44 for omission errors and commission errors, respectively). For example, a child who obtained 3 omission errors in the block that had 12 possible omission errors obtained a converted score of 2.75.

Table 1. Descriptive Statistics for Behavioral and Cognitive Measures.

Assessment tool	Mean	SD	Range		Skew
			Minimum	Maximum
CTRS
Inattention	5.96	5.81	0	31	1.07*
Hyperactivity/Impulsivity	12.66	12.29	0	51	1.07*
Oppositional Behavior	6.57	7.04	0	31	1.31*
CPT # Omission Errors
Total	13.47	10.19	0	41	0.72*
Block 1	2.10	2.17	0	8	1.12*
Block 2	2.92	2.69	0	10	0.92*
Block 3	3.69	3.31	0	11	0.88*
Block 4	4.53	3.62	0	11	0.53*
CPT # Commission Errors
Total	13.92	19.89	0	141	2.87*
Block 1	3.12	5.17	0	36	3.53*
Block 2	3.75	6.40	0	44	3.13*
Block 3	3.85	6.86	0	40	2.88*
Block 4	3.20	5.99	0	35	3.15*
S-B IV Copying Subtest scores	4.19	2.11	0	14	1.41*

Note. N = 399; CTRS = Conners' Teacher Rating Scale; CPT = Continuous Performance Test; S-B = Stanford Binet.

^*p < .05.

Partial correlations between teacher-ratings and CPT errors controlling for age, sex, and S-B IV Copying subtest scores are presented in Table 2. Teachers' ratings of inattention, H/I, and ODB were strongly correlated with each other (rs = .49 to .75). In contrast, overall numbers of omission and commission errors on the CPT were not correlated. For earlier blocks of the CPT, teachers' ratings were generally associated with commission errors, but for later blocks of the CPT, teachers' ratings of behaviors were generally associated with omission errors.

Table 2. Partial Correlations between Teacher Ratings of Behavior and the CPT Controlling for Age, Sex, and SB-IV Copying Subtest Scores.

	1	2	3	4	5	6	7	8	9	10	11	12
1. CTRS Inattention	--
2. CTRS H/I	.69***	--
3. CTRS ODB	.49***	.75***	--
4. Block 1 Omissions	.08	-.07-	.07	--
5. Block 2 Omissions	.18***	.08	.05	.54***	--
6. Block 3 Omissions	.16**	.18***	.10+	.38***	.69***	--
7. Block 4 Omissions	.12*	.17**	.09+	.35***	.61***	.72***	--
8. Total Omissions	.17**	.13*	.07	.64***	.86***	.88***	.85***	--
9. Block 1 Commissions	.12*	.19***	.14**	.04	.13	.20**	.20**	.19***	--
10. Block 2 Commissions	.14**	.13**	.09+	-.01	.00	.06	.16**	.07	.59***	--
11. Block 3 Commissions	.06	.07	.06	-.01	-.07	-.12*	.01	-.06	.47***	.54***	--
12. Block 4 Commissions	.05	.05	.07	-.05	-.06	-.07	-.15	-.10*	.34***	.35***	.62***	--
13. Total Commissions	.12*	.13*	.13*	-.01	.03	.05	.08	.06	.71***	.73***	.81***	.70***

Note: N = 399; SB = Stanford-Binet. CTRS= Conners' Teacher Rating Scale; CPT = Continuous Performance Test; H/I = hyperactivity/impulsivity; ODB= Oppositional Behavior.

⁺marginally significant.

^*p< .05.

^**p < .01.

^***p < .001.

Growth Models for Errors on the CPT

Latent growth curve analyses were conducted to examine the associations between teacher-rated behaviors and errors across blocks of performance on the CPT while controlling for age, gender, and S-B IV Copying subtest scores. Children's initial performance and change across blocks of trials in terms of omission and commission errors on the CPT was modeled in Mplus version 5.1 (Muthén & Muthén, 2008). The Maximum Likelihood estimator with robust standard errors (MLR) was used in analyses to account for the non-normality in the data. A sandwich estimator was used to account for the clustering of children within schools. Analyses were conducted for commission errors and omission errors separately.

Unconditional models

An initial examination of the growth models was conducted to determine whether growth in errors was better characterized as linear or curvilinear. A nonsignificant negative residual variance for Block 1 was fixed to zero in both the commission errors and omission-errors models. Results of Satorra-Bentler Chi-Square (S-B χ²) difference tests indicated that the curvilinear growth models for both omission errors and commission errors fit significantly better than did the linear growth models (omission errors: Δ S-B χ² [df = 4] = 64.73, p = .001; commission errors: Δ S-B χ² [df = 4] = 34.74, p = .001). Models containing a quadratic component provided a good fit to the data, with CFI values close to 1.0 (see Table 3). For the omission-errors model, the intercept, linear, and quadratic terms all had significant overall effects and significant variance. The intercept was not significantly correlated with the quadratic term. The linear and the quadratic term were negatively correlated (i.e., children with larger increases overall also had the largest deceleration in change). The intercept and the linear term were negatively correlated such that children who had a lot of initial omission errors demonstrated slower increases in omission errors over the course of the task. For the commission-errors model, the intercept, linear, and quadratic terms all had significant overall effects and significant variance. The intercept was not significantly correlated with the linear or the quadratic term, and the linear and the quadratic term were negatively correlated.

Table 3. Model Fit Indices for Latent Growth Curve Models Predicting Omission Errors and Commission Errors from Teacher Ratings of Behavior Problems.

	Omission Errors
Model	df	S-B χ2	p	CFI	TLI	RMSEA	SRMR
Unconditional Model	2	10.63	.00	.981	.942	.104	.022
Model with Control Variables	5	16.04	.01	.985	.946	.074	.015
Inattention Model	6	16.51	.01	.986	.949	.066	.014
H/I Model	6	17.53	.01	.985	.945	.069	.014
ODB Model	6	16.57	.01	.987	.949	.066	.014
All Predictors Model	8	21.59	.00	.983	.938	.066	.012
	Commission Errors
Model	df	S-B χ2	p	CFI	TLI	RMSEA	SRMR
Unconditional Model	2	5.37	.07	.979	.936	.065	.024
Model with Control Variables	5	10.48	.06	.985	.945	.052	.018
Inattention Model	6	13.16	.04	.982	.934	.055	.019
H/I Model	6	10.68	.10	.989	.957	.044	.017
ODB Model	6	10.29	.11	.989	.960	.042	.016
All Predictors Model	8	11.58	.17	.992	.969	.033	.017

Note: N = 399. H/I = Hyperactivity/Impulsivity; ODB = Oppositional/Defiant Behavior. S-B χ²= Satorra-Bentler Chi-Square. CFI = Comparative Fit Index.TLI = Tucker Lewis Index.RMSEA = Root Mean Square Error of Approximation.SRMR = Standardized Root Mean Square Residual.

An additional analysis was conducted in which omission errors and commission errors were modeled simultaneously to examine the relations between these two types of errors. Number of initial omission errors was not significantly associated with number of initial commission errors (r = .10, p = .13). The linear and quadratic terms for number of omission errors were inversely related to the linear and quadratic terms for number of commission errors (r = -.46, p < .001, r = -.72, p < .001, respectively), demonstrating that an increase in one type of error during the task was associated with a decrease in the other type of error during the task.

Conditional models with control variables

Growth models that included only the control variables as predictors of initial status and growth were examined. The models for both omission errors and commission errors demonstrated good fit to the data (see Table 3). For the omission-errors model, only age predicted intercept (p < .001), and no control variable was uniquely associated with the linear or the quadratic terms. For the commission-errors model, age, sex, and S-B IV Copying scores each significantly and uniquely predicted the intercept (ps < .05) such that older children, children with higher S-B IV copying scores, and girls had lower initial commission errors. No control variable was associated with the linear or the quadratic terms.

Conditional models including teacher-ratings

Growth models that included both control variables and teacher-ratings as predictors were used to examine the relations between teachers' rating of problem behaviors and initial status and increases in errors on the CPT. Models in which each of the three types of teacher-rated problem behaviors was entered alone as a predictor were used to evaluate the general associations between each behavior and CPT performance. Models in which all three teacher-rated behaviors were entered simultaneously as predictors were used to evaluate whether the associations between teacher-rated behaviors and CPT performance were unique or overlapping. All models demonstrated good fit as indicated by CFI values ≥ .97, TLI values ≥ .93, and SRMR values ≤ .03 (see Table 3).

Standardized parameter estimates for prediction of the intercept, linear, and quadratic terms in the omission-errors models are shown in Table 4. Control variables accounted for 12%, 13%, and 12% of the variance in the inattention-only, H/I-only, and ODB-only models, respectively. Teacher-rated inattention marginally predicted the intercept, and it significantly predicted the linear and the quadratic terms for omission errors. Higher ratings of inattention were (marginally) associated with a higher initial number of omission errors, a greater linear increase in number of omission errors over the course of the task, and a greater deceleration in the rate of increase in omission errors over the course of the task. Teacher-ratings of inattention accounted for .4%, 3%, and 5% of the variance in the intercept, linear, and quadratic term, respectively. Neither teacher-rated H/I nor teacher-rated ODB was predictive of the initial number of omission errors; however, teacher-rated H/I and teacher-rated ODB were associated with a greater linear increase in the number of omission errors during the task, with H/I and ODB accounting for 4% and 2% of the variance in the linear term, respectively, when entered separately as predictors.

Table 4. Standardized Estimates for Latent Growth Curve Models Predicting Omission Errors from Teacher Ratings of Behavior Problems.

Teacher-Rated Behavior Predictorsin Model
	Inattention Only			H/I Only			ODB Only			All Predictors
Predictor Variables	intercept	linear	quad	intercept	linear	quad	intercept	linear	quad	intercept	linear	quad
Control Variables
Age	-.30***	-.01	-.04	-.31***	.01	-.05	-.32***	.01	-.05	-.33***	.01	-.04
Copy	-.08	-.03	-.04	-.09	-.06	.00	-.09	-.07	.01	-.06	-.04	-.03
Gender	.02	.00	-.03	-.01	-.01	.00	.01	-.04	.02	.03	.00	-.04
Teacher Rating Predictors
Inattention	.07+	.20*	.23*							.23*	.08	-.25+
H/I				-.07	.22*	-.16				-.20+	.16	.05
ODB							-.06	.16+	-.13	-.02	.00	-.06
Pseudo-r2ˆ	.00	.03	.05	.00	.04	.02	.00	.02	.02	.03	.05	.05

Note: N = 399. Intercept is centered at Block 1. All predictor variables centered at the mean. H/I = hyperactivity/impulsivity; ODB = oppositional/defiant behavior; intercept = intercept estimates; linear = linear term estimates; quad = quadratic term estimates.

r²ˆ= proportion of variance accounted for by behavior predictors in the model.

⁺marginally significant.

^*p< .05.

^**p < .01.

^***p < .001.

Standardized parameter estimates for prediction of the intercept, linear, and quadratic terms in the models for commission errors are shown in Table 5. Control variables accounted for 10%, 11%, and 13% of the variance in the inattention-only, H/I-only, and ODB-only models, respectively. As individual predictors, teacher-rated inattention, H/I, and ODB were all significantly associated with a higher initial number of commission errors, accounting for 2%,3%, and 2% of the variance in the intercept, respectively. Teacher-rated behaviors were not predictive of either the linear or the quadratic terms in the models of commission errors.

Table 5. Standardized Estimates for Latent Growth Curve Models Predicting Commission Errors from Teacher Ratings of Behavior Problems.

Teacher-Rated Behavior Predictors in Model
	Inattention Only			H/I Only			ODB Only			All Predictors
Predictor Variables	intercept	linear	quad	intercept	linear	quad	intercept	linear	quad	intercept	linear	quad
Controls
Age	-.14***	-.07	.06	-.12**	-.07	.06	-.12**	-.07	.06	-.12**	-.07	.06
Copy	-.09+	-.02	.06	-.11*	-.03	.07	-.12*	-.03	.07	-.11*	-.01	.06
Gender	-.22***	-.09	.07	-.21***	-.10	.08	-.24***	-.10	.08	-.21***	-.08	.07
Teacher Rating Predictors
Inattention	.13**	.04	-.06							-.01	.10	-.07
H/I				.19***	-.02	-.02				.20*	-.08	.00
ODB							.13*	-.02	.00	-.01	-.01	.04
Pseudo-r2ˆ	.02	.00	.00	.03	.00	.00	.02	.00	.00	.03	.01	.00

Note: N = 399. Intercept is centered at Block 1. All predictor variables centered at the mean. H/I = hyperactivity/impulsivity; ODB = oppositional/defiant behavior; intercept = intercept estimates; linear = linear term estimates; quad = quadratic term estimates.

r²ˆ= proportion of variance accounted for by behavior predictors in the model.

^*p < .05.

^**p < .01.

^***p < .001.

To examine whether the associations between teacher-rated behaviors and CPT performance were unique or overlapping, all three teacher-rated behaviors were entered simultaneously as predictors in the models. Control variables accounted for 13% of the variance in the omission-errors model and 9% in the commission-errors model. For omission errors (see Table 4), teacher-ratings accounted for 3%, 5%, and 5% of the variance in the intercept, linear, and quadratic terms, respectively. Teacher-rated inattention was uniquely associated with higher initial levels of omission errors, and H/I was marginally associated with lower initial levels of omission errors. No teacher-rated behaviors were uniquely associated with the linear or the quadratic terms, with the exception of teacher-rated inattention which was marginally associated with the quadratic term. For commission errors (see Table 5), the teacher-ratings accounted for 3%, 1%, and 0% of the variance in the intercept, linear, and quadratic terms, respectively. Only teacher-rated H/I was uniquely associated with higher initial levels of commission errors, and teacher ratings were not uniquely associated with either the linear or the quadratic terms.¹

Multi-group analysis

Although a community sample was used in the present study, a multi-group analysis was conducted to provide a basic examination of whether teacher ratings of inattention, H/I, and ODB predicted performance trajectories differently for children with and without elevated levels of problem behaviors. T-scores for the ADHD index were computed from the items on the CTRS-R. Children with t-scores above 56 (i.e., levels of problem behaviors that should raise concern; Conners, 1997) were classified as having elevated levels of problem behaviors. Of the 399 children, 78% (n = 310) were classified as having typical levels of problem behaviors and 22% (n = 89) were classified as having elevated levels of problem behaviors. The mean t-score for the group with elevated levels of behavior problems was 63, which is in the “mildly atypical” range. Children in the elevated group were significantly younger (M = 54.73 months, SD = 6.11 months) than children in the non-elevated group (M = 56.60 months, SD = 6.40 months), t (397) = 2.46, p = .014, and were more likely to be male, χ² (1) = 14.71, p < .001. There also was a significant effect for child ethnicity χ² (2) = 6.47, p < .039. Of the children for whom ethnicity was reported, children identified as African American were more likely than were those identified as Caucasian, B (1) = 0.77, p = .023, and those identified as another ethnicity, B (1) = 1.01, p = .029, to be classified as having elevated levels of problem behaviors. There was no significant difference in the likelihood of classification between children identified as Caucasian and children identified as an ethnicity other than Caucasian or African American.

The baseline multi-group models for both omission errors (S-B χ²= 13.10, df= 4, p = .01; CFI = .982; SRMR = .022) and commission errors (S-B χ²= 6.72, df = 4, p = .15; CFI = .993; SRMR = .027) demonstrated adequate fit to the data. For omission errors, the models in which the effects of the predictor variables on the growth parameters were freed across groups did not fit the data significantly better than did the models in which the effects were constrained to equality across groups for the inattention-only model (Δ S-B χ² [df= 3] = 0.06, P= .99), the H/I-only model (Δ S-B χ² [df= 3] = 2.11, p = .55), the ODB-only model(Δ S-B χ² [df= 3] = 2.20, p= .53), and the combined model(Δ S-B χ² [df = 9] = 5.36, p = .80). Similarly, there were no differences in fit between the constrained versus unconstrained models for commission errors: inattention-only model (Δ S-B χ² [df = 3] = 1.96, p = .58), H/I -only model (Δ S-B χ² [df = 3] = 6.95, p = .07), ODB-only model (Δ S-B χ² [df = 3] = 1.75, p = .62), and combined model (Δ S-B χ² [df = 9] = 9.15, p = .42). These results indicate that teacher-rated problem behaviors did not operate differently as predictors of performance across groups of children with and without elevated levels of problem behaviors.

Discussion

The purpose of this study was to understand how different aspects of children's behavior relate to their performance across time during a CPT task. Results of this study revealed that the rate of children's omission and commission errors on the CPT increased over the course of the task. The peak error rate for omission errors was in the last block of trials, and the peak error rate for commission errors was in the penultimate block of trials. Generally, results examining the independent relations between each teacher-rated behavior and CPT errors yielded evidence of poor discriminant validity; however, stronger evidence of convergent and discriminant validity emerged when the three teacher-rated behaviors were included simultaneously in the model, which controlled for the high degree of overlap between teachers' ratings of these behaviors. These findings have implications for the measurement and conceptualization of problem behaviors in young children.

Support for the hypotheses in the present study was mixed. Only teacher-rated inattention was associated with higher initial rates of omission errors. All three teacher-rated problem behaviors were associated with higher initial rates of commission errors; however, only teacher-rated H/I was uniquely related to initial commission errors. Thus, the hypotheses that only teacher-rated inattention would uniquely predict initial number of omissions errors and that only teacher-rated H/I would uniquely predict initial number of commission errors were supported. Higher rates of teacher-reported problem behaviors were associated with increasing rates of omission, but not commission, errors during the CPT task; however, the relation was not specific to one type of problem behavior. As such, the hypotheses that only teacher-rated inattention would uniquely predict change in the number of omission errors and that teacher-rated H/I would uniquely predict initial number of commission errors were not supported.

Response Trajectories on the CPT

The results of this study indicated that preschool children's performance on the CPT was not constant across the duration of the CPT task. For both types of errors, there were significant increases in errors across the blocks of trials. This is consistent with prior research reporting that older children and adults, both with and without elevated levels of problem behaviors, exhibit increases in omission errors over the course of the CPT (Collings, 2003; Corkum, Schachar, Siegel, 1996). In contrast, prior research with samples of older children and adults has reported little change in commission errors over the course of low target frequency versions of the CPT (Ballard, 2001). In a developmental study of CPT performance, Kanaka et al. (2008) reported significant variability in the number of commission errors up until age six, at which point performance stabilized. Older children and adults typically make few commission errors on low target-frequency versions of the X-CPT. Incorrect hits are more likely to be observed in CPTs with high target-frequencies (Ballard, 2001). The executive skills of preschool-age children are still in an early phase of development (Zelazo & Carlson, 2012), making the inhibitory processes required to avoid incorrect hits more difficult for some young children, even in low target-frequency versions of the X-CPT that are relatively easy for older children and adults.

Commission errors demonstrated a trajectory showing an initial increase in the number of errors followed by a decrease in the number of errors toward the end of the task. Omission errors demonstrated a trajectory showing an increase in the number of errors over the course of the task characterized by a deceleration in the rate of increase toward the end of the task. Although both types of errors generally increased over the course of the task, there were individual differences in the change of the rate of increase in errors over the course of the task. For commission errors, there was no association between the initial numbers of errors and changes in number of errors during the task. For omission errors, higher initial levels of omission errors were associated with slower increases in omission errors during the task. This is possibly due to a ceiling effect in which it was not possible for children with high levels of initial errors to obtain higher numbers of errors in subsequent blocks of the task.

The finding that initial number of omission errors was not significantly related to initial number of commission errors but that the linear and the quadratic terms for the omission-errors model were inversely related to the linear and the quadratic terms for the commission-errors model indicates that increases in one type of error were associated with decreases in the other type of error. This pattern may have resulted, in part, because of a downward turn in number of commission errors that occurred toward the end of the task. That is, children who became disengaged from the task and exhibited an increase in number of omission errors also exhibited a decrease in the number of commission errors--not because their abilities to withhold responses improved but because their motivation to complete the task had waned. Because all children were actively engaged at the beginning of the CPT, this inverse relation was not present at the onset of the task. Therefore, in samples of young children, error rates obtained at the beginning of the task when engagement is high may be more meaningful than error rates obtained later in the task when disengagement, which may not be specifically related to difficulties with attention or H/I, has begun to influence performance.

In terms of control variables, age was significantly associated with initial omission errors and initial commission errors. This finding is consistent with developmental studies of the CPT showing that younger children typically perform worse than older children (Berwid et al., 2005; Brocki et al., 2010). The finding that gender was uniquely associated with commission errors (i.e., boys committed more commission errors than did girls) but not omission errors is partially consistent with research in samples of older children demonstrating that boys show worse performance in terms of both error types (Brocki et al., 2010; Greenberg & Waldman, 1993). For both omission and commission errors, there were no effects of age or gender on the change in number of errors during the tasks. This finding is in keeping with research on school-age children (Brocki et al., 2010) reporting no effects of gender or age on performance decrements over the course of the task in terms of commission errors and no effects of age on performance decrements in terms of omission errors. However, in contrast to our findings, Brocki et al. reported a developmental effect on omission errors during the tasks such that younger children demonstrated greater deterioration than did older children. It may be that the range of ages included in this sample was not large enough to detect developmental differences.

Relations to Teacher-Ratings of Behavior

The results of this study indicate that the pattern of errors on the CPT in preschool samples is complex and may be determined by several factors. Specific indices from the CPT, such as omission and commission errors, represent the joint product of these factors. The pattern of relations between errors on the CPT and teachers' ratings of children's problem behaviors suggests that lower omission errors are a function of attention; however, omission errors are also influenced by higher levels of H/I. Children with higher H/I respond more frequently, and although they make more commission errors as a result, the higher rate of responding reduces the number of omission errors. In this study, the relation between teacher-rated inattention and initial omission errors became substantially stronger when teacher-rated H/I was included in the model and teacher-rated H/I was negatively related to initial omission errors.

The pattern of relations between errors on the CPT and teachers' ratings of children's problem behaviors suggests that commission errors are a function of H/I and do not appear to be substantially influenced by problems of inattention or ODB. In this study, all three forms of teacher-rated problem behaviors were associated with initial commission errors; however, only teacher-rated H/I was uniquely associated with initial commission errors when all three forms of teacher-rated behavior were considered simultaneously. These results suggest that each subscale on the CTRS assesses, in part, behaviors associated with H/I, but that the H/I subscale does assess unique aspects of H/I not assessed by the other subscales. The unique association between teacher-rated H/I and commission errors is consistent with the assertion that commission errors reflect hyperactive/impulsive behaviors specifically (Barkley, 1991).

The results of this study suggest that increases and decreases in omission and commission errors over the course of the CPT task were not related specifically to the underlying processes these errors are presumed to reflect (i.e., inattention, H/I). Increasing omission errors were associated with all three forms of teacher-rated problem behaviors when considered separately, but none of the three subscales was uniquely associated with increasing rates of omission errors when they were considered simultaneously. These results indicate that although levels of inattention were associated specifically with initial number of omission errors, overall problem behaviors contributed to decrements in performance and may have reflected disengagement from the task. It may be that behaviors measured by the inattention subscale of the CTRS (i.e., “inattentive, easily distracted”) influence the ability to respond consistently to the target stimulus from the onset of the task. However, behaviors measured by the H/I (e.g., “cannot remain still”) and ODB (e.g., “defiant”) subscales may not impact performance until time progresses and the novelty of the task diminishes. This finding that overall problem behaviors were associated with decrements in performance across the CPT implies that if sustained attention in young children is defined as decrements in performance over time (van der Meere, & Sergeant, 1988), multiple childhood behavior problems may impact sustained attention capacities on the CPT.

Consistent with results from studies of older children (Brocki et al., 2010; Epstein et al., 2003), teachers' ratings were unrelated to changes in commission errors over the course of the CPT in this study. The absence of a relation between H/I and change in commission errors across the CPT suggests caution in using change in commission errors on the CPT for the purpose of identifying or representing hyperactive/impulsive behaviors in preschool-age children. A likely explanation for the lack of a relation between teacher-rated H/I and temporal changes in commission errors is that H/I is associated with more than one pattern of change in the number of commission errors over the course of the task. Some hyperactive children may begin the task with moderate rates of commission errors and exhibit an increase in these errors as they become increasingly restless. Other hyperactive children may begin the task with high rates of commission errors and become increasingly restless to the extent that they stop responding regularly and exhibit a decrease in these errors toward the end of the task. Research using person-centered data analytic approaches, such as cluster analysis or latent class analysis, would be useful for exploring possible heterogeneity in CPT response patterns. Although the results of the present study suggest that there is not a linear relation between teacher-rated H/I behaviors and changes in commission errors across performance on the CPT, specific patterns of change in performance may provide clinically-relevant information about different manifestations of problem behaviors in preschool children.

The poor discriminant associations between teacher-ratings and neuropsychological measures of inattention and H/I typically reported in the literature (Chhabildas et al., 2001; Epstein et al., 2003) may be due, in part, to a failure to account for the strong interrelatedness of teachers' ratings of distinct behavior domains. In this study, results concerning the individual relations between each teacher-rated behavior and CPT errors generally provided evidence of poor discriminant validity. That is, all three teacher-rated behaviors were associated with both initial number of commission errors and increases in omission errors over time. However, evidence of convergent and discriminant relations between teacher-ratings and performance on the CPT emerged when the three teacher-rated behaviors were included simultaneously in the models. These results show that the strong overlap between teacher-ratings of different behaviors, if not accounted for in analyses, will obscure the associations between specific behavioral attributes and specific aspects of CPT performance. In fact, the poor discriminant relations typically observed between direct-measures and informant ratings of the same behavioral construct, may be the result of the informant ratings.

The substantial interrelatedness among subscales suggests that the multi-dimensionality of the CTRS-R, and other informant-rating scales with highly correlated subscales, may be modeled most accurately using techniques that account for the hierarchical structure of these measures. Much of the item-level variance in these measures may be associated with a general problem behavior factor, rather than a specific factor. Bi-factor models may be ideally suited for modeling both the general and the specific behavioral constructs captured by informant-ratings of problem behaviors (Reise, 2012). Research using this technique could elucidate the underlying structure of informant-ratings and may increase the precision of both research-related and clinically-oriented work using these measures.

Conclusions and Future Directions

The findings of this study indicate that when the interrelations among teachers' ratings of problem behaviors were accounted for, initial error rates on the CPT were associated with teacher-ratings of the problem behaviors they are presumed to reflect. Such findings indicate that initial performance on the CPT is valid for assessing specific problem behaviors. However, the finding that multiple teacher-rated problem behaviors were associated with increases in omission errors over the course of the task challenges the validity of omission errors on the CPT as an indicator of sustained attention. Rather, a variety of general problem behaviors may contribute to the occurrence of omission errors on later blocks of the task. This finding has both research-related and clinical implications for the use of the CPT with preschool children. Whether the CPT is included in a study as the primary measure of interest or as a control variable, researchers should be cautious when using overall omission errors or increases in omission errors over the course of the task as an indicator of sustained attention in young children. Similarly, although clinical decisions are not typically made based solely on CPT scores, clinicians should refrain from interpreting increases in omission errors on the CPT as a specific indicator of problems with sustained attention in preschool-age patients.

This study demonstrated how different types of problem behaviors may influence initial performance and response trajectories during the CPT in a relatively large sample of preschool children; however, it is not without limitations. This study was conducted using a community-based sample of children. Although our results indicated that the relations between teacher-rated behaviors and CPT performance did not differ across groups of children with and without elevated levels of problem behaviors, it may be useful to replicate this study in a clinical sample. Additionally, behavioral ratings completed by other important informants, such as parents and observers, were not available. It may be useful to replicate the present study using behavior ratings completed by other informants to examine how the relations between these measures differ as a function of informant. The results of this study are based on a low target-frequency CPT, which requires response only when a single target image is observed. Because it is not known whether different versions of the CPT (e.g., high target-frequency CPTs) measure the same or different behavioral constructs, it may be important to examine whether these results hold across variations of the CPT.

Given the relatively modest amount of variance accounted for in CPT scores, it may also be useful to conduct analyses including other variables (e.g., working memory; Raiker, Rapport, Kofler, & Sarver, 2012) that may influence CPT performance. Additionally, we also did not have information regarding the disability status or use of medication for this sample of children. Consequently, it is unknown how these factors may have impacted results; however, no child in the sample had frank neurological, sensory, or cognitive impairment, and it seems likely that few preschoolers recruited from a community setting were on medication for problem behaviors. It should also be noted that the examiners remained in the room with the children during task administration. Given that examiner presence may improve task performance for some children (e.g., Power, 1992), the children in our study, particularly those with behavior problems, may have demonstrated poorer performance on the task had examiners not been present.

The results of this study indicated that preschoolers' performance trajectories on the CPT were both similar to and different from those of older children and adults. That is, although preschoolers' trajectories of omission errors mirrored those typically observed in samples of older children, the pattern of commission errors revealed greater variability and deterioration of performance over the course of the task for preschool children than typically reported for older children. Overall, higher levels of teacher-reported inattention and H/I were associated with higher rates of initial omission errors and initial commission errors, respectively. Higher levels of teacher-rated problem behaviors, in general, were associated with greater increases in the number of omission errors over the course of the CPT. Although poor discriminant relations between teacher-ratings and CPT performance were observed when the relations were examined independently, stronger evidence of convergent and discriminant relations across these measures was apparent when the strong overlap between teachers' ratings of different behaviors was accounted for. These results imply that to best understand the relations between these two types of measures, it is necessary to examine multiple aspects of performance on the CPT (i.e., initial performance, change in performance across the course of the task) and to control for the strong interrelatedness found in teachers' ratings of different problem behaviors.