Preschool Neuropsychological Measures as Predictors of Later Attention Deficit Hyperactivity Disorder

Abstract

The present study examined preschool neuropsychological measures as predictors of school-age attention deficit hyperactivity disorder (ADHD). Participants included 168 children (91 males) who completed neuropsychological measures at ages 3 and 4, and who were evaluated for ADHD and oppositional defiant disorder at age 6. The Conners’ Kiddie Continuous Performance Test (K-CPT), NEPSY Statue subtest, and a delay aversion task significantly distinguished at-risk children who later did and did not meet criteria for ADHD, with poor to fair overall predictive power, specificity, and sensitivity. However, only the K-CPT ADHD Confidence Index and battery added incremental predictive validity beyond early ADHD symptoms. This battery approach, which required impairment on at least 2 of the 3 significant measures, yielded fair overall predictive power, specificity, and sensitivity, and correctly classified 67% of children. In addition, there was some support for the specificity hypothesis, with evidence that cool executive function measures (K-CPT and Statue subtest) tended to predict inattentive symptoms. These findings suggest that neuropsychological deficits are evident by preschool-age in children with ADHD, but neuropsychological tests may still misclassify approximately one-third of children if used alone. Thus, neuropsychological measures may be a useful component of early ADHD assessments, but should be used with caution and in combination with other assessment methods.

Attention deficit hyperactivity disorder (ADHD) is one of the most commonly diagnosed childhood disorders, and is characterized by developmentally deviant levels of inattention, hyperactivity, and/or impulsivity. Children with ADHD are classified under three presentations: (i) primarily hyperactive/impulsive, (ii) primarily inattentive, or (iii) combined, depending on whether hyperactivity/impulsivity or inattentive symptoms, or both, are more predominant (American Psychiatric Association, 2013). ADHD symptoms can cause significant impairment in children’s social, emotional, and academic functioning (e.g., Mash & Barkley, 2006). Although ADHD is typically diagnosed at school age, ADHD symptoms are thought to emerge during the preschool years (Applegate et al., 1997). In recognition of this, the American Academy of Pediatrics recently extended downward the recommended earliest age for ADHD diagnosis to age 4 (American Academy of Pediatrics, 2011). However, accurate early diagnosis, which would allow access to early intervention, is complicated by the fact that hyperactivity/impulsivity and inattention are common in early childhood, and some preschoolers who display these behaviors outgrow their symptoms (Harvey, Youngwirth, Thakar, Errazuriz, 2009; Lahey et al., 2004).

Neuropsychological theories of ADHD posit that a variety of neuropsychological deficits underlie the development of ADHD, including working memory, response inhibition, cognitive flexibility, delay aversion, interference control, planning, and sustained attention (see Nigg, 2005; Pauli-Pott & Becker, 2011; Willcutt, Doyle, Nigg, Faraone, & Pennington, 2005). Abnormalities in the neural circuits that control these functions may result in difficulty adapting behavior to contextual circumstances. Furthermore, these circuits are thought to be fine-tuned with experience in typically developing children, allowing early hyperactive/impulsive and inattentive behaviors to consolidate into more sophisticated self-regulation skills. Children with early deficits in neuropsychological functions may be unable to benefit from experience, instruction, and correction, and may therefore continue to display problematic hyperactive/impulsive and inattentive behaviors later in childhood (Nigg, 2005).

Associations Between Neuropsychological Deficits and ADHD in Preschoolers

Consistent with neuropsychological theories of ADHD, two meta-analyses of studies using a wide range of preschool neuropsychological measures have found that delay aversion, response inhibition, and sustained attention measures were consistently associated with ADHD specifically (Pauli-Pott & Becker, 2011), and externalizing behavior problems more generally (Schoemaker, Mulder, Deković, & Matthys, 2013). The mean effect sizes for delay aversion tasks (e.g., delay-of-gratification and resistance-to-temptation tasks) in both meta-analyses were medium (r = .38 and r = .34, respectively). The mean effect sizes for response inhibition tasks such as the Continuous Performance Test (CPT), A Developmental NEuroPSYchological Assessment (NEPSY) Statue subtest, and Go/No Go tasks were somewhat smaller but still of medium magnitude in both meta-analyses (r = .29 and r = .26, respectively; Pauli-Pott & Becker, 2011; Schoemaker et al., 2013). The mean effect size for sustained attention (vigilance/arousal) measures (collected using computerized CPTs) in the Pauli-Pott and Becker (2011) meta-analysis was also of medium magnitude (r = .27). Results of more recent studies, published after, or not included, in these meta-analyses, also demonstrate associations between ADHD symptoms and delay aversion (Dougherty et al., 2011; Sjöwall, Bohlin, Rydell, & Thorell, 2015), response inhibition (Healey, Marks, & Halperin, 2011; Martel, Roberts & Gremillion, 2013), and sustained attention (Healey et al., 2011; Sjöwall et al., 2015). Mean effect sizes for working memory in the Pauli-Pott and Becker (2011) and Schoemaker et al. (2013) meta-analyses were found to be small, but significant (r = .18 and r = .17, respectively).

Few studies have examined whether preschool neuropsychological deficits are predictive of later ADHD symptoms, and these have generally focused on older preschoolers. Rajendran et al. (2013) found that, among children with ADHD, neuropsychological functioning at age 3–4 and at age 4–5 predicted ADHD severity at ages 5–6 and 6–7. Similarly, in a large community sample (N = 776), 4-year-old children’s performance on response inhibition, sustained attention, and delay-of-gratification tasks predicted ADHD symptoms in 1^st and 3^rd grade (Campbell & von Stauffenberg, 2009; von Stauffenberg & Campbell, 2007). However, Wåhlstedt, Thorell, and Bohlin (2008) found that among 4- to 6-year-old children, impairment on a composite of executive function measures, including working memory, response inhibition, and verbal fluency, was not predictive of hyperactivity 2 years later, and was not predictive of inattention when controlling for IQ. A few additional studies have focused on older preschool/early elementary children. In a national population-based sample of preschool children, inhibition measured at age 5 predicted ADHD symptoms 3 years (Berlin, Bohlin, & Rydell, 2003) and 13 years later (Sjöwall et al., 2015). In the same sample, working memory measured at age 6.5 was predictive of ADHD symptoms at age 18; however, only the relation between working memory and inattention symptoms remained significant when controlling for ADHD symptoms in preschool (Sjöwall et al., 2015). In a community sample of 5-year-olds oversampled for disruptive behavior symptoms, poor inhibition (Bohlin, Eninger, Brocki, & Thorell, 2012; Brocki, Eninger, Thorell, & Bohlin, 2010; Brocki, Nyberg, Thorell, & Bohlin, 2007) and selective attention (Brocki et al., 2010) predicted ADHD symptoms 2 years later; however, the predictive ability of working memory was less consistent, with one study finding it to be predictive of later ADHD symptoms (Brocki et al., 2010) and another failing to find this relation (Brocki et al., 2007). A study using a community sample of 452 children, oversampled for risk of externalizing and internalizing disorders, found that children assigned ADHD diagnoses based on assessments at an 18-month follow up were impaired in working memory, inhibition, and sustained attention performance at ages 5 and 6, compared to typically developing controls (Kalff et al., 2005; Kalff et al., 2002). Three of these studies controlled for early ADHD symptoms (Sjöwall et al., 2015; von Stauffenberg & Campbell, 2007; Wåhlstedt et al., 2008) and two of these studies (Bohlin et al., 2012; Campbell & von Stauffenberg, 2009) controlled for early general externalizing symptoms (including impulsivity, aggression, and oppositional defiance). In summary, longitudinal research suggests that preschool neuropsychological deficits are part of a developmental pathway to ADHD. However, studies are needed to build on this small body of research with a particular focus on investigating the extent to which neuropsychological deficits at an early age are predictive of ADHD diagnoses, and to what extent neuropsychological deficits predict ADHD beyond the influence of early behavior problems.

Predicting ADHD Diagnostic Status using Preschool Neuropsychological Tests

Relatively few studies have examined the power of neuropsychological tests for discriminating children with and without ADHD, and those have generally focused on older children. These studies have found that tests of sustained attention, working memory, response inhibition, and set shifting had fair to poor overall classification rates (e.g., Berlin, Bohlin, Nyberg, & Janols, 2004; Doyle, Biederman, Seidman, Weber, & Faraone, 2000; Kaufmann et al., 2010; Perugini, Harvey, Lovejoy, Sandstrom, & Webb, 2000; Wada, Yamashita, Matsuishi, Ohtani, & Kato, 2000). Moreover, studies have found that neuropsychological test batteries that assess multiple neuropsychological functions are better than individual tests at discriminating children with ADHD from controls, with good overall classification rates (Gupta, Kar, & Srinivasan, 2011; Hale et al., 2009). In an earlier study, we (Youngwirth, Harvey, Gates, Hashim, & Friedman-Weieneth, 2007) presented the first examination of the predictive power of neuropsychological tests for preschool-aged children and found that measures of behavioral inhibition, working memory, and sustained attention had fair to good overall predictive power (OPP) to distinguish between non-problem preschool children and children with elevated symptom reports of hyperactivity at age 4. The study, however, did not examine the power of these tests to predict future ADHD diagnoses.

Identifying Children Who Develop ADHD Among Preschoolers with Behavior Problems

To date, studies have almost exclusively focused on examining the power of neuropsychological measures for distinguishing between children with ADHD and non-problem controls (e.g., Doyle et al., 2000; Perugini et al., 2000). However, of perhaps even greater clinical utility is the determination of whether tests distinguish preschoolers who are displaying transient behavior problems from preschoolers with behavior problems who will later be diagnosed with ADHD. Clinically, mental health specialists will more often be faced with the task of distinguishing these two groups, rather than having to distinguish preschoolers with ADHD from children who present with no behavior problems. There is evidence that among preschoolers with elevated inattention/hyperactivity symptoms, poor neuropsychological functioning is predictive of later ADHD symptom severity (e.g., Rajendran et al., 2013). However, studies with older children have found that although neuropsychological functioning is predictive of ADHD a few years later, it is not able to differentiate ADHD persistence or remittance in adulthood (for a review see van Lieshout, Luman, Buitelaar, Rommelse, Oosterlaan, 2013). To our knowledge no studies have examined whether preschool neuropsychological functioning predicts later ADHD diagnoses among children with behavior problems. The present study extends the existing literature by examining the power of neuropsychological tests to distinguish between at-risk preschool children who later outgrow their behavior problems, and at-risk preschool children who later meet criteria for ADHD.

Distinguishing Between ADHD and ADHD/ODD Using Neuropsychological Measures

Between one-third and one-half of children with ADHD also meet criteria for oppositional defiant disorder (ODD; Bendiksen et al., 2014; Nock, Kazdin, Hiripi, & Kessler, 2007). A few studies have investigated whether children with “pure” ADHD have neuropsychological deficits that are different from children with co-morbid ADHD/ODD. Some of these studies have found that children with ADHD/ODD perform comparably to children with pure ADHD on neuropsychological tests (Barnett, Maruff, & Vance, 2009; Clark, Prior, & Kinsella, 2000; Kalff, et al., 2002; Nigg, Hinshaw, Carte, & Treuting, 1998; Skogan et al., 2014; Qian, Shuai, Cao, Chan, & Wang, 2010), whereas others have found that children with the co-morbid diagnoses have more severe deficits (Jennings, van der Molen, Pelham, Debski, & Hoza, 1997; Skogan et al., 2014; Van der Meere, Marzocchi, & De Meo, 2005). Still other studies have found that children with ADHD/ODD perform better on neuropsychological measures than children with pure ADHD, and similarly to controls (Munkvold, Manger, & Lundervold, 2014; Shuai, Chan, & Wang, 2011). These studies have all been cross-sectional, with mostly samples of elementary/middle school-aged children. The only study to use a preschool sample found differences depending on the type of neuropsychological measure (Skogan et al., 2014). Specifically, children with ADHD and ADHD/ODD both performed significantly worse than typically developing children on measures of inhibition, but children with ADHD/ODD performed significantly worse than both typically developing children and children with ADHD on measures of working memory. A longitudinal examination of whether neuropsychological functions in at-risk preschoolers can predict which children will develop ADHD/ODD and which will develop only ADHD would add to this literature.

Predicting Hyperactive/Impulsive and Inattentive ADHD Symptoms

Some have suggested that differential patterns of neuropsychological deficits may underlie different phenotypic presentations of ADHD, such that deficits in “cool” executive functions (e.g., response inhibition and working memory) would be related to inattentive symptoms, whereas performance on “hot” motivation-related tasks, or “hot” executive functions (e.g., delay aversion and resistance-to-temptation), would be related to hyperactive/impulsive symptoms (Castellanos, Sonuga-Barke, Milham, & Tannock, 2006; Sonuga-Barke, 2005). Results of studies testing this “specificity hypothesis” have been mixed. Some have found that “cool” executive function measures were related only to inattentive symptoms (Jarrett, Gilpin, Pierucci, & Rondon, 2015; Miller, Miller, Healey, Marshall, & Halperin, 2013). Others have found that both “hot” and “cool” executive functions were related to inattentive symptoms (Chhabildas, Pennington, & Willcutt, 2001; Martel, Nigg, & von Eye, 2009; Martel, Roberts, Gremillion, von Eye, & Nigg, 2011) and hyperactive/impulsive symptoms (Chhabildas et al., 2001; Jarrett et al., 2015; Martel et al., 2009; Martel et al., 2011; Miller et al., 2013). However, there is some evidence that when inattention symptoms are controlled for, hyperactive/impulsive symptoms are not significantly related to impairment on neuropsychological measures (Chhabiladas et al., 2001). An examination of the ability of different preschool neuropsychological measures to differentially predict the development of later hyperactive/impulsive versus inattentive symptoms would extend the literature, which again, has been mostly cross-sectional.

The Present Study

Only a handful of longitudinal studies have examined whether preschoolers’ performance on neuropsychological measures predict later ADHD symptoms (e.g., Bohlin et al., 2012; Campbell & von Stauffenberg, 2009; Wåhlstedt et al., 2008). Even fewer studies have controlled for early ADHD symptoms to examine the extent to which neuropsychological deficits predict ADHD beyond the influence of early symptomatology (Sjöwall et al., 2015; von Stauffenberg & Campbell, 2007; Wåhlstedt et al., 2008). To our knowledge, there are not yet any longitudinal studies that have examined whether preschool tests of neuropsychological functioning during the early preschool years can predict later ADHD diagnoses made using clinical, multimodal assessments. Moreover, longitudinal studies are needed to determine whether neuropsychological deficits can predict ADHD diagnoses among at-risk children. The present study adds to the current literature by examining the following questions:

1) Do preschool neuropsychological measures predict later ADHD diagnoses?

The present study examined the power of neuropsychological measures to distinguish preschoolers with behavior problems who receive ADHD diagnoses at school age (ADHD group) from those who exhibited behavior problems during their preschool years, but did not receive a diagnosis at age 6 (at-risk non-ADHD group). Additionally, it sought to test whether neuropsychological measures are predictive of later ADHD, above and beyond preschool hyperactivity/impulsivity and inattention symptoms. We hypothesized that measures of response inhibition, sustained attention, working memory, and delay aversion would distinguish the ADHD group from the at-risk non-ADHD group, and that these measures would add incremental predictive validity above early ADHD symptoms.

2) Do preschool neuropsychological measures predict which children who later develop ADHD will also meet criteria for ODD?

The present study further examined whether preschool neuropsychological measures predict which children who develop ADHD also develop ODD at age 6. Given the mixed research findings on this topic, specific hypotheses were not made.

3) Are preschool neuropsychological measures differentially predictive of later hyperactive/impulsive and inattentive symptoms?

The present study examined whether measures of response inhibition, sustained attention, working memory, and delay aversion are differentially predictive of inattentive versus hyperactive/impulsive symptoms. It was hypothesized that preschool measures of response inhibition and working memory (“cool” executive function measures) would predict inattentive symptoms, whereas delay aversion (a “hot” executive function measure) would predict hyperactive/impulsive symptoms at age 6.

Method

Participants

Participants were 168 children (77 girls, 91 boys) and their families drawn from a larger sample who participated in a longitudinal research study focused on ADHD and ODD development among preschool-aged children. Children presented with elevated levels of externalizing problems at age 3. The present study focuses on measures collected from children at age 3 (M = 44.36 months, SD = 3.32), age 4 (M = 56.81, SD = 3.66), and age 6 (M = 80.80, SD = 5.12). The sample was ethnically diverse and consisted of 90 European American (53.6%), 17 African American (10.1%), 38 Latino/a (predominately Puerto Rican; 22.6%), and 23 multi-ethnic participants (13.7%). Years of education ranged from 8 to 20 for mothers (M = 13.41, SD = 2.91) and fathers (M = 13.40, SD = 2.76). Participants’ median annual family income was $47,108.

Procedure

All participants were recruited through local pediatricians’ offices, community and child care centers, and birth record listings in western Massachusetts. The parents of 1,752 3-year-old children completed a screening packet including the Behavior Assessment System for Children (BASC; Reynolds & Kamphaus, 1992) and a questionnaire assessing exclusion criteria, parental concern for externalizing symptoms, and demographic information. Children with externalizing problems (N =199) were recruited using the following inclusion criteria: (a) Parent responded “yes” or “possibly” to “Are you concerned about your child’s activity level, defiance, aggression, or impulse control?” and (b) BASC Hyperactivity and/or Aggression subscale T-scores were at least 65 (92^nd percentile). Exclusion criteria included: intellectual disability, deafness, blindness, language delay, cerebral palsy, epilepsy, autism, and/or psychosis. A smaller group of children without behavior problems was also recruited but were not included in this study. Children and parents were administered assessments during annual home visits from age 3 to age 6. Children were not asked to discontinue medication for the home visits. At age 3, none of the children were on medication for ADHD; however, two children were taking Depakote and one child was taking Collicy, a sedative. At age 4, seven children were on a stimulant medication for ADHD, two children were on Clonidine, one child was on Depakote, and one child was on Risperdal. Children were assigned diagnoses of ADHD and ODD based on multimodal assessments at age 6. The 168 externalizing children who completed follow-up assessments at age 6 are the focus of this study. The 168 children in the present study did not differ from the 31 children with behavior problems who did not complete an assessment at 6 years of age on gender; age; age 3 inattention, hyperactivity/impulsivity, or ODD symptoms; maternal or paternal education; or family income (ps > .22). However, the 31 children who dropped out were more likely to be African American or multi-ethnic and less likely to be Latino or European American, compared to the 168 children included in the study, χ² (3)= 12.64, p = .01. Payment was given to families who participated in the study. Parents provided written informed consent for their child’s participation. This study was conducted in compliance with the authors’ Institutional Review Board.

Measures

M&M'S® Task

The M&M'S® delay task was adapted from the Snack Delay task developed to measure inhibitory control in preschoolers (Kochanska, Murray, Jacques, Koenig, & Vandegeest, 1996). The M&M'S® task was administered to children when they were 3 years old. For this task, the experimenter placed two M&M'S® under one of three identical cups that were placed in front of the child, within arm’s length. The child was instructed to wait to find the M&M'S® until the experimenter rang a bell. Six trials of this task were conducted, with the following delay periods: 10, 15, 25, 35, and 45 seconds. Children were given a score of 1 for each trial in which they were able to wait until the bell rang before finding the M&M'S®, and a score on 0 for each trial in which they responded before the bell rang. Overall scores for this task were therefore out of a total of 6, with lower scores indicating greater delay aversion.

Present Task

The present task was adapted from Kochanska and colleagues’ (1996) battery of tasks designed to measure various aspects of effortful control and was administered at age 3. For this task, the experimenter presented the child with a colorfully wrapped package, and then put it to the side at arm’s length of the child, informing the child that they may open it later. The experimenter then pretended to complete paperwork for 5 minutes, while the child completed a simple puzzle. If the child opened the present, the task was terminated, the time elapsed recorded, and the child was allowed to keep the present. After 5 minutes, if the child had refrained from opening the present, the child was told that he/she could now open the gift. The length of time (in secs) children were able to delay opening the present is the score for this task.

NEPSY subtests

Children’s neuropsychological abilities were assessed during home visits when they were 4 years old using subtests from the first edition of the NEPSY (Korkman Kirk, & Kemp, 1998). This study focused on the Statue, Sentence Repetition, and Visual Attention subtests as measures of inhibition, working memory, and sustained attention, respectively. The Statue subtest requires the child to maintain a body position with eyes closed during a 75-second time interval and to inhibit the impulse to respond to sound distracters. In the Sentence Repetition subtest, the child is read sentences and asked to recall each sentence immediately after it is presented. Finally, the Visual Attention subtest assessed the child’s ability to focus and maintain attention to a visual target within an array. Test-retest reliability for the Sentence Repetition and Visual Attention subtests are reported to be good for 4-year-old children (.84 and .76 respectively), though it is relatively low for the Statue subtest (.50; Korkman et al., 1998). Scaled scores were used for all three NEPSY subtests.

K-CPT

The K-CPT was administered when the children were 4 years old. It is designed to differentiate between 4- and 5–year-olds with or without ADHD and has been shown to do so reliably (Conners & Staff, 2001). The task lasts 7.5 minutes and consists of objects flashing on the screen in either 1.5s or 3s stimulus intervals during which the child is supposed to press the spacebar for every picture unless the picture is of a ball. The K-CPT divides the test into five equal time blocks and then scores the child’s proficiency on 11 dimensions. This study focuses on 2 dimensions: omissions is the number of pictures the child incorrectly refrained from pressing the space bar and commissions is the number of the times the child incorrectly pressed the spacebar when a ball appeared. T-scores were used for both omissions and commissions. Additionally, the computer program calculates an ADHD Confidence Index. This Confidence Index gives a prediction of the likelihood that the child has ADHD. The ADHD Confidence Index is produced by a discriminant function analysis consisting of the following variables: % Omissions; gender; age; Standard Error by ISI; Hit Reaction Time; Response Style (Beta); Attentiveness (d′); and Reaction Time by Block (Conners & Staff, 2001).

McCarthy Numerical Memory Subtest

The Numerical Memory I subtest of the McCarthy Scales of Cognitive Abilities (McCarthy, 1972) was used as a measure of working memory and was administered when the children were 3 years old. The Numerical Memory I subtest requires children to immediately repeat a list of numbers back to the experimenter. This subtest is one of four subtests making up the memory scale, which has demonstrated adequate test-retest reliability (McCarthy, 1972). Raw scores were used for this subtest as only half of the subtest was used in the current study¹.

Kaufman-Assessment Battery for Children (K-ABC) Hand Movements Subtest

The Hand Movements subtest of the K-ABC (Kaufman & Kaufman, 1983) was administered to children when they were 3 years old and served as a measure of working memory. The Hand Movement subtest requires children to copy a series of hand movements, ranging in length from two to six movements per sequence. Seventeen sequences of increasing difficulty were presented. The K-ABC has been found to demonstrate good reliability and strong construct, predictive, and concurrent validity for children ages 2 to 12 (Kaufman & Kamphaus, 1984). Scaled scores were used for this subtest.

BASC-PRS

The BASC-PRS (Reynolds & Kamphaus, 1992) is a comprehensive rating scale measuring many dimensions of both adaptive and problem behaviors in children between 2.5 and 21 years of age. This study utilized the Hyperactivity and Aggression subscales. The BASC-PRS has previously demonstrated good reliability and validity (Reynolds & Kamphaus, 1992), and reliability for this sample was also good (α = .85 for Hyperactivity and α = .81 for Aggression). The BASC has been shown to distinguish children with and without ADHD (Doyle, Ostrander, Skare, Crosby, & August, 1997). The BASC was used to screen children for enrollment in the study, and was also used to inform ADHD diagnoses.

Diagnoses

Trained psychology graduate students assigned diagnoses of ADHD and ODD based on the following measures administered at age 6: Diagnostic Interview Schedule for Children–IV (NIMH DISC-IV; Shaffer, Fisher, Lucas, Dulcan, & Schwab-Stone, 2000), BASC (for mother, father, and teacher), and Disruptive Behavior Rating Scale (for mother and father; Barkley & Murphy, 1998). The full computerized version of the DISC-IV was administered, with the ADHD and ODD sections used to inform decisions about diagnoses of ADHD and ODD. Convergent evidence of clinically significant ADHD symptoms (≥6 hyperactive/impulsive and/or ≥6 inattentive symptoms; BASC Hyperactivity or Attention problems T-score of at least 65) either at home or at school was required for an ADHD diagnosis. Convergent evidence of clinically significant ODD symptoms (≥4 ODD symptoms; BASC Aggression T-score of at least 65) was required for an ODD diagnosis. Additional criteria included evidence of impairment and no evidence that the symptoms were better accounted for by another clinical disorder; evaluation of these criteria were made by clinicians using information collected during the diagnostic interview and a psychosocial interview. Two clinicians independently made diagnoses after reviewing materials. Discrepancies were discussed, and a consensus diagnosis was reached. Kappa was .78 for ADHD and .75 for ODD. Of the 168 behavior problem children who completed T4 assessments, 36 (21%; 20 boys, 16 girls) met criteria for ADHD only, 22 (13%; 13 boys, 9 girls) met criteria for ODD only, and 39 (23%; 26 boys, 13 girls) met criteria for ADHD and ODD. Of the 75 children meeting criteria for ADHD, 6 met criteria for ADHD-primarily inattentive type (8%), 13 met criteria for ADHD-primarily hyperactive/impulsive type (17%), and 56 met criteria for ADHD-combined type (75%). A significant minority of children also met criteria for other comorbid psychopathology based on the NIMH DISC-IV, including specific phobia (n = 25; 15.1%), nocturnal enuresis (n = 13), separation anxiety disorder (n = 12; 7.2%), and social phobia (n = 5; 3%). The following disorders were seen in fewer than 5 children in the sample: post-traumatic stress disorder (n = 4; 2.4%), encopresis (n = 3; 1.8%), tic disorder (n = 3; 1.8%), obsessive compulsive disorder (n = 2; 1.2%), major depression (n = 2; 1.2%), generalized anxiety disorder (n = 1; 0.6%), dysthymia (n = 1; 0.6%).

Analytic Plan

Descriptive statistics, including means, standard deviations, and intercorrelations were first examined. To test whether preschool neuropsychological measures discriminate children who later meet criteria for ADHD, a series of Chi-square tests and classification analyses were conducted. Odds-ratios were calculated as measures of effect size. Dichotomous variables were created for each neuropsychological measure, using cutoffs that corresponded to the base rate of the disorder in this sample. Specifically, 45 percent of children met criteria for ADHD at age 6, so for each neuropsychological measure a cutoff score was selected such that approximately 45 percent of children scored below that score (or above the score for measures where high scores indicated poor performance)². The following dummy-coded variables were constructed as outcome variables: ADHD (1; 75 children) vs. at-risk non-ADHD (0; 93 children); and ADHD only (1; 36 children) vs. comorbid ADHD and ODD (0; 39 children). To measure the power of neuropsychological tests for predicting ADHD diagnosis of children at age 6, we conducted classification analyses using five measures of predictive power: (a) positive predictive power (PPP), which is the proportion of subjects who receive an impaired score who have the disorder; (b) negative predictive power (NPP), which is the proportion of subjects who do not have an impaired score who do not have the disorder being tested; (c) OPP, which is the proportion of subjects who are correctly classified by the test; (d) sensitivity, which is the proportion of individuals with the disorder who are correctly classified by the test; and (e) specificity, which is the proportion of individuals who do not have the disorder who are correctly classified. Based on previous studies (Grodzinsky & Barkley, 1999), we classified predictive power greater than .8 as high, from .7 to .8 as good, from .6 to .7 as fair, and less than .6 as poor.

Next, for measures that were able to significantly distinguish between at-risk children with and without ADHD, we examined if these measures remained significant controlling for family income, and if they would exhibit incremental predictive validity above and beyond the predictive validity of preschool hyperactivity/impulsivity and inattention symptoms. A series of hierarchical logistic regressions were run for these analyses. For the first model (Model A), Step 1 included family income; Step 2 included the neuropsychological measure or battery variable. For the second model (Model B), Step 1 included family income, age 3 NIMH DISC-IV hyperactivity/impulsivity symptoms, and age 3 NIMH DISC-IV inattention symptoms; Step 2 included the neuropsychological measure or battery variable.

Finally, Poisson regressions were conducted to examine the relation between the neuropsychological measures (as continuous variables) and age 6 NIMH DISC-IV hyperactivity/impulsivity and inattention symptoms. Regressions were run separately for age 6 NIMH DISC-IV hyperactivity/impulsivity and inattention symptoms, and were conducted both with and without controlling for the other dimension (i.e., predicting age 6 hyperactivity/impulsivity symptoms controlling for age 6 inattention symptoms and predicting age 6 inattention symptoms controlling for age 6 hyperactivity/impulsivity symptoms). Because family income was associated with several neuropsychological measures and with ADHD symptoms, all Poisson regressions were run again with family income in the model to determine whether observed relations could be accounted for by socioeconomic status.

Missing Data

A portion of the children were missing data on at least one neuropsychological measure. Sample sizes for each measure were as follows: NEPSY Sentence Repetition: n = 150; NEPSY Statue: n = 145; NEPSY Visual Attention: n = 144; McCarthy Numerical Memory: n = 143; Present task: n = 138; M&M'S® task: n = 131; K-ABC Hand Movements: n = 127; and the K-CPT: n = 102³. For the Present and M&M'S® tasks and K-ABC, McCarthy, and NEPSY subtests, missing data were due to test refusals or invalid administrations. A number of children (n = 46) had invalid data on the K-CPT (either stopped partway through, or they did not complete enough blocks to generate a score), with an additional 15 children having missing data due to technical difficulties (e.g., the computer battery died in the middle of administration). The remaining children were missing data because the family did not participate in visits when children were 4 years old or for a variety of other reasons (e.g., child left to use bathroom or refused testing). The proportion of children who refused or were unable to complete each subtest did not differ significantly across the groups for any of the subtests (all ps > .20). To address missing data, multiple imputation was used. Five imputations were run with 1,000 iterations. For Chi-square analyses, the pooled contingency table was used for predictive power; to calculate the Chi-Square statistic an average of the statistics from the five imputations was used. For the Poisson regressions, the pooled output from the five imputations was used.

Results

Descriptive Statistics

Intercorrelations and descriptive statistics for predictor and outcome variables are presented in Table 1. Intercorrelations among neuropsychological measures ranged from small to large. Income was significantly correlated with ADHD symptoms and neuropsychological measures. Means and standard deviations for each neuropsychological measure are presented for each of the five groups in Table 2.

Table 1.

Descriptive Statistics and Intercorrelations for Predictor and Outcome Variables

	M (SD)	Range	1	2	3	4	5	6	7	8	9	10	11	12
1. Income	55,600 (40,609)	55,000–380,000	--
Delay Aversion
2. Present a	232.28 (102.94)	5–300	.18*	--
3. M&M'S® a	4.97 (1.60)	1–6	.19*	−.05	--
Inhibition
4. NEPSY Statue b	10.42 (2.56)	4–14	.14†	.23*	.14	--
5. K-CPT Commission Errors b	55.45 (8.60)	30.0–72.5	−.02	−.15	.06	−.01	--
6. NEPSY Sentence Repetition b	10.33 (2.58)	4–16	.28***	.18*	.26*	.29**	−.13	--
7. K-ABC Hand Movement a	9.84 (2.85)	4–18	.07	.10	.15†	.24*	−.25*	.26**	--
8. McCarthy Numerical Memory a	4.65 (1.76)	0–10	.21*	.25**	.36**	.30**	−.13	.63***	.24**	--
Attention
9. K-CPT Omission Errorsb	53.32 (12.19)	36.9–95.3	−.17†	−.25**	−.16	−.39*	−.18†	−.35***	−.27*	−.23*	--
10. NEPSY Visual Attention b	10.45 (1.95)	4–15	.01	−.04	−.11	.03	.09	.09	.19†	.00	−.06	--
Inhibition/Attention
11. K-CPT ADHD Confidence Index b	52.38 (17.64)	1.8–98.7	−.12	−.36	−.04	−.42**	.01	−.33***	−.42**	−.21†	.77***	−.17†	--
Symptom Counts
12. DISC Att Symptoms c	2.90 (2.87)	0–9	−.16*	−.19*	−.02	−.33***	−.06	−.22**	−.16†	−.15†	.26**	−.13	.32***	--
13. DISC HI Symptoms c	3.79 (2.62)	0–9	−.23**	−.19*	−.14	−.24**	−.01	−.19*	−.14	−.12	.27**	−.03	.23***	.60

Note. NEPSY = A Developmental NEuroPSYchological Assessment, K-CPT = Kiddie Continuous Performance Test, K-ABC= Kaufman-Assessment Battery for Children, DISC = Diagnostic Interview Schedule for Children, Att = Inattention, HI = Hyperactivity/Impulsivity,

^ameasure was administered when children were 3 years old,

^bmeasure was administered when children were 4 years old,

^cmeasure administered when children were 6 years old.

^†p < .10,

^*p < .05,

^**p < .01,

^***p < .001

Table 2.

Means, Standard Deviations, and Effect Sizes for Four Diagnostic Groups

	1. All ADHD M(SD) n = 75	2. ADHD only M(SD) n =36	3. ADHD+ ODD M(SD) n =39	4. At-Risk No-ADHD M(SD) n =93	1 vs. 4 Cohen’s d	2 vs. 3 Cohen’s d
Delay Aversion
Present a	206.23 (111.81)	212.16 (113.12)	200.31 (111.98)	254.80 (89.38)	.48	.11
M&M'S® a	4.68 (1.75)	4.62 (1.86)	4.74 (1.65)	5.24 (1.41)	.35	.09
Inhibition
NEPSY Statue b	9.52 (2.84)	9.34 (2.65)	9.68 (3.02)	11.11 (2.09)	.64	.12
K-CPT Commission Errors b	54.71 (7.94)	55.03 (7.82)	54.34 (8.29)	55.95 (9.04)	.15	.09
Working Memory
NEPSY Sentence Repetition b	9.74 (2.41)	9.65 (2.55)	9.82 (2.32)	10.79 (2.62)	.42	.07
K-ABC Hand Movement a	9.43 (2.61)	8.85 (2.84)	9.91 (2.35)	10.19 (3.02)	.27	.41
McCarthy Numerical Memory a	4.52 (1.80)	4.63 (2.12)	4.44 (1.54)	4.74 (1.73)	.12	.10
Attention
K-CPT Omission Errors b	58.66 (15.21)	60.11 (17.03)	56.98 (13.05)	49.72 (7.94)	.74	.21
NEPSY Visual Attention b	10.28 (1.87)	10.23 (1.88)	10.32 (1.89)	10.59 (2.01)	.16	.05
Inhibition/Attention
K-CPT ADHD Confidence Index b	60.45 (20.50)	61.40 (20.85)	59.35 (20.61)	46.96 (13.02)	.79	.10

Note. NEPSY = A Developmental NEuroPSYchological Assessment, K-CPT = Kiddie Continuous Performance Test, K-ABC = Kaufman-Assessment Battery for Children, DISC = Diagnostic Interview Schedule for Children,

^ameasure was administered when children were 3 years old,

^bmeasure was administered when children were 4 years old.

^†p < .10,

^*p < .05,

^**p < .01,

^***p < .001

Do Preschool Neuropsychological Measures Predict Later ADHD Diagnosis?

Chi-square tests were conducted to examine whether neuropsychological measures significantly distinguish at-risk children with ADHD from those without ADHD. The NEPSY Statue subtest, the Present task, and the K-CPT Omission Errors and ADHD Confidence Index⁴ significantly predicted whether at-risk children later met criteria for ADHD. We examined the clinical utility of these measures by calculating their predictive power to classify individuals into these two groups. The predictive power of neuropsychological measures in discriminating at-risk children with ADHD from at-risk children without ADHD is presented in Table 3. The four measures that significantly predicted group membership between these groups (Statue, Present, Omission Errors, and ADHD Confidence Index) exhibited poor to fair OPP, PPP, NPP, sensitivity, and specificity.

Table 3.

Predictive Power of Neuropsychological Measures in Discriminating At-Risk ADHD vs. At-Risk Non-ADHD Children

Neuropsychological Measure	Sensitivity	Specificity	NPP	PPP	OPP	χ2	Odds Ratio	95% Confidence Interval for Odds Ratio	Cut-Off
Delay Aversion
Present a	0.55	0.66	0.64	0.57	0.61	7.41**	2.36	1.26 – 4.40	297
M&M'S® a	0.53	0.61	0.61	0.53	0.57	3.37†	1.78	0.95 – 3.33	5
Inhibition
NEPSY Statue b	0.62	0.56	0.65	0.53	0.59	5.42*	2.08	1.12 – 3.86	10
K-CPT Commission Errors b	0.45	0.62	0.58	0.49	0.54	1.06	1.32	0.71 – 2.44	54
Working Memory
NEPSY Sentence Repetition b	0.54	0.54	0.59	0.49	0.54	1.23	1.38	0.75 – 2.55	10
K-ABC Hand Movement a	0.53	0.58	0.60	0.51	0.55	1.85	1.52	0.82 – 2.83	9
McCarthy Numerical Memory a	0.66	0.49	0.64	0.50	0.56	3.51†	1.81	0.96 – 3.42	4
Attention
K-CPT Omission Errors b	0.55	0.64	0.64	0.55	0.60	6.37*	2.20	1.18 – 4.09	54
NEPSY Visual Attention b	0.45	0.63	0.59	0.49	0.55	0.95	1.36	0.73 – 2.55	10
Inhibition/Attention
K-CPT ADHD Confidence Index b	0.62	0.68	0.69	0.61	0.65	14.93***	3.44	1.82 – 6.50	53
Battery
Battery	0.65	0.69	0.71	0.63	0.67	19.52***	4.15	2.17 – 7.92	--
Battery + age 3ADHD Symptoms c	0.64	0.75	0.72	0.67	0.70	33.57	5.34	2.74 – 10.36	--

Note. PPP = positive predictive power, NPP = negative predictive power, OPP = overall predictive power, NEPSY = A Developmental NEuroPSYchological Assessment, K-CPT = Kiddie Continuous Performance Test, K-ABC = Kaufman-Assessment Battery for Children, DISC = Diagnostic Interview Schedule for Children,

^ameasure was administered when children were 3 years old,

^bmeasure was administered when children were 4 years old,

^cincluded the battery and age 3 hyperactivity/impulsivity and inattention symptoms. Battery of measures included NEPSY Statue, Present, and CPT ADHD Confidence Index. For analyses, at-risk ADHD children were coded 0 and at-risk no-diagnosis children were coded 1.

^†p < .10,

^*p < .05,

^**p < .01,

^***p < .001

To examine whether a battery of neuropsychological measures may be useful in discriminating children with ADHD from at-risk children without ADHD, a battery was examined consisting of measures that significantly distinguished at-risk children with and without ADHD: the NEPSY Statue subtest, Present task, and the ADHD Confidence Index. Although the K-CPT Omissions measure was also significant, it was not included because the ADHD Confidence Index was comprised in part by Omissions. First, a dichotomous variable was constructed where 0 represented children with impairment on zero or one measure (124 children) and 1 represented children with impairment on two or three measures (44 children). The neuropsychological battery predicted which at-risk children developed ADHD with good NPP, and fair OPP, PPP, specificity, and sensitivity. To examine if the battery had greater utility in combination with early ADHD symptoms, a logistic regression was run with the battery and age 3 hyperactivity/impulsivity and inattention symptoms entered as predictors. This combined assessment method predicted which at-risk children developed ADHD with good OPP, NPP, and specificity, and fair PPP and sensitivity (Table 3).

Logistic regression analyses indicated that after taking family income into consideration, the Present task, K-CPT Confidence Index, and battery were all significantly predictive of ADHD diagnosis at age 6 (Table 4), and the Statue subtest approached significance (p = .05). When age 3 hyperactivity/impulsivity and inattention symptoms were added to Step 1 in Model B, only the ADHD Confidence Index and the battery added incremental predictive validity.

Table 4.

Validity of Neuropsychological Measures in Predicting Age 6 ADHD Diagnoses

		NEPSY Statue b (SE)	Present Task b (SE)	K-CPT Omission Errors b (SE)	K-CPT ADHD Confidence Index b (SE)	Battery b (SE)
Model A	Step 1
	Family Income	.00 (.00)**	.00 (.00)**	.00 (.00)**	.00 (.00)**	.00 (.00)**
	Step 2
	Neuropsychological Measure	.66 (.34)†	.73 (.35)*	.60 (.37)	1.12 (.34)**	1.32 (.38)**
Model B	Step 1
	Family Income	.00 (.00)*	.00 (.00)**	.00 (.00)*	.00 (.00)*	.00 (.00)*
	Age 3 Inattention	.02 (.08)	.03 (.08)	.02 (.08)	.02 (.09)	.02 (.09)
	Age 3 Hyperactivity/Impulsivity	.33 (.11)**	.30 (.11)**	.32 (.11)**	.30 (.11)**	.28 (.11)*
	Step 2
	Neuropsychological Measure	.56 (.36)	.55 (.37)	.52 (.41)	.98 (.36)**	1.11 (.38)**

NEPSY = A Developmental NEuroPSYchological Assessment, K-CPT = Kiddie Continuous Performance Test

^†p < .10,

^*p < .05,

^**p < .01

Do Preschool Neuropsychological Measures Discriminate Children with Pure ADHD from Children with Comorbid ADHD and ODD?

No measures significantly discriminated children who had ADHD only from children with ADHD and ODD (Table 5), but the K-ABC Hand Movement test approached significance, χ² (1) = 3.71, p = .07. This measure demonstrated fair OPP, NPP, and sensitivity, but poor PPP and specificity.

Table 5.

Predictive Power of Neuropsychological Measures in Discriminating Children with ADHD vs. Children with ADHD+ODD

Neuropsychological Measure	Sensitivity	Specificity	NPP	PPP	OPP	Chi-Square	Odds Ratio	Cut-Off
Delay Aversion
Present a	0.54	0.44	0.51	0.47	0.49	0.06	0.90	297
M&M'S® a	0.47	0.40	0.45	0.42	0.43	1.26	0.59	5
Inhibition
NEPSY Statue b	0.67	0.43	0.59	0.52	0.55	1.04	1.55	10
CPT Commission Errors b	0.44	0.55	0.52	0.48	0.50	0.12	0.97	54
Working Memory
NEPSY Sentence Repetition b	0.62	0.53	0.60	0.55	0.57	1.66	1.80	10
K-ABC Hand Movement a	0.65	0.58	0.65	0.59	0.61	3.71†	2.57	9
McCarthy Numerical Memory a	0.62	0.31	0.48	0.45	0.46	0.54	0.76	4
Attention
CPT Omission Errors b	0.54	0.45	0.52	0.48	0.50	0.03	0.98	54
NEPSY Visual Attention b	0.47	0.57	0.54	0.50	0.52	0.19	1.17	10
Inhibition/Attention
CPT ADHD Confidence Index b	0.63	0.39	0.53	0.49	0.51	0.30	1.10	53

Note. PPP = positive predictive power, NPP = negative predictive power, OPP = overall predictive power,

^ameasure was administered when children were 3 years old,

^bmeasure was administered when children were 4 years old. For analyses, children with ADHD were coded as 1 and children with ADHD+ODD were coded as 0.

^†p < .10

Are Preschool Neuropsychological Measures Differentially Predictive of Later Hyperactive/Impulsive and Inattention Symptoms?

Using the age 6 NIMH DISC-IV symptom counts for hyperactivity/impulsivity and inattention symptoms, Poisson regressions were used to examine the relation between “hot” and “cool” executive function measures and hyperactivity/impulsivity and inattention symptoms (see Table 6). Higher age 6 NIMH DISC-IV hyperactivity/impulsivity and inattention symptom counts were predicted by lower scores on the Statue subtest, Sentence Repetition subtest, and Present task, and higher scores on the Omission Errors and ADHD Confidence Index. Controlling for hyperactivity/impulsivity symptoms, lower scores on the Statue subtest and higher K-CPT ADHD Confidence Indexes were related to higher levels of inattention symptoms. These two measures are both “cool” executive function measures. However, when controlling for inattention symptoms, no measures predicted age 6 NIMH DISC-IV hyperactivity/impulsivity symptoms. Changes to results when controlling for family income are footnoted in Table 6.

Table 6.

Relation Between Neuropsychological Measures and Age 6 Hyperactive and Inattentive Symptoms

	DISC Symptom Count		DISC Symptom Count (Controlling for Other Symptom Domain)
Measure	Inattention b (SE)	Hyperactivity/Impulsivity b (SE)	Inattention b (SE)	Hyperactivity/Impulsivity b (SE)
Delay Aversion
Present a	−0.002 (0.001)*c	−0.001 (0.001)*c	−0.001 (0.001)	0.000 (0.000)
M&M'S® a	−0.011 (0.053)	−0.055 (0.032)†d	0.045 (0.046)	−0.047 (0.027)†
Inhibition
NEPSY Statue b	−0.114 (0.027)***	−0.061 (0.020)**	−0.077 (0.024)**	−0.013 (0.017)
CPT Commission Errors b	0.006 (0.012)	−0.001 (0.006)	0.005 (0.010)	−0.004 (0.006)
Working Memory
NEPSY Sentence Repetition b	−0.085 (0.029)**	−0.050 (0.020)*c	−0.045 (0.026)†d	−0.018 (0.017)
K-ABC Hand Movement a	−0.054 (0.028)†	−0.034 (0.022)	−0.022 (0.027)	−0.017 (0.021)
McCarthy Numerical Memory a	−0.079 (0.047)†d	−0.046 (0.030)	−0.039 (0.038)	−0.014 (0.025)
Attention
CPT Omission Errors b	0.018 (0.006)**	0.014 (0.004)**	0.006 (0.005)	0.006 (0.004)
NEPSY Visual Attention b	−0.061 (0.035)†	−0.010 (0.026)	−0.069 (0.035)†e	0.012 (0.024)
Inhibition/Attention
CPT ADHD Confidence Index b	0.017 (0.004)***	0.009 (0.003)*	0.010 (0.004)**	0.002 (0.003)

^ameasure was administered when children were 3 years old,

^bmeasure was administered when children were 4 years old,

^csignificant relation becomes marginally significant (p < .10) when controlling for income,

^dsignificant relation becomes non-significant (p > .10) when controlling for income,

^emarginally-significant trend becomes significant (p < .05) when controlling for income.

^†p < .10,

^*p < .05,

^**p < .01,

^***p < .001

Discussion

This study examined the extent to which performance on neuropsychological tests during the preschool years were predictive of later ADHD. Preschool performance significantly distinguished at-risk children who later meet criteria for ADHD from at-risk children who did not meet diagnostic criteria at age 6. The predictive power of these measures ranged from poor to fair, and when used as a battery, accurately classified approximately 70% of children. Moreover, the ADHD Confidence Index and battery were able to predict later ADHD above and beyond early ADHD symptoms. Neuropsychological measures were associated with both later hyperactivity/impulsivity and inattention symptoms, but there was only modest support for the notion that “hot” and “cool” measures are differentially predictive of symptoms. Several “cool” neuropsychological measures were predictive of later inattention controlling for hyperactivity, but not vice versa.

Preschool Neuropsychological Measures Predict Later ADHD Diagnosis

The present study is one of only a handful of longitudinal studies that have examined preschoolers’ performance on neuropsychological tests as predictors of later ADHD symptoms (Berlin et al., 2003; Bohlin et al., 2012; Brocki et al., 2007, 2010; Campbell & von Stauffenberg, 2009; Sjöwall et al., 2015; von Stauffenberg & Campbell, 2007; Wåhlstedt et al., 2008). It is one of even fewer studies to examine the predictive ability of neuropsychological measures in young preschoolers (Campbell & von Stauffenberg, 2009; von Stauffenberg & Campbell, 2007). Extending the findings of these studies, the present study found that preschool children with deficits in delay aversion, sustained attention, and inhibition were more likely to meet criteria for ADHD diagnoses in the future. This study also extended the findings of previous studies by demonstrating that these measures significantly discriminated children with ADHD from at-risk children without ADHD. This finding is of great clinical utility, as mental health specialists are often faced with the task of distinguishing preschoolers who are displaying transient behavior problems from preschoolers with behavior problems who will later be diagnosed with ADHD.

The measures of memory used in this study are commonly used to measure working memory in preschoolers (e.g., Kalff et al., 2002; Sjöwall et al., 2015; Skogan et al., 2014); however, one might argue that these are in fact simply measures of short-term memory (i.e., short-term recall rather than manipulation). Some previous studies of preschoolers have used just short-term recall tasks (Sjöwall et al., 2015), some have used a mix of short-term recall and manipulation tasks (Berlin et al., 2003; Brocki et al., 2007; Kalff et al., 2002; Skogan et al., 2014; Wåhlstedt et al., 2008), and some have used only manipulation tasks (Brocki et al., 2010) to measure working memory. The McCarthy Numerical Memory II subtest, a more traditional measure of working memory, was also administered to children in this sample, but the majority of 3-year-olds were unable to complete a single item. Thus, our lack of findings could be explained by our use of short-term recall tasks as opposed to measures of working memory that require manipulation. However, previous studies of preschoolers have found a relation between working memory and later ADHD symptoms using short-term recall tasks (Sjöwall et al., 2015; Skogan et al., 2014). Alternatively, our lack of findings could be explained by our younger sample of 3- and 4-year-old children; however, there has been some evidence of a relation between working memory and later ADHD symptoms in children as young as 3 years old (Skogan et al., 2014). Finally, and most likely, our lack of results may be due to our sample size. Because effect sizes have been small for working memory (Pauli-Pott & Becker, 2011; Schoemaker et al., 2013), it could be that only studies with large samples can find significant effects. A comparison of the effect sizes found in the present study for two of our working memory measures (Sentence Memory and Hand Movement) are similar to those found by Skogan et al. (2014), who found significant results but had a much larger sample size than the present study. Thus, it may be that deficits in working memory in young children with ADHD are more subtle and require greater power to detect.

Within this at-risk sample, we found effect sizes that were mostly consistent with those reported by previous meta-analyses (Pauli-Pott & Becker, 2011; Schoemaker et al., 2013), with a few important exceptions. Specifically, our study found mean effect sizes for delay aversion tasks (d = .48 and d = .35) that were just slightly smaller than the medium effect sizes found in the meta-analyses. The effect sizes of some of our response inhibition tasks, including the K-CPT and Statue subtest (d = .64 for Statue and d = .79 for ADHD Confidence Index), were larger than the medium effect sizes found in both meta-analyses, though the effect size of Commission Errors was small in the present study, d = .15. Similarly, the effect sizes of some of our sustained attention measures (Omission Errors, d = .74; ADHD Confidence Index; d = .79) were larger than the magnitudes found in the Pauli-Pott & Becker (2011) meta-analysis, but our effect size was small for the Visual Attention subtest (d = .16). Finally, our effect sizes for working memory were generally consistent with the small effect sizes found in the meta-analyses (Sentence Repetition, d = .42; Hand Movement, d = .27; and Numerical Memory, d = .12).

Our findings stand in contrast to previous studies with older children that have found that neuropsychological deficits in childhood did not predict ADHD persistence (for a review see van Lieshout et al., 2013). It may be that neuropsychological measures can predict which preschoolers will develop ADHD, but not whether children with ADHD will outgrow their diagnoses. It is likely that the processes involved in outgrowing ADHD symptoms are quite different for preschoolers than for older children. Many preschool-aged children may be showing ADHD symptoms as part of a developmental phase and outgrow these difficulties during normal maturation. In contrast, it may be that ADHD symptoms in older children are more likely a result of a clinical disorder rather than a developmental phase, and subsequent remittance of this disorder is a result of neurological changes over time. Neuropsychological tests may be sensitive in detecting differences between preschool children whose behavior problems are due to ADHD and preschool children who are exhibiting behavior problems as part of a developmental phase, but not in predicting subsequent neurological changes in children once they have ADHD.

Although several neuropsychological tests significantly predicted later ADHD diagnosis, the accuracy of these tests in predicting which children would later have ADHD was generally modest. The K-CPT ADHD Confidence Index demonstrated the strongest ability to distinguish children with ADHD from at-risk children who did not later have ADHD, accurately classifying 65% of children. However, many children struggled to complete the K-CPT, limiting the clinical utility of this measure. The NEPSY Statue subtest exhibited the second-highest overall predictive power, accurately classifying 59% of children, despite its low test-retest reliability. Consistent with prior research (Youngwirth et al., 2007; Lovejoy et al, 1999), combining multiple measures resulted in somewhat higher predictive power, accurately classifying 67% of children. However, completing a battery as opposed to just completing a single measure, such as the K-CPT, only improved our ability to accurately classify children by 2 percentage points. Thus, if clinicians are limited on time, administering the K-CPT would be relatively comparable to a battery. Even with a battery, neuropsychological measures still misclassify 33% of preschool children with behavior problems, and almost 40% of children who perform poorly on neuropsychological measures do not later meet criteria for ADHD. These rates are somewhat higher than the false positive rates found when predicting later ADHD diagnosis from preschool ADHD symptoms alone, which is approximately 20 to 30% (Harvey et al., 2009; Lahey et al., 2004; Lahey, Pelham, Loney, Lee, & Willcutt, 2005). In our sample, the OPP found for our battery was somewhat lower than the OPP for age 3 DISC-IV ADHD symptoms alone (which correctly classified 69% of children) and the DISC-IV ADHD symptoms in combination with the maternal BASC (which correctly classified 76% of children; Harvey et al., 2009). Although classification rates of the battery are modest, they are higher than one might expect given the challenge of discriminating among children with behavior problems, rather than simply distinguishing clinical from non-clinical samples. Additionally, these neuropsychological measures may be useful when a clinician is concerned about the validity of an individual parent’s reports of child symptoms, or when there are discrepancies between two parents or between a parent and a teacher. Further research is needed to evaluate whether neuropsychological tests may provide greater utility in these circumstances.

Preschool Neuropsychological Measures Add Incremental Predictive Validity Above Early ADHD Symptoms

Although neuropsychological measures were predictive of later ADHD, only the K-CPT ADHD Confidence Index and battery provided incremental validity above age 3 inattention and hyperactivity/impulsivity symptoms. Specifically, children who scored high on the K-CPT Confidence Index or who showed impairment on at least two out of three measures were significantly more likely to later receive a diagnosis of ADHD. Few studies have controlled for early ADHD symptoms when examining the relation between performance on neuropsychological measures and later ADHD, though other studies have controlled for externalizing problems more broadly. Our findings are somewhat consistent with those of Sjöwall and colleagues (2015), von Stauffenberg and Campbell (2007), and Wåhlstedt and colleagues (2008) who found inhibition, working memory, and sustained attention to be predictive of later ADHD symptoms, after controlling for earlier ADHD symptoms.

Preschool Neuropsychological Measures Do Not Discriminate Children with “Pure” ADHD from Children with Comorbid ADHD and ODD

This was the first study to examine whether performance on neuropsychological tests during the preschool years can predict the development of later ADHD versus ADHD+ODD. In this study, none of the neuropsychological measures discriminated children with “pure” ADHD from children who were diagnosed with comorbid ADHD and ODD at age 6. Corroborating previous cross-sectional research with older children (Barnett et al., 2009; Clark et al., 2000; Nigg et al., 1998; Qian et al., 2010) and extending research to a preschool population, these findings suggest that children with ADHD and ODD perform comparably to children with pure ADHD on neuropsychological measures. However, the findings are in contrast to other cross-sectional research demonstrating that children with comorbid ADHD and ODD have more severe deficits than children with pure ADHD (Jennings et al., 1997; Van der Meere et al., 2005), and those demonstrating that children with comorbid ADHD and ODD perform better than children with pure ADHD (Munkvold et al., 2014; Shuai et al., 2011). These mixed findings may be due to use of different measures of ADHD across studies (formal diagnosis, clinical symptom levels in a community sample), different sample characteristics, or differences in neuropsychological measures. It is also possible that our lack of findings is partially a result of our sample size, which was smaller than for our comparisons of children with and without ADHD. However, examination of effect sizes suggests that any differences were small in size, with the exception of the K-ABC Hand Movement subtest (d = .41), which was marginally significant in distinguishing children with ADHD only from children with ADHD and ODD.

“Hot” and “Cool” Executive Functions

Consistent with previous research, there were significant bivariate relations between both “hot” and “cool” executive functions and inattentive symptoms (Chhabildas et al., 2001; Martel et al., 2009, 2011) and hyperactive/impulsive symptoms (Chhabildas et al., 2001; Jarrett et al., 2015; Martel et al., 2009; Martel et al. 2011; Miller et al., 2013). However, when relations were examined controlling for the other dimension of ADHD, the present study found modest support for the notion that “hot” and “cool” executive functions may play different roles in the phenotypic expression of ADHD (Castellanos et al., 2006). In particular, two “cool” executive function measures predicted inattentive symptoms, controlling for hyperactive/impulsive symptoms: the Statue subtest and the K-CPT ADHD Confidence Index. These results are consistent with previous studies finding that “cool” executive function measures were related only to inattentive symptoms (Jarrett et al., 2015; Miller et al., 2013). In contrast, and consistent with the findings of Chhabiladas and colleagues (2001), no measures predicted hyperactivity/impulsivity, controlling for inattentive symptoms. However, inattention and hyperactivity/impulsivity were so highly correlated that it may be difficult to tease apart differential relations with neuropsychological functioning. Indeed, in several cases, measures that exhibited simple relations with inattention and hyperactivity/impulsivity were not significantly related to either one when the other was controlled.

Limitations

The results of the present study should be interpreted in the context of several limitations. First, testing was conducted in children’s homes rather than at the laboratory; future research should replicate this study in the laboratory setting. Second, some measures were administered when the children were 3 years old, and others when the children were 4 years old. Since the neural circuits underlying neuropsychological functions are thought to be fine-tuned with experience, it is possible that the same ability measured at different time points may have differential predictions, due to developmental maturation. Third, the Statue subtest, while having relatively high predictive power, displayed low test-retest reliability, which might limit the clinical utility of this measure. Finally, the sample of children with ADHD was too small to examine if predictive power varied for different subtypes of ADHD.

Future Directions and Implications

Despite these limitations, the present study contributes to the existing literature by investigating the extent to which neuropsychological functions at an early age are predictive of later clinically significant ADHD problems, above and beyond early behavior problems. This is an important first step towards clarifying the role of early neuropsychological functions in the development of ADHD, as well as improving methods of early identification of risk for ADHD. Additionally, the findings of this study support neuropsychological theories of ADHD positing that neuropsychological deficits underlie the development of ADHD symptoms (see Nigg, 2005; Pauli-Pott & Becker, 2011; Willcutt et al., 2005). In conclusion, this study adds to the growing body of research suggesting that neuropsychological measures are moderately useful for predicting future ADHD in preschoolers and is the first to suggest that they may be useful in discriminating between at-risk preschoolers with transient behavior problems, and those who will later meet clinical criteria for ADHD.