Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2017 Apr 1.
Published in final edited form as: Assessment. 2016 Jul 27;24(3):279–289. doi: 10.1177/1073191116646443

Psychometrically Informed Approach to Integration of Multiple Informant Ratings in Adult ADHD in a Community-Recruited Sample

Michelle M Martel 1, Joel T Nigg 2, Ulrich Schimmack 3
PMCID: PMC5085895  NIHMSID: NIHMS778226  PMID: 27126924

Abstract

Although Diagnostic and Statistical Manual of Mental Disorders–Fifth edition requires that attention-deficit/hyperactivity disorder (ADHD) symptoms are apparent across settings, assessed by multiple informants, there remains no standardized approach to integration of multiple sources in adult ADHD diagnosis. The goal of the study was to evaluate informant effects on adult ADHD symptom ratings. Participants were 406 adults, ages 18 to 37, and identified second reporters, recruited from the community, and completing a comprehensive diagnostic and cognitive assessment, including a clinician-administered diagnostic interview and self- and other-report questionnaires of ADHD symptoms. Structural equation modeling indicated good fit for a trifactor model of ADHD, including general ADHD, specific inattention and hyperactivity–impulsivity, and self- and other-perspective factors. Yet there were a number of symptoms on the specific hyperactive–impulsive and self-factors that exhibited nonsignificant loadings. Significant differential item functioning across self-ratings and informant ratings was also noted. The external validation indices of laboratory executive function and diagnostic team-rated impairment was significantly correlated with the specific inattentive factor. While executive function was marginally significantly correlated with the other perspective factor, impairment was associated with the self-perspective factor. Overall, inattentive symptoms may be more sensitive measures of adult ADHD, and other and self-ratings may provide different information in relation to external criteria.

Keywords: assessment, diagnosis, adults, structural equation modeling, ADHD

The Diagnostic and Statistical Manual of Mental Disorders–Fifth edition (DSM-5) continues to operationalize attention-deficit/hyperactivity disorder (ADHD) with a list of nine inattentive and nine hyperactive–impulsive behavioral symptoms, of which diagnosis requires that individuals must manifest six in one of the two symptom domains for children, or five for adults, as well as substantial interference in functioning occurring in two or more settings (e.g., at home and at school or work; American Psychiatric Association [APA], 2013). The DSM-5 text strongly encourages clinicians to obtain information from more than one informant covering more than one setting. The text (APA, 2013, p. 36) explains that “several inattentive or hyperactive-impulsive symptoms are present in two or more settings (e.g., at home, school, or work; with friends or relative; in other activities)” and further states that “confirmation of substantial symptoms across setting typically cannot be done accurately without consulting informants who have seen the individual in those settings” (pp. 37–38). A key gap in the DSM is that it does not specify how the information from the different informants should be combined. This issue is particularly problematic for adults since they may have a more difficult time identifying a second reporter besides themselves depending on their living situation and occupational setting.

Adult ADHD diagnosis is valid and substantially stable from childhood (Faraone & Biederman, 2005; Kessler et al., 2006), now typified in the DSM-5, but evidence-based guidelines for diagnosis of ADHD in adulthood are still emerging (APA, 2013). Although prior research, conducted mostly in childhood, has established moderate convergent validity of multiple informant ratings of psychopathology (De Los Reyes, 2013; De Los Reyes et al., 2011; De Los Reyes et al., 2015; Dirks, De Los Reyes, Briggs-Gowan, Cella, & Wakschlag, 2012), there is less work on multiple informant ratings of psychopathology in adulthood. Some work on ADHD symptom report in adulthood suggests similar, moderate convergence across raters of adult ADHD, usually the self and some other, typically a parent or partner (Adler et al., 2006; Barkley, Knouse, & Murphy, 2011; Biederman et al., 2006; Erhardt, Epstein, Conners, Parker, & Sitarenios, 1999; Kessler et al., 2007). However, it should be noted that this remains controversial as some studies question the validity of adult self-report of adult ADHD symptoms, and there is some suggestion that adults with ADHD underreport or overreport on their symptoms in particular (see seminal book chapters by Barkley, Murphy, & Fischer, 2008; see also Barkley, Fischer, Smallish, & Fletcher, 2002; Klein et al., 2012; Mannuzza, Klein, Bessler, Malloy, & LaPadula, 1993, 1998; Molina & Sibley, 2014; Sibley et al., 2012). In all, these prior studies have provided crucial documentation that examining convergence across self-report and other report of ADHD symptoms in adulthood is critically important to advancing clinical practice and diagnosis of ADHD in adulthood.

There is currently no consensus about how to assess for such convergence or about the optimal way to integrate symptom ratings across multiple informants to diagnosis adult ADHD. At present, researchers and clinicians use a variety of ad hoc approaches. Following the approach taken in the DSM-IV field trials (Lahey et al., 1994), the most common approach utilized to date in research studies is the “or” algorithm. The “or” algorithm specifies that a symptom is present if at least one rater endorses a symptom as present. After determining whether a symptom is present, the manifest symptoms are then added up to create a sum score (e.g., five out of nine inattentive symptoms endorsed by either rater). This approach to integration of symptom ratings across multiple informants is simple, easily standardized, and reasonably easy to utilize in clinical and research settings. Yet this approach also has several problems that may lower the validity of ADHD diagnoses (Solanto & Alvir, 2009; Valo & Tannock, 2010). For example, if one rater overreports or agrees to all symptoms, ratings of the other rater are essentially ignored because it is sufficient for one rater to endorse a symptom. This defeats the intent of demonstrating or confirming that symptoms are substantially present in two or more settings by confirmation of two or more raters. Furthermore, depending on the nature of the second reporter’s ratings, underreporting may not be able to be detected. These undesirable limitations of the “or” algorithm thus can undermine the intended benefit of multimethod assessments where adding more data sources should increase validity (Campbell & Fiske, 1959; Connelly & Ones, 2010). Thus, despite its convenience, it is far from clear that the “or” algorithm yields the most valid diagnosis of ADHD. Other methods have their own weaknesses, however, so the question remains as to whether alternatives can do better.

Innovations in advanced modeling approaches have the potential to shed light on the best way to integrate cross-informant ratings, but have been little utilized in this context. A trifactor model provides one example of a viable measurement model with particular relevance to multiple informant ratings of a construct. In this model, use of specific items rated by multiple informants allows simultaneous estimation of common (consensus) views of target behaviors (or trait aspects of behavior), the unique perspectives of each informant, as well as specific variance associated with each symptom domain or individual item (Bauer et al., 2013). In principle, the results, if established, could be converted to a clinical algorithm. In line with the utility of such a model, prior work has established that a related bifactor model may provide the best approximation of ADHD (Martel, von Eye, & Nigg, 2010; Toplak, West, & Stanovich, 2013). A bifactor model allows for individual ADHD symptoms to simultaneously load onto an overall, or “general” (“g”), ADHD factor along with completely or partially distinct (“specific”; “s”) inattention and hyperactive–impulsive latent components. A trifactor model builds on that foundation to add individual informant perspectives to the model.

The current study sought to extend prior work by providing, to our knowledge, the first empirical comparison of how to best integrate multiple informant ratings of ADHD through an advanced trifactor model of ADHD that allows for examination of the relatively well-established bifactor model of ADHD with inclusion of informant perspectives. Differential item functioning of symptoms as rated by self and other were examined. External validation of model factors using impairment and laboratory executive function tasks, commonly associated with ADHD, were explored.

Method

Participants

Overview

Participants were 406 adults (197 men), 18 to 37 years old (M = 23.65, SD = 4.53), recruited from the community via media advertisements and general mailings/flyers for a study of the development of attention abilities and mailings to clinics targeting individuals with and without possible attention problems. Therefore, these are not principally clinic-referred individuals but rather primarily reflect ADHD as it appears in the community; treatment rates and sex ratios seen in the current sample reflect what is expected in a community, not a clinic, sample. However, we deliberately overenrolled individuals with ADHD or serious attention problems to ensure adequate coverage and representation of the clinical construct.

Volunteering adults were thoroughly evaluated for study eligibility and diagnostic status, based on DSM-IV-TR criteria (APA, 2000) as described below (note that DSM-IV required six symptoms of either inattention or hyperactivity/impulsivity, whereas DSM-5 requires only five). After that evaluation, they were either excluded, assigned to one of two groups: ADHD (n = 145) and non-ADHD comparison (“controls,” n = 201), or classified as having situational or subthreshold ADHD (n = 60); some of the latter would meet DSM-5 criteria as having five symptoms. This sampling was intended to achieve the goals of providing full representation of the ADHD syndrome, as well as full representation of the dimensional trait space of ADHD symptoms (Haslam et al., 2006; Marcus & Barry, 2011). Based on their historical and current symptoms, the ADHD group included predominantly inattentive type (ADHD-PI; n = 64) defined as met criteria for six or more inattentive symptoms, plus impairment, onset, and duration, and never in the past met criteria for combined type on the basis of self-interview and parent/other interview as detailed below), combined type (ADHD-C; n = 63, defined as met criteria for six or more inattentive symptoms and six or more hyperactive–impulsive symptoms, plus impairment, onset, and duration), and hyperactive–impulsive type (ADHD-PHI; i.e., n = 18, met criteria for six or more hyperactive–impulsive symptoms, plus impairment, onset, and duration). A total of 6% of the entire sample, including those with and without ADHD, met criteria for major depressive disorder, 11% for generalized anxiety disorder, and 10% for borderline personality disorder.

The sample was broadly representative of the community from which they were recruited, approximately 15% ethnic or racial minority and a wide range of socioeconomic status. Approximately 10% of the ADHD group were currently prescribed psychostimulant medication. All participants completed written informed consent, and study procedures were approved by the university institutional review board.

Identification and Recruitment

As described in prior studies of this sample (Nigg et al., 2005; Stavro, Ettenhofer, & Nigg, 2007), recruitment and ascertainment was conducted using a diverse set of recruitment strategies including radio, newspaper, and movie theater advertisements and general mailings/flyers targeting individuals who think they might have attention problems and/or advertising a study of the development of attention, as well as mailings to local clinics (although less than 10% of the sample came from clinic advertisements), in order to recruit a sample of community volunteers with and without attention problems with deliberate oversampling of attention problems and true ADHD compared with the general population.

Prospective participants then underwent a standard multi-gate screening process. At Stage 1, participants completed a telephone screen to assess eligibility. To be eligible to participate in the study, participants had to be a native English speaker and not have a sensorimotor disability, neurological illness, or be on a current prescription for antidepressant, antipsychotic, or anticonvulsant medication. These eligibility criteria were chosen to ensure study participants could adequately understand task instructions and to eliminate the confounds of comorbid conditions and medication use that could affect cognitive performance. Participants who passed this stage of screening went on to a second stage of screening.

At Stage 2, eligible participants completed semistructured interviews and standardized normative rating scales, described below, to ascertain DSM-IV-TR ADHD and comorbid psychopathology. Participants completed a retrospective Kiddie Schedule for Affective Disorders and Schizophrenia (K-SADS-E; Puig-Antich & Ryan, 1986) to assess current and past ADHD symptoms. Adult participants also completed the Barkley and Murphy’s (1998) Current ADHD Symptoms Rating Scale, the Conners, Erhardt, and Sparrow’s (1999) Adult ADHD Rating Scale, the Achenbach’s (1991) Young Adult Self-Report, and the Brown’s (1996) Adult ADHD Rating Scale (all measures were the most current available at the time of data collection).

Crucially given concerns with veridicality of adults self-report of ADHD symptoms, and as recommended by best practices and expert recommendations (Barkley et al., 2011), two other informants also reported on every participant’s ADHD symptoms, as well as their potential mood symptoms, impairment, and drug use. One informant who knew the participant well when they were a child (usually a parent) reported on childhood ADHD symptoms via a structured interview (retrospective K-SADS ADHD module) and a standardized rating scale (the ADHD Rating Scale). The second informant knew the participant well currently, usually a partner or friend, that is, whoever the participant identified as having the most knowledge of their daily behavior. This informant completed a structured clinical interview (K-SADS ADHD module) on current symptoms, and a standardized rating scale (Conners Adult ADHD Rating Scale), and the Barkley and Murphy (1998) scale. They also completed a brief screen of antisocial behavior and drug and alcohol use.

A clinical diagnostic team consisting of a board-certified child psychiatrist and licensed clinical child psychologist, each with more than 10 years of experience, then used this information to arrive at a “best estimate” diagnosis (Faraone, 2000). Each member reviewed ADHD symptom counts from the semistructured interviews, interviewer observation notes, and raw scores and T scores from the rating scales, and independently judged whether ADHD was present or absent, ADHD subtype (if applicable), and comorbid disorders. Each member reviewed information individually to reach a diagnostic decision based on aggregation of all aforementioned information, and then these decisions were compared. In the case of disagreement, consensus was reached by discussion. Interrater agreement prior to conference was satisfactory on presence or absence of ADHD (κ ≥ .80) and ADHD subtype (κ ranged from .74 to .89). The diagnostic team also came up with a Global Assessment of Functioning (GAF) impairment rating (described below).

Measures

For the analytic models, the following measures were abstracted and used.

ADHD Symptom Counts

Current ADHD symptoms were obtained via self-report and other report on the Barkley and Murphy (1998) ADHD Rating Scale. Individual symptoms were coded on a 1 (absent) to 3 (present) rating scale. These individual symptom scores were utilized in structural equation modeling analyses.

Impairment

The diagnostic team generated a GAF estimate of individual impairment after reviewing all diagnostic data. Such estimates ranged from 1 to 100, as described and anchored in DSM-IV-TR (APA, 2000). Lower scores indicate worse impairment. This index was reliable in the current sample.

Executive Function

For external validation purposes, we utilized neuropsychological testing scores that were available on this sample. All testing was done after a minimum medication washout period of ~ 7 half lives (i.e., 24 hours for short-acting preparations and 48 hours for long-acting preparations). The testing battery included measures of response inhibition, set shifting, and working memory (see Nigg et al., 2005, for details; also Huang-Pollock, Nigg, & Carr, 2005). They were administered in a fixed order as follows.

Response suppression/inhibition: Go-Stop task

The Go-Stop task (Logan, 1994) was administered to assess response inhibition; it requires the suppression of a prepotent motor response. During this Choice Reaction time task, participants see an X or an O on a computer screen and respond rapidly with one of two keys to indicate which letter they had seen (called Go Response trials). On 25% of trials, a tone sounds shortly after the X or the O is displayed, indicating that participants are to withhold their response. A stochastic tracking procedure was used; stop signal reaction time was computed as an index of how much warning each participant needs to interrupt a response. Trials are presented across eight blocks. Stop signal reaction time was calculated by subtracting the average stop signal delay from the average Go Response time (Logan, 1994) after cleaning the data to remove anticipations (<100 ms), excessive slow responses (>1,500 ms), or blocks in which the participant appeared not to be doing the task (X/O accuracy <80% or probability of stopping <20% or >80% on average for a block of trials).

Reaction time variability: Go-Stop task

The within-person variability of the reaction time on the Go Response trials from the Go-Stop task was retained as a measure of response variability, again after removing responses <100 ms or >1,500 ms.

Set shifting: Trail Making Test

The Trail Making Test is a widely used timed paper-and-pencil test consisting of two parts (Reitan, 1958). Part A requires the participant to draw a line connecting numbered circles in sequential order. Part B requires the participant to draw a line connecting numbered and lettered circles in alternating sequential–alphabetical order. Time taken to complete Part B was utilized as the outcome measure of set shifting (Arbuthnott & Frank, 2000).

Perseveration: Wisconsin Card Sorting Task

The Wisconsin Card Sorting Task is a widely used, computer-administered task assessing working memory and set shifting (Heaton, Chelune, Talley, Kay, & Curtiss, 1993). Up to 64 trials were administered in which the participants sorted cards on the basis of shifting criteria of color, shape, and numbers. The participant is not told how to sort, but receives feedback from the computers regarding whether they sort correctly or incorrectly. Number of perseverative errors was the outcome variable.

Data Reduction and Analysis

Confirmatory factor analysis was conducted on all executive function summary scores described above to form a single latent factor we termed executive function for purposes of this report (root mean square error of approximation [RMSEA] = .11; comparative fit index [CFI] = .82). All executive function measures loaded significantly on this factor (all p < .05) with set shifting exhibiting the highest loading of .82. This executive function latent factor was used as an external validation for latent factors.

To fit the trifactor model of ADHD ratings, structural equation modeling was conducted in Mplus (version 6.11) using theta parameterization and weighted least squares means and variance adjusted estimation, as recommended for ordinal items (Muthen & Muthen, 2013). As a data analytic check, a fully constrained and freely estimated trifactor model was fit across ADHD and non-ADHD groups to examine invariance across diagnostic groups. Based on power concerns (each diagnostic group comprised slightly over 100 individuals), main analyses focus on the full group, but differences in results based on meaningful ADHD and non-ADHD variance are noted where relevant.

Results

Utilizing self-ratings and other ratings of ADHD symptoms, a trifactor CFA model was fit. This model, shown in simplified form in Figure 1, separates general ADHD trait variance from inattentive and hyperactive–impulsive symptom domain variance and informant (i.e., self and other) perspective variance. The initial model exhibited good model fit to the data (Hu & Bentler, 1999; McDonald & Ho, 2002) with χ2(525) = 607.51 (p < .01), an RMSEA of .021 (90% confidence interval [CI] [.012, .028]) and CFI of .99, suggesting that—once the unique perspectives of each informant and variance unique to each individual symptom domain are accounted for—all individual informant reports on individual items contributed significantly to the general ADHD construct. A fully constrained, or invariant, trifactor model based on ADHD diagnostic status fit reasonably well with χ2(1186) = 2119.02 (p < .01), an RMSEA of .075, and CFI of .64. The freely estimated trifactor model across both diagnostic groups fit slightly better with χ2(1081) = 1867.40 (p < .01), an RMSEA of .072, and CFI of .69.

Figure 1.

Figure 1

Open in a new tab

Simplified trifactor model.

Note. ADHD = attention-deficit/hyperactivity disorder.

Across the full group and as shown in Table 1, factor loadings across raters were uniformly statistically significant on the general ADHD factor with self-rated item loadings slightly higher (.51–.88) than other-rated item loadings (.18–.67). On the specific inattentive factor, other-rated symptom loadings were all significant, but self-rated “mental effort,” distracted,” and “forgetful” symptoms were not significant. For the specific hyperactive–impulsive factor, “fidgets” and “on the go” did not load significantly as rated by self or others. Other-rated “plays quietly” and “waiting turn” did not significantly load on the specific hyperactive–impulsive factor, and self-rated “leaves seat” and “runs or climbs” did not load significantly on the specific hyperactive–impulsive factor. Thus, there appear to be some hyperactive–impulsive symptoms that were not good items across raters, as well as some hyperactive–impulsive items that exhibited differential effects based on rater. Finally, for the rater-specific factors, while all other-rated symptom items exhibited significant loadings on the other perspective factor, self-ratings for the symptoms of “fidgets,” “leaves seat,” runs or climbs,” “plays quietly,” “on the go,” “talks,” and “blurts out” were nonsignificant on the self-perspective factor.

Table 1.

ADHD Symptom Loadings in Trifactor Model.

ADHD general factor Inattention factor Hyperactivity–impulsivity factor Self-factor Other-factor
Self-report
Close attention .50** .50** .50** .34**
Sustain attention .88** .31** .27** .49**
Listens .76** .32** .25** .37**
Follow through .65** .34** .35** .38**
Organizing .53** .45** .46** .36**
Mental effort .62** .13 .37** .25**
Loses things .53** .45** .30** .33**
Distracted .86** .20 .27** .53**
Forgetful .68** .23 .50** .38**
Fidgets .85** .07 −.10 .49**
Leaves seat .77** .15 −.06 .67**
Runs or climbs .91** .09 −.16 .64**
Plays quietly .74** .33** .06 .56**
“On the go” .60** .14 .04 .46**
Talks .51** .54** .07 .36**
Blurts out .62** .51** .13 .58**
Waiting turn .69** .29** .20* .53**
Interrupts .62** .51** .20* .54**
Other report
Close attention .34** .63**
Sustain attention .56** .45**
Listens .34** .36**
Follow through .36** .47**
Organizing .28** .59**
Mental effort .44* .45**
Loses things .18* .68**
Distracted .62** .38**
Forgetful .29** .62**
Fidgets .65** −.15
Leaves seat .46** −.20*
Runs or climbs .67** −.16*
Plays quietly .38** .20
“On the go” .50** −.16
Talks .41** .49**
Blurts out .36** .51**
Waiting turn .51** .04
Interrupts .43** .46**
Open in a new tab

Note. ADHD = attention-deficit/hyperactivity disorder.

*

Loadings significant at p < .05.

**

Loadings significant at p < .01.

Threshold parameter estimates for the full model are shown in Table 2. These are akin to difficulty or severity parameters in item response theory, in that they represent differences between items in the probability of a given response option being endorsed, independent of the latent trait. There is one threshold parameter for each boundary between one response option and the next (i.e., one for the boundary between responding “0” and “1” [absent], another for the boundary between “1” and “2,” and another for the boundary between “2” [subthreshold] and “3” [present]). For many symptoms, the subthreshold option was not endorsed for any participants, leading to only 2 thresholds being available. The thresholds shown in Table 2 suggest that, for many symptoms, the full range was not utilized by either the self or others, but—when the full range was utilized—self and others were more likely to rate individuals more highly based on higher general ADHD problems. Both self and others exhibited this pattern of utilizing the whole range with higher ratings indicating more ADHD problems for the symptom “follow through.” Self-ratings also followed this pattern for the symptom “sustained mental effort.” Other ratings followed this pattern for the symptoms “sustained attention,” organization,” and “loses things.” Therefore, “follows through,” “sustained attention,” “organization,” and “loses things” seemed to be particularly sensitive items, especially as rated by others.

Table 2.

ADHD Threshold Parameters in Trifactor Model.

Self
Other
b1 b2 b3 b1 b2 b3
Close attention 0.39 0.56 0.48 0.68
Sustained attention −0.20 −0.06 0.004 0.32 2.7
Listens 0.20 0.34 0.48 0.76
Follow through 0.38 0.52 2.77 0.56 0.83 2.7
Organization 0.23 0.39 0.15 0.36 2.7
Sustained mental effort 0.03 0.21 2.77 0.32 0.6
Loses things 0.13 0.28 0.17 0.32 2.7
Easily distracted −0.20 −0.03 −0.09 0.23
Forgetful 0.20 0.32 0.31 .056
Fidgets −0.19 −0.05 0.12 0.32
Leaves seat 0.70 0.82 0.67 0.89
Runs or climbs 0.26 0.37 0.35 0.52
Plays quietly 0.74 0.86 0.82 1.07
“Driven by a motor” 0.16 0.34 −0.09 0.25
Talks a lot 0.24 0.46 .12 0.40
Blurts 0.38 0.56 0.33 0.66
Waiting turn 0.53 0.67 0.48 0.80
Interrupts/intrudes 0.22 0.56 0.26 0.65 2.7
Open in a new tab

Note. ADHD = attention-deficit/hyperactivity disorder. Nonsignificant, or p > .05.

Differential Item Functioning Across Self-Ratings and Other Ratings of Symptoms

Differential item functioning was next evaluated for all levels of trifactor model sequentially. First, differential item functioning at the general factor level was tested by constraining the general factor loadings for the same item across informants to be the same; that is, we constrained the loading of inattentive symptom #1 (and all symptoms) on the general factor to be equivalent across the self and other. This model fit significantly worse (Δχ2[17] = 99.21, p < .01; RMSEA = .029; 90% CI [.023, .035]; CFI = .99) than the first model where all loadings were freely estimated. This suggests that informants are not equivalent in their ability to rate the overall ADHD factor after controlling for specific informant perspectives and specific ADHD symptom domains.

We next tested differential item functioning at the specific inattention and hyperactivity–impulsivity factor level by constraining the inattentive and hyperactive–impulsive factor loadings for the same item across informants to be the same; that is, we constrained the loading of inattentive symptom #1 (and all symptoms) on the inattentive factor (and hyperactive–impulsive factor) to be equivalent across self and others. This model also fit significantly worse than the baseline model (Δχ2[25] = 115.75, p < .01; RMSEA = .029; 90% CI [.023, .035]; CFI = .99), suggesting that self and others did not agree in their report of individual items in the specific ADHD symptom domains of inattention and hyperactivity–impulsivity.

Finally, we tested differential item functioning at the level of thresholds, or informant differences in ratings of severity of individual symptoms. This model also fit significantly worse than the baseline model (Δχ2[36] = 88.27, p < .01) with an RMSEA of .026 (90% CI [.019, .032]) and a CFI of .99, suggesting that different raters did not similarly endorse levels of symptoms. Tests of diagnostic invariance across ADHD and non-ADHD subgroups suggested that differential item functioning across diagnostic groups was particularly evident at the level of specific inattentive and hyperactive–impulsive factors (Δχ2[50] = 153.57, p < .01).

Therefore, self-ratings and other ratings of ADHD symptoms were significantly different from one another at the general, specific, and threshold levels.

External Validation With Executive Function and Impairment

Bivariate correlations were next conducted between the freely estimated trifactor latent factors (i.e., general ADHD, specific inattention, specific hyperactivity–impulsivity, self, and other) and the executive function latent factor. As shown in Table 3, the executive dysfunction latent factor was significantly correlated with increased specific inattention (r = .43, p < .01, driven by non-ADHD group) and marginally with the general ADHD latent factor (r = .19, p = .09, not seen in either ADHD or non-ADHD group when examined separately) and high ratings of ADHD based on the other perspective (r = .20, p = .081, seen in both ADHD and non-ADHD groups). However, the executive function latent factor was not significantly correlated with specific hyperactivity–impulsivity (r = −.15, p = .16) or the self-perspective (r = .06, p = .49). Significant and nonsignificant correlations significantly differed from one another (p < .05). As a secondary check on these results, correlations were also conducted between the executive function task exhibiting the highest loading on the executive function latent factor, set shifting, and the trifactor latent factors. Set shifting was significantly correlated with specific inattention (r = .28, p < .01) and higher ratings of ADHD symptoms based on the other perspective (r = .20, p < .05), but not with any of the other latent factors (r = −.11−.13, p > .05).

Table 3.

Correlations Between Trifactor Latent Factors and Executive Function and Impairment.

Executive function
Impairment
r p r p
General ADHD .19 .09 −.55 .00
Specific inattention .43 .00 −.29 .00
Specific hyperactivity–impulsivity −.15 .16 −.06 .25
Self .11 .49 −.27 .00
Other .20 .08 .01 .87
Open in a new tab

Note. ADHD = attention-deficit/hyperactivity disorder.

Bivariate correlations were also conducted between the freely estimated trifactor latent factors (i.e., general ADHD, specific inattention, specific hyperactivity–impulsivity, self, and other) and the GAF impairment index generated by the diagnostic team. As shown in Table 3, more GAF impairment (i.e., low ratings of functioning) was significantly correlated with increased general ADHD latent factor (r = −.55, p < .01, driven by non-ADHD group) and specific inattention (r = −.29, p < .01, driven by ADHD group) and high ratings of ADHD based on self-perspective (r = −.27, p < .01, driven by non-ADHD group). However, GAF impairment was not significantly correlated with specific hyperactivity–impulsivity (r = −.06, p = .25) or the other perspective (r = .01, p = .87). Again, significant and nonsignificant correlations significantly differed from one another (p < .05).

Therefore, both executive function and impairment were significantly associated with general ADHD and specific inattention, but while executive function was significantly associated with other ratings of symptoms, impairment as rated by a diagnostic team was significantly associated with self ratings.

Discussion

To advance the field’s consideration of the optimal strategy for combining reports from multiple informants in the assessment of ADHD in adults, the current study provided the first use of an advanced trifactor model of ADHD. This model has the advantage of allowing for examination of the relatively well-established bifactor model of ADHD with inclusion of informant perspectives. Results suggested that self reports and other reports of adult ADHD symptoms provide different information about the latent ADHD construct, function differentially, and are endorsed differentially based on similar levels of ADHD-related problems. External validation with laboratory executive function tasks and impairment rated by a diagnostic team suggest that inattention may be a more developmentally sensitive manifestation of ADHD during adulthood and that others and self provide different information in relation to external criteria. These results have implications for clinical practice and diagnosis that we discuss below.

In line with prior work (e.g., Amador-Campos, Forns-Santacana, Martorell-Balanzo, Guardia-Olmos, & Pero-Cebollero, 2006; Martel et al., 2010; Toplak et al., 2009), once the unique perspectives of each informant and variance unique to each individual symptom domain was accounted for, all individual informant reports on all individual symptoms contributed significantly to the general ADHD construct. This is consistent with theory of cross-situational convergence of ADHD-related behaviors (APA, 2013). Notably, on the general ADHD factor, self-rated symptoms loaded somewhat higher than informant-rated symptoms. This may suggest a form of bias in which individuals may be somewhat more likely than others to rate individual symptoms in line with their general idea of whether they believe they have global ADHD-related problems.

Other informant ratings of all inattentive symptoms on were significant, suggesting that other (vs. self) informants may be especially important raters of inattentive behaviors. Although self-ratings for most symptoms on the specific inattention symptom were significant, self-ratings for certain symptoms, that is, “mental effort,” distracted,” and “forgetful” symptoms were not significant, suggesting that other ratings on these particular symptoms may be especially informative. “Fidgets” and “on the go” symptoms did not load significantly on the specific hyperactive–impulsive factor as rated by self or others, suggesting these symptoms may not be as sensitive to ADHD symptom manifestation in adulthood, in line with some prior work (Kessler et al., 2010; Martel, von Eye, & Nigg, 2012), although it should be noted that the expanded form of these items just recently published in DSM-5 were not utilized in the current study. Furthermore, “plays quietly” and “waiting turn” did not load significantly on the specific hyperactive–impulsive factor, as rated by others, and “leaves seat” and “runs or climbs” did not load significantly on the specific hyperactivity–impulsivity factor, as rated by the self, suggesting these items may also be less sensitive to adult manifestation of ADHD symptoms or differentially accessible or notable to different raters.

Importantly, there were striking differences in self- and other- item loadings on their respective perspective factors, after controlling for general ADHD and specific inattention and hyperactivity–impulsivity. While all other-rated symptom items exhibited significant loadings on the other informant–perspective factor, self-rated item loadings for the symptoms of “fidgets,” “leaves seat,” “runs or climbs,” “plays quietly,” “on the go,” “talks,” and “blurts out” were nonsignificant on the self-perspective factor. Therefore, the other informant perspective seemed more coherent across symptom ratings, compared with the self-perspective.

In line with this idea that self and others provide different information about ADHD symptoms, differential item functioning was found for ratings at the general ADHD level and the specific inattention and hyperactivity–impulsivity level. That is, self and others appear to rate the same symptoms quite differently. This supports prior work that also suggested that informants provide unique information about ADHD symptoms in adults (Barkley et al., 2002; Barkley et al., 2008; Mannuzza et al., 1993, 1998; Molina & Sibley, 2014; Sibley et al., 2012). In our data, threshold parameter estimates (similar to severity parameters in item response theory), suggested that other informants exhibited more sensitive ratings in that they rated individuals as higher on symptoms given higher levels of ADHD problems, particularly on the following ADHD inattentive items: “follow through,” “sustained attention,” “organization,” and “loses things.” Therefore, these particular symptoms, as rated by others, may have utility for screening or short form ADHD ratings. Thus, we concur with Kessler et al. (2010) and Barkley et al. (2008) that differential validity of self reports and other reports of ADHD symptoms merit further examination. In any case, differential item functioning tests indicated that self and others did not similarly rate symptoms given comparable levels of ADHD problems.

Finally, external validation using executive function tests and impairment as rated by a diagnostic team supported the utility of a focus on inattentive symptoms in adulthood (Barkley et al., 2008; Kessler et al., 2010). Inattentive symptoms may be more sensitive to adult ADHD symptom manifestation, in line with prior work (Martel et al., 2012). Yet, while other ratings (but not self ratings) of adult ADHD symptoms exhibited significant correlations with laboratory executive function tasks, self-ratings (but not other ratings) of ADHD symptoms exhibited significant correlations with impairment rated by a diagnostic team. Thus, results are consistent with the idea that self and others provide different information about symptom manifestation in relation to external criteria (Achenbach, 2006; De Los Reyes et al., 2015) and therefore that preserving their unique inputs in some fashion is important. Therefore, it is likely that ultimately a weighted combination of self-symptom and other-symptom ratings with an emphasis on inattentive symptoms might be useful for diagnostic purposes. Models like that shown here can be helpful in developing such algorithms, which would likely be implemented by clinicians with the aid of handheld computers. That is, particular symptoms might warrant differential weighting by informant. For example, in the current data, other informant ratings of the inattentive symptoms of “follow through,” “sustained attention,” “organization,” and “loses things” would be prioritized over self ratings on these symptoms. Of course, such an idea would first need to be replicated in other samples and appropriate normative and standardization data prior to clinical application.

Despite the promise and implications of our approach, several limitations of the study deserve mention. ADHD is a complex, multifaceted disorder with substantial interindividual heterogeneity, currently represented by three presentations, or subtypes (APA, 2013; Sonuga-Barke, 2003); the model may not accurately capture clinical heterogeneity. In addition, the inclusion of executive function and impairment as external validators is a study strength; yet other external validation measures (e.g., academic performance, social functioning, observational ratings), including additional psychological tasks (e.g., reward or choice delay) that are conceptually related more to impulsivity than inattention, might yield different findings. Although we used a community recruitment strategy and detected no obvious selection bias in relation to socioeconomic status, race, income, comorbidity, or sex ratios, and this method protects to some extent against feigning of ADHD, this was a volunteer sample which may have introduced other unknown bias in relation to clinical features relative to the entire population of individuals with ADHD. For example, some individuals with ADHD deny their symptoms and/or may be unwilling to participate in a study, and this may introduce unknown bias relative to the entire population of individuals with ADHD. Nonetheless, these data illustrate the promise in utilizing a more psychometrically informative approach to build toward next generation diagnostic algorithms.

Since ADHD is a neurodevelopmental disorder, it will be particularly important for future work and replications to evaluate possible developmental change in this model across the lifespan; results by Martel, Schimmack, Nikolas, and Nigg (2015) in childhood sample, for example, suggest that parent and teacher reports of childhood ADHD symptoms may, in many instances, be best averaged. Furthermore, consideration of the expanded developmental descriptors for DSM-5 adulthood ADHD symptom items is also an important future direction (see Matte et al., 2015).

Overall, study results suggest that improved, empirically based algorithms for combining self reports and informant reports are tractable for ADHD. Furthermore, the inattentive symptom domain appears more sensitive than hyperactivity–impulsivity to adult ADHD manifestation, and self-ratings versus other ratings appear to provide different information in relation to external validation criteria, here, executive function and impairment. Future diagnostic algorithms should incorporate such insights into next generation diagnostics.

Acknowledgments

The authors thank all participants for making this work possible.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The project was supported by award number R01-MH070004-01A2 from the National Institute of Mental Health to Joel T. Nigg. Michelle M. Martel was supported by K12 DA 035150.

Footnotes

The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of Health or Drug Abuse.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

References

  1. Achenbach TM. Manual for the young adult self-report and young adult behavior checklist. Burlington: University of Vermont; 1991. [Google Scholar]
  2. Achenbach TM. As others see us: Clinical and research implications of cross-informant correlations for psychopathology. Current Directions in Psychological Science. 2006;15:94–98. [Google Scholar]
  3. Adler LA, Spencer T, Faraone SV, Kessler RC, Howes MJ, Biederman J, Secnik K. Validity of pilot adult ADHD Self-Report Scale (ASRS) to rate adult ADHD symptoms. Annals of Clinical Psychiatry. 2006;18:145–148. doi: 10.1080/10401230600801077. [DOI] [PubMed] [Google Scholar]
  4. Amador-Campos JA, Forns-Santacana M, Martorell-Balanzo B, Guardia-Olmos J, Pero-Cebollero M. DSM-IV attention deficit hyperactivity disorder symptoms: Agreement between informants in prevalence and factor structure at different ages. Journal of Psychopathology and Behavioral Assessment. 2006;28:23–32. [Google Scholar]
  5. American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 4. Washington, DC: Author; 2000. [Google Scholar]
  6. American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 5. Washington, DC: Author; 2013. [Google Scholar]
  7. Arbuthnott K, Frank J. Trail Making Test, Part B as a measure of executive control: Validation using a set-shifting paradigm. Journal of Clinical and Experimental Neuropsychology. 2000;22:518–528. doi: 10.1076/1380-3395(200008)22:4;1-0;FT518. [DOI] [PubMed] [Google Scholar]
  8. Barkley RA, Fischer M, Smallish L, Fletcher K. The persistence of attention deficit hyperactivity disorder into young adulthood as a function of reporting source and definition of disorder. Journal of Abnormal Psychology. 2002;111:279–289. [PubMed] [Google Scholar]
  9. Barkley RA, Knouse LE, Murphy KR. Correspondence and disparity in the self-and other ratings of current and childhood ADHD symptoms and impairment in adults with ADHD. Psychological Assessment. 2011;23:437–444. doi: 10.1037/a0022172. [DOI] [PubMed] [Google Scholar]
  10. Barkley RA, Murphy KR. Attention-deficit hyperactivity disorder: A clinical workbook. 2. New York, NY: Guilford Press; 1998. [Google Scholar]
  11. Barkley RA, Murphy KR, Fischer M. ADHD in adults: What the science says. New York, NY: Guilford Press; 2008. [Google Scholar]
  12. Bauer DJ, Howard AL, Baldasaro RE, Curran PJ, Hussong AM, Chassin L, Zucker RA. A trifactor model for integrating ratings across multiple informants. Psychological Methods. 2013;18:475–493. doi: 10.1037/a0032475. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Biederman J, Faraone SV, Spencer TJ, Mick E, Monuteaux MC, Aleardi M. Functional impairments in adults with self-reports of diagnosed ADHD: A controlled study of 1001 adults in the community. Journal of Clinical Psychiatry. 2006;67:524–540. doi: 10.4088/jcp.v67n0403. [DOI] [PubMed] [Google Scholar]
  14. Brown TE. Brown Attention-Deficit Disorder Scales. San Antonio, TX: Psychological Corporation; 1996. [Google Scholar]
  15. Campbell DT, Fiske DW. Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin. 1959;56:81–105. [PubMed] [Google Scholar]
  16. Connelly BS, Ones DS. An other perspective on personality: Meta-analytic integration of observers’ accuracy and predictive validity. Psychological Bulletin. 2010;136:1092–1122. doi: 10.1037/a0021212. [DOI] [PubMed] [Google Scholar]
  17. Conners CK, Erhardt D, Sparrow E. Adult ADHD Rating Scales: Technical manual. Toronto, Ontario, Canada: Multi-Health Systems; 1999. [Google Scholar]
  18. De Los Reyes A. Strategic objectives for improving understanding of informant discrepancies in developmental psychopathology research. Development and Psychopathology. 2013;25:669–682. doi: 10.1017/S0954579413000096. [DOI] [PubMed] [Google Scholar]
  19. De Los Reyes A, Augenstein TM, Wang M, Thomas SA, Drabick DAG, Burgers D, Rabinowitz J. The validity of the multi-informant approach to assessing child and adolescent mental health. Psychological Bulletin. 2015;141:858–900. doi: 10.1037/a0038498. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. De Los Reyes A, Youngstrom EA, Pabon SC, Youngstrom JK, Feeny NC, Findling RL. Internal consistency and associated characteristics of informant discrepancies in clinical referred youths age 11 to 17 years. Journal of Clinical Child & Adolescent Psychology. 2011;40:36–53. doi: 10.1080/15374416.2011.533402. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Dirks MA, De Los Reyes A, Briggs-Gowan M, Cella D, Wakschlag LS. Embracing not erasing contextual variability in children’s behavior: Theory and utility in the selection and use of methods and informants in developmental psychopathology. Journal of Child Psychology and Psychiatry. 2012;53:558–574. doi: 10.1111/j.1469-7610.2012.02537.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Erhardt D, Epstein J, Conners C, Parker J, Sitarenios G. Self-ratings of ADHD symptoms in adults: Reliability, validity, and diagnostic sensitivity. Journal of Attention Disorders. 1999;3:153–158. [Google Scholar]
  23. Faraone SV. Attention deficit hyperactivity disorder in adults: Implications for theories of diagnosis. Current Directions in Psychological Science. 2000;9:33–36. [Google Scholar]
  24. Faraone SV, Biederman J. What is the prevalence of adult ADHD? Results of a population screen of 966 adults. Journal of Attention Disorders. 2005;9:384–391. doi: 10.1177/1087054705281478. [DOI] [PubMed] [Google Scholar]
  25. Haslam N, Williams B, Prior M, Haslam R, Graetz B, Sawyer M. The latent structure of attention-deficit/hyperactivity disorder: A taxonomic analysis. Australian and New Zealand Journal of Psychiatry. 2006;40:639–647. doi: 10.1080/j.1440-1614.2006.01863.x. [DOI] [PubMed] [Google Scholar]
  26. Heaton RK, Chelune GJ, Talley JL, Kay GG, Curtiss G. Wisconsin Card Sorting Test manual. Odessa, FL: Psychological Assessment Resources; 1993. [Google Scholar]
  27. Hu L, Bentler PM. Cutoff criteria for fit indices in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling. 1999;6:1–55. [Google Scholar]
  28. Huang-Pollock CL, Nigg JT, Carr TH. Deficient attention is hard to find: Applying the perceptual load model of selective attention to attention deficit hyperactivity disorder subtypes. Journal of Child Psychology and Psychiatry. 2005;46:1211–1218. doi: 10.1111/j.1469-7610.2005.00410.x. [DOI] [PubMed] [Google Scholar]
  29. Kessler RC, Adler L, Barkley R, Biederman J, Conners CK, Demler O, … Zaslavsky AM. The prevalence and correlates of adult ADHD in the United States: Results from the National Comorbidity Survey Replication. American Journal of Psychiatry. 2006;163:716–723. doi: 10.1176/appi.ajp.163.4.716. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Kessler RC, Adler LA, Gruber MJ, Sarawate CA, Spencer T, Van Brunt DL. Validity of the World Health Organization Adult ADHD Self-Report Scale (ASRS) Screener in a representative sample of health plan members. International Journal of Methods in Psychiatric Research. 2007;16:52–65. doi: 10.1002/mpr.208. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Kessler RC, Green JG, Adler LA, Barkley RA, Chatterji S, Faraone SV, … Russo LJ. Structure and diagnosis of adult attention-deficit/hyperactivity disorder: Analysis of expanded symptom criteria from the Adult ADHD Clinical Diagnostic Scale. Archives of General Psychiatry. 2010;67:1168–1178. doi: 10.1001/archgenpsychiatry.2010.146. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Klein RG, Mannuzza S, Olazagasti MA, Roizen E, Hutchison JA, Lashua EC, Castellanos FX. Clinical and functional outcome of childhood attention- deficit/hyperactivity disorder 33 years later. JAMA Psychiatry. 2012;69:1295–1303. doi: 10.1001/archgenpsychiatry.2012.271. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Lahey BB, Applegate B, McBurnett K, Biederman J, Greenhill L, Hynd GW, … Shaffer D. DSM-IV trials for attention deficit hyperactivity disorder in children and adolescents. American Journal of Psychiatry. 1994;151:1673–1685. doi: 10.1176/ajp.151.11.1673. [DOI] [PubMed] [Google Scholar]
  34. Logan GD. A user’s guide to the stop signal paradigm. In: Dagenbach D, Carr TH, editors. Inhibitory processes in attention, memory, and language. San Diego, CA: Academic Press; 1994. pp. 189–239. [Google Scholar]
  35. Mannuzza S, Klein RG, Bessler A, Malloy P, LaPadula M. Adult outcome of hyperactive boys: Educational achievement, occupational rank, and psychiatric status. JAMA Psychiatry. 1993;50:565–576. doi: 10.1001/archpsyc.1993.01820190067007. [DOI] [PubMed] [Google Scholar]
  36. Mannuzza S, Klein RG, Bessler A, Malloy P, LaPadula M. Adult psychiatric status of hyperactive boys grown up. American Journal of Psychiatry. 1998;155:493–498. doi: 10.1176/ajp.155.4.493. [DOI] [PubMed] [Google Scholar]
  37. Marcus DK, Barry TD. Does attention-deficit/hyperactivity disorder have a dimensional latent structure? A taxometric analysis. Journal of Abnormal Psychology. 2011;120:427–442. doi: 10.1037/a0021405. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Martel MM, Schimmack U, Nikolas M, Nigg JT. Integration of symptom ratings from multiple sources in ADHD diagnosis: A psychometric model with clinical utility. Psychological Assessment. 2015;27:1060–1071. doi: 10.1037/pas0000088. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Martel MM, von Eye A, Nigg JT. Revisiting the latent structure of ADHD: Is there a “G” factor? Journal of Child Psychology and Psychiatry. 2010;51:905–914. doi: 10.1111/j.1469-7610.2010.02232.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Martel MM, von Eye A, Nigg J. Developmental differences in structure of attention-deficit/hyperactivity disorder (ADHD) between childhood and adulthood. International Journal of Behavioral Development. 2012;36:279–292. doi: 10.1177/0165025412444077. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Matte B, Rohde LA, Turner JB, Fisher PW, Shen S, Bau CH, … Grevet EH. Reliability and validity of proposed DSM-5 ADHD symptoms in a clinical sample of adults. Journal of Neuropsychiatry & Clinical Neurosciences. 2015;27:228–236. doi: 10.1176/appi.neuropsych.13060137. [DOI] [PubMed] [Google Scholar]
  42. McDonald RP, Ho MHR. Principles and practice in reporting structural equation analyses. Psychological Methods. 2002;7:64–82. doi: 10.1037/1082-989x.7.1.64. [DOI] [PubMed] [Google Scholar]
  43. Molina B, Sibley MH. The case for including informant reports in the assessment of adulthood ADHD. ADHD Report. 2014;22(8):1–7. [Google Scholar]
  44. Muthen LK, Muthen BO. Mplus user’s guide. 4. Los Angeles, CA: Author; 2013. [Google Scholar]
  45. Nigg JT, Stavro G, Ettenhofer M, Hambirck DZ, Miller T, Henderson JM. Executive functions and ADHD in adults: Evidence for selective effects on ADHD symptom domains. Journal of Abnormal Psychology. 2005;114:706–717. doi: 10.1037/0021-843X.114.3.706. [DOI] [PubMed] [Google Scholar]
  46. Puig-Antich J, Ryan N. Kiddie Schedule for Affective Disorders and Schizophrenia. Pittsburgh, PA: Western Psychiatric Institute; 1986. [Google Scholar]
  47. Reitan RM. Validity of the Trail Making Test as an indicator of organic brain damage. Perceptual & Motor Skills. 1958;8:271–276. [Google Scholar]
  48. Sibley MH, Pelham WE, Molina BSG, Gnagy EM, Waxmonsky JG, Derefinko K, … Kuriyan AB. When diagnosing ADHD in young adults emphasize informant reports, DSM items, and impairment. Journal of Consulting and Clinical Psychology. 2012;80:1052–1061. doi: 10.1037/a0029098. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Solanto MV, Alvir J. Reliability of DSM-IV symptom ratings of ADHD: Implications for DSM-V. Journal of Attention Disorders. 2009;13:107–116. doi: 10.1177/1087054708322994. [DOI] [PubMed] [Google Scholar]
  50. Sonuga-Barke EJS. The dual pathway model of AD/HD: An elaboration of neuro-developmental characteristics. Neuroscience and Biobehavioral Reviews. 2003;27:593–604. doi: 10.1016/j.neubiorev.2003.08.005. [DOI] [PubMed] [Google Scholar]
  51. Stavro GM, Ettenhofer ML, Nigg JT. Executive functions and adaptive functioning in young adult attention-deficit/hyperactivity disorder. Journal of the International Neuropsychological Society. 2007;13:324–334. doi: 10.1017/S1355617707070348. [DOI] [PubMed] [Google Scholar]
  52. Toplak ME, Pitch A, Flora DB, Iwenofu L, Ghelani K, Jain U, Tannock R. The unity and diversity of inattention and hyperactivity-impulsivity in ADHD: Evidence for a general factor with separable dimensions. Journal of Abnormal Child Psychology. 2009;37:1137–1150. doi: 10.1007/s10802-009-9336-y. [DOI] [PubMed] [Google Scholar]
  53. Toplak ME, West RF, Stanovich KE. Practitioner review: Do performance-based measures and ratings of executive function assess the same construct? Journal of Child Psychology and Psychiatry. 2013;54:131–143. doi: 10.1111/jcpp.12001. [DOI] [PubMed] [Google Scholar]
  54. Valo S, Tannock R. Diagnostic instability of DSM-IV ADHD subtypes: Effects of informant source, instrumentation, and methods for combining symptom reports. Journal of Clinical Child & Adolescent Psychology. 2010;39:749–760. doi: 10.1080/15374416.2010.517172. [DOI] [PubMed] [Google Scholar]

RESOURCES