Cognitive heterogeneity in adult Attention Deficit / Hyperactivity Disorder: a systematic analysis of neuropsychological measurements

Abstract

Attention Deficit / Hyperactivity Disorder (ADHD) in childhood is associated with impaired functioning in multiple cognitive domains: executive functioning (EF), reward and timing. Similar impairments have been described for adults with persistent ADHD, but an extensive investigation of neuropsychological functioning in a large sample of adult patients is currently lacking. We systematically examined neuropsychological performance on tasks measuring EF, delay discounting, time estimation and response variability using univariate ANCOVA's comparing patients with persistent ADHD (N = 133, 42% male, mean age 36) and healthy adults (N = 132, 40% male, mean age 36). In addition, we tested which combination of variables provided the highest accuracy in predicting ADHD diagnosis. We also estimated for each individual the severity of neuropsychological dysfunctioning. Lastly, we investigated potential effects of stimulant medication and a history of comorbid major depressive disorder (MDD) on performance. Compared to healthy adults, patients with ADHD showed impaired EF, were more impulsive, and more variable in responding. However, effect sizes were small to moderate (range: 0.05 – 0.70) and 11% of patients did not show neuropsychological dysfunctioning. The best fitting model predicting ADHD included measures from distinct cognitive domains (82.1% specificity, 64.9% sensitivity). Furthermore, patients receiving stimulant medication or with a history of MDD were not distinctively impaired. To conclude, while adults with ADHD as a group are impaired on several cognitive domains, the results confirm that adult ADHD is neuropsychologically heterogeneous. This provides a starting point to investigate individual differences in terms of impaired cognitive pathways.

INTRODUCTION

Attention Deficit / Hyperactivity Disorder (ADHD) is a common and highly heritable neuropsychiatric disorder in childhood that is strongly persistent over time. At least 35% of all childhood patients still meet full ADHD criteria in adulthood (American Psychiatric Association, 2000), and this percentage is much higher (78%) when partial remitted patients are included (Biederman et al., 2010). ADHD has an average prevalence of 2.5–4.9 % in the adult population (Simon et al., 2009). The clinical phenotype of ADHD is characterized by persistent, age-inappropriate symptoms of inattention, and / or hyperactivity and impulsivity (American Psychiatric Association, 2000).

ADHD has been associated with neurocognitive dysfunctioning, and over the years, several neuropsychological theories about ADHD etiology have been put forward. One of the most influential theories proposed ADHD to arise from a single core deficit in behavioral inhibition, which leads to secondary impairments in several executive functions (Barkley, 1997). However, this assumption of a central deficit was challenged by data showing that ADHD patients are impaired in multiple neuropsychological domains. It has therefore been proposed that there are distinct pathways to dysfunction, including executive function (EF) deficits, delay aversion, and timing problems (Castellanos et al., 2006; Sonuga-Barke et al., 2010). Although not included in the multiple pathway model, another characteristic of ADHD is performance variability. The inconsistency in performance and the high prevalence of moment-to-moment variability in reaction times is one of the most consistently reported manifestations of ADHD. Reaction time variability (RTV) received extensive discussion as an indicator of cognitive performance, although the exact nature of high RTV in ADHD is still uncertain (Kofler et al., 2013; Tamm et al., 2012).

Studies of cognitive functioning in adults with ADHD suggest that cognitive impairments found in adults resemble those observed in children with ADHD, showing equally moderate effects sizes (for meta-analytic reviews, see (Boonstra et al., 2005; Hervey et al., 2004; Schoechlin and Engel, 2005). Similar results were derived from qualitative reviews (Seidman, 2006; Woods et al., 2002). Recent meta-analyses in adult ADHD focused solely on deficits found in working memory (Alderson et al., 2013) and long-term memory (Skodzik et al., 2013). Furthermore, recent experimental studies on adult ADHD show deficits in attention (Fuermaier et al., 2015; Grane et al., 2014), set-shifiting (Boonstra et al., 2010; Hallehand et al., 2012; Rohlf et al., 2012) inhibition (Boonstra et al., 2010; Fuermaier et al., 2015), (working) memory (Fuermaier et al., 2015; Lundervold et al., 2015; Rohlf et al., 2012), delay discounting (Marx et al., 2010), and increased reaction time variability (Feige et al., 2013; Gmehlin et al., 2014; Grane et al., 2014).

From the childhood literature, we know that ADHD is characterized by large heterogeneity at the neuropsychological level, which means that only a minority of ADHD patients shows deficits in each domain and that some patients with ADHD will perform in the normal range (Nigg et al., 2005b). Such heterogeneity was illustrated in a recent study on boys with ADHD (Coghill et al., 2013). Per cognitive domain merely 18–36% of the patients had an impairment, while 25% of the sample did not show deficient performance in any of the cognitive domains.

Heterogeneity in cognitive performance within a sample of ADHD patients may also arise from differences in medication use or comorbidity. Stimulants are effective for the treatment of clinical symptoms in adult ADHD (Faraone et al., 2004) and also in neuropsychological studies medication is usually seen as a potential moderator. Many neuropsychological studies in ADHD have included patients who had previously taken, or were receiving stimulant medication at the time of the study. To eliminate the acute effects of medication, most studies used a washout period (24h or 48h). However, stimulants may act longer than 48h (McCarthy et al., 2014). Similarly, ADHD patients with a comorbid psychiatric disorder showed greater neuropsychological deficits than ADHD patients without comorbidity (Hervey et al., 2004) and may represent a distinct subgroup, with different cognitive profiles (Fischer et al., 2007). However, it has also been shown that cognitive deficits in adult ADHD cannot be accounted for by comorbid disorders (Nigg et al., 2005a; Silva et al., 2013). Major depressive disorder (MDD) is the most frequently observed comorbidity, and can co-occur with ADHD in up to 50% of the cases (Wilens et al., 2009). MDD has been associated with cognitive difficulties in memory, attention and problem-solving. Only two studies examined comorbid MDD in ADHD to date, both suggesting that current comorbid MDD symptoms may not influence neuropsychological profiles in ADHD (Katz et al., 1998; Riordan et al., 1999). While potential effects of comorbid MDD on cognition are often controlled for by excluding patients with current MDD from a study, many included patients will have remitted MDD. It is currently not known whether adult ADHD patients with MDD in remission are distinctively impaired on cognitive performance, although it has been shown that ADHD symptom severity increases in association with lifetime occurrence of comorbid MDD (Simon et al., 2013).

Reviewing the literature of adult ADHD shows that experimental studies and meta-analyses are limited by relying on relatively small samples with different inclusion criteria and tasks. Those studies had limited power to investigate confounding effects on neuropsychological functioning such as comorbidity or treatment. Also, the investigation of different tasks or functions in different samples has limited the possibility to construct a comprehensive picture of impairments associated with adult ADHD. To improve confidence in the findings, replication/validation in a large cohort of adult ADHD patients is thus desirable. Lastly, except for studies by Seidman et al. (1998), Boonstra et al. (2010), and Fuerrmaier et al. (2015), most studies assessed only a narrow range of neuropsychological tasks. Therefore, we investigated case-control differences on a wide range of well-described neuropsychological tasks using the largest sample of adult ADHD patients to date. Neuropsychological tasks were chosen based on the multiple pathway model and RTV literature described above and measured motor speed, sustained attention, inhibition, delay discounting, time estimation, set-shifting, verbal fluency, working memory, and response variability. We expected effect sizes to be moderate, with strongest effects on RTV as this is a pervasive characteristic observable across tasks (Kofler et al., 2013). Furthermore, we were interested in the diagnostic relevance of these tasks. From a clinical perspective it is interesting to know the predictive importance of neuropsychological measurements in ADHD classification. Previous literature showed however that neuropsychological measurements have a relatively poor ability to discriminate between children with ADHD and typically developing controls (Sjowall et al., 2013) or adults with ADHD and psychiatric patients without ADHD (Holst and Thorell, 2013). It remains an open question how discriminative the investigated neuropsychological tasks are in a sample of healthy adults with and without ADHD. We further investigated heterogeneity in performance and severity by computing the number of deficient test scores per participant as was previously done in childhood ADHD (Coghill et al., 2013). Additionally, we explored the potential effect of stimulant medication and a history of comorbid MDD on performance.

EXPERIMENTAL PROCEDURES

Participants

The study population was the Dutch cohort of the International Multicenter persistent ADHD CollaboraTion (IMpACT - http://impactadhdgenomics.com (Franke et al., 2010)). This is an ongoing study that at the time of analysis (1 January 2014) included 298 participants (155 adult ADHD cases, 143 healthy comparison participants). Patients and healthy control participants were recruited at the department of Psychiatry of the Radboud university medical center in Nijmegen and through advertisements. Patients were included if they had previously been diagnosed with persistent ADHD, i.e. present since childhood, by a psychiatrist according to the Diagnostic and Statistical Manual of Mental Disorders (4th edition; DSM-IV-TR; American Psychiatric Association, 2000). Exclusion criteria for participants were psychosis, alcohol or substance addiction in the last six months, current major depression, full-scale IQ estimate <70, neurological disorders, sensorimotor disabilities, non-Caucasian ethnicity, medication use other than psychostimulants, atomoxetine or bupropion and failure to withhold stimulant medication 24 hours prior to testing (see procedure below). Additional exclusion criteria for healthy controls were a current or lifetime neurological or psychiatric disorder in either the proband or his/her first-degree relatives. From the total sample, 33 participants (22 patients, 11 controls) had to be excluded because they met at least one of these exclusion criteria (see Supplementary Table 1).

This study was approved by the regional ethics committee (Centrale Commissie Mensgebonden Onderzoek: CMO Regio Arnhem – Nijmegen; Protocol number III.04.0403). Written informed consent was obtained from all participants.

Procedure

Subjects were invited for two sessions (Supplementary Figure 1), one including a detailed psychiatric assessment and blood withdrawal for biobanking of DNA, RNA and serum. A second session consisted of cognitive testing and neuroimaging procedures. The genetic and neuroimaging data are described elsewhere (i.e. (Franke et al., 2010; Hoogman et al., 2011)). For session 2, participants were requested to withhold stimulant medication 24 hours prior to testing.

Psychiatric assessment

Both patients and controls were assessed using the structured Diagnostic Interview for ADHD in Adults (DIVA, (Kooij, 2010)). This interview focuses on the 18 DSM-IV symptoms of ADHD and uses concrete and realistic examples to thoroughly investigate whether a symptom is currently present or was present in childhood. In addition, a self-report questionnaire on current symptoms was obtained using the ADHD Rating Scale-IV (Kooij et al., 2005). The Dutch version of the Structured Clinical Interview for DSM-IV, SCID-I and SCID-II (Groenestijn et al., 1999; Weertman et al., 2000) was used to identify lifetime Axis I and II disorders. Twenty-two patients and 12 controls did not participate in the clinical interview. These participants were included in the main analysis based on a prior diagnosis of ADHD by a psychiatrist and if they reached clinical threshold for ADHD based on the self-report scale. They were excluded from the analysis of comorbidity (see below).

Neuropsychological measurements

The neuropsychological test battery included measures tapping into EF (working memory, attention, inhibition, set-shifting, verbal fluency), delay discounting, and time estimation. Details about tasks and main outcome measures are described in Table 1 and the supplementary text. To estimate IQ, Vocabulary and Block Design of the Wechsler Adult Intelligence Scale (WAIS-III) were administered (Wechsler, 1997). The tests were always administered in the same order.

Table 1.

Tasks and outcome measures of the neuropsychological test battery

Task	Task description*	Cognitive domain	Outcome measure
1. Baseline speed task	Participants respond with a button press as quickly as possible when a fixation cross changes into a block-shape	Motor speed & reaction time	Mean RT SD of RT
2. WAIS-III Digit span task	Participants repeat strings of digits that are read aloud by the experimenter. In the backward condition, strings are repeated in reverse order. Each trial the working memory load increases.	Executive functioning: Working memory	Forward digit span score Backward digit span score
3. Flanker task	Participants respond with a button press to the color of the center block (yellow or blue), flanked by other blocks. In part 1, the center block is flanked by blocks of the same color (congruent trial) or a different color (green, neutral trial). In part 2, the neutral trials are replaced by incongruent trials (flanking blocks with the color of the alternative response).	Executive functioning: Inhibition	Total mean RT (average over part 1 and 2) Total SD of RT (average over part 1 and 2) Inhibition RT (difference in RT on congruent and incongruent trials in part 2) Inhibition errors (difference in error rate between congruent and incongruent trials in part 2)
4. Sustained attention dots task (SA-dots)	Three, four or five dots are presented on the screen. Participants respond with a button press with the dominant hand to four dots and with the non-dominant hand to three or five dots. An erroneous button press to three or five dots is a false alarm; an erroneous button press to four dots is a miss. For analysis, the task is split up into ten blocks, or series, in order to compute variance in performance over time. The duration of task is 20 minutes.	Executive functioning: Attention & inhibition	Mean series completion time SD series completion time SD series errors (SD of the errors made across blocks) Response bias (the difference between the number of misses and the number of false alarms across the entire task)
5. Sustained Attention to Response Task (SART)	Go/No-Go task. Participants respond with a button press to single digits presented on the screen (1–9), but to withhold a response when the digit 3 is presented.	Executive functioning: Attention & inhibition	Number of commission errors Number of omission errors Mean RT hits SD of RT hits
6. Trailmaking task	Participants need to connect dots containing numbers in consecutive order (part A) or alternating between numbers and letters in consecutive order (part B).	Executive functioning: Motor control & set-shifting	Time to complete part A Time to complete part B Difference in time to complete part B and time to complete part A
7. Semantic category and initial letter fluency	Participants name as many animals or professions they can think of in one minute. Next, they name as many words starting with a `D', `A' or `T' as they can think of in one minute.	Executive functioning: Verbal fluency	Number of words mentioned in category animals Number of words mentioned in category professions Number of words mentioned in category letters (total of 3 letter-trials)
8. Delay discounting task	Participants repeatedly have to choose between two hypothetical incentives that differ in the value (money) and delay (time until the money would be received). The impulsivity parameter (k) is computed from the present value of the delayed reward (V), the real value of the delayed reward (a) and the delay in days (D) with the formula: V = a/(1+kD).	Delay aversion & impulsivity	K 100 (impulsivity high rewards) K 30 (impulsivity intermediate rewards) K 10 (impulsivity low rewards)
9. Time estimation task	Participants have to respond with a button press exactly one second after hearing a sound beep. First, during a training session the length of a second is shown several times. During the experiment, feedback is given (`too slow', `correct', `too fast').	Timing	Median response time Absolute deviation of the median response time from 1000 ms

RT = reaction time; SD = standard deviation; ms = milliseconds.

^*More detailed information about the tasks, including references, can be found in the Supplementary Materials.

Data analysis of neuropsychological tasks

All measures were entered as raw scores in the analyses. Performance on each neuropsychological measure was entered as the dependent variable in separate univariate ANCOVA's, testing the difference between patients and controls. Age and gender were entered as covariates of no interest in order to reduce error variance (Miller & Chapman 2001). This was justified as age and gender did not differ between the groups. We therefore also did not investigate interactions between diagnosis and age or gender. As IQ is correlated with performance on many neuropsychological tasks, we investigated whether adding estimated IQ as an additional covariate would influence the findings. As IQ also did not differ between groups, this analysis using ANCOVA was justified and did not serve to control for IQ. Assumptions with respect to the residuals were checked and neuropsychological measures were transformed if necessary. Outliers were defined as having a score more extreme than four times the standard deviation above or below the mean per group (Leth-Steensen et al., 2000; Nigg et al., 2005a). This threshold guarded against artifacts and chance level performance, while still including cases performing at the extreme of the normal distribution. If a participant's score was an outlier on one outcome variable of a task, his/her scores on all outcome variables from that task were excluded. Effect sizes were computed as Cohen's D, using the corrected means from the ANCOVA's (Cohen, 1988).

Multiple comparison correction was performed by estimating the effective number of independent tests (Meff) (Li and Ji, 2005). This method takes into account the correlation structure between measures and calculates the Meff based on the observed eigenvalue variance of the different neuropsychological measures using the matSpD interface (http://genepi.qimr.edu.au/general/daleN/matSpD). The p-value for significance was determined as 0.05 divided by Meff. Twenty-seven measures resulted in twenty-two independent tests and therefore, only effects with a p-value < 0.0023 were considered significant.

Second, to investigate discriminating ability of the neuropsychological test battery, we used a step-wise backward logistic regression model. To maximize power, with our sample size, we included only those neuropsychological measures that were nominally significant in the case-control comparison to determine the model with the highest prediction accuracy of diagnostic status. Variables were retained in the model when they significantly contributed to the likelihood ratio statistic, all other variables were excluded.

To investigate heterogeneity in cognitive impairments, we computed the number of deficient test scores for each participant. Similar to previous studies, a deficient score was defined as performance below the 10^th percentile of the performance distribution of the control group (Coghill et al., 2013; Nigg et al., 2005b). For variables where higher scores indicated worse performance, deficiency was defined as a score above the 90^th percentile of performance distribution of the control group. For the variable `time estimation median response time' performance at both lower and upper extreme was scored as deficient. As not all participants had completed data for all tasks, we computed the relative number of deficient test scores as a percentage of the total number of scores for that participant. We labeled between 1% and 20% deficient test scores as `mildly impaired', between 20% and 40% as `impaired' and above 40% as `severely impaired'. The difference between cases and controls in the number of relative deficient test scores was computed using an ANCOVA with age and gender as covariates. In addition, we repeated the same analysis in a restricted group of only those participants with complete data (N = 168).

Effects of stimulant medication and history of MDD

We conducted two exploratory analyses. First, in order to investigate stimulant medication effects on neuropsychological measures, we used separate ANCOVA's for each neuropsychological measure comparing medication naïve patients (N = 20), medicated patients (N = 83), and healthy control participants (N = 132), with age and gender as covariates. Second, we conducted a similar analysis comparing patients with at least one lifetime MDD episode (now in remission, N = 55), patients without a history of MDD (N = 68), and healthy controls without prior episodes of MDD (N = 112). Twenty healthy control participants reported to have experienced depressive episodes in the past and were therefore excluded from this analysis. For both analyses, in the case of a main effect of group on the neuropsychological measure, we tested post-hoc the differences between groups. These post-hoc tests were Bonferroni corrected for multiple testing.

RESULTS

Demographics

A total of 265 participants (132 healthy controls and 133 ADHD patients) were included in the analyses. Demographic information is provided in Table 2. Patients and controls did not differ in age, handedness, and estimated IQ. Gender was equally distributed across groups. Patients had received fewer years of education than controls. As expected, patients had significantly more ADHD symptoms based on the diagnostic interview and self-report. Information about psychiatric comorbidities and medication is summarized in Supplementary Tables 2 and 3.

Table 2.

Demographics (N = 265^a)

	Healthy controls (N = 132)	ADHD patients (N = 133)	p-value
Gender	53 (40.2%) male	56 (42.1%) male	n.s.
Age	36.30 (11.75), range 19–63	35.56 (10.40), range 18 – 59	n.s.
Estimated IQb	109.97 (14.90)	107.83 (14.28)	n.s.
Educationc	5.16 (0.81), range 3 – 7	4.70 (0.80), range 2 – 7	< 0.001
Repeated school years (once or more)	53 (40.2%)	77 (57.9%)	0.005
Non-completed education programs (one or more)	40 (33.3%) (N = 128)	87 (67.4%) (N = 129)	< 0.001
Handedness	115 (87.1%) right, 13 (9.8%) left, 3 (2.3%) ambidextrous (N = 131)	113 (85%) right, 16 (12%) left, 4 (3%) ambidextrous	n.s.
Inattentive symptoms (DIVA)	0.39 (0. 83), 0 – 4 (N = 120)	7.38 (1.55), 3 – 9 (N = 112)	< 0.001
Hyperactive / impulsive symptoms (DIVA)	0.52 (0.98), 0 – 4 (N = 120)	5.76 (2.27), 0 – 9 (N = 112)	< 0.001
Total symptoms (DIVA)	0.91 (1.43), 0 – 8 (N = 120)	13.14 (2.76), 7 – 18 (N = 112)	< 0.001
Inattentive symptoms (selfreport)	0.53 (0.98), 0 – 5 (N = 131)	6.40 (2.09), 0 – 9	< 0.001
Hyperactive / impulsive symptoms (selfreport)	0.89 (1.44), 0 – 6 (N = 131)	5.58 (2.26), 0 – 9	< 0.001
Total symptoms (selfreport)	1.42 (2.14) 0 – 9 (N = 131)	11.98 (3.37), 1 – 18	< 0.001

Data show as: mean (standard deviation), minimum – maximum. P-values represent the significance of the group difference, tested with independent samples t-tests for continuous data or Pearson Chi-square tests for categorical data;

^a32 subjects from the total sample were excluded from analyses according to our exclusion criteria;

^bIQ was estimated based on performance on the WAIS-III block pattern and vocabulary tasks;

^cEducation level was coded from 1 (unfinished primary school) to 7 (post-university);

^dDIVA interview data was missing for 22 patients.

Effect of diagnosis on cognitive performance

Findings from the case-control comparison of neuropsychological performance are summarized in Table 3. In the domain of EF patients were impaired on working memory and attention, but no group differences were found for inhibition (Flanker and SART task) and verbal fluency. In the domain of delay aversion patients performed worse than controls on the delay discounting task, but not in the domain of timing (time estimation task). Across several tasks, patients were also more variable in their reaction times than controls. Response speed did not differ between patients and controls in most tasks, except for both conditions of the Trailmaking tasks. Effect sizes were in the small to medium range, with the largest effect on the SA-dots task where patients showed more fluctuation in errors across blocks (effect size = −0.71). Adding the covariate IQ, in addition to age and gender, did not significantly alter the results.

Table 3.

Case/control analysis of cognitive performance (N = 265)

NPO Task	Variable	Healthy Controls Mean (SD)	ADHD Mean (SD)	ANCOVA F (df), p-value	Effect size Cohen's D
1. Digit span test HC: N =132 ADHD: N = 128	Forward score	9.83 (2.36)	8.99 (1.95)	10.45 (1, 256), p = 0.001*	0.40
	Backward score	7.49 (2.34)	6.69 (2.22)	8.62 (1,256), p = 0.004	0.37
2. Baseline speed (ANT) HC: N =130 ADHD: N = 129	Mean RT	313.28 (49.42)	316.83 (55.29)	0.40 (1, 255), n.s.	−0.07
	SD of RT †	4.08 (0.53)	4.26 (0.57)	7.47 (1,255), p = 0.007	−0.34
3. Flanker task (ANT) HC: N =127 ADHD: N = 123	Total mean RT	525.37 (73.00)	537.93 (92.87)	1.85 (1, 246), n.s.	−0.18
	Total SD of RT	93.74 (37.04)	118.39 (58.87)	15.90 (1, 246), p < 0.001*	−0.51
	Inhibition RT	28.44 (28.25)	23.13 (40.76)	1.45 (1, 246), n.s.	0.14
	Inhibition errors	0.68 (1.48)	0.63 (1.47)	0.11 (1, 246), n.s.	0.05
4. SAdots (ANT) HC: N =128 ADHD: N = 123	Mean series completion time	899.05 (129.21)	944.71 (186.08)	5.03 (1,247), P = 0.026	−0.28
	SD completion time †	3.81 (0.44)	4.07 (0.53)	16.82 (1, 247), p < 0.001*	−0.52
	SD errors †	0.70 (0.19)	0.86 (0.26)	32.03 (1,247), p < 0.001*	−0.71
	Response bias	5.05 (6.10)	9.16 (9.51)	18.35 (1,247), p < 0.001*	−0.52
5. SART HC: N = 110 ADHD: N = 104	Commission errors	9.31 (5.03)	10.51 (4.87)	3.02 (1,210), n.s.	−0.25
	Omission errors	2.63 (3.57)	4.04 (4.97)	5.72 (1,210), p = 0.018	−0.32
	Mean RT hits	315.50 (57.48)	326.09 (60.74)	2.44 (1,210), n.s.	−0.21
	SD or RT †	4.35 (0.36)	4.56 (0.44)	14.17 (1,210), p < 0.001*	−0.53
6. Fluency HC: N = 132 ADHD: N = 131	Category; Animals	27.76 (5.77)	25.85 (5.97)	6.84 (1,259), p = 0.009	0.32
	Category: professions	20.27 (5.23)	19.81 (5.09)	0.48 (1,259), n.s.	0.10
	Letters	41.91 (10.51)	38.95 (10.87)	5.15 (1,259), p = 0.024	0.29
7. Time estimation HC: N = 126 ADHD: N = 116	Median response time	1007.09 (67.61)	997.94 (82.21)	1.13 (1,238), n.s.	0.14
	Absolute deviation of the median response time from 1000 ms	49.38 (46.38)	63.92 (51.40)	5.49 (1,238), p = 0.020	−0.30
8. Delay Discounting HC: N = 123 ADHD: N = 109	K 100 †	−5.25 (1.54)	−4.50 (1.65)	12.85 (1,228), p < 0.001*	−0.48
	K 30 †	−4.76 (1.65)	−4.38 (1.66)	3.15(1,228), n.s.	−0.23
	K 30 †	−4.39 (1.43)	−3.97 (1.67)	4.43(1,228), p = 0.036	−0.27
9. Trailmaking task HC: N = 132 ADHD: N = 128	Part A	23.70 (7.51)	26.80 (8.24)	11.60 (1,256), p = 0.001*	−0.43
	Part B	50.06 (17.30)	57.89 (20.30)	12.50 (1,254), p < 0.001*	−0.44
	Part B - A	26.33 (13.51)	31.00 (18.38)	5.88 (1,254), p = 0.016	−0.30

ANCOVA testing the effect of group for each neuropsychological measure, with age and gender as covariates.

^†log-transformed variable to a normal distribution.

^*indicates p-values surviving correction for the effective number of independent tests conducted (N = 22, significance threshold (type 1 error rate at 5%) = 0.0023)(Li and Ji, 2005).

Variables predicting ADHD diagnosis

A stepwise backwards logistic regression identified six out of 17 variables to significantly contribute to a model predicting diagnosis: Digit span (forward), Flanker (total SD of RT), SAdots (SD series errors and response bias), Delay discounting (k100) and Time estimation (absolute median deviation from 1000ms). The entire model significantly distinguished patients from controls (Log-likelihood = 174.13, R² (Nagelkerke) = 0.39, χ² = 57.54 (6 df), p < 0.001) and had a sensitivity (correctly predicting patients) of 64.9% and a specificity (correctly predicting controls) of 82.1%. Model details are shown in Supplementary table 4.

Number of deficient test scores across all outcome measures

Patients had deficient test scores on a significant larger proportion of variables than controls (mean controls = 9.16% (SD = 9.23), mean ADHD = 15.82% (SD = 13.55), F = 22.34 (1,261), p < 0.001). This effect remained when only including participants with complete data (F = 13.08 (1,164), p < 0.001). As apparent from Figure 2, there was a large variability between individual patients, with some patients not having any deficient scores (11%), while others were severely impaired (5%). The majority (62%) of patients was mildly impaired, and 23% was impaired. This variability was also present in the control group, although here the majority of participants (64%) had deficient scores on 10% or less of the outcome variables.

Figure 2. Deficient test scores across participants.

For each participant, deficient test scores were computed as the number of test scores that were below the score of the bottom 10% of the control group, divided by the total number of test scores of that participant and multiplied with 100%. The sections indicate the percentage of participants that had a deficient test score within a certain bin. `Mildly impaired' are participants with 1–20% deficient test scores, `impaired' are those with 20–40% deficient test scores and those with more than 40% deficient were labeled as `severely impaired'.

Effects of stimulant medication and history of MDD on neuropsychological measures

We additionally investigated the effect of stimulant medication on neuropsychological performance by comparing medication naïve patients, medicated patients and controls. Group effects where all in the same direction as in the main case-control analysis, although smaller (Supplementary Table 5). Post-hoc comparisons indicated that on the time estimation task medication naïve patients responded faster than medicated patients and controls.

The main effects from the case-control analyses were also reproduced when comparing healthy controls to ADHD patients with and without a history of MDD (Supplementary Table 6). On none of the neuropsychological measures did patients with a history of MDD differ from patients without this comorbidity. However, on several measures patients with a history of MDD did not differ from controls and patients without MDD, despite a main effect of group.

DISCUSSION

In this study we examined the neuropsychological performance of a large group of patients with persistent ADHD and healthy adult control participants on a broad range of neuropsychological tasks. As a group, patients with ADHD showed impaired EF, especially working memory and sustained attention, were more sensitive to delay aversion, and had increased response variability as compared to healthy controls. Stepwise logistic regression analysis showed that measures from distinct cognitive domains collectively contributed to the predictive model explaining variance in ADHD. Despite this, the model had limited predictive power for diagnostic status. Cognitive heterogeneity of the sample was also apparent from large inter-individual variability in the number of deficient test scores, especially in the ADHD group, but also in controls. Strikingly, no case-control differences were found in tasks measuring inhibition and timing in our test battery. Effect sizes were small to moderate, and medication and a history of MDD comorbidity did not explain differences in performance in adult ADHD.

As described, a popular model of childhood ADHD implicates three neuropsychological pathways in childhood ADHD, one involving EF deficits, one involving altered reward processing, and one involving temporal processing deficits (Castellanos et al., 2006; Sonuga-Barke et al., 2010). We report evidence for impairment in EF and reward processing, but not in temporal processing in patients with persistent ADHD. Our finding that EF deficits are primarily related to working memory and sustained attention is in agreement with the adult ADHD literature (Boonstra et al., 2005; Hervey et al., 2004). This result stresses the significance of attentional problems in adult ADHD and may reflect the fact that the symptoms of hyperactivity and impulsivity decrease as ADHD children approach adulthood (Biederman et al., 2000). It has been suggested that IQ may play a role in explaining working memory and attention deficits in adult ADHD (Boonstra et al., 2010; Murphy et al., 2001). However, as the groups did not differ on IQ, and covarying for IQ did not alter the results, this explanation is unlikely.

Contrary to expectations, we did not find EF differences related to inhibition, set-shifting, or verbal fluency. The ability to inhibit a response has been posited as a core domain impaired in ADHD (Barkley, 1997), and has been found in several studies of adult ADHD (Boonstra et al., 2010; Boonstra et al., 2005; Hervey et al., 2004), though not in others (Gmehlin et al., 2014; Halleland et al., 2012). We used the SART and Flanker tasks to measure inhibition, but are cautious to interpret our null findings as strong evidence against inhibition deficits in adult ADHD. First, the Flanker task showed a ceiling effect in inhibition errors, which is consistent with findings in early adolescence (Drechsler et al., 2005; Harms et al., 2014). This task may therefore lack sensitivity to measure inhibition impairments in adult ADHD. Second, on the SART, the number of commission errors (measuring inhibition) did not differ between patients and controls, nor did the number of omission errors (measuring attention). To better characterize inhibition deficits in adult ADHD, a more sensitive measure would be the stop signal reaction time as measured with a stop signal task. Such a task is unique in that is has variable inter-stimulus intervals often at a rapid pace that requires participants to interrupt an already ongoing response. This task design may provoke impulsive responses among participants more strongly and may thus be more sensitive to inhibition problems in ADHD (Epstein et al., 2001). Indeed, manipulation of response prepotency was effective in evoking response inhibition difficulties in adult ADHD patients (Grane et al., 2014).

Set-shifting is another component of EF, which we measured using part B of the Trailmaking task. Even though patients were slower on this part of the task, they were equally slow on part A, which measures motor speed (Nigg et al., 2005a). This finding is in line with other studies in adult ADHD suggesting that deficits in set-shifting are explained by impaired processing speed (Rohlf et al., 2012). We thus conclude that set-shifting as measured with the Trailmaking task was not impaired in the adult ADHD group. Lastly, patients did not differ from controls in verbal fluency measures, which contradicts previous findings (Boonstra et al., 2005; Hervey et al., 2004). This could be due to the good IQ-matching between patients and controls in our sample, whereas in other studies patients had lower IQ than controls. In children it was found that IQ significantly correlated with verbal fluency (Ardila et al., 2000). Hence, previously reported differences in verbal fluency may be more attributable to differences in IQ than to ADHD.

Delay aversion may represent a second neuropsychological pathway towards ADHD, linked to altered processing of rewards (Sonuga-Barke, 2002). Our results of stronger delay discounting in patients are in line with other evidence of increased impulsive decision making in persistent ADHD (Paloyelis et al., 2009)(Marx et al., 2010). The tendency to prefer immediate (smaller) over delayed (larger) rewards is also considered to be an aspect of impulsivity potentially important for the development of substance use disorders (Dick et al., 2010). Therefore, stronger delay aversion might represent a vulnerability marker for substance abuse in ADHD (Bickel et al., 2012). A third pathway involves temporal processing deficits (Sonuga-Barke et al., 2010). In the present study, patients did not differ from controls on timing accuracy using a time estimation task with an interval of one second. These findings are supported by a recent study using the same task, which showed deficits in time estimation accuracy were present in adolescents with ADHD, but not in adults (Thissen et al., 2014). However, another study, which examined time estimation in adults with ADHD using several time intervals (2, 6, 12, 24, 36 and 48 seconds), found that the patients produced errors predominantly at interval durations of 36 and 48 seconds (Marx et al., 2010). This may suggest that tasks using an interval of one second may not be sensitive enough to measure existing timing deficits in adult ADHD.

In the analyses comparing patients and controls the largest effect sizes were observed for measures of performance variability, both in terms of fluctuations in errors as in reaction times. This confirms our hypothesis, which was based on previous studies identifying RTV as one of the most robust features of ADHD (Kofler et al., 2013; Tamm et al., 2012). Notably, the average reaction time on the tasks used to measure RTV did not differ between patients and controls, supporting the notion that RTV is not attributable to differences in processing speed (Kofler et al., 2013). Rather, RTV is thought to reflect lapses in attention that produce a skewed reaction time distribution with a large tail (Leth-Steensen et al., 2000). More thorough investigation of RTV used ex-Gaussian modeling and showed that increased RTV is partly due to overly slow responses (Feige et al. 2013; Gmehlin et al., 2013; Wolfers et al., 2015). These slow responses are reflected by the ex-Gaussian parameter tau, which represents the exponential component of the reaction time distribution. Recently, we showed that the tau parameter was associated with the microstructural integrity of the right superior longitudinal fasciculus, a white matter tract implicated in both attention and ADHD (Wolfers et al., 2015). Taken together, such findings suggest a neurobiological basis for within-subject variability in ADHD. Interestingly, we observed the largest effect size for the variance in errors made during the SA-dots task. This is a promising novel measure for future studies on sustained attention in ADHD using a continuous performance task.

We achieved limited accuracy in predicting ADHD diagnosis from neuropsychological performance, despite the large number of cognitive test variables available. This is consistent with what was previously found in children with ADHD (Sjowall et al., 2013). The best fitting predictive model included six measures from different cognitive domains (EF, response variability, timing and delay aversion) and reached 82.1% specificity and 64.9% sensitivity. This rather low sensitivity makes a test based on cognitive measures insufficient as a diagnostic tool for ADHD in clinical practice. The variables retained in the final model of the logistic regression could be influenced by outliers, as these can be expected to contribute strongly to the model. However, all extreme outliers were removed from the data before data analysis, reducing the effect of erroneous data on the model. Rather, the variables in the model are likely to be most sensitive to behavioral impairments associated with ADHD, as was also reflected in the effect sizes of most of these variables in the case-control analysis. Importantly, measures from distinct cognitive domains collectively contributed to the model, indicating that there is not a single cognitive task or domain sufficient for explaining ADHD on the group level. This is in agreement with the theory of multiple pathways leading to impairment in ADHD (Sonuga-Barke et al., 2010). Besides that heterogeneity can be explained by impairments in multiple cognitive pathways, we also observed differences in severity of impairments between individuals. The majority of patients were impaired on less than 20% of all cognitive measures, and while a small proportion of patients had more than 40% deficient test scores, 11% of patients did not show any deficit. This is in line with studies in childhood ADHD (Coghill et al., 2013; Nigg et al., 2005b; Sonuga-Barke et al., 2010). Importantly, only 23% of our healthy control participants did not show any deficits, which is much lower than the previously reported 53% and 60% (Coghill et al., 2013; Nigg et al., 2005b). However, these differences between studies can be explained by the fact that the current study included many more variables (27 instead of four and six). Furthermore, the majority of controls fell in the `mildly impaired' group, which means they performed deficiently on 1–20% of the tasks. Seeing that the criterion for having a deficient test score was performing at the extreme of the control distribution, it would be expected that controls perform deficiently on some tasks.

The current findings provide a starting point to investigate individual differences in terms of impaired cognitive pathways, for instance by using clustering analyses on the neuropsychological data (Fair et al., 2012). Such an approach follows the recently proposed strategy by the NIMH, called Research Domain Criteria (RDoC) to investigate mental disorders in a dimensional instead of categorical manner (http://www.nimh.nih.gov/research-priorities/rdoc/index.shtml). Neurocognitive measures can be used to characterize psychopathology without being restricted to current disorder categories. This will aid in the understanding of the neurobiological and behavioral underpinnings of mental disorders. Furthermore, neuropsychological investigations may be helpful for clinicians in characterizing individual differences, allowing more personalized treatments.

We did not find evidence for subgroups within the patient group, neither due to stimulant medication treatment nor history of comorbid MDD, which could explain the observed cognitive heterogeneity. Medication use did not influence task performance in our exploratory analysis; medication naïve patients performed similar to medicated patients. Mechanisms linking pharmacological actions of stimulants to neuropsychological processes are speculative, although our results support observations that, in adult ADHD, stimulants seem to produce little improvement on a variety of neuropsychological tasks (Advokat, 2010; Turner et al., 2005). Similarly, the group of patients with a comorbidity in the form of a history of MDD did not seem to differ greatly from the group without this comorbidity in terms of neuropsychological functioning. This extends earlier findings and suggest that ADHD patients diagnosed with current or remitted MDD show similar neuropsychological profiles as patients diagnosed with ADHD alone (Katz et al., 1998; Riordan et al., 1999). It should be noted however that this study was not set up to investigate the effects of stimulant medication or differences between patients with and without a history of comorbid MDD, hence these effects should be investigate further.

The findings presented here should be considered in light of several strengths and weaknesses. This study is unique in its large, well-defined naturalistic sample of patients and a well-matched control sample. We have used a large battery of tasks covering EF, timing, and delay aversion domains. This allows our findings to be interpreted on the scale of cognitive domains instead of on a task-specific level. Our sample was large enough to investigate effects of (at least one) comorbidity. However, our investigation of the effect of stimulant medication was likely underpowered as there were only 20 medication naïve patients in our sample. Investigating the effects of stimulant medication in adults is challenging, as by definition these patients have been symptomatic for a long period. It would therefore be more relevant to investigate the effect of medication duration across patients, but this requires well-documented medication use history, which was not available. Additionally, our findings are limited by the tasks included in our testing battery. We did not include measures tapping into the domains of planning or decision making, which are also important in ADHD psychopathology. Furthermore, our measures of time estimation could be improved by having longer timing intervals. Similarly, inhibition could be measured by computing stop-signal reaction times from a stop signal task. Including such measures might improve the predictive power for diagnostic status.

To conclude, our study provides novel insights into adult ADHD neuropsychology as well as confirmation of findings observed in earlier, smaller studies. In summary, our study adds to the literature in the following ways: 1) compared with previous studies, our sample size is almost two (Seidman et al. 1998) or three times larger (Boonstra et al. 2010; Fuermaier et al. (2015); 2) we also examined delay aversion and timing deficits which was not sufficiently covered by previous work; 3) while other studies investigated variability in reaction times only, we also investigated variability in errors made during a continuous performance task; 4) we investigated confounding effects of depression history and stimulant treatment (the ADHD patient samples from Seidman et al. (1998) and Boonstra et al. (2010) were all medication-naïve); 5) ours was the first study in adult ADHD to calculate the number of deficient test scores per participant as was previously done in childhood ADHD (Coghill, 2013); 6) we studied not only simple group differences but also measures of sensitivity and specificity to examine the discriminatory ability of the neuropsychological test battery in adult ADHD. Our comprehensive analysis of cognitive performance in a large sample of patients with persistent ADHD and well-matched healthy control participants confirms that several cognitive domains are affected in the adult ADHD population, with moderate effect sizes. Both the ADHD and the control sample were heterogeneous in their cognitive performance, with large differences in the number of tasks on which participants scored deficient. In line with this, a predictive model including measures from several domains had limited power to predict diagnostic status. Neuropsychological tasks may therefore be more relevant for characterizing individual impairments that can specifically be targeted with personalized treatment. Future studies focusing on inter-individual differences in performance of patients may aid in a better understanding of ADHD etiology and its persistence, also in terms of the underlying biology.

Figure 1. Differences in performance between ADHD patients and controls on measurements from several cognitive domains.

Bar graphs indicate the average performance per group for each neuropsychological measure (time estimation absolute median deviation from 1000ms is not shown); error bars represent the standard error of the mean. Dark grey bars represent the healthy control group, lighter grey bars represent ADHD patient group. An asterix (*) indicates measures where patients differed significantly from controls.