A Quick Test of Cognitive Speed for Comparing Processing Speed to Differentiate Adult Psychiatric Referrals With and Without Attention-Deficit/Hyperactivity Disorders

Abstract

Objective: This retrospective study used A Quick Test of Cognitive Speed (AQT) to compare processing speed and efficiency measures by adults with attention-deficit/hyperactivity disorder (ADHD) or non-ADHD psychiatric disorders and healthy controls.

Method: Color, form, and color-form combination naming tests were administered to 104 adults, ages 17–55 years, referred for psychiatric evaluation of possible ADHD. Thirty healthy adults were controls. Psychiatric intake procedures identified 64 adults with ADHD (ICD-10 and DSM-IV criteria) and 40 with mild psychiatric disorders without ADHD. The study was conducted from 2008 through 2010.

Results: At intake, color, form, and color-form combination naming times (seconds) were longer and overhead [color-form combination – (color + form)] was larger for patients with ADHD than for non-ADHD patients and controls. In the ADHD group, color and form measures were in the normal range. Color-form combination was in the slower-than-normal speed (60–70 seconds) and overhead, a processing-efficiency measure, in the atypical range (> 10 seconds). In the non-ADHD patient and control groups, all AQT measures were in the normal range. Analysis of variance with post hoc analysis of log-normal values for color, form, and color-form combination and time for overhead indicated significant (Bonferroni P < .01) mean differences between the ADHD and other groups, but not between the non-ADHD and control groups. When using fail criteria for either color-form combination or overhead, the sensitivity for the ADHD group was 89%.

Conclusions: Results support AQT as a possible complement to psychiatric intake procedures to differentiate adults with ADHD from those with mild psychiatric disorders, and they suggest that a controlled prospective study might be productive.

Clinical Points

• ▪ Measures of processing speed in A Quick Test of Cognitive Speed may complement standard psychiatric intake procedures to differentiate adults with attention-deficit/hyperactivity disorder from referrals with mild psychiatric disorders.

• ▪ Current evidence indicates a high degree of clinical utility (sensitivity 89%) when applying fail criteria for dual-dimension processing of color-form combinations and overhead, a measure of processing efficiency.

• ▪ A Quick Test of Cognitive Speed may provide objective, clinical baselines for evaluating the effects of treatment with pharmaceuticals.

Behavioral and quality of life ratings are measurement standards in psychiatric intake procedures and for evaluating pharmacologic treatment effects in adolescents/adults with attention-deficit/hyperactivity disorders (ADHDs).^1–6 A diagnosis of ADHD is based on observations of impulsivity, inattention, and hyperactivity, and distinguishing between ADHD and mild psychiatric disorders without ADHD based on behavioral ratings can be difficult.⁷ This retrospective study explored whether A Quick Test of Cognitive Speed (AQT) might provide complementary, objective measures of cognitive control to differentiate adults with ADHD from those with milder, nonorganic forms of psychiatric disorders without ADHD.^8,9 A Quick Test of Cognitive Speed color-form naming has proven useful in assessing executive dysfunction secondary to neurologic/neuropsychiatric disorders in older adults.^10–14 A prospective study provided preliminary evidence that the AQT additive model for processing speed and efficiency differentiated adults with ADHD and healthy controls.^15,16

The use of processing-speed tests in clinical studies of ADHD symptomatology in children and adults has increased in recent years.^17–21 This study used AQT color, form, and color-form combination naming to assess processing speed and efficiency. Dual-dimension color-form combination naming by healthy adults activates bilateral temporal-parietal and subcortical brain regions, including the hippocampus.^8,9,13,21 The temporal-parietal lobes bilaterally have been related to working memory and white matter atrophy has been related to cognitive processing deficits.^22,23 Other evidence supports that these regions are associated with focusing and reorienting attention, keeping visual items active in working memory, set shifting, and cognitive control.^24–26 These executive functions are now widely substantiated to be impaired in children and adults with ADHD and to affect work performance negatively.^{2–4,27–33}

This study evaluated the clinical utility of AQT in differentiating psychiatric referrals for evaluation of possible ADHD as either true or false. Adults with ADHD were expected to exhibit significantly slower processing speed and reduced processing efficiency compared to healthy controls.¹⁵ Adults with mild, nonorganic psychiatric disorders without ADHD (non-ADHD) were hypothesized to exhibit similar processing speed and efficiency as healthy controls. We anticipated that the AQT additive model would result in similar levels of sensitivity and specificity as a means for differentiating adults with ADHD from healthy controls.^15,16

METHOD

Participants

Intake protocols, covering a 3-year period from 2008 through 2010, for 127 adults who were referred to a regional, urban outpatient clinic for psychiatric evaluation and who completed the AQT provided the database. All psychiatric diagnoses were determined primarily according to World Health Organization (WHO) ICD-10 criteria, as required by state regulations, and secondarily to DSM-IV-TR (314.01) criteria for validation.^34,35 Of the 64 referrals, 28 female participants and 36 male participants (ages 17 to 54 years; mean = 31.14) received ADHD diagnoses (WHO ICD-10 code F90.0 or F90.9) and follow-up pharmacologic treatment. Five adults with ADHD diagnoses who did not complete treatment within the setting were excluded. Participants in the ADHD group exhibited impaired academic achievement, difficulties with employment, and comorbidities such as substance abuse and mild personality disorders or depression, all of which are commonly found in adults with ADHD.^3,7,36–41 Attention-deficit/hyperactivity disorder subtypes, as delineated by DSM-IV-TR, were not further explored due to time constraints imposed by the social-medicine environment.

Forty referrals, 12 female and 28 male patients (ages 18 to 44 years; mean = 32.3), received diagnoses that included personality disorders (ICD-10 codes F60.0, F60.1, F60.31, F60.5, F60.6, F60.7, or F60.9), addiction (F10.1, F12.2, F19.1, F19.2, or F19.9), affective disorders (F32.9, F33.8, or F38.1), or obsessive-compulsive disorder (F42.9), but excluded ADHD. Some participants in this group experienced impaired academic performance; specific learning disabilities, such as dyslexia; and/or difficulties with employment. Eighteen psychiatric referrals with autism spectrum syndrome (F84.0, F84.1), organic brain disorders (F04.9, F06.2, and F07.0), or bipolar disorder/schizophrenia (F20.0, F20.6, F20.9, F21.9, F23.1, F31.9, and F45.1) were excluded from the study. All participants with psychiatric disorders consented to be assessed with AQT at intake before medication or other treatment.

Thirty healthy adults, 14 female and 16 male, ranging in age from 18 to 43 years (mean = 28.27), who participated in an earlier study, were controls.¹⁵ Controls were recruited from the urban community in response to announcements to personal contacts. Single-factor analysis of variance (ANOVA) indicated that participants in the 3 groups did not differ significantly in age (F_1,131 = 2.29, P = .11).

Materials and Administration

The color, form, and color-form combination naming tests of A Quick Test of Cognitive Speed were administered individually, in standard order (ie, color, form, and then color-form combination) to all psychiatric referrals at intake without medication and to all controls. The single-dimension color task requires rapid naming of 40 randomly sequenced, colored squares (black, blue, red, yellow), and the form task requires naming of black shapes (circles, lines, squares, triangles). The color and form tests measure reaction, retrieval, and response time. The dual-dimension color-form combination test requires rapid naming of combinations of the colors and shapes (eg, red circle). In healthy adults, the color-form combination test measures an additional “switch cost” derived from articulating 2 words in rapid sequence.^8,9,16 In neuropsychiatric disorders, there is an additional penalty that is caused by increased demands on attentional control, visual working memory, and cognitive set shifting.^10–13 The naming times for each test (seconds) and a measure of the difference (overhead) between color-form combination naming and the sum of color and form naming times [color-form combination – (color + form)] provided the data. Overhead measures the relative efficiency of dual-dimension processing, including switch cost, and it was norm-referenced with 270 healthy adults, ages 18 to 65 years.¹⁶ A Quick Test of Cognitive Speed color, form, and color-form combination tests are highly reliable (r = 0.91–0.95), and there is no evidence of learning or habituation during repeated (10 minute) naming.^8,9 In healthy development, maximum color-form combination processing speed is reached at age 15 years, with minimal change with advancing age (1 s/decade).^42,43

The same psychiatrist administered AQT concurrently with psychiatric interviews and completion of Adult ADHD Self-Report Scale (ASRS-v1.1)⁴⁴ during a standard psychiatric work-up that concurred with prevailing Danish and European psychiatric practice and started with a referral for a diagnostic evaluation of possible ADHD symptomatology.⁴⁵ Intake procedures consisted of (1) a structured psychiatric interview, (2) administration of ASRS-v1.1, (3) administration of AQT for baseline purposes, (4) and acceptance of possible treatment with informed consent. At intake, participants with ADHD diagnosis (n = 64) received ASRS-v1.1 ratings, based on all items, ranging from 22 to 76 (mean = 50.69, SD = 11.56) and indicating probable ADHD (ie, > 20 points). Participants with non-ADHD diagnoses (eg, personality disorders, addiction, mild depression) received ASRS-v1.1 ratings ranging from 22 to 68 (mean = 49.41, SD = 11.85). Comparison of group mean values indicated no significant difference in the ASRS-v1.1 ratings in the 2 psychiatric groups (t = 0.47, P = .64).

Statistical Analyses

All analyses of data were performed with the Microsoft Excel StatPlus, Mac 2009 version (Microsoft, Redmond, Washington). One-way ANOVA with post hoc analysis (Bonferroni) compared mean differences in processing speed, as measured by color, form, and color-form combination, between the 3 groups. Paired-sample t tests, not assuming equal variance, compared group mean values for the overhead [color-form combination – (color + form)]. Null hypotheses were rejected at P < .01.

RESULTS

Descriptive statistics for the AQT naming times and overhead measures (seconds) for the 3 groups (ADHD, non-ADHD, and controls) are shown in Table 1. At intake, color and form naming-time mean values were within the normal ranges for all groups (ie, < 25 and < 30 seconds, respectively). The ADHD group mean for color-form combination naming was in the upper range of slower-than-normal speed (ie, between 60–70 seconds) and, for overhead, in the atypical/pathological range (ie, > 10 seconds). In the non-ADHD and control groups, the mean values for color-form combination and overhead were within the normal range (ie, < 60 seconds and < 6 seconds, respectively).^8,9,16 Figure 1 illustrates the naming-time differences (seconds) between groups for each measure.

Table 1.

Descriptive Statistics for A Quick Test of Cognitive Speed Naming Times and Overhead for Adolescents and Adults in the ADHD, Non-ADHD Psychiatric Referral, and Healthy Control Groups

Group	n	Color, Mean (SD), s	Form, Mean (SD), s	Color-Form Combination, Mean (SD), s	Overhead, Mean (SD), sa
ADHD	64	24.63 (4.45)	29.50 (6.09)	66.69 (11.12)	13.00 (7.93)
Non-ADHD	40	21.20 (3.64)	24.95 (4.34)	48.15 (7.29)	2.13 (3.94)
Control	30	20.73 (2.94)	23.53 (3.50)	47.53 (5.82)	3.27 (3.29)

^aOverhead equals color-form combination − (color + form).

Abbreviations: ADHD = attention-deficit/hyperactivity disorder.

Figure 1.

Medians and Interquartile Ranges for Color, Form, and Color-Form Naming and Overhead (seconds) for Participants in the ADHD (n = 64), Non-ADHD Psychiatric Referral (n = 40), and Control (n = 30) Groups

Abbreviation: ADHD = attention-deficit/hyperactivity disorder.

Tests for normality of the sampling distributions for each AQT processing-speed variable accepted this assumption for all measures. Assumptions for homogeneity of variance were rejected, and we applied log-normal transformations to all color, form, and color-form combination naming times. One-way ANOVA with post hoc analysis (Bonferroni) evaluated the significance of mean differences in the AQT log-normal measures between groups (Table 2). The mean values for color (F_1,131 = 13.46, P = .0000, η² = 0.17), form (F_1,131 = 17.78, P < .0000, η² = 0.21), and color-form combination naming (F_1,131 = 77.91, P < .0000, η² = 0.54) differed significantly between groups. Effect sizes ranged from small for color and form to large for color-form combination naming.

Table 2.

One-Way Analysis of Variance With Post Hoc Analyses (Bonferroni) for Log-Normal Mean Differences Between Participants in the ADHD (n = 64), Non-ADHD Psychiatric Referral (n = 40), and Healthy Control Groups (n = 30)

AQT Test	df	Sum of Squares	Mean Squares	F	P	Bonferroni Test Statistic
Color	2	0.8112	0.4056	13.46	.0000	ADHD vs non-ADHD = 4.06* accepted
	131	3.9467	0.0301			ADHD vs control = 4.33* accepted
						non-ADHD vs control = 0.40 rejected
Form	2	1.2069	0.6035	17.78	.0000	ADHD vs non-ADHD = 4.20* accepted
	131	4.4452	0.0339			ADHD vs control = 5.20* accepted
						non-ADHD vs control = 1.34 rejected
Color-form combination	2	3.9321	1.9660	77.91	.0000	ADHD vs non-ADHD = 9.99* accepted
	131	3.3059	0.0252			ADHD vs control = 10.02* accepted
						non-ADHD vs control = 0.66 rejected

^*P < .01.

Abbreviations: ADHD = attention-deficit/hyperactivity disorder, AQT = A Quick Test of Cognitive Speed.

Overhead times were not transformed to log-normal values because they could be either positive or negative. The significance of mean differences was tested with 2-tailed t tests, not assuming equivalence of variance. Overhead mean values differed significantly between the ADHD and non-ADHD groups (t_1,103 = 9.29; P = .0000; Cohen d = 1.74, r = 0.63) and between the ADHD and control groups (t_1,93 = 8.40; P < .0000; Cohen d = 1.60, r = 0.63), and effect sizes were large. There was no significant mean difference between the non-ADHD psychiatric and control groups (t1,69 = 1.32; P > .05; Cohen d = −0.31, r = −0.16). On the basis of the ADHD overhead mean, the mean switch cost plus penalty overhead for 40 color-form combination items was 330 ms/stimulus, indicating a sizeable effect of demands on attention, visual working memory, and set shifting. In comparison, the mean switch cost was 80 ms/stimulus in the control group and 50 ms/stimulus in the non-ADHD group.

As results were analyzed, the question emerged whether the intake measures would be consistent if AQT were administered twice before the initiation of pharmacologic treatment. On examination, 45 adults in the ADHD group and 27 in the non-ADHD group were given the AQT tests with short time intervals ranging from a day to a week. Mean values for each AQT variable for these administrations were compared with simple 1-way ANOVA, as the paired sample distributions met assumptions for normality and homogeneity of variance. In the ADHD group, paired mean values for first and second administrations did not differ significantly (P > .05) for color (25.22 and 25.51 seconds, F_1,88 = 0.0531), form (29.87 and 29.31 seconds, F_1,88 = 0.1424), color-form combination (65.96 and 64.29 seconds, F_1,88 = 0.4330), or overhead (11.47 and 10.07 seconds, F_1,88 = 0.4024). In the non-ADHD group, paired mean values for first and second administrations also did not differ significantly (P > .05) for color (20.46 and 21.78 seconds, F_1,52 = 0.7379), form (24.48 and 24.52 seconds, F_1,52 = 0.0011), color-form combination (47.11 and 48.63 seconds, F_1,52 = 0.7816), and overhead (1.48 and 2.37 seconds, F_1,52 = 0.5075).

Sensitivity and specificity were determined by comparing individual naming times to criterion-referenced cut-off times at +1 SD of the mean for the upper limits of healthy performance.^9,10,16 Table 3 shows the pass/fail percentages for each AQT measure in the 3 groups. The sensitivity in identifying adults with ADHD (n = 64) at intake was 80% for the overhead and 89% when fail criteria for either color-form combination naming or overhead were applied in combination. Specificity proved highest for the AQT color-form combination processing-speed measure (100%).

Table 3.

Pass and Fail Percentages and Participant Counts for Normal (< 1 SD of mean values) Naming and Overhead Times (seconds) for Participants in the ADHD, Non-ADHD Psychiatric Referral, and Healthy Control Groups

AQT Test	ADHD (n = 64), % (n)		Non-ADHD (n = 40), % (n)		Control (n = 30), % (n)
	Pass	Fail	Pass	Fail	Pass	Fail
Color	62 (40)	38 (24)	93 (37)	7 (3)	97 (29)	3 (1)
Form	61 (39)	39 (25)	90 (36)	10 (4)	100 (30)	0 (0)
Color-form combination	31 (20)	69 (44)	98 (39)	2 (1)	100 (30)	0 (0)
Overhead	20 (13)	80 (51)	88 (35)	13 (5)	87 (26)	13 (4)
Color-form combination or overhead	11 (7)	89 (57)	98 (39)	2 (1)	87 (26)	13 (4)

^aOverhead equals color-form combination − (color + form).

Abbreviations: ADHD = attention-deficit/hyperactivity disorder, AQT = A Quick Test of Cognitive Speed.

We used linear least squares regression analyses to establish how well the sum of the color + form naming times would predict color-form combination naming for each population. The resulting coefficients of determination (R²) were 0.980 for the ADHD group, 0.986 for the non-ADHD group, and 0.951 for the control group. Figure 2 indicates extensive separation of the regression lines between the ADHD and the non-ADHD and control groups, which are similar. The differences in the slopes indicate that, in the ADHD group, the increase in switch cost is proportionally larger as these participants obtain slower processing-speed measures than those in the non-ADHD and control groups. These results indicate that the sum of the single-dimension naming times (color + form) may have strong predictive capability of whether an individual has ADHD or is a psychiatric referral without ADHD or is healthy.

Figure 2.

Individual Data Points and Linear Least-Squares Regression Lines and Equations for the ADHD (n = 64), Non-ADHD Psychiatric Referral (n = 40), and Control (n = 30) Groups

Abbreviation: ADHD = attention-deficit/hyperactivity disorder.

DISCUSSION

Some of the limitations of this study were imposed by time restrictions set by the social-medicine system within which it was conducted. The greatest limitation appears to be that subtypes of ADHD and existing comorbidities were not explored in greater detail. However, all participants with ADHD completed pharmacologic treatment with either methylphenidate or atomoxetine, with positive outcomes in that dual-dimension processing speed and efficiency (overhead) were restored to normal levels at stabilization of the ADHD symptomatology. The heterogeneity among participants with mild psychiatric disorders and associated difficulties presents a second limitation. It should be noted that only 2 participants in this group exhibited specific learning disabilities, such as dyslexia. The results should be considered from a clinical perspective and viewed as preliminary until independently validated. Notwithstanding, the results supported some of the stated hypotheses while negating others. Moreover, the present findings concur with a previous study of adults with ADHD and age- and sex-matched healthy controls.¹⁵ The outcomes of this and the previous study will therefore be compared throughout the discussion. We will also discuss the outcome of the comparison of ASRS-v1.1 ratings in the 2 psychiatric groups, although we did not form a priori hypotheses about these findings. Rapid naming deficits can occur with psychiatric and psychoeducational disorders, and it is appropriate to compare AQT color-form with the Stroop and Rapid Automatized Naming and Rapid Alternating Stimulus Test (RAN/RAS) naming tests.^46,47

The design of AQT is similar to the Stroop Color-Word Test, as both assess single-dimension naming in 2 tasks and combine the 2 dimensions for dual-dimension naming. The executive function and the associated brain regions involved in performing the dual-dimension tasks on the 2 tests differ, however. The Stroop assesses interference inhibition deficits that reflect fronto-striatal underactivation associated with, for example, ADHD,^48,49 whereas AQT color-form combination is associated with bilateral temporal-parietal activation.^8,9,13,21 The RAN component of RAN/RAS requires sequential single-dimension naming, and total pause time is a strong predictor of early reading accuracy and fluency.⁵⁰ The RAS component requires rapid naming of sequentially arranged alternating stimuli (letters, numbers, and colors). The RAN/RAS does not feature stimuli that combine previously tested single-dimension naming, and heavy emphasis is placed on naming symbolic stimuli that appear to be critical in differentiating dyslexia and ADHD.¹⁹

On the basis of ASRS-v1.1 ratings, all participants with ADHD obtained ratings above the cut-off criterion for normalcy, and the average was well within the range considered to indicate possible ADHD. Moreover, the ASRS-v1.1 ratings were not significantly different for adults in the ADHD or non-ADHD psychiatric groups. This finding indicates that no clear differentiation between ADHD and mild psychiatric disorders without ADHD could be made on the basis of the ASRS-v1.1 self-ratings.

The outcomes of group comparisons of the AQT processing speed and efficiency measures generally supported a priori predictions. It was, however, unexpected that participants with ADHD were significantly slower at processing and naming the single-dimension stimuli, even though the group mean values were in the normal range. It was anticipated that adults with ADHD would be slower and less efficient in processing dual-dimension stimuli than healthy controls. These findings concurred with our hypotheses and with the earlier comparison study of adults with ADHD and healthy controls.¹⁵ We tested the consistency of processing speed and efficiency measures, obtained at intake with 2 administrations of AQT, and found that group mean values were consistent over time in the ADHD and non-ADHD psychiatric groups. It is notable that, for adults with ADHD at intake without medication, the mean dual-dimension naming speed was in the slower-than-normal range (ie, 60–70 seconds) and the average overhead in the atypical/pathological range (> 6 seconds) compared to criteria for normalcy.^8,9,16 These findings indicate that cognitive control of attention, working memory, and set shifting were significantly reduced by the ADHD symptomatology. On average, cognitive control was reduced by a factor of 4 compared to normative controls as a result of the increased demands on executive functions by color-form combination naming, and this agrees with previous findings.¹⁵ The sensitivity with which AQT dual-dimension processing speed (color-form combination) and efficiency (overhead) in combination identified adults with ADHD was somewhat lower (89%) in this than in the previous study (93%), but it was still in the high range.¹⁵

It was unexpected that the non-ADHD group performed within the normal range on all AQT measures. On the basis of ASRS-v1.1 self-ratings, it would be assumed that there would be some degree of reduction of cognitive control. This appeared not to be the case, as there were no significant differences in processing speed or efficiency in the non-ADHD compared to the control group. The average switch cost imposed by dual-dimension color-form combination naming in the non-ADHD group was relatively short at 50 ms/stimulus and compared well with the control group and normative data.^15,16 The percentages that passed dual-dimension color-form combination naming criteria for normalcy in the non-ADHD and control groups were similar and high.

Our findings suggest that AQT dual-dimension naming speed and overhead measures can differentiate psychiatric referrals with ASRS-v1.1 ratings that indicate possible ADHD and differentiate true- from false-positive results. The illustration of results from linear least-squares regression analyses (see Figure 2) indicates that the switch cost imposed by ADHD symptomatology clearly separates adults in the ADHD group from those in the non-ADHD and control groups and thus supports the AQT additive model. Our findings should be validated in a prospective study with control of psychiatric and psychoeducational comorbidities, as these may have additive effects on the presenting ADHD symptomatology and executive dysfunctions.^36–41 Independent validation could have vast clinical implications for daily psychiatric practice, costs, and integration of adults with ADHD in the work force of a complex society.²⁷ Adult psychiatric referrals identified to fit the AQT additive-model profile for ADHD could be immediately placed on pharmacologic treatment trial for responsiveness.^15,16 Adult referrals without organic brain disorders or pervasive developmental disorders, identified to fit the normal AQT performance profile (non-ADHD), could be referred for cognitive-behavioral therapy or other forms of rehabilitation. Within the limitations of this study, these preliminary findings suggest that validation studies with greater imposed control of variables might be productive.

Drug names: atomoxetine (Strattera), methylphenidate (Daytrana, Ritalin, and others).

Potential conflicts of interest: Drs Wiig and Nielsen receive royalty shares for A Quick Test of Cognitive Speed from Pearson Assessment.

Funding/support: None reported.