Cross-Disorder Cognitive Impairments in Youth Referred for Neuropsychiatric Evaluation

Abstract

Objective

Studies suggest that impairments in some of the same domains of cognition occur in different neuropsychiatric conditions, including those known to share genetic liability. Yet, direct, multi-disorder cognitive comparisons are limited, and it remains unclear whether overlapping deficits are due to comorbidity. We aimed to extend the literature by examining cognition across different neuropsychiatric conditions and addressing comorbidity.

Method

Subjects were 486 youth consecutively referred for neuropsychiatric evaluation and enrolled in the Longitudinal Study of Genetic Influences on Cognition. First, we assessed general ability, reaction time variability (RTV) and aspects of executive functions (EFs) in youth with non-comorbid forms of attention-deficit/hyperactivity disorder (ADHD), mood disorders and autism spectrum disorder (ASD) as well as in youth with psychosis. Second, we determined the impact of comorbid ADHD on cognition in youth with ASD and mood disorders.

Results

For EFs (working memory, inhibition and shifting/ flexibility), we observed weaknesses in all diagnostic groups when participants’ own ability was the referent. Decrements were subtle in relation to published normative data. For RTV, weaknesses emerged in youth with ADHD and mood disorders, but trend-level results could not rule out decrements in other conditions. Comorbidity with ADHD did not impact the pattern of weaknesses for youth with ASD or mood disorders but increased the magnitude of the decrement in those with mood disorders.

Conclusions

Youth with ADHD, mood disorders, ASD, and psychosis show EF weaknesses that are not due to comorbidity. Whether such cognitive difficulties reflect genetic liability shared among these conditions requires further study.

Historically, conceptual models of neuropsychiatric disorders have emphasized relationships between particular conditions and cognitive decrements that are potentially pathognomonic (e.g., inhibitory control in ADHD [Barkley, 1997]). Yet, as Pennington (2006) and colleagues (Willcutt, Doyle, Nigg, Faraone, & Pennington, 2005) point out, models that include multiple cognitive deficits that overlap are more consistent with growing evidence that neuropsychiatric disorders are complex, multifactorial conditions that share some of their underlying genetic risk (PGC, 2013). The literature to date supports these more complex models. Although not all studies are consistent, meta-analyses that summarize these data implicate weaknesses in general cognitive ability, executive functions and reaction time variability (RTV) in a range of conditions (see Table 1 for examples).

Table 1.

Meta-analyses, conceptual reviews and studies of relatives support the relevance of impairments in general ability, reaction time variability and executive functions to neuropsychiatric conditions known to share genetic underpinnings.

	Neuropsychological weaknesses found in….
	IQ/ General Cognitive		RTV		Executive Functions
					Working Memory		Shifting/ Flexibility		Inhibition/ Interference Control
	Affected Individuals	Relatives	Affected Individuals	Relatives	Affected Individuals	Relatives	Affected Individuals	Relatives	Affected Individuals	Relatives
ADHD	Willcutt et al., 2008a	Kuntsi et al., 2004d; Oerlemans et al., 2015c	Kofler et al., 2013a; Lipszyc & Schachar, 2010a	Wood et al., 2011c; Thissen et al., 2014c	Kasper et al., 2012a; Alderson et al., 2013a	Bidwell et al., 2007c; Oerlemans et al., 2015c	Weak/ inconsistent Willcutt et al., 2008a	Bidwell et al., 2007c	Willcutt et al., 2008a; Lipszyc & Schachar, 2010a	Slaats-Willemse et al., 2007c
BPD	Joseph et al., 2008a; Stefanopoulou et al., 2009*	Morgan et al., 2012c	Not well-studied Bora et al., 2006; Krukow et al., 2017; Kaiser et al., 2008	Brotman et al., 2009	Bora et al., 2009a; Bourne et al., 2013a	Weak/trend-level findings; Doyle et al., 2009; Volkert et al., 2016	Bourne et al., 2013a; Bora et al., 2009a; Stefanopoulou et al., 2009a	Bora et al., 2009a,c	Lipszyc et al., 2010a; Bora et al., 2009a	Bora et al., 2009a,c
ASD	Willcutt et al., 2008a	Fombonne et al., 1997c	Weak evidence; suggest impaired in the presence of ADHD Karalunas et al., 2014b	--	Craig et al., 2016	Inconsistent Mosconi et al., 2010c; McLean et al., 2014c	Willcutt et al., 2008a	Inconsistent Wong et al., 2006c; McLean et al., 2014c	Geurts et al., 2014*	No support; Wong et al., 2006c; McLean et al., 2014c
MDD	--	Morgan et al., 2012c	Kaiser et al., 2008	--	Snyder, 2013; Wagner et al., 2015	--	Stefanopoulou et al., 2009a	--	Lipszyc & Schachar, 2010a	--
SCZ	Willcutt et al., 2008a; Stefanopoulou et al., 2009a; Fioravanti et al., 2012a	Agnew-Blais and Seidman, 2013a,c; Owens et al., 2011d	Kaiser et al., 2008	--	Forbes et al., 2009a; Fioravanti et al., 2012a	Agnew-Blais and Seidman, 2013a; Toulopoulou et al., 2010d; Toulopoulou et al., 2007d	Willcutt et al., 2008a; Stefanopoulou et al., 2009a; Fioravanti et al., 2012a	Agnew-Blais and Seidman, 2013a; Owens et al., 2011d	Lipszyc & Schachar, 2010a; Westerhausen et al., 2011a	Snitz et al., 2006a,c

Note: Rather than an exhaustive review, Table 1 includes representative studies from the literature and emphasizes meta-analyses in youth. Where such analyses are not available, we include meta-analyses of adults or, alternatively, reviews and individual studies. We also note cells where evidence for cognitive impairment is weak or limited. ADHD = attention-deficit/hyperactivity disorder; BPD = bipolar disorder; ASD = autism spectrum disorder; MDD = major depressive disorder; SCZ = schizophrenia.

^aIndicates meta-analysis.

^bIndicates review.

^cIndicates study of first-degree relatives.

^dIndicates twin study.

Despite this evidence, studies that directly compare cognition across multiple psychopathological diagnoses are limited. Although there are some exceptions (e.g. Goldberg et al., 2005) the bulk of the data supporting cross-disorder cognitive impairments reflects extrapolation from studies examining single conditions versus controls (Willcutt, Sonuga-Barke, Nigg, & Sergeant, 2008). The few cross-disorder meta-analyses have extended the evidence for domains of common weakness (Lipszyc & Schachar, 2010; Stefanopoulou et al., 2009; Willcutt et al., 2008). Yet, as Willcutt et al. (2008) have noted, direct within-sample analyses are needed to estimate the relative magnitude of decrements on a common metric and also to clarify whether comorbidity accounts for or exacerbates decrements across conditions.

Gaining a better understanding of the domains of cognitive weaknesses across conventional diagnostic boundaries is important for both clinical and research purposes. In the child clinical assessment field, while neuropsychiatric diagnoses are not made based on scores from psychometric tests, cognitive decrements are often taken into consideration; yet, the degree to which clinicians should expect cognitive weaknesses to contribute to differential diagnosis is not clear. Clarifying the extent of overlapping deficits across conditions will improve the evidence base regarding the implications of particular cognitive weaknesses. In the research literature, there is growing evidence from molecular genetic studies that different forms of neuropsychiatric illness share aspects of their underlying risk, and family and twin studies suggest that cognitive decrements may index liability in at least some conditions (Table 1). Confirming domains of cognition that are compromised across different forms of psychopathology will facilitate the use of cognitive constructs in studies aiming to examine cross-disorder risk mechanisms (Craddock et al., 2009). In conjunction with emerging genomic findings, such studies may help to incorporate cognition into a more biologically-informed psychiatric nosology, as advocated by the NIMH’s Research Domain Criteria framework (Cuthbert, 2015).

The current study aimed to address gaps in the literature by examining cognitive weaknesses and the impact of comorbidity in youth with different neuropsychiatric conditions known to share genetic underpinnings (Lee et al., 2013; Malhotra & Sebat, 2012; PGC, 2013). Specifically, we focused on youth ascertained from a single cohort with ADHD, mood disorders, ASD and psychotic symptoms. We predicted decrements in general cognitive ability, executive functions (EFs) and reaction time variability (RTV) across multiple conditions because weaknesses in these constructs are implicated in meta-analyses of affected individuals and because family and twin studies suggest their role in underlying disease liability.

Method

Subjects

Participants were from the Longitudinal Study of Genetic Influences on Cognition (LOGIC). LOGIC recruits youth referred for evaluation at a pediatric assessment clinic within the Psychiatry Department at Massachusetts General Hospital (MGH). Patients with neuropsychiatric symptomatology are referred to this clinic for cognitive and psychiatric evaluation to assist with differential diagnosis and/or treatment or educational planning. To enroll, youth must contribute their clinical data. They are also asked to provide a DNA sample and to supplement assessments to create a uniform cognitive and psychiatric battery across subjects. Study procedures were in compliance with the Partners Institutional Review Board and the Helsinki Declaration. Parents and youth 18 and older provide written informed consent after a description of risks and benefits; youth 7–17 provide written assent.

Subjects in the current analysis were consecutively enrolled patients meeting the following criteria: 1) full scale IQ > 70; 2) ages 8 to 21 years old (i.e. eligible to be assessed on measures reflecting cognitive domains of interest); and 3) a DSM-IV-TR diagnosis from one of the following categories: ADHD, mood disorders (major depressive disorder, bipolar disorder or mood disorder- not otherwise specified [NOS]), autism spectrum disorder (pervasive developmental disorder [PDD] NOS, Asperger’s Syndrome, or autistic disorder) or positive symptoms of psychosis (i.e. hallucinations and/or delusions). These four groups were selected because recent large-scale genomic studies indicate that these forms of psychopathology share common genetic variation that contributes to their risk (Lee et al., 2013; Malhotra & Sebat, 2012; PGC, 2013). LOGIC is an ongoing project. At the time of these analyses, there were 486 unrelated youth who met these criteria. Their mean age was 11.8 + 3.1 years and 34.8% are female.

Cognitive Assessments

Tests were administered using published instructions by licensed psychologists or by advanced trainees or psychometricians under their supervision. Based on the literature (Table 1), we examined the following constructs, using measures with robust psychometric properties that are commonly used in child clinical practice and research:

IQ/ General cognitive ability

We assessed cognitive ability using the general ability index (GAI) from the Wechsler Intelligence Scale for Children – Fourth Edition (Wechsler, 2004) for youth 8 to 16 and the Wechsler Adult Intelligence Scale – Fourth Edition (Wechsler, 2008) for youth 17 to 21. We used GAI because this score estimates ability without the use of processing speed and working memory (WM) tests which may show relative weaknesses in clinical populations (Prifitera, Weiss, & Saklofske, 1998; Tulsky, Saklofske, Wilkins, & Weiss, 2001). Additionally, WM was examined separately in our analyses.

RTV

RTV represents intra-individual consistency in reaction time. Increased variability is often considered to reflect failures of sustained attention; however, it may additionally reflect the regulation of arousal or executive allocation of attentional resources (Tamm et al., 2012). Although RTV has been studied extensively in ADHD, it may be relevant to other forms of psychiatric illness (Kaiser et al., 2008; Lipszyc & Schachar, 2010). Our measure of this construct was obtained from the Conners’ Continuous Performance Test – Second Edition (Conners, 2000) based on the Hit Reaction Time Standard Error.

EFs

By definition, EFs support goal-directed behavior and environmental adaptation (Loring, 1999). We targeted components of EFs (i.e., WM, inhibition and shifting/mental flexibility) that overlap with major domains of the overarching EF construct (Miyake & Friedman, 2012; RDoC Cognitive Group, 2011). We operationalized WM using the Working Memory Index from the Wechsler Intelligence scales (Wechsler, 2004, 2008). We operationalized inhibition using the Commission Errors score on the Conners’ Continuous Performance Test – Second Edition (CPT II; (Conners, 2000). We examined shifting/ mental flexibility using the Switching condition on the Delis-Kaplan Executive Function System (D-KEFS) Trail Making Test (Delis, Kramer, Kaplan, & Holdnack, 2004).

Diagnoses

DSM-IV-TR Axis I diagnoses were made by or under the supervision of MGH/ Harvard Medical School (HMS) faculty who were licensed clinical psychologists. Our source clinic is a training site for neuropsychiatric assessment for pre- and post-doctoral Clinical Psychology Fellows. Thus, accurate and thorough diagnostic assessment is one of the “deliverables” of the clinic. Diagnostic procedures include: 1) clinical interviews with a parent/ legal guardian and patient; 2) review of available medical records and 3) review of omnibus and targeted behavioral rating scales (including the Child Behavior Checklist/6–18 or Adult Behavior Scale, and the Child Symptom Inventory-IV, which includes specific DSM-IV-TR criteria).

Diagnoses were made if full DSM-IV-TR criteria were met based on information from these sources. The one exception was that we allowed for a diagnosis of ADHD in the presence of an ASD, as in DSM-5. This strategy was implemented in our clinic prior to the publication of DSM-5 because of the academic/ therapeutic implications of high levels of inattention and hyperactivity/impulsivity and our group’s interest in studying comorbidity in child neuropsychiatric conditions. We examined the reliability of the diagnostic process by having four independent licensed clinical psychologists blindly review and rate a subsample of 30 youth per diagnosis. These cases were randomly selected regardless of comorbidity.

Following the guidelines of Landis and Koch (1977) we interpreted kappa coefficients between .61–.80 as representing a substantial agreement and kappa coefficients between .81–1.00 as indicative of almost perfect agreement. The inter-rater reliability using Cohen’s Kappa was .93 for ADHD, ASD, and mood disorders (95% CI: [.80–1.06]) indicative of almost perfect agreement, and .80 for the presence or absence of psychosis (95% CI: [.59–1.01]), indicative of substantial agreement. Further corroboration of clinician diagnoses of ADHD from our source clinic occurred in twelve youth who were not part of the current study. These youth received the KSADS-E as part of a separate research project that recruited ADHD cases from our clinic. This structured diagnostic interview confirmed clinician diagnoses of ADHD in 100% of these cases.

Diagnostic characteristics of the sample

Four hundred and eighty-six youth met criteria for one or more of our target diagnoses. As noted, we ascertained n=383 individuals with ADHD. There were n=106 youth with an ASD. As per DSM-IV-TR, 8.5% of these youth had autistic disorder, 51.9% had Asperger’s Syndrome, and 39.6% had PDD NOS. There were 157 youth with mood disorders (13.4 % with bipolar disorder, 41.4% with major depressive disorder, 4.5% with dysthymia and 40.8% with mood disorder NOS). Finally, n=29 of the individuals exhibited positive symptoms of psychosis (i.e. hallucinations and/or delusions). Of these, 24% had a diagnosis of schizophrenia and 76% were categorized as Psychotic Disorder -NOS given the emerging, fluid nature of the symptoms. The breakdown of comorbidity within these four categories is shown in Table 2.

Table 2.

Distribution of 486 individual patients across comorbid disorders conditions

	ADHD	Mood disorders	ASD	Psychosis
ADHD	253	66	35	1
Mood disorder	66	41	12	10
ASD	35	12	34	1
Psychosis	1	10	1	3
> 1 Comorbid disorder*	28	28	24	14
Total	383	157	106	29

Note. Youth with a single (non-comorbid) diagnosis are represented in the shaded cells on the diagonal. Frequencies above the diagonal in gray are included, despite their redundancy, to allow easier calculation along each vertical column of total numbers of subjects with and without comorbid conditions in the overall cohort.

^*Because n=30 youth have > 1 comorbid disorder, youth may appear more than one time across this row (e.g. the ADHD, mood disorders and ASD columns include the same n=16 youth who met criteria for all three conditions). Thus, while column totals at the bottom reflect the total n for each diagnosis, they cannot be added together to reach 486 because they include overlapping subjects.

Other characteristics of the sample

Detailed data (dose, type, onset, offset) regarding use of psychotropic medication was obtained as part of the clinical evaluation. A total of n=154 (31.7%) children were taking stimulants, n=71 (14.6%) were on non-stimulant medication to treat ADHD (e.g. atomoxetine), n=60 (12.3%) were taking an atypical antipsychotic, n=88 (18.1%) were taking a Selective Serotonin Reuptake Inhibitor (SSRI), n=32 (6.6%) were taking a non-SSRI antidepressant, n=17 (3.5%) were taking a benzodiazepine, and n=39 (8.0%) were taking another type of psychotropic medication. Totals exceed 100% because some youth were taking more than one type of medication. Based on this information, we created a binary variable to indicate current use of one or more types of psychotropic medications versus non-use. This variable yielded a total of n= 273 (56.2%) youth using psychotropic medication.

Analytic Approach

Our goals were to determine the presence and magnitude of cognitive weaknesses across youth in the four target diagnostic groupings and to clarify the impact of comorbidity.

Phase 1: Youth with non-comorbid diagnoses

First, we aimed to focus on patients from each psychopathology group who were free of comorbidity; however, because only three youth with psychosis were free of comorbidity, we included all 29 youth with psychosis in our fourth category in order to study youth with this relatively rare but severe presentation. Within this group, 45% met criteria for ADHD, 31% met criteria for diagnosis within the ASD category and 52% meet criteria for a mood disorder (21% major depressive disorder, 14% bipolar disorder and 17% mood disorder NOS). Rates exceed 100% due to youth with multiple conditions. For all four groups in this phase of inquiry, we operationalized cognitive weaknesses in two ways: first relative to population norms (Phase 1a) and then as a discrepancy relative to their own general ability (Phase 1b).

Covariates

Table 3 shows demographics for subjects analyzed in Phase 1. Full Scale IQ is provided for descriptive purposes because components of this measure (i.e. GAI and WM) are outcome measures. Significant differences in the distribution of sex were found across groups (χ² (3)=16.41, p=.001), with more boys in the ASD group (85.3%) and more girls in the mood disorders group (58.5%). For age, we found a significant group effect (F(3,353)=23.02, p<.001), with participants with psychosis and mood disorders slightly but significantly older than youth with ASD and ADHD, respectively (ASD vs. psychosis p=.005; ASD vs. mood disorders p<.001; psychosis and mood versus ADHD both p<.001). Finally, significant differences in rates of medication use were found (χ² (3)=19.67, p<.001), with more youth taking medication in the psychosis (75.9%) and mood disorder (68.3%) groups and fewer in the ASD group (35.3%). We thus controlled for age, sex and current medication use in subsequent analyses.

Table 3.

Demographic characteristics of participants included in Phase 1 analyses.

Diagnosis	N (%a)	Sex (%boys)	Mean age (SD)	Full Scale IQ	Any meds (%)
ADHD	253 (55.2%)	65.6%	11.0 (2.5)	99.4 (11.9)	43.9%
Mood disorder	41 (9.0%)	41.5%	14.1 (3.0)	102.9 (14.8)	68.3%
ASD	34 (7.4%)	85.3%	11.6 (3.0)	99.3 (17.5)	35.3%
Any psychosis	29 (6.3%)	58.6%	13.9 (3.3)	94.0 (14.6)	75.9%

Note. These include youth with non-comorbid ADHD, mood disorders, and ASD, as well as youth with psychotic symptoms regardless of comorbidity.

^aPercentage of the total sample of 486 participants

1a. Comparisons between groups and comparison of groups vs. normative data

To compare cognition between groups and in relation to published norms, we used analysis of covariance (ANCOVA), controlling for age, sex and medication use. Comparisons between groups on each of the five cognitive domains (GAI, RTV and the three EF constructs) allowed us to estimate marginal means (adjusted for potential confounders) that could be used post hoc to compare the performance of youth in each diagnostic group with published norms. We note that the normative samples from our measures are considered generally representative of the US population (Conners, 2000; Delis et al., 2004; Wechsler, 2004, 2008) and are large for the ages (8 to 21) overlapping our participants (i.e. WISC-IV n=1600, WAIS-IV n=600, D-KEFS n=1050 and CPT-II n=1632).

1b. Comparisons vs. GAI

Second, we used a mixed modeling approach to determine the presence of weaknesses on RTV and EFs compared to participants’ own GAI for each diagnostic grouping. Mixed modeling is an extension of regular regression appropriate when data are hierarchically structured (as in our five cognitive domains nested within a subject). This statistical technique does not require the data to be balanced (i.e., not every subject will have a score on each domain) presuming missing data are random (Snijders & Bosker, 2012), and our analyses support this assumption. We note that the Conners’ CPT was included after the start of our study and thus there are greater numbers of missing data in relation to the RTV and Inhibition measures across diagnoses. In the total sample, there were 132 youth (27.2%) that had one or more missing cognitive measures. The group with data on all cognitive measures did not differ from the group with missing data on age (t(484)=1.20, p=.23), FSIQ (t(484)=1.32, p=.19, sex (χ²(1)=1.60, p=.21), a diagnosis of ADHD (χ²(1)=.07, p=.80), or any of the diagnostic groups we examined (χ²(5)=6.79, p=.24). Here, cognitive scores were converted into z-scores in order to compare GAI to RTV and EF. Converted scores were entered as a within-subjects variable, referred to as “cognition,” consisting of five cognitive domains. When a significant main effect was found, we ran post hoc comparisons of RTV and the three EF domains to GAI.

Phase 2: Impact of comorbidity

Next, we examined the influence of comorbidity on cognitive profiles. To reduce the possibility of Type II error, we only examined comorbid groups that exceeded n=30 subjects in this set of analyses. As shown in Table 2, only groups with ASD + ADHD (n=35) and mood disorders + ADHD (n=66) met our threshold. We therefore ran two mixed modeling analyses, comparing ASD with ASD + ADHD groups and then mood disorders versus mood disorders + ADHD groups, controlling for confounders.

In each model, we first estimated a “full” model including a diagnostic group by cognition interaction term. A significant interaction would indicate that the shape of the cognitive profile differed between the comorbid and non-comorbid groups. Given a non-significant interaction effect, we dropped the interaction term and tested the resultant model (via Wald χ² test) to determine a main effect for diagnostic group and/or a main effect for cognition. A main effect for diagnostic group represents a difference in the magnitude of the decrement between the comorbid and non-comorbid groups. A main effect of cognition reflects within-subject differences between particular cognitive domains and GAI. Like Phase 1, post-hoc comparisons determined which cognitive domains differed from GAI, but here include subjects regardless of comorbidity.

Analyses were conducted with STATA 14 (StataCorp, 2015). We used a significance level of .05 except where we applied a Bonferroni correction (with a significance threshold of .0125) in relation to 1) the post hoc comparisons with the norm scores (.05/4 target groups) and 2) the post hoc mixed models in phase 1b and phase 2 (.05/4 cognitive domains vs. GAI).

Results

Phase 1: Cognitive Decrements in Non-Comorbid Diagnostic Groups and Psychosis

Phase 1a: Differences among groups and relative to age-based norms

We examined the extent to which youth from the psychopathology groups differed from one another on the five cognitive domains via an F-test (Table 4). No significant differences were found between groups after controlling for age, sex and medication use. We then compared means generated from these comparisons (adjusted for potential confounders) to population norms (also Table 4). Figure 1 shows effect sizes (Cohen’s d) from these post hoc comparisons of the estimated marginal means. Consistent with the lack of group differences, effect sizes for the diagnostic groups were close in range, with some of the differences from the normative mean reaching statistical significance. For GAI, slightly but significantly higher than normative performance was noted in youth with ADHD and with mood disorders (p<.001 and p=.01, respectively). For EFs, after correction for multiple testing, the ADHD group showed statistically worse performance in all three domains and the psychosis group showed lower WM (all p-values <.001). For RTV, significantly greater variability (worse performance) occurred in youth with ADHD (p<.001).

Table 4.

Results of Phase 1 comparisons, including 1) ANCOVA between group comparisons (correcting for age, sex, and current medication use) and 2) post-hoc comparisons between estimated marginal means (corrected for covariates) and age-based normative data

	ANCOVA – between group comparisons			Adjusted marginal means from ANCOVA - Post-hoc comparisons with normative data
Cognitive domains	F	p-value	Eta2	ADHD	Mood disorder	ASD	Psychosis
General Ability Index	1.61	.19	.013	102.9 (12.4)*	107.8 (15.5)*	103.9 (17.5)	101.7 (13.8)
				(101.2;104.6)	(103.4;112.2)	(99.3;108.4)	(96.7;106.8)
				253	41	34	29
Response Variability**	.38	.77	.004	53.8 (10.8)*	53.2 (11.6)	51.9 (12.6)	55.1 (11.2)
				(52.3;55.3)	(49.3;57.2)	(47.8;56.0)	(49.9;60.1)
				220	35	29	19
Working Memory	1.84	.14	.015	94.8 (12.0)*	97.4 (11.4)	94.9 (16.3)	90.1 (16.2)*
				(93.2;96.4)	(93.2;101.5)	(90.5;99.3)	(85.3;94.9)
				253	41	34	29
Inhibition**	1.07	.36	.010	52.1 (10.5)*	53.6 (10.4)	54.4 (9.1)	55.7 (10.3)
				(50.8;53.5)	(50.0;57.2)	(50.7;58.1)	(51.0;60.4)
				220	35	29	19
Shifting	.77	.51	.008	8.2 (3.5)*	9.2 (3.5)	8.6 (4.2)	8.5 (3.1)
				(7.7;8.7)	(8.0;10.4)	(7.2;10.1)	(7.1;9.8)
				219	38	24	28

Note. Values in table represent adjusted marginal means (SD), (95% CI), and sample size from ANCOVA;

^*p≤0.0125 (significant after Bonferroni correction);

Low scores indicate worse performance, with the exception of measures designated with **, in which high scores reflect greater difficulty.

Fig. 1.

Cognitive performance relative to normative data in referred youth with different neuropsychiatric conditions.

Phase 1b: Comparison to GAI

We then performed mixed modeling analyses for each diagnostic group to examine weaknesses in RTV and domains of EFs within subjects, relative to their own ability and controlling for potential confounders. Per convention, numbers of subjects for these mixed models (Phase 1b) as well as those in Phase 2 (discussed below) are shown in Table 5. Results are shown in Figure 2. Each of these four analyses yielded a significant main effect for the within-subjects factor cognition, indicating differences between cognitive domains (for ADHD [Wald χ² (4)=122.52, p<.001], mood disorders [Wald χ² (4)=31.53, p<.001], ASD [Wald χ² (4)=14.84, p=.005] and psychosis [Wald χ² (4)=13.62, p=.01]).

Table 5.

Distribution of the number of subjects per diagnosis on each cognitive domain for mixed modeling analyses.

	GAI	Response Variability	Working Memory	Inhibition	Shifting
ADHDa	253	220	253	220	219
Mood disordera	41	35	41	35	38
ASDa	34	29	34	29	24
Psychosisb	29	19	29	19	28
ASD + ADHD	35	25	35	25	26
Mood disorder + ADHD	66	53	66	53	57

^aNon-comorbid.

^bPsychosis with any comorbidity.

Fig. 2.

Cognitive impairments relative to participants’ own general ability.

Note. *p ≤0.0125 (significant after Bonferroni correction)

In post hoc comparisons, differences with GAI were significant across RTV and all EF measures for ADHD and mood groups (all p-values <.001). For youth with ASD and psychosis, significant effects were found for WM, inhibition and shifting (for ASD: WM p=.005, inhibition p=.004 and shifting p=.001; for psychosis: WM p =.002, inhibition p =.002 and shifting p =.008). Decrements on RTV versus GAI did not achieve statistical significance after Bonferroni correction (for ASD, p=.04; for psychosis, p=.08). Thus, post-test comparisons indicated that weaknesses in EF versus GAI were significant across the four groups but that RTV was only significantly impaired relative to GAI in the ADHD and mood groups.

Phase 2: Impact of Comorbidity on Cognition in ASD and Mood Disorders

Table 6 shows the results of mixed modeling analyses to determine the effect of comorbidity with ADHD on cognition in youth with ASD and mood disorders. Here, diagnostic group was a between-subjects factor and cognition was a within-subjects factor.

Table 6.

Mixed model analyses of decrements in cognitive functioning in youth with ASD and mood disorders with and without comorbid ADHD (controlling for age, sex, and medication use)

Analysis 1: ASD
Effectsa	β (SD)	p-value	Wald’s test
Main effect cognition			χ2 (4)=28.58, p<.001
GAI (reference category)	-	-
RT variability	−.35 (.17)	.04*
Working Memory	−.74 (.16)	<.001
Inhibition	−.57 (.17)	.001
Shifting	−.70 (.17)	<.001
Main effect diagnosis			χ2 (1)=0.21, p=.65
ASD (reference category)	-	-
ASD + ADHD	.09 (.20)	.65
Analysis 2: Mood disorder
Effectsa	β (SD)	p-value	Wald’s test
Main effect -cognition			χ2 (4)=48.59, p<.001
GAI (reference category)	-	-
RT variability	−.69 (.13)	<.001
Working Memory	−.57 (.12)	<.001
Inhibition	−.72 (.13)	<.001
Shifting	−.70 (.12)	<.001
Main effect -diagnosis			χ2 (1)=5.41, p=.02
Mood disorder (reference category)	-	-
Mood disorder + ADHD	−.33 (.14)	.02

^*Asterisk=non-significant; critical value after Bonferroni correction for multiple cognitive tests is 0.0125.

^aInteraction between diagnosis and cognition is not shown because of a lack of statistical significance.

In youth with ASD, there was no significant group × cognition interaction (Wald χ² (4)=3.89, p=.42), indicating that the shape of the cognitive profile did not differ between the comorbid and non-comorbid groups. When the interaction term was dropped, the final model yielded a significant main effect for cognition (Wald χ² (4)=28.58, p<.001), but not for group (Wald χ² (1)=.20, p=.65). Thus, there were no significant differences in the magnitude of impairment in children with ASD with and without ADHD. Post hoc comparisons suggest that for youth with ASD regardless of comorbidity, performance in the three EF domains, but not RTV (p=.04), was significantly worse than GAI (all 3 EF p-values ≤.001).

Regarding mood disorders, the shape of the cognitive profiles for those with and without ADHD did not differ, as indicated by the non-significant group × cognition interaction (Wald χ² (4)=6.07, p=.19). As with ASD, the model without the interaction yielded a significant main effect for cognition (Wald χ² (4)=48.59, p<.001), with post-hoc comparisons indicating significantly lower performance on all four cognitive domains RTV, WM, Inhibition, and Shifting compared to GAI (all p-values <.001). Additionally, in contrast to the comorbidity analysis for ASD, the group effect in the analyses of mood disorders comorbidity was significant (Wald χ² (1)=5.41, p=.02). Thus, while the shape and relative weaknesses within the cognitive profile were similar for the two groups, cognitive difficulties were of greater magnitude in the group with mood disorders + ADHD compared to the group with mood disorders alone.

Discussion

We extended evidence for cognitive weaknesses that are relevant across disorders by examining youth with different forms of psychopathology from a single cohort and addressing the impact of comorbidity. We focused on ADHD, ASD, mood disorders and psychosis because these conditions share genetic liability and because twin and family studies suggest that cognitive impairments potentially index their underlying risk. Results indicated that aspects of EFs show decrements in a range of conditions and are not simply a result of their comorbidity. Such findings have implications for clinical practice and for studies seeking to understand mechanisms of shared liability.

We first examined cognition in youth with non-comorbid diagnoses where possible in order to associate cognitive weaknesses with specific conditions. Given few non-comorbid cases, youth with psychosis were included regardless of comorbidity to allow this severe form of psychopathology to be included in our analyses. Significant differences in cognition between the four diagnostic groupings were not observed. We then aimed to identify decrements present in multiple disorders. Here, we operationalized impairment in relation to normative data as well as one’s own general ability.

When impairment was defined in relation to participants’ ability, strong evidence for cross-disorder EF weaknesses emerged. Using this approach, significant decrements were observed for all diagnostic groups on measures of WM, inhibition and shifting after correcting for potential confounders and multiple testing. Cross-disorder weaknesses in EF were less robust when normative data was the referent. In this case, after Bonferroni correction, cognitive performance was only significantly worse in youth with ADHD for the three EF domains and in youth with psychosis for WM. Yet, effect sizes based on adjusted marginal means for youth with ASD, mood disorders and psychosis were small to moderate and generally comparable to effect sizes in youth with ADHD (Cohen’s d = −.2 to −.5). Thus, findings were generally consistent with the cross-disorder decrements found in relation to GAI in the mixed models, which benefitted from greater statistical power. For RTV, significant decrements were identified in youth with ADHD and mood disorders compared to their own GAI, and in the ADHD group when normative data was the referent. However, we cannot conclude that weaknesses in this domain were specific to ADHD and mood disorders, given that effect sizes (albeit non-significant) in youth with ASD and with psychosis also fell in the small to moderate range.

Finally, we addressed the impact of comorbidity with ADHD on cognition for Mood Disorders and ASD, using GAI as the referent. In both cases, there was no significant group × cognition interaction and youth exhibited significant weaknesses on all three EF domains, regardless of comorbidity. Thus, in these cases, EF impairment per se was not specific to the groups showing comorbidity with ADHD. RTV was also significantly worse than GAI in both the mood disorders and mood disorders plus ADHD groups, but not in either group with ASD. In youth with mood disorders, however, those with comorbid ADHD showed a greater magnitude of decrement than the group with mood disorders alone, whereas the magnitude of impairment did not differ in youth with ASD with and without ADHD.

These results have implications for research and clinical practice. In the literature, evidence for a shared genetic liability across different neuropsychiatric conditions has sparked interest in phenotypes that may index shared risk mechanisms. Our data support further investigation of the role of EF decrements (and the disturbances in frontally mediated neural networks that they reflect) in the liability shared across different types of psychopathology (Craddock et al., 2009). Given that EFs impact one’s ability to problem solve and adapt to challenges, it is plausible that such functions impact mental health generally. Whether or not EF decrements lie directly in the pathway between genes and a general risk for psychopathology is therefore an important question to examine, since support for this possibility would yield new implications for cognition as a therapeutic target.

These findings also have relevance to evidence-based assessment in child clinical settings. To date, no prior studies have documented cognitive profiles in these four types of psychopathology within a single sample. Our results suggest that youth whose diagnoses fall within these groupings are unlikely to be distinguished from one another based on the pattern or magnitude of weaknesses in the cognitive domains we investigated. For example, although ADHD is strongly associated with inhibition decrements in the theoretical literature, our data highlight the lack of specificity of weaknesses in inhibition to ADHD, particularly when considered in relation to patients’ own GAI.

Although RTV performance was more uneven across groups, results did not support diagnostic specificity for impaired RTV, despite its strong association with ADHD in the literature (Kuntsi et al., 2006). Rather, our findings echo Pennington’s conceptualization of multiple overlapping deficits across different conditions (2006). This pattern is consistent with multifactorial inheritance and suggests that psychometric test scores in isolation do not speak to differential diagnosis. Although neuropsychologists are generally aware of this conclusion, those referring youth for evaluations may benefit from education regarding the limited diagnostic specificity of test scores per se. Importantly, these data do not negate the value of testing. A large literature associates impaired test scores to real world academic, occupational and emotional functioning (e.g. Biederman et al., 2006; Dajani, Llabre, Nebel, Mostofsky, & Uddin, 2016; Green, 2006) thus highlighting the value of identifying strengths and weaknesses for academic, rehabilitative or treatment planning (Dajani et al., 2016; Seidman, Bruder, & Giuliano, 2008).

This study supplemented clinical assessments already being undertaken. Despite the advantages of this approach for amassing a large, well-characterized multi-diagnostic sample in a cost-efficient manner, it is not without limitations. First, we used clinician diagnoses to create diagnostic groupings. Although confirmation by structured diagnostic interviews would have enhanced our approach, these were not available for diagnoses beyond ADHD. Nonetheless, given the role of our source clinic in training at an academic teaching hospital, considerable attention is given to whether patients fulfill specific diagnostic criteria. Moreover, our blinded ratings showed high levels of inter-clinician agreement.

Second, our conclusions about cognitive weaknesses in youth with psychosis are limited by the comorbidity and sample size of this group. We cannot rule out the possibility that the impairments in this particular group are due to co-occurring conditions. However, given that 1) comorbidity within youth-onset psychosis appears to be the rule rather than the exception in our cohort as well as in the literature (Buckley et al., 2009) and also that 2) there is a significant gap in literature involving cross-disorder comparisons, the exclusion of this group would have omitted potentially informative data that domains of cognitive weaknesses overlap not only between more common conditions of youth but also in relation to this less common but severe manifestation of psychopathology. Additionally, impairments versus normative data for domains other than WM did not reach statistical significance despite generally comparable effect sizes to other groups, potentially due to the smaller sample size of this group. Nonetheless, effect sizes are informative, and comparisons versus GAI, which capitalize on the power of within-individual comparisons, were more robust.

Third, we acknowledge that the heterogeneity within our mood disorders group. Although the majority of subjects received a Mood Disorder NOS diagnosis, this group included youth with Major Depressive Disorder and Bipolar Disorder. While combining across youth with these diagnoses is reasonable given that mood symptoms in these children may evolve across diagnostic boundaries over time, the field would benefit from comparison of youth with specific forms of mood disorders to other psychopathology.

Fourth, the conclusion that these findings may have implications for clinical practice presumes reasonable generalizability of our findings. Impairments in our clinical cohort are generally comparable with the literature (Table 1) and thus may help to bridge the gap between research and clinical samples. Although performance on GAI was slightly but significantly better than normative samples in the ADHD and mood disorder groups, high average to above average ability estimates are often observed in child clinical research populations (Seidman et al., 2006), potentially in part due to exclusion of subjects with FSIQ < 70, as in our study. We note that the mean Full Scale IQ of four groups ranged from 94.0 to 102.9 and that GAI is expected to be higher than FSIQ in neuropsychiatric samples (Iverson, Lange, Viljoen, & Brink, 2006). Thus it is reasonable to expect generalizability of our findings. Moreover, discrepancy with overall ability is considered functionally important (Sattler, 2008) and asymmetric cognitive performance within individuals may help to identify difficulties obscured by group means (Jacobson, Delis, Bondi, & Salmon, 2002). Thus, our use of mixed models to capitalize on patients’ own ability as a referent for identifying areas of weakness (Denckla, 1994) not only augments statistical power but helps to extend the generalizability of our data.

Fifth, a medication-naϊve sample would have been preferable; however, cognitive impairment is notoriously unresponsive to psychotropic medications (Frazier et al., 2012), and we conservatively controlled for medication use in all analyses. Sixth, as in any youth sample, subjects have not passed through the age of risk for mood disorders and psychosis, which could reduce differences between youth with ADHD and ASD and those already manifesting these conditions if cognitive impairments are trait- rather than state-related phenomena. Studies of adults across these different diagnostic groupings and/or longitudinal follow up of youth in this sample would therefore complement the current findings.

Despite these issues, this study advances the literature by examining multiple diagnostic groups and cognitive domains in a single youth sample with attention to comorbidity. By investigating conditions known to share genetic liability, we aimed to highlight cognitive functions suitable for consideration in studies of cross-disorder risk mechanisms. Our findings suggest that domains of EFs including working memory, inhibition and shifting/mental flexibility show decrements in ADHD, mood disorders, ASD and psychosis that are not simply a function of comorbidity. Further, although comorbidity with ADHD increases the magnitude of deficits in mood disorders, the overall profile of findings for GAI, RTV and EFs does not differ in youth with ASD and mood disorders in the presence of comorbidity with ADHD. These data support a complex relationship between cognition and psychopathology, as described in Pennington’s (2006) multiple deficit models and set the stage for further investigation of the role of EF weaknesses, and the disrupted neural networks they reflect, in the heritable risk mechanisms common to different forms of neuropsychiatric illness. They further highlight that test scores themselves do not speak to differential psychiatric diagnosis, despite having other utility in the context of neuropsychiatric evaluations.