Specificity of Executive Functioning and Processing Speed Problems in Common Psychopathology

Abstract

Objective

Interest continues in neuropsychological measures as cross-disorder intermediate phenotypes in understanding psychopathology. A central question concerns their specificity versus generalizability to particular forms of psychopathology, particularly for executive functioning (EF) and response speed. Three conceptual models examining these relationships were tested to clarify this picture at different levels in the diagnostic hierarchy.

Method

Participants (total n = 641, age 18–60) yielded complete structured diagnostic interviews and a neuropsychological test battery comprising measures of executive function, processing speed, and IQ. Repeated measures MANOVA, linear regression, and structural equation modeling (SEM) were used to test (1) a Specificity model, which proposes that individual disorders are associated with component EF processes and Speed; (2) a Severity model, which proposes that the total number of comorbid disorders explain poor EF and/or slow Speed; and (3) a Dimensional model, which proposes that internalizing versus externalizing disorders are differentially related to EF or Speed.

Results

EF effects were best explained by a Specificity model, with distinct aspects of EF related to ADHD versus antisocial substance use disorders. Speed, on the other hand, emerged as a general indicator of externalizing psychopathology in the Dimensional model, as well as overall severity of psychopathology in the Severity Model.

Conclusions

Granular approaches are likely to be most productive for linking EF to psychopathology, whereas response speed has underutilized potential as an endophenotype for psychopathology liability. Results are discussed in terms of an integrated conceptualization of neuropsychological processes and putative neural systems involved in general and specific aspects of psychopathology.

Neuropsychological abilities remain of keen interest in relation to psychopathology either as potential markers of genetic or other disease liability (endophenotypes) or as components of pathophysiology (e.g., “intermediate phenotypes” or “transdiagnostic phenotypes” (Nolen-Hoeksema & Watkins, 2011)). A growing movement driven in part by recent leadership at the NIMH has called for improving the mental disorders nosology by utilization of neurobiologically relevant dimensional measures of cognitive and emotional functioning called the research domain criteria (RDoC)(Insel et al., 2010). These are conceptually related to intermediate phenotypes except that they involve cross cutting process-related, not just descriptive, correlates. Neuropsychological abilities fit well within this framework.

Under the RDoc and other rubrics, it is assumed that there is no 1:1 association of process breakdown to the existing disorders in the nosology. Indeed, it is widely recognized conceptually that different psychiatric disorders likely entail overlapping lower level functional abilities (e.g., “emotion regulation”, “executive functioning”, “information processing” and others). Yet, many if not most studies of psychopathology focus these dimensional and alternative phenotype measures on individual disorders. A key challenge is to determine what is in fact the overall correlational structure of intermediate phenotypes with different existing disorders. We use the term “intermediate phenotype” as a shorthand herein while recognizing that it remains unknown to what extent EF or Speed (explained below) are actually endophenotypes, pathophysiological indicators, or liability indicators for different disorders and to what extent they serve as shared markers across or within disorders or groups of disorders. The present study aims to shed light on that discussion.

Thus, the present paper takes a closer look at just two of the many important candidate intermediate phenotypes relevant to this larger effort to improve nosology, in relation to multiple disorders: executive functions (EF) and processing or response speed (Speed)ⁱ. We define both terms below.

Executive Functioning

EF refers to top-down, goal directed cognitive processing; the construct emerged from studies of frontal lobe functioning and clinical neuropsychology of adults (Luria, 1966; Pribram, 1973; T. Shallice, 1982; T. Shallice & Burgess, 1991; D.T. Stuss & Benson, 1986). It has accrued a welter of overlapping definitions, as recently reviewed by Barkley (Barkley, 2012). Contemporary definitions emphasize a set of at least partially independent top-down functions that support goal-directed action (Marie T. Banich, 2009; Blair, Raver, & Finegood, 2016; Diamond, 2013; Friedman & Miyake, 2016; Miyake et al., 2000) as well as complex cognition (Barkley, 1997; Diamond, 2013). EF are invoked when automatized routines will not work or are not possible (e.g., novel situations).

EF is important to psychopathology research across numerous disorders due to their relevance to a wide range of functional capacities including problem solving, impulse control, and emotion regulation. They have therefore been proposed as intermediate phenotypes, endophenotypes, or measures of mechanistic dysfunction for attention-deficit hyperactivity disorder or ADHD (Murphy, Barkley, & Bush, 2001; Nigg et al., 2005), antisocial personality disorder (Morgan & Lilienfeld, 2000), alcoholism (Adams et al., 1993; Brokate et al., 2003), substance use disorders (Jovanovski, Erb, & Zakzanis, 2005; Pope & Yurgelun-Todd, 1996), depression (Austin et al., 1999; Fossati, Amar, Raoux, Ergis, & Allilaire, 1999; Kaiser et al., 2003; Moritz et al., 2002), and anxiety (Eysenck & Calvo, 1992), as well as schizophrenia, autism, and learning disorders.

Potential EF capacities and component abilities vary across major theoretical models (Marie T Banich, 2004; Barkley, 1997; Lezak, Howieson, & Loring, 2004; Tim Shallice, 2002; Donald T Stuss & Alexander, 2000); for a review see (Jurado & Rosselli, 2007). For the present study, we selected a subset of key abilities and measures that are cited in numerous models: (1) set-shifting (and maintenance), (2) interference control, (3) response inhibition, and (4) working memory. While not always included in older EF models, there is considerable contemporary interest in response consistency/variability (Karalunas, Geurts, Konrad, Bender, & Nigg, 2014), which may be a correlate of either EF or Speed. We included it and determined empirically its association with latent variables for our main constructs.

EF thus comprises multiple processes, which complicates the discussion of EF as unitary (Burgess, 1997; Duncan, Johnson, Swales, & Freer, 1997; Hanes, Andrewes, Smith, & Pantelis, 1996; Miyake et al., 2000; Salthouse, Atkinson, & Berish, 2003). On the other hand, there also appears to be an underlying shared mechanism or ability that contributes to performance across different EF tasks lending support to the unity of the EF construct (Duncan et al., 1997; Hanes et al., 1996; Miyake et al., 2000; Salthouse et al., 2003) (McGrath et al., 2015). The unity and diversity of EF processes provides the opportunity to examine EF from a holistic perspective, with a focus on the combined measurement of EF, as well as from a component perspective.

Understanding EF has taken on increasing urgency with the growing market of various forms of cognitive retraining to improve components of executive functioning, such as working memory, as a proposed way to reduce psychopathology, although the efficacy of these interventions has yet to be proven (Cortese et al., 2015; Redick et al., 2013). Better understanding of how and when such interventions should be considered relies on clarifying how and in which respects executive functioning and slow processing speed relate to psychopathology at different levels of analysis and granularity.

Output Speed

We use the term ‘speed’ to refer to a composite of processes that generate speed of response on a task or output speed. This output speed in fact entails component processes such as speed of perception, efficiency of information processing and accumulation, speed of and bias toward response preparation, as well as response execution. The final outcome of these processes is speed of responding, which we call for shorthand “speed.” Separating Speed from efficiency of EF is an important aspect of evaluating shared phenotypes for cognition (Salthouse, 1996) and thus for psychopathology.

Although it is less heavily theorized as a specific marker for psychopathology, response Speed has garnered considerable interest in part as an index of genetic liability and white matter integrity (Kochunov et al., 2015; Schneider et al., 2015). The literature has related processing speed to intelligence (IQ) and to health and psychopathology (Betjemann et al., 2010; Knowles, David, & Reichenberg, 2010; Shanahan et al., 2006). Slow Speed is a ubiquitous finding in mental health research and is a widely used outcome measure in intervention studies for various health conditions (Kelly et al., 2014; Lampit, Hallock, & Valenzuela, 2014). However, studies of EF generally fail to consider processing speed. This is notable because hierarchal models of human cognition (e.g.(Botvinick, 2008) suggest that lower order processes, such a processing speed, inform higher order processes, such as EF. It has been hypothesized that Speed may underlie EF (Salthouse, 1996, 2005); it was an indicator of EF in one latent variable study (McGrath et al., 2015). Response speed has emerged therefore as a less-studied, but potentially reliable and easily measured marker.

While distinctions can be made between perceptual speed, reaction time, processing speed, and output speed, in the present report we focus on common measures of motor response speed that represent the sum of information processing and output speed on simple but commonly used neuropsychological processing tasks. It is recognized that reaction time and speed measures can be decomposed into multiple psychological processes (Karalunas et al. 2015), but doing so was beyond the scope of the present study. Here, we consider only a simple, clinically-used set of measures.

Selection of Psychopathologies

Although we sought to broaden the range of disorders included over those in prior studies, some constraints were necessary to maintain a focus on common, co-occurring disorders. The following disorders were selected: Attention deficit/hyperactivity disorder (ADHD), antisocial personality disorder (ASPD), alcoholism, substance use disorder, depression (mood disorders, cumulative), and anxiety disorders (cumulative). These disorders frequently co-occur (Kessler, Berglund, et al., 2005; Kessler, Chiu, Demler, Merikangas, & Walters, 2005) and often can be grouped into shared liability dimensions (below). We limited the investigation to these disorders, ignoring psychotic disorders, largely on feasibility grounds. Similarly, we ignore here personality disorders to constrain the scope of the report, although these are clearly expected to be present in the populations studied. Thus, we focus here on a selected sub-set of the psychopathology domain that is fairly common and frequently overlapping.

Speed, EF, and structure of psychopathology

Weak EF and slow Speed are identified as potentially important to multiple disorders that are part of the common nosology. Clearly EF dysfunction is not a necessary and sufficient cause for various mental disorders (Insel et al., 2010; Pennington & Ozonoff, 1996). It is implausible that a single cognitive deficit is sufficient to cause all cases of what are now recognized to be multifactorial psychiatric disorders (Garber & Hollon, 1991; Insel et al., 2010). What then is the appropriate way to conceptualize their cross-disorder role and degree of specificity? Although key reviews have attempted to compare EF across disorders (Pennington & Ozonoff, 1996; Sergeant, Geurts, & Oosterlaan, 2002), they agree that comparing effects across different studies using different methodologies makes interpretation difficult. Studies have compared EF across a small number of disorders (Airaksinen, Larsson, & Forsell, 2005; Boldrini et al., 2005; Fossati et al., 1999; Moritz et al., 2002; Selby & Azrin, 1998; Uekermann, Daum, Schlebusch, Wiebel, & Trenckmann, 2003; Weyandt, Rice, Linterman, Mitzlaff, & Emert, 1998). However, few studies have empirically examined EF with the intention of understanding the specificity versus generalizability of EF or Speed deficits to many different disorders within the same study.

The present study attempts to clarify, using a subset of common existing disorders, the structure of their joint relationship with EF and Speed. In doing so, it is hoped, can provide clues on how to integrate intermediate phenotypes with the nosology in future, as well as clarify the constraints and probable models most promising in relation to both EF and Speed particularly.

One possibility, in fact, is that Speed accounts for some of the EF effects, because EF measures are often confounded with Speed. We consider that by modeling EF and Speed simultaneously in some models. Three other hypotheses can be proposed.

The first model, here termed the “Specificity model,” proposes that different disorders are associated with different types of EF deficits. That is, this model assumes EF has to be considered not as a single construct but more granularly, as components. Substantial literature has considered the component structure of EF (Friedman & Miyake, 2016; Miyake et al., 2000). Application of this concept may reveal, for example, that ADHD may be related to poor working memory while antisocial behavior may be related to poor response inhibition. Another possibility is that specificity will occur at a less granular level, in relation to higher order liability dimensions that are described below (Snyder, Miyake, & Hankin, 2015; Young et al., 2009).

The second, “Severity model,” proposes that EF or Speed impairment are related nonspecifically to overall severity of psychopathology rather than a specific form of psychopathology. Severity may vary within disorder (e.g., severity of depression), in terms of impairment (overall functional level), and in other ways. Here, this was examined in an initial way using two simple proxies for severity. The first one that has been used in the literature is simply the number of co-occurring disorders (Angold, Costello, & Erkanli, 1999; Kessler, Chiu, et al., 2005). While imperfect, this index has the advantage of helping to clarify whether the appearance of a cognitive function association with a disorder may actually simply be due to that disorder often being nested in a cluster of many disorders (comorbidity). The second approach used is the clinician rating of Global Assessment of Functioning (GAF) (Jones, Thornicroft, Coffey, & Dunn, 1995). Some studies have supported its validity (Kohler, Horsdal, Baandrup, Mors, & Gasse, 2016; Samara et al., 2014; Startup, Jackson, & Bendix, 2002; Suzuki et al., 2015), although others have been less sanguine (Aas, 2011). Here we were able to obtain adequate inter-clinician reliability (ICC> .9) on the GAF ratings in the ADHD cohort (although not tested in the alcohol cohort) and therefore used it as second measure of severity.

The third, “Dimension” model, proposes that EF or Speed deficits are related to one or more shared, underlying psychopathology liability dimensions rather than specific disorders. For example, Krueger (1999) found that two broad, superordinate factors accounted for the pattern of correlations among liabilities to common mental disorders, which he labeled as an internalizing factor and an externalizing factor. Internalizing was related to depression and anxiety disorders, and externalizing to anti-social and substance use disorders. These labels are not meant to connote the traditional mechanistic meaning (tendency to direct negative affect inward versus outward) but simply as shorthand labels for mood versus behavioral types of problems. We therefore retain that formulation here. However, whereas the subsequent literature often supports these two latent factors, it questions whether ADHD belongs on its own factor (Lahey et al., 2008). Further, additional factors may also exist (Wittchen et al., 2009). Finally, recent work has suggested there may also be a general psychopathology “g” or “p” factor (Caspi et al., 2014; Lahey et al., 2012; Lahey et al., 2015) using a hierarchical or bifactor model. Here, we consider the internalizing and externalizing factors (recognizing ADHD’s placement may be uncertain) and the P-factor based on the empirical structure within our data.

Methods

Two community-based adult samples from largely overlapping local communities were combined. These were initially recruited for different goals, which we outline below. However, they were capitalized on here because (a) they had extensive clinical evaluations of comorbid disorders; (b) were community-recruited, avoiding the bias of clinic-referred samples, and (c) contained a broad representation of the disorders of interest. We discuss their limitations later. Here we describe and then consider the validity of combining them for the present report.

Sample 1: ADHD Adults and Controls, “Sample 1”

Initial Purpose

This cohort was initially recruited in the period from 1998 to 2003 for a study aimed at evaluating the relationship of executive functioning to ADHD in young adults (aged 18–40). To enable an unambiguous definition of ADHD, psychotic disorders were excluded.

Participants

Participants were recruited from the community via public advertisements that asked both for individuals with possible or suspected history of ADHD and for individuals for a study of attention. Volunteers were evaluated in a standard multistage screening and diagnostic evaluation procedure to select those who met inclusion criteria for ADHD and non-ADHD adults. Prospective participants contacted the project office at which point key inclusion criteria were checked (age 18–40, no sensory-motor handicap, no neurological illness, no head injury with loss of consciousness, and native English-speaking to enable valid completion of normed clinical tests at that time). They then proceeded through a multi-stage evaluation process to confirm eligibility and final diagnostic status. A federal certification of confidentiality was obtained to increase participant comfort in veridical reporting of substance use, antisocial and other behavioral history.

Psychopathology evaluation

Eligible participants were assessed during a face-to-face interview with a trained masters-degree level clinician using the Structured Clinical Interview for DSM-IV Axis I Disorders (SCID-I; (First, Spitzer, Gibbon, & Williams, 1997), including MDD, dysthymic disorder, bipolar disorder, substance abuse and dependence, ASPD, psychotic symptoms, GAD, PTSD, panic disorder, agoraphobia, social phobia, and eating disorders. OCD and Specific phobia were assessed, but occurred very rarely (<10 cases). Disorders that are typically seen in childhood, such as ADHD, CD, and ODD, were assessed with the Kiddie Schedule for Affective Disorders and Schizophrenia (K-SADS; (Puig-Antich & Ryan, 1996)) modified for adults, following prior studies in the literature that had evaluated the validity of this procedure (Biederman et al., 1992; Biederman, Faraone, Keenan, Knee, & Tsuang, 1990). Interviews were administered by masters-degree level clinicians (social work, psychology, or counseling) following extensive training. For ADHD, which requires childhood onset, a parent or adult who knew the participant as a child completed a structured interview about childhood symptoms. A diagnostic team that included a licensed clinical psychologist, and a board certified psychiatrist evaluated all available information to arrive at a best estimate clinical diagnostic judgment following DSM-IV criteria (APA, 1994). Inter-clinician agreement on all disorders was satisfactory (all kappa>.75) (Nigg et al., 2005).

IQ Assessment

IQ was estimated with a reliable and valid five subtest short form of the WAIS-III (current at the time of data collection) (Sattler & Ryan, 1999; Wechsler, 1997): Picture Completion, Vocabulary, Similarities, Arithmetic, and Matrix Reasoning.

Exclusion criteria

Participants were excluded from neuropsychological testing if they were in a current episode of major depression or mania/hypomania, were unable to remain sober during testing, had an IQ < 75, had a history of psychosis or autism, or were currently prescribed anti-psychotic, anti-depressant, or anti-convulsant medication. They were also excluded if there was lack of convergence among reporters (e.g., childhood history informants failed to verify any problems), if estimated IQ<80, or if ADHD symptoms were borderline (at that time, 5 symptoms of inattention or hyperactivity/impulsivity).

Final sample

A total of 363 adults passed through the initial screen and completed the screening rating scale and the diagnostic screen visit and other eligibility. About half were screened out by the exclusion criteria above, yielding a final sample of n=193 for the current report. The retained group did not differ from the larger group on age (p=23.8 vs 23.5, p=.55), gender (51.2% vs 50.8%, p=.94), % Caucasian (85% vs 86%).

Medication Washout

Individuals who were prescribed regular psychostimulant medications (Adderall, Ritalin, Concerta, and Focalin in this sample) were tested after a minimum of 24 hours (for short acting preparations) to 48 hour washout (for long acting preparations); actual mean washout time was 63.8 hours.

Sample 2: Substance Abuse Study (“Sample 2”)

Initial Purpose

This cohort was selected to be at high risk for developing substance use disorders, for the purpose of study the etiology of substance use disorders from the preschool years until the time of greatest substance use involvement (Zucker et al, 2000).

Participants

The second sample consisted of parents from families who participated in the Michigan Longitudinal Study (Substance Abuse Study), an ongoing longitudinal study of the development of alcohol and other substance use disorders (Zucker et al., 2000). Families were recruited from the community based upon the alcoholism status of the father during the period of 1985 to 1993, when their target child was in preschool and the parents (focus of the present study) were aged 20 to 53. The recruitment community was the same as the first study above. Data were collected at the initial recruitment (Wave 1) and at 3-year intervals thereafter. At Wave 5 (12 years following recruitment), neuropsychological testing was administered to all parents participating in the study, including 30 stepparents who joined the study when they married a participant, and is the focus of the current report when the parents were aged 26 to 66, during the years 1996 to 2005. The battery used the same measures due to close collaboration among the investigators on the two studies (below). Examiners traveled to the families’ homes in order to administer the neuropsychological test battery. Privacy and minimal distractions were ensured throughout testing.

Court alcoholic families (n=159) were recruited when the father was convicted of drunk driving and had a high blood alcohol content, but was not currently undergoing litigation. A Certification of confidentiality was obtained to further support veridical reports of behavior and substance use. Control families (n=91), in which neither parent had a current or former diagnosis of substance use disorder were recruited using door-to-door canvassing in the neighborhoods of the court alcoholic families. Canvassing also uncovered community alcoholic families (n=61) in which the father met criteria for probable or definite alcoholism but had no recent drunk driving or drug-involved arrest. Both parents participated in nearly all instances. The participants sample thus comprised 607 men and women from these 311 families but not all were included here as described under “final sample.” Because parents were not genetically related and we had no evidence of assortative mating patterns that would affect our modeling, we treated these as independent observations for present purposes.

Psychopathology Evaluation

The SMAST (Selzer, Vinokur, & van Rooijen, 1975), the Drinking and Drug History Questionnaire (DDHQ;(Fitzgerald, Zucker, & Noll, 1990)), the Antisocial Behavior Inventory (ASB; (Zucker, Ellis, Bingham, & Fitzgerald, 1996; Zucker, Ellis, Fitzgerald, Bingham, & Sanford, 1996; Zucker, Noll, Ham, Fitzgerald, & Sullivan, 1993)) and the Diagnostic Interview Schedule, Version III (DIS; (Robins, Helzer, Croughan, & Ratcliff, 1981)) were completed and DSM–IV alcohol-related and ASPD diagnoses were established by two masters-degree or higher clinical raters with excellent inter-rater reliability (Zucker, Ellis, Bingham, et al., 1996; Zucker, Ellis, Fitzgerald, et al., 1996). For other disorders, the DIS diagnosis was utilized. Lifetime diagnoses were created by combining the retrospective data and information gathered at these five different time points (i.e., Wave 1 – Wave 5) for the disorders of interest in the present study. “Current” diagnoses were those for which the participants met criteria within one year of the neuropsychological assessment. ADHD was assessed by retrospective reporting of ADHD symptoms in childhood, using the DIS-IV Lifetime Form (Robins et al., 2000)

IQ Assessment

IQ was estimated with a four subtest short form of the WAIS-R(Wechsler, 1981)), current at that time: Picture Completion, Information, Arithmetic, Block Design. The WAIS R and WAIS-III have a full scale estimated IQ correlation > .90 (Wechsler, 2003).

Severity

We counted total disorders (below). Secondly, interviewers scored the Global Assessment of Functioning scale (GAF) on a scale from 0 (severely impaired) to 100 (high functioning). A subset of cases rated by two clinicians yielded acceptable agreement (intraclass r > .9).

Exclusion criteria

Any participants who met criteria for lifetime diagnoses of psychosis, bipolar disorder, and IQ < 75 at any time point were removed from the data set. In reality, this led to exclusion for these criteria of approximately 2% of those who would otherwise qualify. In addition, participants were not allowed to have drunk more than two alcoholic beverages during the hour prior to testing, nor to be drunk from earlier alcohol use during the day. Information was collected on medication and drug use prior to testing but medication status was free to vary.

Final sample

607 men and women from 311 families participated in some part of the substance abuse study. Since neuropsychological tests were administered only at Wave 5, individuals who had not completed at least half of the four tasks included in this study (i.e., Trail Making Test, Stroop Interference, Stop Signal Test, and WCST) were eliminated from further analyses (137 had completed none of the neuropsychological measures due to not having reached that wave), leaving 470 participants. Further rule outs were implemented from these 470 participants: schizophrenia-spectrum disorder or history of psychosis (n = 10), bipolar I disorder (n = 9), epilepsy (n = 1), and anti-psychotic and anti-mania medications (n = 2). This left a final n of 448.

Table 1 summarizes the similarities and differences in these two samples; the validity of combining them is evaluated empirically, below.

Table 1.

Comparison of the two samples included in the study

	ADHD Study	Alcoholism Study
N	193	448
Recruitment method	Advertisements	Agreement with courts; Canvassing
Clinical Interview	SCID	DIS
ADHD Evaluation	Modified KSAD	DIS-IV Lifetime Form
IQ	WAIS-III short form	WAIS-R short form
Age at evaluation	18–40	26–66
% Male	51%	48%
GAF mean (st.dev)	78.7 (9.3)	69.3 (7.6)
EF measures
Set shifting	Trailmaking B	Trailmaking B
Interference control	Stroop CW	Stroop CW
Flexibility	WCST-64	WCST-64
Response inhibition	Stop task	Stop task
Speed measures	Trails A	Trails A
	Stroop C,W	Stroop C,W
	Go RT	Go RT

Note:

Stroop C,W=Stroop color naming and Stroop word naming condition, with the interference condition score partialed. Trails A=Trailmaking A, with the interference condition (B) partialed. Go RT=go trial reaction time on the stop-go task.

Neuropsychological Test Battery in Both Samples

The primary neuropsychological measures were the same for both samples; each of the variables created from the neuropsychological tests were coded such that a higher value represents better performance.

Set Shifting

To isolate set shifting we administered the Trail Making Test (Trails). This is a well-known timed paper-and-pencil test consisting of two parts (Reitan, 1958). On part A participants connect numbered circles in sequential order “as fast as they can without making errors.” Part B entails switching between numbers and letters in alphabetic-numerical order. Part A is generally recognized as a measure of output speed, while part B entails additional demands on scanning and motor speed in addition to switching; however it’s validity in relation to other switching measures is recognized (Strauss, Sherman, & Spreen, 2006). To better isolate executive function effects, various ways of removing the Part A component from Part B have been proposed such as difference, ratio, or residual scores (Arbuthnott & Frank, 2000). Here, we adopted a residual score approach to avoid the psychometric problems inherent in difference scores and to reduce the overall speed-EF correlation.

Interference Control

To isolate interference control we administered the paper and pencil Stroop Color-Word Test (Stroop) (Golden, 1978). In the Word test the participant reads the black-on-white words “red”, “green,” “blue,” in random order as fast as possible. In the Color test they name the color ink of the letters XXXX printed in varying red, blue, or green ink. In the interference or color-word condition, they name the color of the ink when the ink and the word conflict (e.g., the word “red” in blue ink.). Various methods of creating interference scores that remove the word reading speed and color naming speed or both have been proposed, including subtraction, residual, and other formulas (Strauss et al., 2006). Here, to avoid controversies over the Golden scoring method and psychometric limitations of difference scores, as well as maximize speed and EF separation, an interference residual score was created by regressing the speed for Word Reading and Color Naming from Color-Word score.

Set Maintenance and Working Memory

To estimate higher level cognitive control we administered the computerized 64-item Wisconsin Card Sorting Test (WCST) (Heaton, Chelune, Talley, Kay, & Curtiss, 1993). Participants viewed a computer screen with four patterned “key” cards to which they needed to match a stimulus card, which appeared at the bottom of the screen. Using feedback from the computer (“right” or “wrong”), participants had to deduce how to sort the cards (color, shape, or number). The category remained the same for ten cards, at which point the sorting principle switched. Categories and perseverative errors were correlated, but because perseverative errors provided better model fit (below), it was used as the outcome variable. This score is conceptually related to rule detection, set maintenance, shifting, interference, and working memory updating. We refer to it as working memory for simplicity, recognizing its complexity.

Response Inhibition

The Go-Stop task (Logan, 1994) was administered for this purpose. It is a dual-task computer paradigm to assess response suppression or inhibition in a rapid decision context. Procedures were identical to those used by Logan, Schachar, & Tannock (1997) and Nigg (1999) and so are only summarized here. The computer screen displayed an X or an O on a black and white screen and individuals were required to respond to these stimuli by pressing designated buttons labeled ‘X’ and ‘O’ as quickly as possible using their dominant hand. They were to withhold responding when they heard a tone. Each trial consisted of 1000ms fixation, 1000 ms display, and 1000 ms additional time response (ITI). Four blocks of 64 trials were administered following two practice blocks of 32 trials each. The time of the stop signal tone was varied in a stochastic procedure to maintain accuracy at 50%, so that stop signal reaction time (SSRT) was computed as the difference between stop signal delay and go reaction time (Logan, 1994). To ensure validity of the measure and control for performance variation (e.g., the effort to perform correctly), data were carefully screened to eliminate responses < 100 ms (anticipations), > 1500 ms (potentially anticipation of next trial), blocks in which response accuracy < 75%, and blocks in which the probability of stopping was either < 25% (suggesting not attempting to stop) or > 75% (suggesting ignoring the cue); these procedures follow the literature (Logan, et al., 1997; Nigg, 1999).

Response time (RT) variability

RT variability was computed as within-subject standard deviation on the “go-trials” of the stop-go task. This measure is of increasing interest in psychopathology research and can be interpreted either as a measure of EF or of arousal state (Karalunas, Geurts, et al., 2014). Here, it loaded on the EF latent variable (see results). Abstracting the RT variability measure from the dual task design requires consideration of whether the stop trials interfered with or altered the variability responses. In one prior paper we modeled this to confirm that the RT variability index is valid (Karalunas, Huang-Pollock, & Nigg, 2012). The task validity was subjected to the same cleaning rules as the stop measure, in that blocks with invalid accuracy or stop probabilities were excluded from the final estimate.

Data Preparation and Analysis

Data cleaning

As recommended in methodological texts (see (Wilcox, Keselman, & Kowalchuk, 1998)), extreme outliers (z > 4.0 and more than .5 SD from the next score) were truncated to within .5 SD of the next nearest score to prevent undue influence of single scores on linear models and to reduce type I and type II errors. Data preparation was completed in the two study samples separately. Multiple imputation was used to impute missing data (1.6% in the ADHD study sample, and ~11% in the substance abuse study sample). We generated 10 imputed data sets, using Proc MI FCS Method for categorical variables in SAS. 9.4. ADHD data were missing for 10% of the substance use study.

Data Reduction

Composite and latent variables were employed for different analyses. We used confirmatory factor analysis to create latent variables for internalizing and externalizing dimensions, as well as for EF and Speed, and used these results to justify the creation of composite variables for executive function and Speed when needed. Composite scores were created by averaging the standardized scores for each item that loaded on that factor. For the latent variable analyses, Mplus 7.2 was employed. Multiple fit statistics were interpreted as outlined by Kline (2004): (1) Pearson chi-square for which nonsignificant values signify good fit, and a χ²/df ratio < 3 is acceptable; (2) Comparative Fit Index (CFI;(Bentler, 1990)) for which a value > .90 is considered a good fit; and (3) Root Mean Square Error of Approximation (RMSEA; (Steiger, 1990)) for which a value of <=.08 is considered acceptable and <= .05 is considered good. Because χ² is dependent on sample size, we focused on the CFI and RMSEA to determine adequacy of fit. Since binary diagnosis variables were used as indicators of the latent variables, the CATEGORICAL statement was used in MPlus and the WLSMV estimator was used. Chi-square difference testing between nested models was implemented with DIFFTEST. The SAS 9.3 statistical package was used to perform all other statistical analyses. Table 2 shows the correlations among the cognitive measures and Figure 1 shows the final measurement model for internalizing and externalizing disorders. Based on standard criteria of CFI>.90 and RMSEA<.05, the resultant model fit satisfactorily χ² (13) = 29.4, p = .006, CFI = 0.93, RMSEA = .04 and all factor loadings were significant. In the alcohol sample, model fit was marginal (CFI = .87, RMSEA = .06); whereas the fit was excellent for the ADHD sample (CFI = .97, RMSEA = .02). All factor loadings were significant for both samples. Figure 2 shows the bifactor model with a general psychopathology factor (“P”) (Caspi et al., 2014; Lahey et al., 2012). This model also fit well (χ²(7) = 14.9, p = .04; CFI= .97; RMSEA = .042) and we discuss it further in results, but we retained the two factor (externalizing and internalizing) model as well for reasons explained in results. Figure 3 shows the resulting measurement model for EF and Speed. To achieve adequate model fit (χ²(8) = 93.0, p = .000, CFI=.92, RMSEA=0.08), a residual covariance was allowed between Response inhibition and Reaction time variability (these are both measurements taken from the same the Stop Signal test). All factor loadings were significant.

Table 2.

Correlations among cognitive measures (n=641)

	Speed	IQ	Word	Color	Trails A	Stroop CW	Trails B	SSRT	RT variability	WCST
EF composite	0.30	0.31	0.26	0.29	0.18	0.52	0.54	0.58	0.46	0.46
Speed composite	1.00	0.33	0.85	0.85	0.72	0.05	0.21	0.29	0.19	0.12
IQ Estimated		1.00	0.26	0.29	0.34	0.20	0.31	0.17	0.32	0.17
Word reading (Word)			1.00	0.68	0.38	0.00	0.25	0.25	0.16	0.09
Color naming (Color)				1.00	0.41	0.00	0.25	0.22	0.19	0.13
Trails A					1.00	0.12	0.00	0.24	0.26	0.10
Stroop CW residual						1.00	0.12	0.09	0.09	0.11
Trails B residual							1.00	0.08	0.10	0.19
Stop signal RT (SSRT)								1.00	0.58	0.08
RT variability									1.00	0.12
WCST										1.00

Stroop color-word score is the score for the interference condition, with color and word reading scores regressed out. Trails B is a residual score with Trails A score regressed out. See text for explanation. At this sample size, r <= .11 yields p <. 01.

Figure 1.

Measurement model for psychiatric disorders, separated into internalizing and externalizing latent variables, omitting ADHD. (GAD= Generalized anxiety disorder, MDD = Major Depressive Disorder; ASPD = antisocial personality disorder)

Figure 2.

The bi-factor model, with a general psychopathology (p) factor and the specific factors externalizing and internalizing, which are orthogonal in this model. Model fit was good (χ²(7) = 14.9, p = .04; CFI= .97; RMSEA = .042). Alcohol disorder=alcohol use disorder; drug disorder=drug use disorder. P=General Psychopathology factor.

Figure 3.

Measurement model for cognitive test performance as executive functions and processing Speed latent variables.

Resp Inhibition=Response Inhibition as measured by Stop Signal Response Time; RT Variability=Response time variability; WCST PE=Wisconsin Card Sorting Test perseverative errors standard score.

Final psychopathology variables

The primary focus for all analyses was absence/presence of lifetime psychiatric diagnoses (coded as 0/1). However, all analyses were repeated to assess current disorders (i.e., within the past year) for completeness.

Response Speed

We used a composite measure of response speed comprising reaction time for Trails A, Stroop Color Naming, and Stroop Word reading. These measures were standardized and averaged to create the speed composite as explained below.

Sample Pooling

Pooling the samples offered the advantage of providing a large, diverse sample across a range of disorders to test our hypotheses, but required empirical justification. Statistical covariation of sample source was deemed inappropriate because the two samples had different distributions of disorders, and controlling for the sample of origin could result in unintentionally controlling for presence of disorders; in other words, controlling for the variable of interest. To justify sample pooling we did the following. First, we controlled for key demographic variables that differed between the samples in all analyses (age and sex). Second, where appropriate, we standardized scores separately in the two samples. Third, we conducted confirmatory factor analyses (CFAs) between the groups to examine comparability of the two covariance structures. For each latent variable shown in Figures 1 and 2, we fit models: (1) to the combined sample; (2) separately in the two samples, and then (3) conducted a multiple-group analyses to evaluate the measurement model in both groups simultaneously with varying levels of cross-group equality constraints. Metric invariance requires that factor loadings are equivalent across samples whereas scalar invariance for models with categorical indicators requires factor loadings and thresholds to be equivalent across samples (Kline, 2004). Partial measurement invariance requires that factor loadings are equivalent for some of the factors but not for others. Measurement invariance was tested in Mplus. Full invariance was tested with the convenience feature (MODEL= CONFIGURAL SCALAR) and partial invariance was tested with DIFFTEST. For the model of internalizing and externalizing disorders (Figure 1), measurement invariance testing between the two groups showed evidence for scalar invariance (Δχ² (3) = 6.5, p = 0.092), justifying our decision to combine the two samples. We repeated this for the model of EF and Speed in Figure 2. Between the two samples, metric invariance was found for this measurement model (Δχ² (6) = 9.4, p = 0.15), again justifying combining samples.

Statistical Analyses

To test the “Specificity Model” we conducted a repeated measures MANOVA to evaluate whether different disorders are associated with differential patterns of strengths and weaknesses in EF. To do this we grouped the disorders according to what our sampling could handle into several non-overlapping groups based on the hierarchical structure of disorders in which anxiety and mood disorders cluster on related lower order dimensions (Krueger, 1999). From there we were able to create non-overlapping groups with no undescribed comorbidity as described in results. This basic analysis simply seeks to determine if a particular combination of scores, rather than one individual score, characterizes a group. This is one version of “profile analysis” which at a general level refers to clustering on multiple indicators simultaneously (Atchison, Bradshaw, & Massman, 2004; Tabachnick & Fidhi, 1996). If the model were correct, we would predict task x disorder interaction on the MANOVA. We then planned to follow up with post-hoc pairwise tests to examine which EF functions are lower in each disorder. Multiple testing was handled by application of the Duncan multiple range test for pairwise group comparisons within each neurocognitive measure.

To test the Severity model, multiple regression was used to regress all of the cognitive measures on a score for (a) total number of disorders and (b) the Global Assessment of Functioning (GAF) score. Secondary analyses examined follow up questions also using multiple regression.

To test the higher order Dimension, structural equation modeling was conducted with the latent EF, Speed, Externalizing, and Internalizing factors and the P-factor shown in Figures 1, 2 and 3.

Results

Sample Descriptions

Descriptive and clinical data for the two samples and the pooled sample are presented in Table 3. As is apparent, the substance use sample was older, had lower IQ, and was more likely to be married. Given the differences in sample age, study recruitment objectives, and inclusion/exclusion criteria, differences in the rates of disorders were expected.

Table 3.

Demographic and Clinical Summary of Samples Used

	Pooled	ADHD Sample	SA Sample
N	641	193	448
N(%) male	336 (52.4%)	103 (53.4%)	233 (52.0%)
N (%) white	615 (95.9%)	167 (86.5%)	439 (98.0%)
N (%) married	405 (69.5%)	44 (28.9%)	361 (83.8%)
Average Age (years)	38.1 (10.39)	24.2 (4.5)	44.1 (5.0)
Education (years)	14.1 (2.13)	14.3 (1.9)	14.0 (2.2)
Estimated full scale IQ (SD)	106.3 (12.85)	112.0 (11.0)	103.9 (12.8)
Lifetime Disorders
Major Depressive Disorder	212 (33.1%)	43 (22.3%)	169 (37.7%)
Dysthymia	63 (9.8%)	11 (5.7%)	52 (11.6%)
Mood Subtotal	232 (36.2%)	49 (25.4%)	183 (40.9%)
Generalized Anxiety Disorder	39 (6.1%)	17 (8.8%)	22 (4.9%)
Agoraphobia	8 (1.2%)	0 (0%)	8 (1.8%)
Social Phobia	50 (7.8%)	16 (8.3%)	34 (7.6%)
Specific Phobia	89 (13.9%)	6 (3.1%)	83 (18.5%)
PTSD	26 (4.1%)	4 (2.1%)	22 (4.9%)
Anxiety subtotal	174 (27.2%)	35 (18.1%)	139 (31.0%)
ADHD	133 (20.75%)	103 (53.4%)	30 (6.70%)
Antisocial Personality Disorder	72 (11.2%)	5 (2.6%)	67 (14.9%)
Alcohol Dependence	218 (34.0%)	22 (11.4%)	196 (43.8%)
Drug Dependence	44 (6.9%)	14 (7.3%)	30 (6.7%)
No disorder	175 (27.3%)	67 (34.7%)	108 (24.11%)

Primary Analyses

Specificity model

We evaluated the Specificity model with repeated measures MANOVA as described earlier (the simplified “profile” analysis). Each test score was converted to a z-score so that all scores were on the same scale. For the lifetime analysis, individuals were assigned to discrete, non-overlapping groups for analysis as follows: (1) No disorder (control) (n=173), (2) alcoholism with antisocial personality disorder (hereafter, “antisocial alcoholism” (n = 45)); (3) any anxiety disorder without depression (n=70); (4) ADHD without depression, anxiety, or substance use disorder (n=48); (5) other substance use disorder without anxiety disorder (n = 123); (6) depression without anxiety disorder or substance use disorder (n=82); (7) anxiety+ depression (n = 97). Finally, n=3 with ASPD who did not fit in other groups were omitted from this analysis.

For the primary analysis of lifetime disorders, we saw a significant main effect of group (F (6, 631) = 2.7, p < .01) and test (F (4, 2512) = 4.4, p < .05) and the crucial test-by-disorder interaction was again significant (F(24, 2512) = 2.2, p < .01). Figure 4 displays the results, controlling for sex, IQ and age. Post-hoc pairwise tests corrected for multiple comparisons indicated that the ADHD group was distinguished by poor performance on interference control and set shifting, although the corrected effect was only different from all other groups for set shifting (all p<.05 corrected). The ADHD and the antisocial-alcoholic groups were characterized by poor response inhibition and excess response variability, with the worst performance seen in the antisocial-alcoholic group. Formally, on response inhibition the alcoholic-antisocial group differed from all other groups (p<.05) except ADHD, and the ADHD group differed from all the internalizing groups (p<.05). On variability, both ADHD and antisocial alcoholics were worse than controls and all internalizing groups (p<.05). For working memory, although the antisocial-alcoholic group had the worst performance, corrected p-values were non-significant. For Speed, ADHD, antisocial-alcoholic, and Anxiety+Depression all were slower than controls and slower than simple depression or anxiety (p<.05 corrected).

Figure 4. Repeated Measures Anova (Profile Analysis) for Specificity Model.

Performance on neuropsychological tests for non-overlapping groups defined by lifetime diagnoses. Control group mean scaled to 0.00 on all tests; lower scores indicate worse performance. Y axis indicates standardized z scores. Standard error bars shown. Anti-alc=antisocial alcoholic, Alc=alcoholic without antisocial, Dep=Depression, Anx+Dep=Anxiety and Depression, all disorders lifetime.

For current diagnoses (not shown), the groupings were (1) No disorder (control) (n=323), (2) antisocial alcoholism (n = 23); (3) any anxiety disorder without depression (n=46); (4) ADHD without depression, anxiety, or substance use disorder (n=54); (5) substance use disorder without anxiety disorder (n = 50); (6) depression without anxiety disorder or substance use disorder (n=35); (7) anxiety+ depression (n = 21). In this analysis, the test-by-disorder interaction was again significant, suggesting different deficits by disorder (F (24, 2180) = 1.57, p < .05). However, the interaction did not survive control of covariates. Post-hoc corrected comparisons showed very few group differences, although the general pattern of scores was the same and ADHD again was worse than controls on set shifting and both ADHD and antisocial alcoholic were again worse than controls on response inhibition. Current anxiety and depression were associated with poor interference control, in contrast to lifetime disorders (means and SEM’s available on request from the authors).

Severity Model

This analysis examined the possibility that the number of disorders (a proxy for global psychiatric severity) contributed to EF deficits and slow Speed. For these analyses, we used composite scores, created by averaging across the items that loaded on the executive functioning latent variable and the Speed latent variable (from Figure 2 in Methods). A single variable grouped individuals as: (1) no disorders; (2) one disorder; (3) two disorders; and (4) three or more disorders. Here, rare disorders could be included and so this count drew from MDD, dysthymia, GAD, PTSD, agoraphobia, social phobia, specific phobia, OCD, alcohol dependence, drug dependence, ASPD, and ADHD. Table 4 displays the frequency of number of current and lifetime disorders, and Table 5 shows the results of the regression models for lifetime and current disorders. As shown in Table 4, separate regressions models predicted the composite EF and Speed variables, with the number of disorders entered at step 1 and covariates entered at step 2 (age, sex) and step 3 (IQ). Number of lifetime disorders was not associated with EF, but was related to slower Speed. Current severity was likewise robustly associated with Speed; it was also associated with EF, although not after control for IQ.

Table 4.

Comorbidity: Frequencies of Numbers of Individual Lifetime Disorders

Number of Disorders	Lifetime (n=641)	Current (n=641)
0	175 (27.3%)	387 (60.4%)
1	201 (31.4%)	164 (25.5%)
2	135 (21.1%)	64 (10.0%)
3	79 (12.3%)	19 (3.0%)
4	37 (5.8%)	6 (0.9%)
5 or more	14 (2.2%)	1 (0.2%)

Table 5.

Regression analyses for number of lifetime and current disorders on executive function and Speed composite scores

LIFETIME
	Executive function			Speed
	B1	B2	B3	B1	B2	B3
# disorders	−.04 (ns)	−.05 (ns)	.00 (ns)	−.15 (.000)	−.16 (.000)	−.11 (.002)
age		.01 (.001)	.02 (.000)		−.05 (ns)	−.00 (ns)
Sex		−.07 (ns)	−.14 (ns)		−.32 (.000)	−.38 (.000)
IQ			.35 (.000)			.33 (.000)
R2	.00	.02	.13	.03	.06	.16
CURRENT
	Executive function			Speed
	B1	B2	B3	B1	B2	B3
#	−.15 (.001)	−.14 (.002)	−.06 (ns)	−.18 (.001)	−.19 (.001)	−.11 (.010)
age		.12 (.002)	.18 (.000)		−.08 (ns)	−.02 (ns)
sex		−.07 (ns)	−.14 (ns)		−.29 (.000)	−.36 (.001)
IQ			.33 (.000)			.33 (.000)
R2	.02	.03	.14	.03	.05	.15

As a separate check on the severity model, we used the Global Assessment of Functioning (GAF) score as a proxy for severity of psychiatric disorder. We had collected current GAF scores from both samples. The GAF was a weaker predictor of both EF and speed. GAF did not significantly predict EF (beta = .02, p > .1). While GAF predicted the composite speed score (beta = .04, p = .00), after controlling for age, IQ and sex, the prediction was not significant (beta = .01, p > .1). Overall, these analyses converged in failing to support an association of EF with severity, while retaining an association of speed with some definitions of severity.

We then checked whether the presence of having any disorder was related to EF or Speed. A variable indicating the presence of any disorder (0 = no disorder, 1 = at least one disorder) was entered at step 1 and covariates entered at step 2 and 3 (data not shown). The presence of any disorder predicted poor EF (Lifetime: β = −.19, CI= −.36 – .01, p = .04; Current: β = −.26, CI= −42 – −.10, p = .001) after controlling for age and sex. This effect remained significant after IQ was added to the model for current (β = −.16, CI= −.31 – −.005, p = .04), but not lifetime diagnoses (β = −.08, CI= −.25 – .08, p = .33).

Finally, we conducted a regression to take into account comorbidity, by testing for the effect of each diagnosis, controlling for all other diagnoses. In this analysis, only ADHD significantly predicted EF composite, after controlling for age, sex and IQ (Table 6, ADHD, Lifetime: β = −.54, CI= −.76 – −.32, p = .000; Current (not shown): β = −.51, CI= −.74 – −.28, p = .001). ADHD and Drug Dependence predicted lower speed, after controlling for all other diagnoses, age, sex and IQ.

Table 6.

Regression analysis for lifetime disorders predicting performance on executive function and Speed composite Score

	Executive Function			Speed
Disorders (lifetime)	B1	B2	B3	B1	B2	B3
Depression	.05 (ns)	.03 (ns)	.92 (ns)	−.04 (ns)	−.04 (ns)	01 (ns)
Anxiety	.11 (ns)	.10 (ns)	.10 (ns)	.02 (ns)	.01 (ns)	.02 (ns)
Alcohol Dependence	.03 (ns)	.03 (ns)	.12 (ns)	−.25 (.004)	−.15 (ns)	−.07 (ns)
Drug Dependence	−.08 (ns)	−.07 (ns)	−.06 (ns)	−.27 (ns)	−.31 (.05)	−.30 (.04)
ASPD	−.12 (ns)	−.11 (ns)	.02 (ns)	−.25 (ns)	−.15 (ns)	−.04 (ns)
ADHD	−.54 (.000)	−.53 (.000)	−.54 (.000)	−.29 (.007)	−.47 (.000)	−.47 (.000)
Age		.02 (ns)	.06 (ns)		−.13 (.006)	−.10 (.03)
Sex		−.08 (ns)	−.16 (.04)		−.28 (.000)	−.37 (.000)
IQ			.35 (.000)			.33 (.000)
R2	.06	.06	.17	.05	.08	.18

Notes ASPD=antisocial personality disorder; ADHD=attention-deficit/hyperactivity disorder.

Dimension model

The latent variables described in Methods Figure 1 and 3 were evaluated in an SEM model first. Figure 5 displays the model results for lifetime disorders using the internalizing and externalizing composite variables and excluding ADHD. Age and gender were covaried (but not shown for figure readability). Fit was adequate: χ² (105) = 260.49, p = .000, CFI = .83, RMSEA = 0.05. In this model, externalizing was uniquely related to both poor EF and Speed, but internalizing was not related to either cognitive outcome. When ADHD was added to the model as a predictor of EF and Speed separate from externalizing, it was strongly related to EF (β = −.43, CI= −.74 – −.12, p= .007) and the externalizing dimension was no longer associated with EF (β = −.13, CI= −.40 – .14, p= .36). However, externalizing remained independently associated with Speed (β = −.18, CI= −.34 – −.01, p= .04), even after ADHD was in the model. Removing ADHD from the model and putting IQ in the model also removed the EF association with externalizing (β = −.06, CI= −.17 – .04, p= .24), although Speed again remained associated (β = −.18, CI= −.31 – −.04, p = .009).

Figure 5.

Final structural equation model for the relationships between lifetime internalizing and externalizing disorders and cognitive task performance for executive functions and Speed, controlling for age and gender.

Notes. Fit statistics: ${\chi }^2 \left(105\right)=260.49, p= .000,$CFI = .83, RMSEA = 0.05. Squares represent manifest variables; Circles represent latent variables; GAD=generalized anxiety disorder; ASPD =antisocial personality disorder; Dep=dependence; MDD=major depressive disorder; WCST=Wisconsin Card Sorting Test perseverative errors; Resp Inhib=Response Inhibition.

For current externalizing and internalizing composite scores, fit was again adequate (χ² (116) = 210.34, p = .000, CFI = .87, RMSEA = 0.04), and the model is shown in Figure 6. Internalizing and externalizing latent variables were not significantly related to EF or Speed measures. When ADHD was added to the model, it was related to EF (β = −.50, CI= −.96 – −.05, p= .03).

Figure 6.

Structural equation model examining the relationships between current internalizing and externalizing disorders on cognitive task performance for executive functions and Speed, controlling for age and gender. Model Fit: ( ${\chi }^2 \left(116\right)=210.34, p= .000,$CFI = .87, RMSEA = 0.04), (abbreviations as in Figure 5).

We then attempted to replicate the bifactor model of general psycopathology or “p” factor that has been introduced (Caspi et al., 2014; Lahey et al., 2012). We found support for the bifactor model (Figure 3) having a better fit to the data than the correlated factors model (Figure 1) (delta chi-square = 15.1, df=6, p=.02); whereas a single factor model had very poor fit (RMSEA = .11, CFI = .55). However, when we attempted to use the model in Figure 2 to predict EF, the model failed to converge. Using the model in Figure 2, we found that the p-factor (beta = −.25, p < .05) and externalizing specific factor (beta = −.29, p < .01) predicted speed whereas the internalizing factor (beta = −.01, p > .1) did not significantly predict speed. A second bifactor model with only a specific externalizing factor was also found to be a good fit to the data (CFI = .95, RMSEA = 0.042). This model fit did not differ from the model in Figure 2 (delta chi-square (4 d.f.) = 8.1, p > .05). In this model (not shown), the p-factor marginally predicted speed (beta = −.12, p < .1) and did not significantly predict EF (beta = .02, p >.1); the externalizing factor predicted both EF (beta = −0.28 p < .01) and speed (beta = −0.34, p < .01).

Discussion

The present study examined the relationship of psychiatric disorders with EF and response Speed on common forms of psychopathology that have previously been associated, in some manner, with neuropsychological deficits. This approach is increasingly important as the field considers how to integrate mechanistically relevant intermediate phenotypes within frameworks such as the research domain criteria. Three conceptual models were tested.

Results were complex but helpful. The best model differed somewhat for EF versus Speed. In the case of EF, although the EF composite was related to externalizing (and not internalizing in the dimension model), it was not related to the general psychopathology P factor, although that model was not robust (below). Whereas when ADHD was included it seemed to account for externalizing effects in different models, the more granular picture provided by the specificity model was informative. ADHD was associated with one particular pattern of EF components, whereas antisocial alcoholism (or antisocial substance use disorder) was associated with a somewhat different profile. While we were unable, in these data, to differentiate the contribution of antisocial personality separate from substance use disorders, this distinction nonetheless illustrates that a dimensional approach provides some context but within it, a more granular approach appears the most productive for understanding how EF relates to the ADHD and externalizing domains. The findings fit with prior reports that observed particular EF profiles in ADHD (Willcutt et al, 2012) and in antisocial alcoholism (Giancola & Moss, 1998; Malloy, Noel, Rogers, Longabaugh, & Beattie, 1989).

When taking overlapping/comorbid conditions into account and controlling for their presence in regression analyses, only ADHD was related to the EF composite score. That result is consistent with previous findings that EF deficits associated with ADHD remain robust after controlling for comorbid conditions (Nigg et al., 2005; Seidman, Biederman, Weber, Hatch, & Faraone, 1998; Willcutt, Doyle, Nigg, Faraone, & Pennington, 2005). However, we also found that when ADHD is not present and EF is examined at a componential level, distinct associations are seen with antisocial alcoholism as well.

It is notable that Speed and EF were able to be differentiated and show distinct correlational patterns. Concerns have been raised about whether or not EF processes are differentiable from other perceptual speed a well as the general intelligence factor (g), which appears to underlie performance on IQ tests (Salthouse, 2005; Salthouse et al., 2003). The EF effects that were found in this study withstood controls for processing speed. The use of residual scores likely helped to differentiate these constructs to some degree. On the other hand, some EF effects could not withstand controls for IQ. Some researchers have argued that components of EF underlie IQ (Conway, Kane, & Engle, 2003; Friedman et al., 2006; Kane & Engle, 2002), so the appropriateness of control for IQ in EF studies remains debatable.

The nature of the disorder-specific relationships within the externalizing dimension may reflect developmental variation in the emergence of EF deficits. These externalizing disorders are all highly comorbid, but ADHD is the first to develop in childhood. In this study, ADHD was also the disorder with the strongest and broadest relationship to EF deficits, which remained even after controlling for IQ. Symptoms of ADHD may contribute to and mediate the EF effects associated with other disorders. Thus, the developmental sequence of the externalizing disorders may be contributing to the relationships with EF seen in this study.

The picture for Speed was different. Slow Speed was a robust correlate of psychopathology even at the most abstract level of analysis (externalizing psychopathology), whether at the lifetime or current diagnosis level. Likewise, total severity, measured by total number of comorbid disorders, was also related robustly to response Speed regardless of covariates. Finally, in the overall SEM model, the Speed effect was distinct from the EF effect, rather than one explaining the other. While slow Speed was related to both internalizing and externalizing disorders in the profile analysis, in the conjoint dimensional analysis in which externalizing and internalizing were both in the model, Speed was related only to Externalizing.

Several conceptual implications of these findings can be noted. At the level of phenotypic structure, psychopathology can be conceptualized hierarchically, with particular manifestations (alcoholism, depression) nested within higher order factors (internalizing, externalizing); our data suggest that processing speed is associated with these higher order liabilities for psychopathology and for its severity, particularly in the externalizing domain and the closely related domain of ADHD. (In fact, some factor models include ADHD in the externalizing domain). In contrast, EF appears to operate not as a general factor, but at a componential level in which more specific aspects of cognitive weakness are related to particular manifestations of psychopathology. This finding suggests the operation of a two-stage model of neuropsychology and psychopathology that may guide how we approach intermediate phenotypes. From this perspective, processing or response Speed is an intermediate phenotype for both externalizing psychopathology and overall psychopathological severity. In contrast, EF components are intermediate phenotypes for subsets of that liability.

What might this mean neurobiologically? The neurobiological basis of processing speed is difficult to isolate due to its own multi-componential nature. However, as noted earlier, one interesting possibility is that it reflects, at least in part, delayed or immature white matter development (Kochunov et al., 2015; Schneider et al., 2015). Delays in cortical maturation have been noted in ADHD (Shaw et al., 2013), and diffuse white matter alterations have been noted in the brains of children and adults with ADHD (van Ewijk, Heslenfeld, Zwiers, Buitelaar, & Oosterlaan, 2012) as well as antisocial personality and/or substance abuse (Elofson, Gongvatana, & Carey, 2013; Herting, Schwartz, Mitchell, & Nagel, 2010), although data are still sparse and it remains to be seen how diffuse versus localized these are when larger studies are completed. Nonetheless, diffuse white matter developmental delays may be a common feature of liability for externalizing psychopathologies; within that framework processing or response Speed appears to be an endophenotype for the liability. EF is associated with connectivity and functional responsivity of specific fronto-thalamic-subcortical neural circuits that are also associated with ADHD but are associated in distinct ways with antisocial or other externalizing behavior (Rubia, 2011).

Limitations

While this work helps break new ground in cross-disorder conceptualization of neuropsychological intermediate phenotypes, important limitations temper our conclusions and mandate the need for further work in this area. First, measurement of Speed and EF was not comprehensive. We did not fractionate speed components or isolate, for example, perceptual speed. Meantime, other EF components could include purer measures of working memory or capacity, verbal fluency, and planning that might reveal additional or more differentiated associations (Moscovitch & Winocur, 1995; Pennington & Ozonoff, 1996; West, 1996).

Second, sampling was targeted at externalizing populations, with internalizing disorders identified secondarily. Although we had ample representation of internalizing disorders and examined them without comorbid externalizing, sampling may still have under-represented the range of internalizing psychopathology. This was most likely to affect associations with current, rather than lifetime disorders, and may have led to under-identification of effects in current disorders. In particular, individuals with current MDD were excluded in the ADHD sample. However, this concern is somewhat countered by the observation that while anxiety and depression have been variably associated with EF deficits in past literature, these effects are most robust during current episodes (Culpepper, 2015). Thus, our results would argue that history of depression or anxiety is not related to enduring EF weakness, even though EF is impaired by severe mood episode (Sommerfeldt et al., 2015). It may be that a broader representation of EF measures would show a linkage to aspects of emotion regulation and internalizing disorders more clearly (see discussion by Porter et al (2015)).

We combined two samples that had somewhat different features and utilized somewhat different assessment methods. While our two-group SEM model supported the psychometric validity of the pooling carried out here, it remains possible that combining these two samples led to underestimation of effects, increasing Type II error.

Further, in the case of substance use disorders, several nuances were unaddressed. For instance, number of drug dependencies, presence of additional comorbid disorders, dose-related drug effects, chronicity of use, and recency of intake were relevant to associations between drug dependence and EF impairment in past studies (Bolla, Rothman, & Cadet, 1999; Eldreth, Matochik, Cadet, & Bolla, 2004; Fals-Stewart & Bates, 2003; Goldstein et al., 2004; Gruber & Yurgelun-Todd, 2005; Selby & Azrin, 1998). The inability to account for these factors in the present study may have impeded the ability to detect some EF effects. Further, we relied on self-report to control for use on the day of testing; objective validation was not obtained.

Finally, the models are open to criticism. For example, they are not nested, so we cannot formally pit them against one another. Rather, they reflect different conceptual proposals and their comparison is qualitative and are provided here to stimulate further consideration of conceptual models in comparing disorders on intermediate phenotypes. A specific point here is that conclusions about severity are limited by the possibility that number of disorders and GAF are not adequate markers of severity. Previous studies have found that severity of depressive symptoms and frequency/chronicity of episodes were related to cognitive impairment (Merriam, Thase, Haas, Keshavan, & Sweeney, 1999; Purcell, Maruff, Kyrios, & Pantelis, 1998). Alternative markers of severity not examined here include level of self-reported impairment, chronicity, success of treatments, relapse rates, and other indicators. The present results thus provide only an initial examination possible measures of severity and of these various models in the same data.

Conclusions and Future Directions

Even with these limitations acknowledged, the present study has numerous distinctive features that clarify the literature on EF, Speed, and psychopathology. The high density of many different types of disorders in these at-risk samples provided a unique opportunity to simultaneously evaluate the cognitive effects associated with individual as well as combined classes of disorders. To our knowledge, this is the first study to directly examine EF and Speed together in this manner with this range of different disorders. Thus, it provides the best evidence for EF effects in psychopathology, against which future studies may be evaluated. It also provides support for the effort to conduct cross disorder studies using intermediate phenotypes as intended by the RDoC effort (Insel et al., 2010).

The samples themselves were large, well-defined, and community-based, which provide some advantages compared so some other studies. Although they differed in several ways (Table 1) including rates of screen out and recruitment method, we provided empirical justification for pooling them here and the integration of different sampling methods and a wide range of risk groups improved generalizability (e.g., range of SES, high risk alcoholics and other disorders, control participants that are functioning well across a number of domains and that are demographically similar to affected individuals). The fact that the samples were community recruited largely eliminates the selective bias when using the restricted samples present in treatment populations. Both EF and psychopathology were measured from multiple perspectives. EF was considered as both a global factor as well as one involving multiple distinct component processes. Psychopathology was examined at the level of individual disorders, comorbid conditions, and shared underlying dimensions. This broad approach allowed for an extensive examination of the relationships of interest.

Several future directions are needed, inevitably. Central is the continuation of this work across other sampling strategies, and extension to other disorders, such as personality disorders, for which neuropsychological intermediate phenotypes have also been proposed. Further, existing disorders have biological and genetic overlaps, as well as heterogeneity of neuropsychological and self-regulatory profiles within existing psychiatric syndromes. It is possible this partly explains the mixed findings particularly for EF. Future work can take these into account to fruitfully integrate the approach here with potential revisions to nosology, as illustrated by recent papers (Fair, Bathula, Nikolas, & Nigg, 2012; Karalunas, Fair, et al., 2014).

In conclusion, these results suggest that the relationship between EF and psychopathology involves specific externalizing pathological processes, comorbidity among disorders, specific components of EF, and multiple cognitive domains. In contrast, slow Speed is a non-specific indicator of externalizing psychopathology. These results highlight issues that need to be accounted for in future studies to clarify the interplay between behavioral and cognitive aspects of psychopathology.