Developing Dimensional, Pandiagnostic Inhibitory Control Constructs with Self-Report and Neuropsychological Data

Abstract

Trait markers, or intermediate phenotypes linking different units of analysis (self-report, performance) from the Research Domain Criteria (RDoC) matrix across populations is a necessary step in identifying at risk individuals. In the current study,150 healthy controls (HC) and 456 individuals with Bipolar (BD) Type I or II, NOS (Not Otherwise Specified) or Schizoaffective BD completed self-report neuropsychological tests of inhibitory control (IC) and executive functioning. Bi-factor analyses were used to examine the factor structure of these measures and to evaluate for invariance across groups. Bi-factor analyses found modest convergence of items from neuropsychological tests and self-report measures of IC among HC and BD. The factor scores showed evidence of a general IC construct (i.e. subdomain) across measures. Importantly, invariance testing indicated that the same construct was measured equally well across groups. Groups differed on the general factor for three of the four scales. Convergence on a general IC factor and invariance across diagnosis supports the use of combined dimensional measures to identify clinical risk and highlights how prospective RDoC studies might integrate units of analysis.

Traditionally, psychiatric illnesses have been classified in a categorical manner based on a clinical consensus of signs and symptoms as defined in the Diagnostic and Statistical Manual of Mental Disorders (DSM). However, these diagnostic categories have not aligned with clinical neuroscience and genetic findings and may not capture the underlying mechanisms of dysfunction in psychiatric illnesses (Insel et al., 2010). The National Institute of Mental Health (NIMH) has spearheaded new, complementary efforts to better classify psychiatric illness and the underlying mechanisms that lead to them with the Research Domain Criteria (RDoC) research framework. RDoC aims to study psychiatric illness using multi-layered systems to identify how these systems contribute to psychiatric functioning (NIMH, 2015). These multi-layered systems include 1) transdiagnostic domains that cut across several traditional diagnostic categories and 2) different units of analysis to measure each domain on a dimensional scale. NIMH has defined an initial set of domains in the RDoC research matrix, with units of analysis that include: Circuits (e.g., neuroimaging and animal neurobiological techniques), Physiology (e.g., heart rate, cortisol), Behavior/Performance (e.g., neuropsychological tasks), and Self-reports (e.g., self-report questionnaires) (NIMH website; see (Langenecker, Jacobs, & Passarotti, 2014).

Integrating these units of analysis from RDoC is a crucial step in refining intermediate phenotypes within each domain that link measures of clinical risk together, helping us to better understand vulnerability for and expression of psychiatric illness (Langenecker et al., 2014). Despite the possibility that dimensional intermediate phenotypes are closer to the risks for illness, dimensional models have been underutilized in psychiatric research or in research of illnesses with both performance and symptom change (e.g., Parkinson’s). In addition, it is not clear if these RDoC units of analysis converge on key latent variables, allowing us to understand how each system or domain contributes to underlying factors or mechanisms related to psychiatric illness in a reliable and valid manner (Shankman & Gorka, 2015).

As an example, within the Cognitive Control subdomain, recent meta-analyses found weak convergence of self-report and neuropsychological inhibitory control (IC) measures in clinical and non-clinical samples (Duckworth & Kern, 2011), similar to a previous review (Meyer et al., 2001) and other recent reports (Vasconcelos et al., 2014). However, another study found significant correlations between a self-report measure, the Barratt Impulsiveness Scale (BIS-11; Patton, Stanford, & Barratt, 1995), and neuropsychological measures of IC in healthy adults (Enticott, Ogloff, & Bradshaw, 2006). Due to the fact that many of the self-report measures, including the BIS-11, and neuropsychological measures of IC were normed on healthy adults, it is possible that these measures converge more strongly, or even quite differently, in healthy adults relative to adults with psychiatric illness. For example, those with psychiatric illness may have worse IC behavioral performance, but they may have inaccurate and widely varying self-reported estimates of their inherent abilities, captured by self-report. Integration of differing cognitive profiles and accuracy in self-report could result in a failure of measurement invariance across healthy and disordered populations (e.g., differences in error in measurement that are greater in those with clinical diagnosis, Berenbaum, 2013). Therefore, it is important for us to better understand how and if different units of analysis like neuropsychological data and self-report data contribute to subdomains like IC (e.g., Cognitive Systems) in both clinical and non-clinical populations – invariance criterion.

Cognitive Systems is a domain that is impaired in many psychiatric illnesses, including Bipolar Disorder (BD) (Langenecker, Caveney, et al., 2007; Votruba & Langenecker, 2013). BD is characterized by deficits in IC (Green, Cahill, & Malhi, 2007; Langenecker, Saunders, Kade, Ransom, & McInnis, 2010; Najt et al., 2007; Ryan et al., 2012), one type of Cognitive System that is often defined as a predisposition toward unplanned, rapid reactions without regard to negative consequences (Moeller, Barratt, Dougherty, Schmitz, & Swann, 2001). Several studies have found impairments in IC using neuropsychological measures (Garcia-Blanco, Perea, & Salmeron, 2013; Green et al., 2007; Kravariti et al., 2009; S. A. Langenecker et al., 2010; Larson, Shear, Krikorian, Welge, & Strakowski, 2005; Ryan et al., 2012; Swann, Lijffijt, Lane, Steinberg, & Moeller, 2009b) and using self-report measures (Newman & Meyer, 2014; Reddy et al., 2014; Strakowski et al., 2010; Swann, Lijffijt, Lane, Steinberg, & Moeller, 2009a; Swann, Pazzaglia, Nicholls, Dougherty, & Moeller, 2003) across the active and euthymic states of illness in BD. As such, IC may be an important intermediate phenotype in not only BD, but also other psychiatric illnesses (Langenecker et al., 2014), strengthening the need to develop integrated dimensional markers of IC to use in many clinical contexts.

The present study examined the convergence between neuropsychological measures of executive functioning, including IC, and self-report measures of IC among individuals with BD and healthy controls. Notably, while many ongoing studies are preparing to integrate multiple units of analysis, interim studies like the current study can use existing data sets as intermediate steps to evaluate and validate these analyses. In particular, some strategies require larger sets of data that may be too expensive for full assessment across units of analysis within the RDoC Matrix. As such, an important intermediate step would be to demonstrate that self-report and performance/neuropsychological measures achieve convergence in large samples, and that the challenge of invariance can be surmounted. Then, studies with smaller samples and more units of analysis can use confirmatory strategies, as well as exploratory strategies to define and refine latent variables. The present study capitalized on a large, pandiagnostic, well-characterized sample with multiple self-report and performance measures of executive functioning to attempt to target latent variable definition at a subdomain level, IC. The first aim of the study was to validate the factor structure of inhibitory control measures used in our sample. We hypothesized that the existing factor structures of measures (which have mostly been based on samples comprised of healthy individuals) would fail to replicate in samples with BD and HC individuals. This assumption was based off prior samples with more limited age range and clinical presentation, restricting the distribution in responses and scores. First, we planned to confirm existing factor structures of measures. However, if these failed to replicate in our sample, we would perform split-half exploratory and confirmatory factor analyses for each measure to better understand the factor structure in a pandiagnostic sample (comprised of BD and HC). The second aim of our study was to examine measurement invariance in all the measures, to determine if factor structures were dimensional across diagnoses or if factor structures were specific to diagnosis (i.e., BD or HC). We hypothesized that measures would be invariant across BD and HC individuals. If the factor structures obtained were invariant across diagnostic groups, then we could be more confident that a dimensional approach like RDoC is valid (general dimensionality hypothesis). If the structure varies by diagnosis, such that the factors are different or have substantially different loadings in BD relative to HC, then dimensional approaches may be invalid (disease-specific hypothesis), or still be valid but require more nuanced approaches to measuring latent variables. In particular, the invariance test is critical because most RDoC studies may have difficulty testing invariance, with smaller sample sizes and more expensive deep phenotyping. These alternative approaches are not new, nor is the premise behind the RDoC initiative. However, these strategies are critically important in light of the many sources of evidence suggesting that dimensional and diagnostically non-specific markers are linked to shared risk factors. Finally, at a more exploratory level, the third aim of our study was to examine if measures would converge to capture a multi-measure inhibitory control construct for BD and HC individuals. We hypothesized, based on previous literature, that there would be weak convergence of self-report measures and neuropsychological measures of IC among individuals with BD and healthy controls (Burdick et al., 2005; Vasconcelos et al., 2014). Our goal in this paper is one of illustration and transparency, so additional analyses and strategies are included to frame possibilities and limitations for RDoC papers.

Methods

Participants

Participants included 150 healthy controls (HC), 309 individuals with BD Type I or Type II, and 147 individuals with BD-NOS (Not Otherwise Specified) or Schizoaffective BD who were recruited from the Heinz C. Prechter Longitudinal Study of BD at the University of Michigan (UM) to capture the full spectrum of bipolar illness and the range of impulsivity and executive functioning. HC were recruited using flyers and online postings, and were free of any past or current psychiatric or neurologic disorder, including current substance use disorders. Diagnoses were determined using the Diagnostic Interview for Genetic Studies (DIGS; (Nurnberger et al., 1994). All participants were administered the Hamilton Depression Rating Scale (HDRS) and the Young Mania Rating Scale (YMRS) by a study clinician to better characterize the sample. BD participants were included in this study while in the euthymic, depressed, or mixed state, but not a hypomanic state (there were too few individuals in the hypomanic state to consider the effects of manic symptoms on factor loadings/invariance test). Table 1 provides the demographic, clinical data, and results of group comparisons analyses for participants. Recruitment for this study was approved by Institutional Review Board at UM and written informed consent was obtained, beginning in 2003 and continuing through 2014. As such, the measures selected were not chosen specifically to match RDoC domains and subdomains (they were chosen in relation to potential intermediate phenotypes for bipolar disorder (Langenecker, Saunders, Kade, Ransom, & McInnis, 2010).

Table 1.

Demographics and Neuropsychological Performance

	HC	BD-I & BD-II	BD-NOS	Comparisons
	(n=150)	(n=309)	(n=147)
Demographics
Age*	32.80 (14.20)	39.50 (12.70)	38.70 (14.30)	HC < I & II, NOS
Gender (% female)	55%	63%	61%	ns
Education*	15.55 (2.17)	15.34 (2.24)	14.56 (2.17)	HC, I & II > NOS
Estimated IQ*	113.40 (11.66)	108.65 (12.34)	107.50 (12.49)	HC > I & II, NOS
HDRS*	1.11 (1.61)	8.75 (6.13)	6.75 (6.63)	HC < NOS < I & II
YMRS*	0.23 (0.55)	3.06 (3.86)	2.83 (3.88)	HC < I & II, NOS
Years ill	--	20.92 (12.77)	19.44 (13.14)	ns
	(n=97)	(n=140)	(n=61)
BIS-11 total score*	54.00 (9.42)	65.76 (13.86)	63.38 (11.99)	HC < I & II, NOS
	(n=84)	(n=138)	(n=61)
BDHI total score*	16.99 (7.74)	25.78 (11.98)	24.38 (12.56)	HC < I & II, NOS
	(n=90)	(n=131)	(n=64)
BGA-R total score*	3.00 (3.10)	9.12 (5.78)	9.28 (7.28)	HC < I & II, NOS
Neuropsychological Test Performance
	(n=150)	(n=309)	(n=147)
COWA-Category Fluency Total*	21.62 (5.00)	19.87 (5.97)	19.54 (6.06)	HC > I & II, NOS
COWA-Word Fluency Total*	43.88 (10.98)	40.39 (12.69)	39.29 (11.72)	HC > I & II, NOS
Digit Symbol-Total*	81.65 (14.90)	68.89 (15.83)	70.16 (16.88)	HC > I & II, NOS
Trails A-Time*	−24.83 (8.46)	−31.20 (11.97)	−30.90 (12.18)	HC < I & II, NOS
Trails B-Time*	−57.64 (24.47)	−78.47 (37.24)	−79.94 (36.83)	HC < I & II, NOS
Stroop-Word (T-score)*	48.17 (8.15)	45.85 (8.29)	45.84 (8.05)	HC > I & II, NOS
Stroop-Color (T-score)*	46.97 (7.62)	44.40 (8.23)	44.63 (8.58)	HC > I & II, NOS
Stroop-Interference (T-score)*	55.31 (7.74)	51.59 (7.27)	53.19 (8.17)	HC > NOS > I & II
WCST-Total Cards Correct	108.65 (11.91)	112.01 (12.36)	111.24 (12.45)	ns
PGNG-Mean Correct Hits*	35.82 (3.45)	34.25 (4.09)	33.32 (5.62)	HC > I & II > NOS
PGNG-Mean Correct Rejections*	8.32 (1.88)	7.96 (1.97)	7.56 (2.25)	HC > I & II > NOS
PGNG-Reaction Time*	−456.53 (46.24)	−485.41 (54.21)	−478.38 (60.04)	HC < I & II, NOS

Note. All values are means and standard deviations unless otherwise noted; HDRS, Hamilton Depression Rating Scale; YMRS, Young Mania Rating Scale; BIS-11, Barratt Impulsiveness Scale; BDHI, Buss-Durkee Hostility Inventory; BGA-R, Brown-Goodwin Aggression Scale- Revised; COWA, Controlled Oral Word Fluency; WCST, Wisconsin Card Sort Test; PGNG, Parametric Go/No-go Task; I = BD Type I, II = BD – Type II, NOS = BD Not Otherwise Specified;

^*p <.05.

Self-Report Measures

Barratt Impulsiveness Scale (BIS-11)

The BIS-11 is a 30-item self-report measure of executive functioning that asks individuals to rate frequency of actions or thoughts on a four-point ordinal scale. Previous factor analyses have found that the BIS-11 has six factors: Attention, Cognitive Instability, Motor, Perseverance, Self-Control, and Cognitive Complexity (Patton et al., 1995). A subset of the sample completed this self-report measure (see Table 1; α=.89)

Buss-Durkee Hostility Inventory (BDHI)

The BHDI is a 75-item self-report measure with true-false response designed to assess seven subscales of aggression: Assault, Indirect Hostility, Irritability, Negativisim, Resentment, Suspicion, and Verbal Hostility; and one subscale of Guilt (Buss & Durkee, 1957; Buss, Fischer, & Simmons, 1962). A subset of the sample completed this self-report measure (see Table 1; α=.92).

Brown-Goodwin Aggression Scale- Revised (BGA-R)

The BGA-R is an 11-item self-report measure assessing frequency of aggressive behaviors on a five-point ordinal scale across three stages of life (childhood, adolescence, and adulthood), for a total of 33 total measurements (Brown, Goodwin, Ballenger, Goyer, & Major, 1979). A subset of the sample completed this self-report measure (see Table 1; (α=.91).

Neuropsychological Tests (IQ estimate and Executive Functioning Measures)

Synonym Knowledge Task (SKT; based on Shipley Langenecker & Giordani, 2003))

Estimated premorbid verbal IQ was assessed with the SKT. Participants were presented with a word and then asked to choose most similar one of four additional words.

Controlled Oral Word Fluency (COWA; Benton, Hamsher, & Sivan, 1994)

The COWA assesses verbal phonemic fluency by requiring examinees to produce as many words as possible that begin with a given letter within three 60-second trials. The total number of words was the word fluency score.

Animal Naming (Benton et al., 1994)

This task measures semantic fluency by asking examinees to produce as many words as possible within a certain category (i.e., animals) during a 60-second trial. The total number of words served as the total animal naming score.

Wechsler Digit Symbol subtest (Wechsler, 1997)

In this task, participants were asked to write a symbol that corresponds with a number as quickly as possible within 120 seconds to assess processing speed. The total number of correct symbols served as the score.

Stroop Color Word Test (Stroop, 1935)

The Stroop Color Word Test is a measure of cognitive flexibility and processing speed. In this task participants were asked to name colors or read words across three trials. Scores for the Color trial and Word trial were the total number of correct responses for each trial, standardized to T-scores (α=.79). The Interference T-score was the difference between the total number of correct responses for the Color/Word trial and the predicted Color/Word score, based on each person’s Color trial and Word trail scores. Negative difference scores reflect poorer than predicted ability at suppressing word naming in favor of color naming, while positive difference scores indicate the opposite.

Trail Making Test (TMT; Battery, 1944)

The TMT assesses motor speed, visual scanning, sequencing, and attentional shifting abilities. Participants were asked to draw a line connecting 25 numbers and letters as quickly as possible over two trials. Time to completion served as the score for both trials, and was inverted for ease of interpretation (i.e., less time=better performance; α=.50).

Wisconsin Card Sort Test (WCST; Grant & Berg, 1948)

Examinees are asked to match cards to a set of stimuli based on examiner feedback. Scores derived from this test were created by multiplying the percent of correct cards by the total number of cards possible (128) to get an accuracy score for each participant (Total Cards).

Parametric Go/No-go Task (PGNG; Langenecker et al., 2005; Langenecker, Zubieta, Young, Akil, & Nielson, 2007; Votruba & Langenecker, 2013)

The PGNG is an eleven-minute task which measures attention (hits) and set-shifting, processing speed, and correct (rejections) and incorrect (commissions) responses to lure trials as a part of IC. The PGNG task consists of three separate levels, which were completed in order of ascending difficulty. Scores were computed for the average Correct Hits for Go items across all three levels of the task (α=.64), average Correct Rejections of No-go items across the two more difficult levels of the task (α=.42), and Go Response Time across all three levels of the task (α=.81).

Statistical Procedures

Confirmatory Factor Analysis (CFA) and Alternative Strategy if CFA failed

To assess the structure of the three self-report and the combined neuropsychological measures, the following analyses were performed, addressing aim 1. In Step 1, Confirmatory Factor Analysis (CFA) was conducted on each measure based on the original factor structure derived from the literature (Langenecker et al., 2010; Miyake et al., 2000; Patton et al., 1995). This was done to determine to what extent the factor structures proposed in the literature fit the data from the current sample. If Step 1 succeeded, the original factor structure was retained, and between group tests of invariance and differences were conducted (aim 2 below).

If the CFA failed for a given measure, a Step 2 was employed. In Step 2 for aim 1, the factor structure of each measure was further explored by dividing the sample into two random halves. Exploratory Factor Analysis (EFA) was conducted in the first half with geomin bi-factor rotation (Jennrich & Bentler, 2012), which has a general factor and a number of specific factors. The number of factors to be retained was determined using parallel analysis (Horn, 1965)¹, which compared eigenvalues from an EFA conducted on the original data to eigenvalues from EFAs conducted on 1,000 random datasets with the same properties of the original data. To avoid overextraction of factors, observed factors with eigenvalues greater than 95% of the eigenvalues from the randomly generated data were retained (Glorfeld, 1995). As a Step 3, CFA was conducted also in the first half of the sample to determine if the factor structure obtained through EFA would yield an acceptable model fit within the CFA framework. CFA models were created by allowing all indicators to load on the general factor of the bi-factor model and only one specific factor (i.e., no cross-loadings were allowed among specific factors). Initial assignment of items to specific factors was based on loadings higher than .3 in the EFA solutions. However, when initial model fit was lower than acceptable, modification indices were used to assign additional items to specific factors, but maintaining exclusion of cross-loadings among specific factors. After a final CFA solution was achieved in the first half of the sample, the Step 4 was cross-validation in the second half without further modifications. Step 5 (final step) for aim 1 was a CFA conducted in the whole sample to derive final model parameters to be reported. We note that there is disagreement about whether a whole sample EFA is superior to a split half EFA followed by a CFA. We chose the latter because it is unlikely that there will be a chance for another group to collect an independent sample to perform the CFA, especially of this size. Furthermore, any new sample would be less likely to include the same measures, as more specific RDoC measures are likely to be chosen.

Measurement Invariance and Between Group Comparisons

Measurement invariance testing was also performed on the whole sample to address aim 2. Multiple-group CFA (MGCFA) was used to test for measurement invariance across the BD, BD-NOS, and HC groups. This started by specifying a model in which loadings and intercepts (or thresholds), among factors were allowed to vary across groups. An acceptable model fit at this stage suggested that configural invariance (i.e., same overall factor structure) was found across groups. Then, loadings and intercepts (or thresholds) were sequentially constrained to be equal across groups to determine if weak (i.e., only metric invariance [loadings]) or strong (i.e., both metric and scalar invariance [loadings and intercepts or thresholds]), invariance was supported (Meredith, 1993). At least strong invariance is required to perform meaningful comparisons across groups, which is an important practical implication of a dimensional approach. In the strong models, the HC group was defined as the reference group in which factor means were fixed to zero, while they were freely estimated in the other groups. If factor means in other groups were significantly different from zero, this implied a significant difference with respect to the HC group. Given that in bi-factor models the variance of all factors is fixed to one, weak invariance was not tested in scales with ordered-categorical indicators (Muthén & Muthén, 2013). Level of invariance was determined through model comparison based on chi-square difference tests and CFI decrements lower than .02 (Meade, Johnson, & Braddy, 2008). Although it is generally expected that more restrictive models, such as the weak and strong models, will have poorer fit compared with the configural model, it is possible that fixing parameters could improve model fit when the groups do not differ much on the relevant parameters. In this case, the most parsimonious model should be retained, regardless of the statistical significance in the model comparison.

Exploratory, Supraordinate Factor Structures

Aim 3 was addressed in a set of steps to determine factor structures across all measures, whether self-report or neuropsychological performance based.

All analyses were conducted in MPlus 7 (Muthén & Muthén, 1998–2012) using the maximum likelihood (for neuropsychological measures) and robust (i.e., mean- and variance-adjusted) weighted least squares (WLSMV) estimator to estimate thresholds, given that indicators for the self-report measures are categorical and ordered-categorical. Three goodness-of-fit indices were used to evaluate the models, including the comparative fit index (CFI), the Tucker-Lewis index (TLI), and the root-mean-square error of approximation (RMSEA). Conventional cutoffs for these indices are 0.9 or higher indicating acceptable fit and 0.95 or higher indicating excellent fit for the CFI and the TLI, and .08 or lower indicating acceptable fit and .05 or lower indicating good fit for the RMSEA (Hu & Bentler, 1995).

Results

BIS-11

Estimation and determination of factor structure for BIS-11, aim 1

First, a CFA for the original factor structure of the BIS-11 (Patton et al., 1995) yielded improper solutions (i.e., correlations greater than 1 between higher order factors) in both the first random half of the sample and the full sample. Therefore, an EFA with parallel analysis in the first half of the sample was used to determine the factor structure, yielding a four-factor solution, which corresponded to one general factor and three specific factors (Table 2). The number of factors remained the same after excluding two items (“I make-up my mind quickly” and “I like puzzles”) that lack salient loadings on any factor (i.e., hyperplane items). A CFA cross-validated in the second half of the sample, with adequate fit to the data [χ²=639.5, 328 df, p<.001, RMSEA=.074 (90% CI=.065–.082), CFI=0.92, TLI=0.91], indicating that a robust solution was achieved. Finally, a CFA conducted in the full sample also indicated adequate fit to the data [χ²=1052.6, 328 df, p<.001, RMSEA=.080 (90% CI=.074–.085), CFI=0.92, TLI=0.90]. Standardized loadings for this final model are shown in Table 2. Results from the initial CFA with the first half of the sample can be found in the Supplement.

Table 2.

Confirmatory Factor Analysis for Barratt Impulsiveness Scale (BIS) Using Parallel Analysis

Item	General Factor	Lack of Planning Factor	Impulsive Decision-Making Factor	Hyperactivity Factor
I plan tasks carefully	0.70
I do things without thinking	0.69
I am happy-go-lucky	−0.38	0.15
I don’t “pay attention”	0.63
I have “racing” thoughts	0.60			0.29
I plan trips well ahead of time+	0.60
I am self controlled+	0.75
I concentrate easily+	0.81
I save regularly+	0.61	−0.06
I “squirm” at plays or lectures	0.50			0.69
I am a careful thinker+	0.82		−0.29
I plan for job security+	0.68	−0.11
I say things without thinking	0.65	0.41
I like to think about complex problems+	0.44	−0.48
I change jobs	0.31		0.54
I act “on impulse”	0.74	0.47
I get easily bored when solving thought problems	0.59
I act on the spur of the moment	0.69	0.53
I am a steady thinker+	0.74	−0.38
I change residences	0.16		0.65
I buy things on impulse	0.61		0.37
I can only think about one thing at a time	0.39
I change hobbies	0.37		0.47
I spend or charge more than I earn	0.65		0.17
I often have extraneus thoughts when thinking	0.51		0.36
I am more interested in the present than the future	0.33
I am restless at the theater or lectures	0.60			0.76
I am future oriented+	0.59	−0.30

Note.

⁺reverse scored item.

Invariance testing of disease-specific versus general dimension model fit in BIS-11, aim 2

Invariance testing across the BD, BD-NOS, and HC groups was performed in the full sample. The configural model showed adequate fit to the data [χ²=1666.0, 984 df, p<.001, RMSEA=.077 (90% CI=.071–.084), CFI=0.90, TLI=0.89], but the strong invariance model fit the data better [χ²=1793.6, 1174 df, p<.001, RMSEA=.067 (90% CI=.061–.074), CFI=0.91, TLI=0.91]. This difference in model fit was statistically significant (χ²=277.2, 190 df, p<.001), and the strong invariance model was retained, allowing for estimation of factor mean differences across groups (see Supplemental Table 1).

Between group comparison on final CFA factors for BIS-11

In both the BD and BD-NOS groups, only the general factor was significantly different from zero (and positive), implying significantly higher levels in overall impulsivity with respect to the HC group.

BDHI

Estimation and determination of factor structure for BDHI, aim 1

First, given that, to the best of our knowledge, no previous factor analyses of the BDHI have been published, we did not conduct an initial CFA of its original structure. So we moved to step two, an EFA using parallel analysis yielded a four-factor solution, corresponding to one general factor and three specific factors (Table 3). One item (“Once in a while I cannot control my urge to harm others”) that yielded empty cells in bivariate tables was excluded, together with four additional hyperplane items (“There are a number of people who seem to be jealous of me”, “I never play practical jokes”, “I get into fights about as often as the next person”, and “I generally cover up my poor opinion of others”), but the four-factor solution was retained. This solution was cross-validated in the second half of the sample, with adequate fit to the data [χ²=2600.1, 2306 df, p<.001, RMSEA=.027 (90% CI=.021–.032), CFI=0.91, TLI=0.90].The CFA conducted in the full sample also yielded adequate fit [χ²=2915.9, 2306 df, p<.001, RMSEA=.027 (90% CI=.024–.030), CFI=0.93, TLI=0.92]. Standardized loadings for this final model are shown in Table 3. Results from the initial CFA with the first half of the sample can be found in the Supplement.

Table 3.

Confirmatory Factor Analysis for Buss-Durkee Hostility Inventory (BDHI) Using Parallel Analysis

Items	General Factor	Expressive Aggression Factor	Victimization/ Resentfulness Factor	Rigidity Factor
I seldom strike back, even if someone hits me first+	0.44	0.65
I sometimes spread gossip about people I don’t like	0.39
Unless somebody asks me in a nice way, I won’t do what they want	0.47
I lose my temper easily but get over it quickly	0.45
I don’t seem to get what’s coming to me	0.50		−0.35
I know that people tend to talk about me behind my back	0.67
When I disapprove of my friends’ behavior, I let them know it	0.03		0.65
The few times I have cheated, I have suffered unbearable feelings of remorse	0.26			−0.31
I never get mad enough to throw things+	0.55	0.04
Sometimes people bother me just by being around	0.59
When someone makes a rule I don’t like I am tempted to break it	0.38
Other people always seem to get the breaks	0.71		−0.26
I tend to be on my guard with people who are somewhat more friendly than I expected	0.58
I often find myself disagreeing with people	0.60
I sometimes have bad thoughts which make me feel ashamed of myself	0.62
I can think of no good reason for ever hitting anyone+	0.34			0.69
When I am angry, I sometimes sulk	0.48	−0.44
When someone is bossy, I do the opposite of what he asks	0.39
I am irritated a great deal more than people are aware of	0.68	−0.26
I don’t know any people that I downright hate+	0.41
There are a number of people who seem to dislike me very much	0.67
I can’t help getting into arguments when people disagree with me	0.55		0.45
People who shirk on the job must feel very guilty	0.22			−0.09
If somebody hits me first, I let him have it	0.56	0.70
When I am mad, I sometimes slam doors	0.57	−0.05
I am always patient with others+	0.44
Occasionally when I am mad at someone, I will give him the “silent treatment”	0.29
When I look back on what’s happened to me, I can’t help feeling mildly resentful	0.76		−0.41
I demand that people respect my rights	0.21			0.32
It depresses me that I did not do more for my parents	0.53
Whoever insults me or my family is asking for a fight	0.60	0.37
It makes my blood boil to have somebody make fun of me	0.56			−0.10
When people are bossy, I take my time just to show them	0.41
Almost every week I see someone I dislike	0.57
I sometimes have the feeling that others are laughing at me	0.73
Even when my anger is aroused, I don’t use “strong language” +	0.48		0.34
I am concerned about being forgiven for my sins	0.42			−0.26
People who continually pester you are asking for a punch in the nose	0.53			0.54
I sometimes pout when I don’t get my own way	0.57	−0.20
If somebody annoys me, I am apt to tell him what I think of him	0.40		0.81
I often feel like a powder keg ready to explode	0.80
Although I don’t show it, I am sometimes eaten up with jealousy	0.64
My motto is “Never trust strangers”	0.55			0.04
When people yell at me, I yell back	0.56		0.35
I do many things that make me feel remorseful afterward	0.77			−0.19
When I really lose my temper, I am capable of slapping someone	0.52			0.32
Since the age of ten, I have never had a temper tantrum+	0.46
When I get mad, I say nasty things	0.72
I sometimes carry a chip on my shoulder	0.65
If I let people see the way I feel, I’d be considered a hard person to get along with	0.78
I commonly wonder what hidden reason another person may have for doing something nice for me	0.78
I could not put someone in his place, even if he needed it+	−0.05	0.67
Failure gives me feelings of remorse	0.53	−0.39
I can remember being so angry that I picked up the nearest thing and broke it	0.58
I often make threats I don’t really mean to carry out	0.83
I can’t help being a little rude to people I don’t like	0.66
At times I feel I get a raw deal out of life	0.70		−0.40
I used to think that most people told the truth but now I know otherwise	0.58		−0.16
When I do wrong, my conscience punishes me severely	0.33			−0.41
If I have to resort to physical violence to defend my rights, I will	0.31	0.55
If someone doesn’t treat me right, I don’t let it annoy me+	0.56
I have no enemies who really wish to harm me+	0.55
When arguing, I tend to raise my voice	0.48		0.21
I often feel that I have not lived the right kind of life	0.69
I have known people who pushed me so far that we came to blows	0.65	0.40
I don’t let a lot of unimportant things irritate me+	0.69	−0.20
I seldom feel that people are trying to anger or insult+	0.44
Lately, I have been kind of grouchy	0.59
I would rather concede a point than get into an argument about it+	0.11		0.50
I sometimes show my anger by banging on the table	0.57

Note.

⁺reverse scored item.

Invariance testing of disease-specific versus general dimension model fit in BDHI, aim 2

Invariance testing across the BD, BD-NOS, and HC groups was performed in the full sample. The configural model showed somewhat adequate fit to the data [χ²=7229.6, 6918 df, p<.01, RMSEA=.022 (90% CI=.013–.028), CFI=0.88, TLI=0.88], but the strong invariance model provided better fit to the data [χ²=7410.0, 7114 df, p<.001, RMSEA=.021 (90% CI=.012–.027), CFI=0.89, TLI=0.89]. This difference in model fit was not statistically significant (χ²=218.9, 196 df, p=.12), and the more parsimonious strong invariance model was retained.

Between group comparison on final CFA factors for BDHI

In both the BD and BD-NOS groups, only the general factor was significantly different from zero (and positive), implying significantly higher levels in overall hostility with respect to the HC group (see Supplemental Table 1).

BGA-R

Estimation and determination of factor structure for BGA-R, aim 1

First, given that, to the best of our knowledge, no previous factor analyses of the BGA-R have been published, we did not conduct an initial CFA of its original structure. So we moved to step two, EFA. EFA using parallel analysis yield a seven-factor solution, while EFA using model fit yielded a four-factor solution. However, CFA yielded improper solutions (i.e., correlations greater than 1 between factors) for both models and subsequent steps could not be conducted.

Neuropsychological Measures

Estimation and determination of factor structure for neuropsychological measures, aim 1

In step one, a CFA for the original factor structure of the neuropsychological measures (S. A. Langenecker et al., 2010; Miyake et al., 2000) failed to converge. For step two, EFA using both parallel analysis and model fit yielded the same two-factor solution, corresponding to one general factor and one specific factor comprised of Stroop measures. The number of factors remained the same after excluding two hyperplane indicators (PGNG-Mean Correct Rejections and WCST-Total Cards Correct). An initial CFA conducted in the first half of the sample yielded a model fit that was lower than acceptable, and modification indices suggested the inclusion of Stroop color in the factor defined by the other two Stroop measures. As this modification was theoretically reasonable, it was incorporated in the CFA for the first half of the sample, increasing model fit to an acceptable level.² This solution was cross-validated in the second half of the sample, with adequate fit to the data [χ²=90.3, 32 df, p<.001, RMSEA=.078 (90% CI=.059–.097), CFI=0.94, TLI=0.92]. A CFA conducted in the full sample also indicated adequate fit to the data [χ²=144.1, 32 df, p<.001, RMSEA=.076 (90% CI=.064–.089), CFI=0.94, TLI=0.92]. Standardized loadings for this final model are shown in Table 4. Results from the initial CFA with the first half of the sample can be found in the Supplement.

Table 4.

Confirmatory Factor Analysis for Neuropsychological Measures

Item	General Factor	Stroop Factor
COWA-Category Fluency Total	0.47
COWA-Word Fluency Total	0.53
Digit Symbol-Total	0.85
Trails B-Time	0.74
Trails A-Time	0.62
PGNG-Reaction Time	0.53
PGNG-Mean Correct Hits	0.56
Stroop-Word (T-score)	0.51	0.82
Stroop-Color (T-score)	0.65	0.40
Stroop-Interference (T-score)	0.41	−0.32

Note. COWA, Controlled Oral Word Fluency; PGNG, Parametric Go/No-go Task.

Invariance testing of disease-specific versus general dimension model fit in neuropsychological measures, aim 2

Invariance testing across the BD, BD-NOS, and HC groups was performed in the full sample. The configural model showed adequate fit to the data [χ²=164.3, 99 df, p<.001, RMSEA=.077 (90% CI=.055–.097), CFI=0.93, TLI=0.91], but the weak [χ²=184.3, 121 df, p<.001, RMSEA=.068 (90% CI=.048–.088), CFI=0.94, TLI=0.93] and strong invariance models fit the data better [χ²=213.1, 137 df, p<.001, RMSEA=.070 (90% CI=.051–.088), CFI=0.92, TLI=0.92]. The difference in model fit comparing the configural model with the weak and strong invariance models was not statistically significant (χ²=19.9, 22 df, p=.59 and χ²=48.8, 38 df, p=.11, respectively). In addition, the difference between the weak and strong invariance models was statistically significant (χ²=28.8, 16 df, p<.05), but the CFI decrement was .013, lower than the .02 cutoff, so that the strong invariance model was still retained, allowing for estimation of factor mean differences across groups (see Supplemental Table 1).

Between group comparison on final CFA factors for neuropsychological measures

In both the BD and BD-NOS groups, only the general factor was significantly different from zero (and negative), implying significantly poorer overall performance in the neuropsychology measures with respect to the HC group.

Integrating All Measures to Examine Convergence (Exploratory, Supraordinate Factors), Aim 3

Different attempts were made at exploring the factor structure and patterns of associations of all measures together. First, an attempt was made to conduct EFA and CFA in all items for all measures in the full sample, as an alternative test of aim 1. Initial EFA found seven factors including a separate factor comprised of neuropsychological measures. However, one of the factors did not include any items with highest loadings on that factor, and the corresponding CFA did not converge (see Supplemental Table 2).

Next, EFA with geomin (not bi-factor) rotation was conducted using factor scores derived from CFAs of the different measures (that is, factor scores from the full sample CFAs above), yielding a three-factor solution (Table 5) with good model fit [χ²=19.97, 7 df, p=.006, RMSEA=.055 (90% CI=.003–.084), CFI=0.98, TLI=0.92]. Note, though, that the specific factor (Stroop) from the neuropsychological measures was excluded from the solution set due to lack of salient loadings on any EFA factor. The Expressive Aggression factor of the BDHI was excluded from the solution set due to a negative residual variance. Standardized loadings are shown in Table 5.

Table 5.

Exploratory Factor Analysis Results of Factors Derived from Confirmatory Factor Analysis for Each Measure

	All General Factors	BIS Factors	BDHI Factors
BDHI–General Factor	0.85
BIS–General Factor	0.73
Neuropsychological–General Factor	−0.40
BIS-Lack of Planning Factor		0.73
BIS-Impulsive Decision-Making Factor		0.90
BIS-Hyperactivity Factor		0.34
BDHI-Victimization/Resentfulness Factor			0.59
BDHI-Rigidity Factor			0.96

Note. BDHI, Buss-Durkee Hostility Inventory; BIS, Barratt Impulsiveness Scale. Neuropsychology Stroop Factor did not have any significant loadings in the EFA.

Finally, the correlations among factor scores of the general factors were calculated. Although the BIS and BDHI general factors exhibit the strongest correlation (Table 6), correlations between these and the Neuropsychological Measures general factor were also statistically significant and moderate in magnitude with the BIS and the BDHI general factors. These relationships are negative as lower scores for neuropsychological measures is indicative of worse IC, the inverse of the questionnaire scale factors.

Table 6.

Correlations of All Factors with Clinical Measures in BD

	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15
1. HDRS
2. YMRS	0.35*
3. All General Factors-Factor	0.47*	0.24*
4. BIS Factors-Factor	0.04	0.08	0.09
5. BDHI Factors-Factor	−0.13	0.002	−0.14*	0.19*
6. Neuropsychological–General Factor	−0.19*	−0.16*	−0.29*	0.19*	−0.01
7. Neuropsychological-Stroop Factor	−0.10	−0.06	−0.12	−0.01	−0.03	0.08
8. BIS–General Factor	0.40*	0.20*	0.87*	0.00	−0.22*	−0.17*	−0.07
9. BIS-Lack of Planning Factor	0.00	0.14*	0.07	0.69*	0.16*	0.13*	−0.01	0.02
10. BIS-Impulsive Decision-Making Factor	0.03	0.06	0.09	0.99*	0.18*	0.19*	−0.01	−0.01	0.65*
11. BIS-Hyperactivity Factor	0.06	0.13	0.02	0.28*	0.02	0.17*	0.02	−0.06	0.18*	0.26*
12. BDHI–General Factor	0.45*	0.21*	0.90*	0.17*	0.02	−0.19*	−0.14*	0.58*	0.11	0.16*	0.10
13. BDHI-Expressive Aggression Factor	−0.06	0.02	−0.05	0.12	0.81*	−0.14*	−0.01	−.154*	0.10	0.12	−0.03	0.07
14. BDHI-Victimization/Resentfulness Factor	−0.11	0.00	−0.09	0.14*	0.95*	−0.01	−0.05	−0.16*	0.13*	0.13*	0.02	0.03	0.64*
15. BDHI-Rigidity Factor	−0.09	0.03	−0.06	0.15*	0.79*	−0.08	0.01	−0.17*	0.10	0.15*	0.01	0.08	0.90*	0.55*

Note. HDRS, Hamilton Depression Rating Scale; YMRS, Young Mania Rating Scale; BDHI, Buss-Durkee Hostility Inventory; BIS, Barratt Impulsiveness Scale; higher scores for Neuropsychological General Factor indicates better performance, but higher scores for self-report factors suggests worse inhibitory control;

^*p <.05

Correlations Between Factors and Clinical Measures in the Bipolar Sample

As shown in Table 6, only the general factors from each measure and the All General Factors-Factor (from EFA of all factor scores) were significantly related to symptoms of depression (HDRS) and symptoms of mania (YMRS). Specifically, the All General Factors factor, the BIS-General Factor, and the BDHI-General Factor were positively associated with symptoms of depression and mania, while the Neuropsychological-General Factor was negatively associated with symptoms of depression and mania (see above for interpretation of inverse correlation).

Discussion

We first examined the factor structure of IC measures and whether there was invariance across diagnostic groups for each measure. We then tested the convergence between neuropsychological and self-report measures of IC among individuals with BD and HC. There was very modest convergence of items from neuropsychological tests and self-report measures of IC, including the BIS-11 and the BDHI. In fact, most self-report items tended to hang together within each self-report measure, while the neuropsychological measures also tended to be more closely related with each other. This is in line with previous studies that have found weak convergence between neuropsychological measures and self-report questionnaires (Duckworth & Kern, 2011; Meyer et al., 2001), suggesting method variance plays a large role in these factors. However, when the convergence of factor scores from each measure was examined, there was slightly more overlap. Specifically, the general factors from the bi-factor analysis of each measure (BIS-11, BDHI, and neuropsychological measures, respectively) were strongly related, while the specific factors from each measure were not related. These findings indicate that each measure may be capturing a general IC construct, or a shared amount of variance. Analyses like bi-factor analysis can be used in order to extract this latent construct from the unique, measure specific constructs that may disrupt the convergence of these measures on an item level; therefore allowing us to better understand how each system or domain contributes to underlying factors or mechanisms related to psychiatric illness in a reliable and valid manner. Strategies for dimensional scale development like this in RDoC can strengthen dimensional comparisons to better identify shared and unique aspects of psychiatric illness (Ryan et al., 2015).

The role that method variance seems to have in our results is an important consideration for future studies, especially those attempting to use multiple units of analysis in a manner consist with RDoC. While the “All General Factors” factor in Table 5 from the overall EFA of the subfactors seems to be capturing a latent IC factor across measures, the other two factors capture distinctive elements of each self-report measure, perhaps restricted by method variance. The “BIS Factors” factor captures elements of poor planning, impulsive decision-making, and hyperactivity, all IC constructs that can be found in BD, but can also be seen in other psychiatric illnesses like Attention-Deficit Hyperactivity Disorder (ADHD). On the other hand, the “BDHI Factors” factor captures elements of victimization and/or resentfulness, as well as rigidity, and would not generally be considered to be IC constructs.

Therefore, it is possible that the “All General Factors” factor captures the most meaningful information, especially in light of the fact that this factor showed group differences, while the other two factors may represent nuisance variability or factors less germane to disease. Indeed, “All General Factors” was significantly associated with symptoms of depression and mania in the BD sample (as were the general factors from each measure), while “BIS Factors” and “BDHI Factors” were not (nor were the specific factors from each measure). These “nuisance” factors may not help to measure psychiatric illness due to the fact that the constructs they capture can be present in many different populations and also that they are constrained by measure-specific variance that does not seem to converge with other units of analysis, at least those that were measured in the current study.

Alternatively, the second and third factors from a given measure may be more likely to capture residual variance of less clinical significance – method variance rather than construct variance. For example, the second factor for neuropsychological measures appears to be a word reading speed factor that includes a negatively loading on interference. While the Interference T score loads onto the main neuropsychological factor as expected, and in the expected direction, this secondary factor appears to have limited relevance to inhibition or impulsivity. Indeed, it did not relate to any of the clinical scales, and to the majority of the factor scores within the BD sample.

Additionally, we found that the same construct was measured equally well across groups and the factor structures obtained were invariant across HC and BD groups to address aim 2 – for each of the scales that had a satisfactory factor solution. Therefore, the equal variance in both groups is a promising indicator that a general dimensional approach like RDoC can be used across clinical and non-clinical groups and across different units of analysis. This suggests that for IC, at least, there is not great disease specificity in the way that factor scores are derived, but rather where on the dimension of poor impulse control a given person may fall. This supports the utility of a dimensional RDoC approach to understand normal to abnormal variations along a continuum. For example, individuals with and without BD may have the same score on an IC construct, but may differ on other constructs (e.g., declarative memory, social communication) or domains (e.g., negative valence systems, positive valence systems), and the overall profiles of these individuals can help to determine their diagnoses and level of severity (Langenecker et al., 2014). However, on an individual level, an individual’s score on an IC construct can help inform prevention and/or treatment targets.

Furthermore, we found different factor structures for each measure than has been found in previous studies (aim 1). Specifically, we were not able to replicate the factor structure of the BIS-11 (Patton et al., 1995) or the factor structure of neuropsychological measures (Langenecker et al., 2010; Miyake et al., 2000), notably with a much larger and more heterogeneous sample. In addition, we used state of the art strategies for identifying latent variables and shared variance, which, when combined with a larger and more heterogeneous sample, is likely to result in a more reliable and replicable structure. For example, it is important to note that the BIS factors were found on a sample of undergraduate healthy controls, which may limit the original factors’ generalizability in a more diverse sample including larger age ranges and clinical populations with more variability in responses, like the sample in the current study. Indeed, other studies have found significant correlations between the BIS and neuropsychological measures of IC in healthy adults (Enticott et al., 2006), but this may not be the case in diverse samples with clinical populations (Vasconcelos et al., 2014). In addition, we modified a few of the original measures used in the factor structure of neuropsychological measures (PGNG-Mean Correct Hits, PGNG-Mean Correct Rejections, WCST-Total Cards Correct), to increase variability, as the original measures (PGNG-Attention Accuracy, PGNG-IC Accuracy, WCST-correct responses, WCST-perseverative errors, respectively) were percentage variables, which would need to be handled as count variables with an offset term (Nussbaum, Elsadat, & Khago, 2008). We did not attempt to replicate the original subscales of the BDHI, as construct validity of the seven original subscales on the BDHI has been found to be less than optimal (Biaggio, Supplee, & Curtis, 1981) and other studies with the BDHI have been inconsistent (Buss & Perry, 1992; Spielberger, Jacobs, Crane, & Russell, 1983). We were also unable to find a factor solution for the BGA-R, indicating it may not be a strong measure of IC in clinical and non-clinical populations.

Our results have several implications for utilizing RDoC approaches in future studies. First, units of analyses may only converge weakly across modalities, making it important for future studies to examine both convergence and granularity in units of analysis, especially if multiple modalities for analysis are used (Cuthbert & Kozak, 2013; Sanislow et al., 2010). Our results underscore the importance of using analyses like bi-factor analyses to capture latent constructs across different units of measurement and specific constructs for each unit of measurement that may be crucial in refining intermediate phenotypes within domains, thus helping us to better understand psychiatric illness. Further, it is important for studies to examine invariance, to ensure that the same construct is being measured equally well across clinical and non-clinical populations. If invariance does not exist, then approaches like RDoC may not be valid to use or may require more nuanced approaches to measuring latent variables. It is important to note that the whole sample CFA may be an overly generous assessment of the potential fit in other samples, so potential for generalization should be taken only from model fit in the second half. Finally, factor analysis may tend to overgeneralize, or lump data together in ways that are contradictory to the intent of RDoC, where subdomains are intended to be distinct, correlated factors.

The current study has several strengths, including a large sample size, and a wide range of representation of disease states and variability of performance on IC; maximizing our ability to find meaningful latent factors of IC. However, the study also has limitations, including the fact that the sample and measures were of convenience; the study was designed before RDoC was conceptualized and did not include key biological measures that might strengthen the factor analysis strategies that could be employed. There was also missing data for some self-report measures (see Table 1), which may have diminished our abilities to find more powerful relationships between variables. In addition, the recommended sample size for EFA has been defined in terms of absolute number of cases (e.g., a minimum of 100 or 250), subjects to variables ratio (SVR), level of communalities of the variables, and variables to factors ratio (overdetermination of the factors; MacCallum, Widaman, Zhang, & Hong, 1999). While all of these criteria were generally met or exceeded in the current study, the SVR is considerably lower in the individual item CFA of all measures, so it is possible that this model failed to converge because of sample size. One related concern is that group differences existed for the main bifactor for each measure where the CFA converged (BIS, BDHI, neuropsychological measures), but not for the second and third factors. This could reflect a lack of power with sample size, weakness with the bifactor model overall in detecting stable and meaningful second and third factors. Additional, future analyses should attend to these potential concerns.

Moreover, some of the loadings for the CFA fell below traditional thresholds for inclusion, which may have been due to the fact that the final CFA was conducted in another sample (i.e., the second half of the sample). The data is available through the Heinz C. Prechter Repository (http://www.prechterfund.org/bipolar-research/repository/) with data use agreements, for those interested in alternative or comparative analytic strategies. For example, some items did not load at all on any factors, such as the “happy go lucky” item on the BIS. We note that even the author of the BIS-11 has suggested that at times item loadings are inconsistent (Patton et al., 1995), and that differing factor analytic strategies were employed in prior work – PCA vs bifactor analysis. Furthermore, we add that the Stroop items load differently on the main neuropsychological factor in Table 4 relative to the second factor. In the second factor, Word T has a strong positive loading, whereas Color T and Interference T have modest loadings, and the Interference T is negative. This factor was subsequently excluded due to low loadings in the CFA (Table 5), and notably does not correlate with any relevant clinical factor or most other factors (Table 6). It seems likely that this is a residual variance factor more related to reading and of little clinical relevance to Bipolar Disorder or to executive functioning or impulsivity. Furthermore, a number of items on the BDHI have very low factor loadings on the main factor, or any subfactor. This suggests that some items might be excluded from future iterations of the instrument (e.g., “people who shirk on the job…”, “Occasionally when I am mad…, silent treatment”). Alternatively, these items might need to be reworded to better represent contemporary linguistic norms. Of course, movements toward shortened forms and computer adaptive testing might effectively exclude these lower loading items.

Overall, the structure of the subcomponent scales did not match pre-existing factor structures for measures that had data-driven scales, it also did not match scales that were theoretically based. While potentially confusing, these could reflect the broader age range in our sample (including equivalence in age and education across HC and BD samples). This could also reflect the newer factor analytic techniques differing from older principle components analysis techniques. Thus, the final factor solutions should be interpreted with caution. Overall, it is important to note that the intent of our analysis was to follow a data driven approach like RDoC, and to show all scales. Future studies should use information in this work to subselect critical scales and may want to exclude scales/subscales with very low loadings. Furthermore, RDoC is somewhat predicated on the hypothesis that neural circuit patterns will be a critical medium by which connections between genes that convey risk and disease expression may be made, and we did not have these measures.

In conclusion, we found that there was very modest convergence of items from neuropsychological measures and self-report measures of IC among HC and BD; however, when examining the convergence of factor scores from each measure a general IC construct across measures emerged. These shared and specific constructs may be crucial in identifying subtypes of diseases, as well as capturing and modeling non-linearity, helping us to better understand disease states and traits. Importantly, invariance testing indicated that the same construct was measured equally well across groups, validating the use of a dimensional approach across clinical and non-clinical groups in this sample, as proposed by RDoC. Future studies will implement bi-factor analyses and invariance testing across several units of analyses and psychiatric disorders to better understand the underlying mechanisms that contribute to psychiatric illness.