The Barratt Impulsiveness Scale - 11: Reassessment of its Structure in a Community Sample

Abstract

The Barratt Impulsiveness Scale Version 11 (BIS-11; Patton, Stanford & Barratt, 1995) is a gold-standard measure that has been influential in shaping current theories of impulse control, and has played a key role in studies of impulsivity and its biological, psychological, and behavioral correlates. Psychometric research on the structure of the BIS-11, however, has been scant. We therefore applied exploratory and confirmatory factor analyses to data collected using the BIS-11 in a community sample (N = 691). Our goal was to test four theories of the BIS-11 structure: (a) a unidimensional model; (b) a six correlated first-order factor model, (c) a three second-order factor model, and (d) a bifactor model. Among the problems identified were: (a) low or near-zero correlations of some items with others; (b) highly redundant content of numerous item pairs; (c) items with salient cross-loadings in multidimensional solutions; and ultimately; (d) poor fit to confirmatory models. We conclude that use of the BIS-11 total score as reflecting individual differences on a common dimension of impulsivity presents challenges in interpretation. Also, the theory that the BIS-11 measures three subdomains of impulsivity (attention, motor, and non-planning) was not empirically supported. A two-factor model is offered as an alternative multidimensional structural representation.

Despite much debate regarding the structure and measurement of impulsivity, research on this topic continues to thrive. There are many self-report measures of impulsivity and constructs related to impulse-control; however, the Barratt Impulsiveness Scale Version 11 (BIS-11; Patton, Stanford, & Barratt, 1995) appears to be the gold-standard self-report instrument in this domain, and it has played a major role in research (see Ireland & Archer, 2008; Stanford et al., 2009). Numerous recent studies have used the BIS-11 to explore the social consequences and behavioral correlates of individual differences in impulsivity (Carlson, Johnson, & Jacobs, 2010; Kjome et al., 2010; Piero, 2010; Sweitzer, Allen, & Kaut, 2008) as well as the biological and genetic origins of impulsivity (Benko et al., 2010; Kaladjian, Jeanningros, Azorin, Anton, & Mazzola-Pomietto, 2010; Lee et al., 2009; Stoltenberg & Nag, 2010).

Despite the extensive use of the BIS-11, psychometric research on its internal structure has been relatively scant, especially in large community samples. The primary goal of the present investigation, therefore, was to apply exploratory and confirmatory factor analyses to BIS-11 item response data collected in a community sample (N = 691). Our objective was to test specific, well-known theories of the BIS-11structure, which form the basis of the aggregate scale and subscale scores that are currently used in research (Patton et al., 1995; Stanford et al., 2009).

Specifically, we used exploratory and confirmatory unidimensional and bifactor modeling to evaluate the degree to which the BIS-11 total score reflects a single common latent trait of impulsivity. We also used exploratory and confirmatory factor analyses to evaluate the theory that BIS-11 responses reflect six correlated first-order constructs (attention, motor, self-control, planfullness, cognitive complexity, perseverance, cognitive instability), which in turn, form three second-order factors (attention, motor, and non-planning). Prior to further description of these analyses, the next section reviews the development of the BIS-11.

The BIS-11

In their comprehensive review, Stanford et al. (2009) noted that the BIS, in various versions, had been used in impulsivity-related research for over 50 years. The BIS-11 was designed to be a “multifaceted” measure of trait impulsivity. An original goal in developing the family of Barratt measures was to produce a unidimensional instrument, providing scores that would be relatively uncorrelated with self-reported anxiety and sensation/thrill-seeking measures (Barratt, 1965, 1972). In more recent versions, such as the BIS-10 (Barratt, 1985), however, the emphasis shifted to developing item content that would reflect Barratt’s theory that there are three major subtraits of impulsivity: motor, cognitive, and non-planning.

The current 30-item BIS-11 (Patton et al., 1995) was derived from psychometric analyses of the 34-item BIS-10 responses of 412 undergraduate students, 164 substance abuse patients, 84 general psychiatry patients, and 73 male prison inmates. After deleting four items due to poor psychometric properties (e.g., low item-total correlations), principal components analyses suggested six correlated first-order components. In Table 1, we have listed the items by these subdomains, which are: 1) attention, “focusing on current tasks”; 2) cognitive instability, “intruding thoughts”; 3) motor impulsiveness, “acting quickly”; 4) perseverance, “stable lifestyle”; 5) cognitive complexity, “enjoys mental challenges”; and 6) self-control, “plans and thinks deliberatively”.

Table 1.

BIS-11 Items , Proposed Subdomain Assignment, Item Means, and Item-Test Correlations^a

Item No.	Abbreviated Item Content	Subdomain (First-Order Factor)	Higher-Order Factor	Item Mean	Item- Test rb
5*	don’t pay attention	Attention	Attentional	1.60	.44
9*	concentrate easily	Attention	Attentional	2.14	.50
11	squirm at plays or lectures	Attention	Attentional	1.69	.35
20	Am a steady thinker	Attention	Attentional	2.06	.46
28	am restless at the theater	Attention	Attentional	1.69	.45
6	have racing thoughts	Cog Instability	Attentional	1.69	.37
24	change hobbies	Cog Instability	Attentional	1.72	.29
26	have extraneous thoughts	Cog Instability	Attentional	1.56	.39
2*	do things without thinking	Motor	Motor	1.97	.50
3	make up my mind quickly	Motor	Motor	1.70	−.03
4	am happy-go-lucky	Motor	Motor	2.37	.05
17	act on impulse	Motor	Motor	2.68	.64
19*	act on spur of the moment	Motor	Motor	1.84	.58
22	buy things on impulse	Motor	Motor	2.00	.41
25	spend more than earn	Motor	Motor	1.65	.39
16	change jobs	Perseverance	Motor	1.59	.29
21	change residences	Perseverance	Motor	1.94	.31
23	think about only one thing	Perseverance	Motor	1.75	.18
30	am future oriented	Perseverance	Motor	1.74	.35
10	save regularly	Cog Complexity	Non-Planning	2.20	.39
15	like to think about problems	Cog Complexity	Non-Planning	2.50	.14
18	bored solving problems	Cog Complexity	Non-Planning	2.36	.43
27	interested in present	Cog Complexity	Non-Planning	1.64	.23
29	like puzzles	Cog Complexity	Non-Planning	2.24	.18
1*	plan tasks carefully	Self-Control	Non-Planning	2.47	.41
7	plan trips ahead of time	Self-Control	Non-Planning	2.14	.40
8*	am self-controlled	Self-Control	Non-Planning	2.21	.44
12*	am a careful thinker	Self-Control	Non-Planning	1.80	.49
13	plan for job security	Self-Control	Non-Planning	1.86	.47
14*	say things without thinking	Self-Control	Non-Planning	2.38	.51

Note.

^*Items from Brief-BIS.

^aData are from a sample of 691 participants.

^bItem-Test r is the item-test correlation with the item under consideration removed from the total score.

In turn, mostly consistent with Barratt’s (1985) three-factor theory of the structure of impulsivity, a principal components analysis of the correlations among the six first-order components resulted in a solution described by three second-order factors with two first-order components loading on each second-order component: attention and cognitive instability defined attentional impulsiveness, motor and perseverance defined motor impulsiveness, and self-control and cognitive complexity formed non-planning impulsiveness (see Table 1).

Although many investigators who have used the BIS instruments have ignored the subscale structure, Stanford et al. (2009) argued that if item-response data indeed are separable into subdomains, it may be crucial to report subscale scores. Indeed, in recent years investigators have routinely reported BIS-11 total scores and their correlates, as well as subscale scores and their correlates derived from the second-order components analyses described above. Subscale correlates based on the proposed six first-order components are rarely reported.

Prior Psychometric Studies

Despite 1,548 citations to Patton et al. (1995) as of May 2012 (Google Scholar, n.d.), research specifically focused on testing the proposed six first-order or three second-order factor structure of the BIS-11 is rare, especially confirmatory factor analytic research in community samples. Most relevant to the present study are two recent confirmatory factor analytic investigations of forensic patient samples, and one recent study that was based on item-response theory and conducted in a college-student population.

Haden and Shiva (2009) recently conducted confirmatory analyses of four alternative structural models based on a sample of male mentally ill forensic patients. These researchers argued that a two correlated factors (r = .24) model, based on a subset of 24 BIS-11 items, provided the best fit. They termed these factors motor and non-planning impulsivity. Ireland and Archer (2008) also conducted confirmatory analyses based on data collected in samples of male and female prison inmates. They were also not successful in confirming either a unidimensional or the proposed three-factor structure for the BIS-11. However, note that they did not specifically test a three-factor second-order model, rather they tested a three correlated factors model at the item level.

Most recently, Steinberg, Sharp, Stanford, and Tharp (in press) applied a bifactor item response theory model to a large sample (n = 1,178) of BIS-11 responses collected from a sample of undergraduate students. This bifactor model specified that all 30 BIS-11 items discriminate on a single general factor (reflecting impulsivity) and one of three group factors reflecting Barratt’s three hypothesized subdomains (see Table 1). Based on inspection of the item discrimination parameters (analogous to factor loadings) on the general factor, the authors argued that many BIS-11 items are poor measures of a common impulsivity dimension. They therefore proposed an 8-item, unidimensional alternative, called the BIS-Brief (Note: we have placed asterisks next to BIS-Brief items in Table 1). The authors argued that their item response theory analysis provided no support for scoring or interpreting the three subscales commonly reported in the literature and advocated in Stanford et al. (2009).

Present Investigation

Given the scant research on the latent structure of BIS-11 item responses, the goals of this investigation were to explore the validity of the hypothesized first- and second-order factor structure, and assess the degree to which multidimensionality affects the interpretation of BIS-11 scores as reflecting a single impulsivity dimension. To accomplish these objectives, exploratory and confirmatory factor analyses were applied to data collected as part of the UCLA Consortium for Neuropsychiatric Phenomics (CNP), an ongoing collaborative study of the genetic and environmental bases of variation in psychological and neural system phenotypes (Bilder et al., 2009).

Method

Participants

The sample was composed of 691 healthy adults. These individuals all passed eligibility and screening requirements and completed the BIS-11 in the CNP project. At the time of analysis, 1,000 self-identified Caucasian and/or Hispanic individuals, 21–50 years of age, had been recruited from the local community using flyers, Internet postings (e.g., Craigslist.org), and community presentations by investigators (e.g., in public libraries and to church groups). Recruitment criteria restricted ethnicity to two groups to reduce problems related to ethnic influences on genetic analyses in the larger study. Relevant demographic characteristics of the sample were as follows: gender was 53% male, 47% female; ethnicity was 43% Hispanic, 57% non-Hispanic; and education was 36% with high school degree (2% had less), 52% with bachelor’s degree (8% had more).

There were another 309 participants who consented for the overarching CNP project, but did not undergo BIS-11 testing. Of those, 11.0% were lost to follow-up, and the others were excluded on the basis of diagnostic criteria (70.9%), demographics (10.4%), sensory or communication problems (7.1%), and 0.6% other (e.g., examiner error). The diagnostic criteria that led to exclusion of the 219 participants were based on: (a) an ADHD questionnaire (16.5%), (b) an ADHD interview (5.5%); (c) medical or psychiatric conditions (e.g., diagnosis of an Axis-I disorder using the Structured Clinical Interview for DSM Disorders) (41.8%); (d) psychoactive drugs prescribed (2.6%), and (e) a positive urine screen for drug abuse 4.5%. There were no individuals who started the BIS-11 who did not complete it.

Procedure

All candidates participated in telephone screening followed by an on-site structured clinical interview and self-report symptom questionnaires after giving written informed consent, as approved by the UCLA Office of Protection for Research Subjects. The BIS-11 was completed as part of a larger assessment battery that included more than 50 neuropsychological tests, counterbalanced to control for order effects and spread out over several sessions. The BIS-11 was self-administered via computer, with the responses automatically recorded in a database. Participants were each compensated $15/hr for their participation.

Analytic Plan

The BIS-11 contains 30 items that are self-rated on a scale of “1” to “4”: 1) rarely/never, 2) occasionally, 3) often, and 4) almost always. First, basic psychometric analyses were performed for the entire scale and subscales using the psych library (Revelle, 2012) available in the R software package (R Software Development Core, 2012). These included analyses of response frequencies, item and scale means and standard deviations, item-test correlations (corrected for overlap by eliminating the item under consideration from the total score), and coefficient alpha internal consistency reliability estimates. In addition, a hierarchical clustering algorithm iclust (see Revelle, 1979; Schalet, Durbin & Revelle, 2011), which is available from the psych library, was used to specify three- and six-cluster solutions. The resulting graphs were used to provide preliminary evidence of whether the items group together in a manner that is consistent with the proposed three- and six-factor structures.

We then considered several alternative representations of the latent structure of the BIS-11 item responses: (a) unidimensional, (b) a bifactor structure with a single general factor and six group factors, (c) six correlated factors, and (d) a second-order model with six first-order factors and three (correlated) second-order factors. In turn, two analytic methods were used, namely, exploratory and confirmatory factor analysis. Pearson correlations were used for all exploratory analyses. It was not possible to estimate polychoric correlations in the data because there were few responses in extreme categories.

The psych library (Revelle, 2012) in R (R Development Core, 2012), using minres extraction and promax rotations (for multidimensional solutions), was used for all exploratory factor analyses. These analyses included a Schmid-Leiman bifactor rotation (SL; Schmid & Leiman, 1957) using the schmid command. This exploratory model allows evaluation of the relative strength of the general factor, as well as estimation of the degree to which variance in raw scores can be attributed to a single common factor through computation of coefficient omega hierarchical (ω_H; McDonald, 1999; Revelle & Zinbarg, 2009; Zinbarg et al., 2005).

For each model, we also estimated a confirmatory factor analysis (CFA) model using EQS software (Bentler, 2006). For each model, items were treated as continuous and robust maximum likelihood estimation based on the covariance matrix was used. Fit was judged using robust versions of commonly used indices, such as the comparative fit index (CFI), root mean square error of approximation (RMSEA), standardized root mean residual (SRMR), and the Satorra-Bentler chi-square (SB ). Hu and Bentler (1999) recommended benchmarks of RMSEA ≤ .06, SRMR ≤ .08, and CFI ≥ .95, as indicating good model fit.

Confirmatory Models Estimated

A unidimensional CFA model was specified by allowing all items to load onto a single factor and fixing the variance of the latent variable to 1.0 for identification. This is an important model because many investigators have found significant correlations between the BIS-11 total score and myriad criterion variables (see Stanford et al., 2009, for a review). Thus, a critical question centers on the degree to which the 30 BIS-11 items reflect a single common latent variable (i.e., the common trait of impulsivity) rather than a composite of unequally weighted smaller content dimensions. Evaluating the fit and estimated factor loadings of a unidimensional model, relative to these same values in a bifactor model (described below), can aid in addressing this question (Reise, Morizot & Hays, 2007).

The unidimensional model described above is highly restricted and likely only fits item-response data from measures with highly homogeneous content (see Reise, Moore, & Haviland, 2010, for discussion). When a measure is proposed to assess a common latent variable, but construct-relevant multidimensionality is present due to clusters of items reflecting diverse trait manifestations a bifactor model may be a more plausible alternative (Reise, in press; Thomas, 2011). Therefore, the second CFA model considered was a bifactor structure where each BIS-11 item was allowed to load onto a general factor. In addition, each BIS-11 item was allowed to load onto one of six orthogonal group factors. Estimated loadings on group factors were determined by first-order content classifications (see Table 1). This model was identified by fixing all factor variances to 1.0 and specifying that all factors were orthogonal.

In the third CFA model, we evaluated the six correlated first-order factors model. For this model, six first-order factors were defined with each item loading onto only one factor according to theory (Table 1). Correlations among the factors were freely estimated and the model was identified by setting all factor variances to 1.0.

Finally, a second-order model was estimated by specifying each item to load onto one of six first-order factors according to theory (a loading for one item per factor was set to 1.0 for identification). A disturbance (residual variance for the latent factor) term was also specified for each first-order factor. In turn, three second-order factors were specified with two first-order factors allowed to load onto each second-order factor according to theory (see Table 1). The three second-order factors were allowed to correlate freely.

Results

Basic Descriptive Psychometrics

The item-test correlations suggested that the BIS-11 items varied greatly in their relation to the aggregate total score (Table 1). Items 17 (acts on impulse) and 19 (acts on the spur of the moment) had very strong item-test correlations. On the other hand, there were eight items with item-test correlations below .30, suggesting that they are only marginally related to what is being evaluated by the aggregate score. Items 3 (make up my mind quickly) and 4 (happy-go-lucky) were essentially unrelated to the other items. Coefficient alpha for the total score was .80, and the average item inter-correlation was .13, suggesting that the common variance among the items was weak. The average total raw score was 59.18 (sd = 9.54), which is consistent with other reports in the literature (e.g., Stanford et al., 2009). Finally, correlations within the first-order and second-order scales (Table 2), as well as the reliability estimates and descriptive statistics were, for the most part, highly consistent with those reported for the Stanford et al. (2009) combined college student and healthy adult sample.

Table 2.

Scale Level Descriptive Statistics and Scale Inter-Correlations in the Present Data and Comparable Results From Stanford et al. (2009)

	First-Order
	Present Data (N = 691)			Stanford et al. (2009; N = 1,577)
Sub-Scale	Mean	SD	Alpha	Mean	SD	Alpha
ATT	9.2	2.4	.65	10.4	2.9	.72
CI	5.3	1.7	.54	6.4	1.9	.55
MOT	13.8	3.1	.61	15.0	3.2	.64
PER	7.6	1.9	.32	6.9	1.8	.27
CC	12.0	3.0	.65	11.5	2.6	.48
SC	11.2	2.5	.40	12.1	3.3	.72
	Second-Order
Sub-Scale	Present Data			Stanford et al.
ATT	14.4	3.5	.70	16.7	4.1	.74
MOT	21.5	4.0	.60	22.0	4.0	.59
NP	23.3	4.6	.67	23.6	4.9	.72
	aFirst-Order
Sub-Scale	ATT	CI	MOT	PER	CC	SC
ATT	-	.41	.34	.25	.36	.51
CI	.39	-	.27	.08	.02	.22
MOT	.28	.35	-	.19	.27	.41
PER	.23	.26	.24	-	.30	.31
CC	.47	.22	.37	.38	-	.39
SC	.30	.03	.21	.29	.41	-
	aSecond-Order
Sub-Scale	ATT	MOT	NP
ATT	-	.39	.45
MOT	.42	-	.50
NP	.40	.47	-

Note. Correlations below the diagonal are the present results; correlations above the diagonal are from Stanford et al. (2009). ATT = Attention; CI = Cognitive Instability; MOT = Motor; PER = Perseverance; CC = Cognitive Complexity; SC = Self-Control; NP = Non-Planning; SD = Standard Deviation.

Hierarchical Clustering

Figures 1 and 2 display the results of iclust specifying three and six final clusters, respectively. The numbers in the figures represent correlations, either between two items that are joined to form a cluster, between two clusters, or between a cluster of items and a joined item (see http://personality-project.org/r/r.ICLUST.html) The patterns of item clustering in either figure were not consistent with the proposed three second-order domains or the six first-order domains. When the number of final clusters was set to three, our expectation was that items would first group according to first-level domain, and then form three final clusters consisting of two sub-clusters each. Instead, items that were highly similar in content formed the first 11 clusters (see circles C1 through C11 in Figure 1). These “doublets” then joined to form four intermediate clusters, of which three ultimately joined into a single grouping. When the number of final clusters was set to six, our expectation was that the six clusters would align with the proposed first-order subdomains. However, again the first 11 clusters were item pairs (see circles C1 through C11 in Figure 2). Ultimately, one large cluster of 17 items from different first-order content domains was created, one four-item grouping of Cognitive Complexity items, and four stand-alone doublet or triplet clusters.

Figure 1.

Hierarchical iclust Analysis Specifying Three Clusters

Figure 2.

Hierarchical iclust Analysis Specifying Six Clusters

Unidimensional Model

Estimated loadings for a minres exploratory factor analysis extracting a single factor, and the estimated loadings for a maximum likelihood unidimensional CFA solution differed only trivially, thus only the latter is shown in the first column of Table 3. In either solution, the first factor accounted for 15% of the total item variance. Overall, these results parallel the item-test correlation results described in Table 1. Specifically: (a) items 3 and 4 had loadings near zero; (b) items displayed a very wide range of loadings with numerous items having small (< .30) loadings; and (c) items 9 (concentrates easily), 2 (does things without thinking), 17 (acts on impulse), 19 (acts on spur of moment), and 14 (says things without thinking) had the highest loadings. The content of the higher-loading items appeared to reflect self-descriptions of acting and thinking without careful deliberation, sustained attention, concentration, or self-control.

Table 3.

Unidimensional Solution, Exploratory SL Rotation, and Confirmatory Bifactor Model

		Schmid-Leiman Bifactor							Confirmatory Bifactor
Item	U	Gen	Gr1	Gr2	Gr3	Gr4	Gr5	Gr6	Gen	Gr1	Gr2	Gr3	Gr4	Gr5	Gr6
5*	.45	.34	.33	.14	.00	.01	−.03	−.01	.46	.03
9*	.51	.48	.36	.05	.12	.14	.00	−.24	.55	.15
11	.34	.33	−.04	.14	.60	.02	−.10	−.01	.27	.70
20	.46	.42	.38	−.08	−.02	.10	.02	−.01	.51	−.01
28	.41	.42	−.04	.11	.69	.02	−.04	−.01	.36	.75
6	.41	.28	.11	.45	.13	−.08	.04	−.03	.35		.44
24	.30	.17	.09	.30	−.06	.02	.04	.11	.25		.17
26	.41	.29	.06	.52	.12	−.08	.11	−.13	.36		.73
2*	.52	.37	.36	.13	−.07	−.01	.01	.23	.50			.18
3	.04	−.07	−.02	−.08	−.01	−.12	−.03	.40	−.12			.31
4	.06	−.03	−.01	.17	−.03	.01	−.10	.31	−.04			.35
17	.65	.44	.30	.41	−.01	.03	−.04	.53	.57			.58
19*	.58	.38	.25	.35	.00	.12	−.08	.60	.47			.67
22	.40	.33	.10	.20	−.04	−.15	.33	.09	.37			.16
25	.37	.35	.02	.16	−.08	−.17	.53	−.09	.37			.03
16	.27	.15	−.12	.42	.05	.14	.11	.09	.22				.47
21	.29	.16	−.04	.43	.01	.11	.07	.11	.23				.88
23	.15	.16	.08	−.11	.10	−.09	.08	−.01	.17				−.02
30	.29	.36	.00	−.28	.08	.27	.31	.04	.35				.12
10	.36	.37	.03	.05	−.10	.10	.47	−.13	.39					−.06
15	.11	.20	.30	−.50	.00	−.06	.04	−.05	.17					.47
18	.41	.36	.24	−.05	.20	−.07	−.01	.12	.41					.36
27	.19	.19	−.12	−.09	.13	.05	.22	.28	.19					−.04
29	.14	.19	.09	−.26	.12	.02	.07	−.04	.17					.47
1*	.40	.35	.25	−.01	−.05	.48	−.07	.02	.43						.49
7	.39	.34	.09	.08	.03	.61	−.05	.03	.38						.64
8*	.45	.40	.28	.00	−.04	.17	.09	−.05	.49						.12
12*	.48	.44	.49	−.15	−.12	.10	.01	.04	.53						.12
13	.43	.38	−.06	.17	.01	.36	.30	.06	.42						.20
14*	.53	.39	.32	.16	.04	−.14	.04	.24	.55						−.26

Note.

^*Items from Brief-BIS. U indicates unidimensional model, GEN is the general factor and GR1…GR6 are group factors.

The fit of the unidimensional CFA model was SB chi-square = 2,443.1 (df = 405, p < .01), CFI = .49, RMSEA = .09, and SRMR = .09. These values were all below conventional benchmarks for adequate fit. Inspection of modification indices revealed that a major source of poor fit was due to numerous correlated errors, especially between the doublets identified earlier in the iclust figures. As a follow-up analysis, to explore whether a recently proposed subset of BIS-11 items formed a unidimensional scale, we fit a unidimensional CFA model to the BIS-Brief (Steinberg et al., in press). BIS-Brief items are denoted by an asterick in Table 3 (items 1, 2, 5, 8, 9, 12, 14, and 19). The fit of the unidimensional CFA model was SB chi-square = 112.09 (df = 20, p < .01), CFI = .87, RMSEA = .08, and SRMR = .06. These fit values are greatly improved for this 8-item model relative to the 30-item model, with the caveat that CFI remains below conventional fit benchmarks.

Bifactor Model

Table 3 also displays the results for an exploratory bifactor model derived from a Schmid-Leiman (SL) transformation and a confirmatory bifactor model. In either solution, the general factor loadings for many items (e.g., items 17 and 19) were relatively smaller than their counterparts in the unidimensional model. This occurred because in the unidimensional solution, loadings were biased upwards due to multidimensionality.

The SL pattern of group factor loadings provided little support for the a priori theorized six-content domains in the BIS-11. Specifically, many items failed to load onto their expected group factor, and several items had salient cross loadings, defined as loadings greater than .30 on more than one group factor. As would be expected, confirmatory results rejected this model as inadequate; the fit of the bifactor CFA model was SB chi-square = 1,498.1 (df = 375, p < .01), CFI = .72, RMSEA = .07, and SRMR = .08. These latter fit values bordered on being acceptable, but the CFI was well below even the most liberal of proposed benchmarks.

Finally, a valid ω_H estimate could not be calculated from the confirmatory solution because the model displayed a poor fit and the group factors were not meaningfully interpretable. However, it is appropriate to estimate ω_H from the unrestricted exploratory SL solution (McDonald, 1999). Therefore, ω_H based on the SL solution was estimated to be .61, indicating that 61% of raw score variation on the BIS-11 total scores could be attributed to a single common factor, ostensibly impulsivity. Given that coefficient alpha was previously estimated to be .80, 19% of the reliable variance in raw scores can be attributed to secondary common factors beyond the general impulsivity factor.

Six-Factor Model

Table 4 presents the factor loadings from the exploratory and confirmatory six factor oblique solutions. Consistent with the SL model results, the exploratory six factor structure was also problematic in terms of the a priori theory. In particular, (a) items 6, 4, 22, 23, 25 and 27 had no salient (> .30) loadings on any factor; (b) several items, such as 17 and 19, had salient cross-loadings; and (c) two of the factors were marked primarily by item content doublets (Factors 3 and 6). The confirmatory results appeared more promising (see Factors 1, 2, and 6), but close inspection revealed that the remaining three factors reflected item doublets rather than an interpretable latent variable running among a psychologically homogeneous cluster of items. The loadings of items 17 and 19 on Factor 3, or of items 16 and 21 on Factor 4, exemplified this phenomenon. Moreover, the fit of the correlated-factors model was not acceptable: SB chi-square = 1,948.2 (df = 390, p < .01), CFI = .61, RMSEA = .08, and SRMR = .09.

Table 4.

Six-Factor Explatory solutions with Promax Rotations and Confirmatory Model

		Exploratory						Confirmatory
Item	SUB	1	2	3	4	5	6	1	2	3	4	5	6
5	Att	.44	−.11	−.06	−.07	.00	.00	.48
9	Att	.55	.00	−.21	.05	−.27	.01	.68
11	Att	−.09	−.17	−.73	.02	−.04	−.13	.43
20	Att	.57	.09	.03	.06	−.02	.04	.50
28	Att	−.08	−.15	−.79	.06	−.04	−.08	.51
6	CI	.11	−.41	−.23	−.14	−.03	.11		.64
24	CI	.12	−.30	.06	−.01	.12	.07		.34
26	CI	.05	−.46	−.22	−.12	−.12	.19		.66
2	Mot	.49	−.12	.07	−.06	.25	.05			.49
3	Mot	−.08	.02	.07	−.06	.41	−.04			.10
4	Mot	−.02	−.22	.08	−.03	.30	−.11			.18
17	Mot	.38	−.46	.00	.00	.53	−.03			.84
19	Mot	.32	−.42	.02	.09	.58	−.11			.75
22	Mot	.11	−.17	.08	−.05	.10	.46			.41
25	Mot	−.01	−.09	.09	.02	−.08	.71			.31
16	Per	−.18	−.46	−.10	.22	.10	.04				.65
21	Per	−.07	−.46	−.05	.12	.12	.03				.71
23	Per	.05	.13	−.19	.00	.02	.05				.04
30	Per	.01	.25	−.12	.54	.06	.16				.01
10	CC	.07	−.01	.15	.28	−.13	.54					.30
15	CC	.40	.53	−.01	.00	−.04	.02					.46
18	CC	.27	.06	−.31	−.02	.14	−.05					.49
27	CC	−.22	.04	−.16	.30	.31	.12					.13
29	CC	.10	.26	−.20	.13	−.02	.00					.38
1	SC	.47	−.03	.10	.39	−.02	−.17						.52
7	SC	.28	−.15	.02	.55	−.02	−.18						.48
8	SC	.43	.01	.03	.17	−.06	.09						.52
12	SC	.73	.18	.16	.04	.03	.03						.60
13	SC	−.04	−.22	−.03	.56	.06	.18						.43
14	SC	.36	−.14	−.11	−.13	.27	.07						.44

Note. Att = Attention; CI = Cognitive Instability; Mot = Motor; Per = Perseverance; CC = Cognitive Complexity; SC = Self-Control; SUB = Sub- Scale.

Second-Order Model

Because the six correlated-factors model did not fit the data, we will not report on the results of the second-order model. Moreover, inspection of Table 2 revealed that in neither the CNP data nor in Stanford et al. (2009), are the correlations among subscale scores consistent with the proposed second-order model. For example, Attention was most highly correlated with Cognitive Complexity in the present data, whereas according to theory, it should have been most highly correlated with Cognitive Instability. Likewise, in the Stanford et al. (2009) data, Attention was most highly correlated with Self-Control, not Cognitive Instability.

An Alternative Two-Factor Representation of the BIS

Item-level factor analyses provided no support for the proposed BIS-11 first- or second-order latent structures. We considered the problem of several items having cross-loadings on multiple factors in exploratory solutions, making it difficult to identify the number of major dimensions in the data matrix correctly. When such cross-loadings are set to zero in confirmatory modeling, the fit is inevitably harmed. By far the largest concern, however, was that the BIS-11 contains numerous item doublets and triplets. Although the practice of including items with the same psychological theme phrased in slightly different ways increases the psychometric virtues of a measure (by increasing the inter-item correlations and thus internal consistency), it wreaks havoc on factor or correlational analyses because it makes it impossible to separate common from specific item variance (see McDonald, 1999).

To address this issue, we aggregated BIS-11 items that are near replicates, and then analyzed the resulting item parcels. We used the previously described iclust results (Figures 1 and 2) as an empirical guide to forming content-homogeneous parcels. This procedure resulted in the creation of 11 parcels (see Table 5, top portion). These 11 parcels included all BIS-11 items except items 3 (I make up my mind quickly) and 4 (I am happy-go-lucky). As noted, these items had near-zero loadings in the unidimensional solution and likely do not belong to the same domain as the rest of the BIS-11 items.

Table 5.

Eleven BIS-11 Parcels (top), and One- and Two-Factor Solutions for the BIS-11 Parcels (bottom)

	Parcel	Items Belonging to Parcel
1	Acts impulsively	19	17
2	Not planful	1	7
3	Can’t sit still	11	28
4	Lives in the moment	13	30	27
5	Changes, moves around	21	16	24
6	Extraneous/Racing thoughts	26	6
7	No concentration/self-control	8	9
8	Buying and spending sprees	10	25	22
9	Not a steady thinker	20	12
10	No cognitive mediation	14	2	5
11	Likes complicated things	15	29	18	23
Factor Models
	Parcel	Unidimensional		1a	2
1	Acts impulsively	.61			.53
2	Not planful	.47		.56
3	Can’t sit still	.38			.41
4	Lives in the moment	.41		.41
5	Changes, moves around	.33			.52
6	Extraneous/Racing thoughts	.44			.80
7	No concentration/self-control	.61		.62
8	Buying and spending sprees	.46		.28	.24
9	Not a steady thinker	.57		.76
10	No cognitive mediation	.66		.37	.39
11	Likes complicated things	.31		.49

Note. Items with loadings less than .20 were deleted.

^aCorrelation between Factor 1 and Factor 2 in two-factor solution was .55.

Scree plot analyses of the 11 parcels were inconclusive and indicated that anywhere between one and four factors were needed to account for the common variance. However, promax rotated factor solutions in three and four dimensions were not interpretable and appeared to represent specification of too many factors (e.g., only one or two parcels with a salient loading on a factor). Therefore in Table 5, we present only the one- and two-dimensional (promax rotated) solutions. In the unidimensional solution, acts impulsively (items 19 and 17), no self-control/concentration (items 8 and 9), not a steady thinker (items 12 and 20), and no cognitive mediation (items 14, 2 and 5), parcels have loadings greater than .50, and appear to dominate the factor. A unidimensional CFA model fit to only these four parcels resulted in: normal-theory = 49.7 (df = 2, p < .01), CFI = .91, RMSEA = .19, and SRMR = .06.

When expanding the solution to two correlated factors, however, interpretable dimensions resulted. The first dimension was marked by not a steady thinker (items 20 and 12), no self-control/concentration (items 8 and 9), and not planful (items 1 and 7) parcels. The second dimension was marked by extraneous racing thoughts (items 26 and 6), acts impulsively (items 19 and 17), and changes, moves around (items 21, 16, and 24). The correlation between these dimensions was .55. When a two-factor CFA model was fit to only the three highest loading parcels for each dimension, the correlation among the factors was estimated to be .48, and the resulting fit was: normal-theory χ² = 46.8 (df = 8, p < .01), CFI = .94, RMSEA = .08, and SRMR = .04.

Discussion

The overarching goal of this research was to test four theories of the BIS-11 structure, each defined by a different model: (a) a unidimensional model; (b) a six correlated first-order factor model, (c) a three second-order factor model, and (d) a bifactor model. Such exploration of latent structure can inform theory and is critically important for the valid interpretation of total and subscale scores derived from an instrument. In the following sections, we consider the results of this assessment and their ramifications for the interpretation of scores derived from the BIS-11 and for theories of impulsivity.

Interpreting BIS-11 Total Scores

We first consider interpretation of the BIS-11 total score as reflecting a single construct. In considering this question, it is critical to note the distinction between unidimensionality (existence of one and only one common factor) and the ability to scale individuals on a single dimension. These properties are related, but they answer slightly different questions. When item-response data are strictly unidimensional, scores can be interpreted unambiguously as indicators of a single, common dimension. When item responses are multidimensional, determining the degree to which the total test scores can scale individuals precisely on a single common dimension requires further analyses as reviewed below.

As for the former, confirmatory factor analyses indicated that BIS-11 responses cannot be explained on the basis of one and only one common factor chiefly due to the presence of content doublets as well as other systematic factors. When we considered a shortened version (Brief-BIS) proposed in Steinberg et al. (in press), which eliminated items that provided little discrimination on a general factor and items that were overly redundant, the fit to a unidimensional model was much improved. We therefore recommend that researchers, seeking a univocal measure of impulsivity, consider a brief version.

On the other hand, a failure to meet strict unidimensionality criteria does not necessarily negate the possibility of interpreting total scores as reflecting a common impulsivity dimension. It has been argued that to evaluate the interpretability of a composite score in the presence of item-response multidimensionality, one should estimate statistical indices based on a bifactor structural model (e.g., Gignac, Palmer & Stough, 2007; Gustafsson & Aberg-Bengtsson, 2010; Mohlman & Zinbarg, 2000; Reise, Moore & Haviland, 2010). A specific recommendation was computation of (McDonald, 1999), which can be interpreted as an indicator of general factor saturation, or as an estimator of the percent of total score variance due to a general latent factor.

In the BIS-11 data analyzed here, estimated on the basis of an SL exploratory bifactor model was .61, indicating the BIS-11 total score variance due to variance on a single common dimension (the general factor in the bifactor model) was approximately 61%. The remaining 39% was attributable to additional common dimensions (19%) and random error (20% based on the coefficient alpha estimate). One possible conclusion from the finding that most of the variance (61%) in total BIS-11 scores reflects a singe latent dimension is that prior research that has used the 30-item BIS-11 total score is indeed interpretable because the influence of other systematic factors/dimensions is small (19%). It is important to recognize, however, that BIS-11 scores are made difficult to interpret substantively because they are influenced by multiple sources of systematic variance.

Interpreting BIS-11 Subscales

CNP BIS-11 item responses are clearly multidimensional, but neither exploratory nor confirmatory factor analyses provided any support for the proposed six first-order or three second-order multidimensional structures. In fact, it appears that factors estimated on the BIS-11 are better interpreted as doublets than as valid latent factors that explain the covariance among a set of homogeneous items. Thus, our results provided no support for scoring the BIS-11 instrument by the three proposed subscales, and indicated that the subscale scores are not interpretable meaningfully as indicators of an underlying latent variable or psychological construct (see also Steinberg et al., 2011 for similar conclusions).

To obtain a clearer picture of the BIS-11 structure in the present data, we joined content-similar items into parcels and then conducted modeling on the parcels. Our results supported a model with two correlated factors, composed of three parcels each. Factor one reflected mostly individual differences in cognitive impulsivity - attentional control, concentration, careful and deliberate thinking, planning. Factor two reflected mostly individual differences in behavioral impulsivity (with some cognitive elements) – acts impulsively, changes jobs, moves residences relatively often, and a scattered quick-paced cognitive tempo (extraneous or racing thoughts). We note, however, that an alternative interpretation is that these factors are “method” factors reflecting items that are phrased in terms of constraint (factor one) and items that are phrased in terms of impulsivity (factor two). Further research is required to flesh out the meaning of these factors. Nevertheless, for researchers interested in either using the BIS-11in structural equation modeling research, or in raw scoring subscales, we recommend use of the three parcels with the highest loadings in the two-factor model confirmed in the present study.

Conclusion and Implications for Theory

The BIS-11 was originally developed with Barratt’s three-subdomain theory in mind (attention, motor, non-planning) and the virtues of recognizing the proposed multidimensional structure of the BIS-11, and scoring subscales accordingly, has been championed in Stanford et al. (2009). The results of the present study, as well as those of Steinberg et al. (in press), provide no support for the theory that BIS-11 can be partitioned meaningfully into three subdomains that reflect the three constructs proposed by Barratt. Our two-factor parcel-based solution is highly consistent with Haden and Shiva’s (2009, p. 201) two-factor confirmatory solution, and the item content is consistent with the proposal of Swann, Bjork, Moeller, and Dougherty (2002), that there are two distinct conceptualizations or types of impulsivity ─ onereflecting inability to wait for a reward, and another reflecting a rapid response style. Of course, psychometric analysis can only address the structure of data derived from a particular instrument and does not directly evaluate any theory of the true nature of a psycho-biologically based construct.

In conclusion, for those who have not yet collected data, but desire to study the conceptually broad construct of impulsivity, we suggest that alternative measures, such as the I₇ (Eysenck, Pearson, Easting, & Allsopp, 1985) and the Multidimensional Personality Questionnaire (Tellegen & Waller, 2007), be considered. Those measures have undergone more rigorous psychometric evaluation than the BIS 11, and have held up very well. Indeed, in linking psychological constructs, such as impulsivity, to their biological and/or sociological origins, interpretable, structurally valid measures are needed. Without them, it is not possible to know exactly what is being measured with a particular instrument, and what is being measured differs from what similarly named measures assess. In other words, if the latent variables are not specified correctly, explorations of the relationships among impulsivity-related constructs and important criterion variables are seriously flawed.