Psychometric properties of the Barkley Deficits in Executive Functioning Scale: A Spanish-language version in a community sample of Puerto Rican adults

Abstract

Performance-based measures have shown some limitation in the assessment of Executive Functions (EF) and rating scales have been proposed as an alternative. Our aim was to conduct a comprehensive psychometric evaluation of the Barkley Deficits in Executive Functioning Scale (BDEFS), as administered in 452 Latino community adults (65.5% female). The BDEFS was back-translated into Spanish. We performed exploratory factor analysis (EFA) to assess the structure of the translated BDEFS and to compare it with the original 5-factor structure based on the English language version. Confirmatory Factor Analysis (CFA) was performed to test the original language structure of the instrument, and also a modified version with items that loaded equally in both versions. The Adult Self-Report Scale (ASRS) was used to screen for ADHD symptoms. We assessed invariance on the latent factor’s mean by age and gender, and to estimate associations with ADHD symptom dimensions. The 5-factor structure of the BDEFS was partially supported by EFA/CFA, in which 78 out of 89 items loaded similar to the original English language structure. Factor scores were significantly associated with ADHD symptom dimensions. Model-based contrasts revealed that inattention was primarily associated with disorganization, time-management and motivational aspects of EF; hyperactivity was predominantly related to self-restraint and self-regulation factors. The BDEFS seemingly assesses similar dimensions of the EF construct in English and in the present Spanish language versions. Factor scores were differentially associated with ADHD subtypes. Replication and confirmation of the Spanish language BDEFS in a larger sample is advised.

Attention deficit hyperactivity disorder (ADHD) is a neurodevelopmental disorder associated with deficits in executive functioning (EF) (Barkley, 1997; Barkley & Murphy, 2010; Brown, 2006; Pennington & Ozonoff, 1996). EF deficits are associated with ADHD, as well as other psychiatric and neurological disorders (Frazier, Demaree, & Youngstrom, 2004; Willcutt, Doyle, Nigg, Faraone, & Pennington, 2005). Both ADHD and EF deficits often persist across the lifespan (Faraone, Biederman, & Mick, 2006; Miyake & Friedman, 2012) and may cause significant impairment in adults (Barkley & Fischer, 2011). Therefore, the assessment of EF impairments in adults with ADHD has important clinical and research implications.

Understanding executive functioning is also relevant because it is a core component of self-control/self-regulation and has broader impact across the lifespan. Early life self-control is associated with several adult outcomes in areas including education, health, wealth and crime (Mischel et al., 2011; Moffitt et al., 2011). Performance-based measures (i.e., cognitive tasks), and more recently rating scales have been proposed to assess EF in adults. Most of these rating scales have been almost exclusively developed and available in English.

There is a need for Spanish-language measures for clinical and research use. The Pew Research Center forecasts that the U.S. will be the country with the largest Spanish-speaking population in the world by 2050, and currently Spanish is the most spoken non-English language in U.S. homes, with 37.6 million speakers (Passel & Cohn, 2008). Clinicians and researchers working with Spanish-speaking populations need tools that are in the primary language of the person assessed and whose scores provide valid, reliable indicators of the construct measured (American Educational Research Association, American Psychological Association, & National Council on Measurement in Education, 2014). This in turn will impact the decisions, treatment plans and the provision of services derived from their assessment.

Although some self-report measures of EF have been developed for adults and have been translated into Spanish, such as the BRIEF and DEX (Garcia Fernandez, Gonzalez-Pienda, Rodriguez Perez, Alvarez Garcia, & Alvarez Perez, 2014; Pedrero-Pérez et al., 2011), research on the psychometric properties including factorial structure of these Spanish language versions is limited. There is a gap in the literature of EF measures in adults compared with what is available for children (e.g., BRIEF; Garcia-Fernandez et al. 2011). The DEX is the only adult Spanish-language scale of EF with known published psychometric properties.

In the present study we examine the psychometric properties of a Spanish-language version of the Barkley Deficits in Executive Functioning Scale (BDEFS; Barkley, 2011). We translated the BDEFS into Spanish and conducted exploratory factor analysis (EFA) on the Spanish version. We compare results of EFA from the Spanish-language BDEFS items with the preliminary PCA reported in the scale manual for the original English language version. We then aimed to confirm the factor structure initially reported for the English language version. To further examine the validity of BDEFS scores, we relate the CFA-derived BDEFS factor scores to ADHD symptom dimensions of inattention and hyperactivity/impulsivity.

Our working hypothesis presupposes that the Spanish translation of the BDEFS would have a similar factor structure to the original scale developed in English. We have not identified literature indicating that cultural variations influence the presentation, development, correlates, or outcomes of executive functioning in adults. For example, research such as that by Rey, Feldman, Rivas-Vazquez, Levin, and Benton (1999) does not indicate differences in performance on neuropsychological EF measures between Hispanics and non-Hispanics, or between Hispanic sub-groups such as Puerto Rican and Mexican. Although some differences have been reported in self-assessment of functional status between Hispanics and non-Hispanics (Tomaszewski-Farias, Mungas, Reed, Haan, & Jagust, 2004), authors report that cognitive and imaging predictors of cognitive decline were comparable across groups. Conversely, research on bilingualism has implications for the study of executive functioning, as the “bilingual advantage” literature suggests. For example, Bialystok, Craik, Green, and Gollan (2009) summarize research findings that point to advantages in bilingual persons over monolinguals such as enhanced executive control functions. They also cite preliminary evidence that executive control advantages in turn, may mitigate cognitive decline and contribute to cognitive reserve. Findings from these studies may have implications for EF research with Hispanic individuals.

Nevertheless, certainly not all Hispanic Spanish speakers are bilingual, and this applies to island Puerto Ricans, the population on which the present article is based. Puerto Rico is a U.S. territory located in the Caribbean, with a 98.9% Hispanic/Latino population. According to the U.S. Census, while only 5% of Puerto Rican residents are monolingual English speakers, 95% speak Spanish (U.S. Census Bureau, 2013). Nevertheless, only around 20% of the Puerto Rican population report speaking English at functional levels. Despite English being taught in the state school system from the elementary level, it is inaccurate to state that Puerto Rico is a bilingual territory, when 80% of the population is not bilingual. In this context, we had no a priori reason to expect differences in the factor structure of the Spanish- vs. English-language versions of the BDEFS. There are no population studies in Puerto Rico documenting the prevalence of adult ADHD, but the prevalence in children is high, between 7.5–8% (Bauermeister et al., 2007; Canino et al., 2004). Meta-analytic rates worldwide estimate the adult prevalence at about 5.3% (Polanczyk, de Lima, Horta, Biederman, & Rohde, 2007), and in the US it has been estimated at 4.4% (Kessler et al., 2006).

BDEFS’s formulation for executive functioning

Although numerous definitions of EF are normally used in the literature, there is no official consensus (Barkley, 2011; Jurado & Rosselli, 2007). Naglieri and Goldstein (2014) describe EF as the efficiency with which people acquire knowledge and solve problems across areas that include attention, emotion regulation, flexibility, inhibitory control, initiation, organization, planning, self-monitoring and working memory. The BDEFS is based on the proposed formulation of executive functioning as an ‘extended phenotype’, in which EF is described as a “meta construct”, referring to self-directed actions needed to choose goals and to create, enact and sustain actions toward those goals, or simply put, as self-regulation to achieve goals. Taking an evolutionary perspective, Barkley (2001) argues for an “evolutionary, adaptationist stance” to EF (p. 26). This perspective emphasizes functional aspects of executive functions, and proposes that EF evolved the ability to self-regulate to facilitate biological adaptation to solve problems of a social nature.

Barkley’s theory describes EF as self-regulation directed towards the future; it indicates that EF involves goal-directed actions, and that it includes at least 5 self-directed activities that develop in a process akin to classic internalization of speech processes (Vygotsky, 1962). These include: self-inhibition, self-directed sensory motor action, self-directed private speech, self-directed emotion/motivation, and self-directed reconstitution/play (Barkley, 2001; Barkley, 2011). This conception of EF as a self-directed set of actions intended to alter a future outcome underlies the construction of the BDEFS, and its five subscales reflect these (Barkley, 2011, pp. 9–11). Executive functions are hierarchically organized in five levels of increasing complexity of functioning. An important part of this formulation is that at each level more complex abilities and skills are needed, which are essential to self-regulation across time. These levels include: the pre-executive, the instrumental-cognitive, the methodical-self-reliant, tactical-interactive, and strategic-cooperative.

The pre-executive level refers to various neurocognitive abilities necessary to create an initial level of EF, such as sensory-motor, visuospatial and language abilities. In the instrumental-cognitive level of EF, pre-executive processes become self-directed and internalized for self-control. Performance-based (neuropsychological) EF tests are thought to reflect this type of functioning. The third, methodical-self-reliant level of EF reflects the use of instrumental EFs to solve problems related to survival and welfare on a day-to-day basis. Next is the tactical-interactive level of EF, comprising abilities used in daily social interactions and the accomplishment of short-term (hours to days) goals related to social reciprocity, trade and social skills. These tactical interactive abilities, in turn, are necessary for the strategic-cooperative level of EF, the fifth and last proposed by the theory. The strategic-cooperative level involves the attainment of longer-term (weeks to months) goals in domains such as educational, work, cohabitating, child rearing, financial management, driving and community. These goals span longer time periods, involve larger domains of social collaboration, and require more complex behaviors and interactions with more people. According to this formulation (Barkley, 2012), rating scale measures of EF reflect this level of functioning, and this explains their low relation to performance-based tests. This conception of EF levels seems to agree with the proposition that different methods of assessing EF capture different aspects of behavioral and cognitive functioning (Toplak, West, & Stanovich, 2013).

Assessment of executive functioning: Self-report vs. performance-based measures

Typically, EF deficits have been assessed using neuropsychological, performance-based measures, generally regarded as a “gold standard”. Focusing exclusively on this testing approach presents limitations including reduced ecological validity of scores and “task impurity” (Burgess, Alderman, Evans, Emslie, & Wilson, 1998). The alternative of assessing EF through rating scales has been proposed (Barkley, 2011; Burgess et al., 1998; Roth, Isquith, & Gioia, 2005; Spinella, 2005). Potential advantages to this approach include their ecological validity and predictive capacity in relation with daily life impairment (Barkley & Fischer, 2011; Burgess et al., 1998). Self-report measures of EF predict impairment in daily living (Barkley & Fischer, 2011) and occupational functioning (Barkley & Murphy, 2010; Burgess et al., 1998) better than performance-based measures of EF. Nevertheless, rating scale assessment of EF has the limitations of having to rely on self (or other) reports, as well as those that arise when measuring different constructs in a similar manner (i.e. shared variance). For instance, rating scales of EF might improve the prediction of impairment when compared with neuropsychological tests because both constructs are often assessed through self-reports.

It is argued that measures of EF based on performance, such as neuropsychological-cognitive tests, and EF measures based on ratings seem to assess different levels within the hierarchy of EF, or different levels of analysis (Barkley & Fischer, 2011; Toplak et al., 2013). Low correlations between the two types of measures - self-reports such as the BRIEF, BADS-DEX, and BDEFS and performance tests - seem to indicate that they measure different levels of cognition (Barkley, 2011; Toplak et al., 2013). Whilst Barkley explains the lack of correspondence between self-report and performance-based measures in terms of a hierarchical model of EF, Toplak et al. refer to the distinction between algorithmic and reflective levels of analysis. Both arguments are consistent.

Because of limitations of neuropsychological measures of EF, including that they might not be measuring certain aspects of EF, assessing abilities not inherent to EF, and other concerns about the validity of score interpretation and norms (Burgess et al., 1998; Miyake & Friedman, 2012), it is imperative to include alternate measures of EF such as rating scales in the assessment of ADHD and EF in the context of neuropsychiatric and psychological assessment. Both types of assessment are useful and valuable, as they provide different types of complementary information regarding cognitive and behavioral functioning that are all relevant to clinical assessment (Isquith, Roth, & Gioia, 2013; Toplak et al., 2013).

Rating scales of EF available for adults

Several measures of EF in the form of rating scales have been developed, such as the Behavior Rating Inventory of Executive Function-Adult Version (BRIEF-A) (Roth et al., 2005), the Dysexecutive Questionnaire (DEX) (Burgess et al., 1998), the Executive Function Index (EFI) (Spinella, 2005) and more recently, the Barkley Deficits in Executive Functioning Scale (BDEFS) (Barkley, 2011). These have been studied with clinical and normal populations, are empirically derived and have varying evidence of the validity of their scores for assessing EF. However, measures such as the DEX, EFI, and BRIEF have shown some limitations ranging from low reliability (EFI) (Janssen, De Mey, & Egger, 2009) to inconsistent factor structures (DEX) (Burgess et al., 1998; Chan, 2001; Gerstorf, Siedlecki, Tucker-Drob, & Salthouse, 2008; Mooney, Walmsley, & McFarland, 2006; Pedrero-Pérez et al., 2011; Shinagawa et al., 2007).

Published studies on some self-report measures of EF for adults report variable factor structures across studies, even when the same-language version is compared. For example, Burgess et al. (1998) report that the DEX evidenced a five-factor structure. However, the DEX’s dimensionality has been studied for English (U.S. and Australia)-, Spanish-, Japanese- and Chinese-language versions, with reports of various one-, two-, three-, four- and five-factor solutions (Chan, Shum, Toulopoulou, & Chen, 2008; Gerstorf et al., 2008; Mooney et al., 2006; Pedrero-Pérez et al., 2011; Shinagawa et al., 2007; Simblett & Bateman, 2011). More relevant to our present study, the Spanish-language DEX was examined by Pedrero-Perez (2011) with a sample of 1,013 Spanish adults. Although reporting adequate reliability for the DEX scores, the factor structure varied across studies. Pedrero-Perez and colleagues identified two factors in this Spanish version, which is inconsistent with the 5-factor structure of the English language version (Burgess et al., 1998).

Likewise, the BRIEF-A (Roth et al., 2005) reportedly measures two factors, Behavioral Regulation and Metacognition, but it was recently demonstrated that its factor structure was somewhat inconsistent, showing instead a three-factor model (Roth, Lance, Isquith, Fischer, & Giancola, 2013). Moreover, although five subscales were derived through factor analysis for the EFI (Spinella, 2005), a recent study of a Dutch version for the five-factor structure did not adequately fit their data and reliability coefficients were low (Janssen et al., 2009).

Among these rating scales, we identified only two that have adult versions and are available in Spanish. These are the BRIEF-A (García-Molina, Tormos, Bernabeu, Junqu, & Roig-Rovira, 2012) and the DEX (Pedrero-Pérez et al., 2011). According to standards for educational and psychological testing (American Educational Research Association et al., 2014), when tests are translated, evidence of the validity, reliability and comparability of scores across versions is necessary (pp. 60–61). Garcia-Molina et al. (2012) translated the BRIEF-A into Spanish and examined the association of its scores to performance-based measures of EF and a measure of activities of daily living. They did not report on the psychometric properties of the Spanish BRIEF-A, and we were unable to identify studies documenting its psychometric properties. The Spanish language literature surveyed focuses on the child BRIEF.

The BDEFS (Barkley, 2011) is based on a hierarchical model of executive functioning (Barkley, 1997; Barkley, 2012) and its scores are intended to measure deficits in EF in daily life activities in adults (Barkley & Fischer, 2011). Its 89 items are organized in 5 subscales: Self-Organization/Problem Solving, Self-Management to Time, Self-Regulation of Emotion, Self-Restraint, and Self-Motivation. Subscales are related to 5 self-regulatory activities (Barkley, 2001; Barkley, 2011).

According to Barkley’s EF theory, the assessment of EF by self-report corresponds to the Strategic-Cooperative level of the hierarchy; accordingly the BDEFS assessment would correspond to this level. The content of the scale items organized in subscales corresponds to the 5 self-regulatory, goal-directed activities described in his model of EF. All internal consistency alpha coefficients reported in the scale manual for total and subscale scores are above 0.91. The manual reports norms for adults ages 18–70+, and states that the BDEFS approach to norms was different from the BRIEF-A, for example, since people without internet access, with psychiatric diagnoses, psychiatric drug use, learning disabilities, neurological disorders or medical illness were not systematically excluded (Barkley, 2011). This increases the representativeness of the sample of the U.S. general population, based on the 2000 census.

The BDEFS has been derived from a theoretical conception of EF; it has some evidence of the ecological validity of its scores, at least in terms of correlations with other self-report measures, including impairment in daily living and occupational functioning, and is comprehensive, including aspects of EF that are relevant to daily life functioning. Russell A. Barkley conducted an exploratory analysis that yielded the 5-factor structure reported in the scale manual, for which he used principal components analysis (PCA) and orthogonal (varimax) rotations. No studies have examined the structure using CFA in English or any other language.

In the present study, we explore the factor structure of a Spanish-language BDEFS with a sample of island Puerto Ricans; we also attempt to confirm the preliminary structure reported for the English language version. Besides documenting the translation into Spanish of a potentially useful instrument, to our knowledge, this is the first study to perform independent factor analysis and to attempt confirmation of the English-language structure of the BDEFS through confirmatory factor analysis (CFA).

Method

Participants and sample selection

Final sample were 452 Spanish-speaking adults in Puerto Rico. These included university students and adults from the community recruited at local departments of motor vehicles, churches and community centers. Participant’s recruitment was done by convenience and individual’s availability. Participants completed the questionnaires in paper and pencil form, individually or in groups. In some churches, community and university settings, the participants completed the questionnaires in a classroom or a meeting room, where these were administered to small groups.

Demographic characteristics are described in Table 1. Almost two thirds of the sample was female (65.5%). Mean age for the total sample was 29 with no significant differences by gender (ages 21–76). Most participants had university experience (2 years university; 47%), 38% had completed a university degree, 6% graduate school and 8% high school, and 64% of the sample was currently employed.

Table 1.

Demographic characteristics and descriptive statistics of Spanish-speaking adults who completed BDEFS (n = 452)

Characteristic	Males	Females	Total
	N (%)	N (%)	N (%)
	156 (34.5)	296 (65.5)	452 (100.0)
Marital Status
Single	124 (80.5)	192 (65.5)	316 (70.7)
Married/cohabiting	26 (16.9)	84 (28.7)	110 (24.6)
Separated/divorced	4 (2.6)	17 (5.8)	21 (4.7)
Currently employed	104 (67.5)	184 (62.6)	288 (64.3)
Education
High School	12 (7.7)	26 (8.9)	38 (8.5)
2 years university	78 (50.3)	133 (45.4)	211 (47.1)
4 years university (degree)	56 (36.1)	115 (39.3)	171 (38.2)
Graduate school	9 (5.8)	19 (6.5)	28 (6.3)
Age Mean (SD)	28.3 (11.2)	28.9 (10.9)	28.7 (11.0)
BDEFS total score Mean (SD)	140.3 (28.0)	138.5 (33.0)	139.1 (31.3)
ASRS 18-symptomsa Mean (SD)	3.8 (3.6)	3.6 (3.8)	3.6 (3.7)
Inattention 9 symptoms (SD)	1.8 (2.2)	1.7 (2.1)	1.8 (2.1)
Hyperactive 9 symptoms (SD)	2.0 (2.0)	1.8 (2.1)	1.9 (2.1)

Note: row % for gender, otherwise column %.

BDEFS = Barkley Deficits in Executive Functioning Scale

ASRS = Adult ADHD Self-Report Scale

^a9 missing cases.

ADHD symptoms were screened with the adult Self-Report Scale (ASRS).

Measures

Data on socio-demographic characteristics were collected via self-report questionnaire. This included information about gender, age, marital, employment and educational status.

Executive Functioning

The Barkley Deficits in Executive Functioning Scale (BDEFS; Barkley, 2011) is an 89 item self-report measure of executive functioning. Its scores provide a measure of deficits in executive functioning as they manifest in daily life in adults via items on a four-point scale. It comprises 5 subscales: Self-Management to Time (21 items, sample item: “[I] waste or mismanage my time”); Self-Organization/Problem Solving (24 items, sample item: “I don’t seem to process information as quickly or as accurately as others.”); Self-Restraint (19 items, sample: “[I] make decisions impulsively”); Self-Motivation (12 items; sample: “Others tell me I am lazy or unmotivated”) and Self-Regulation of Emotion (13 items; sample: “I remain emotional or upset longer than others”). Items are answered on a 4-point scale ranging from “1= Never or rarely” to “ 4 = Very often”; α’s range from 0.91 to 0.96. Higher scores are interpreted as indicating greater deficits in EF.

ADHD

The Adult Self-Report Scale (ASRS) (Adler et al., 2006; Kessler et al., 2005) was used to measure participants’ ADHD symptoms. The ASRS has been translated into Spanish and other languages using the World Health Organization (WHO) back translation standard protocol, and is widely used in international research (Daigre et al., 2009; Pedrero-Pérez & Puerta-García, 2007; Sanchez-Garcia et al., 2015). It includes 18 items describing ADHD symptoms, following DSM-IV criteria. Items are scored on a 5-point Likert type scale ranging from 0 (“never”), to 4 (“very often”), with 9 items for inattention, 6 for hyperactivity and 3 for impulsivity. The scale authors reported that for screening purposes, a short 6-item version of the ASRS outperforms the 18-item version in terms of overall predictive accuracy (97.9% vs. 94.5%) and area under the curve (AUC; 0.84 vs. 0.78) (Kessler et al., 2005). However, we decided to use the original 18-item version in order to be able to compute reliable scores of inattention and hyperactivity/impulsivity and total symptom scores (Caci, Bayle, & Bouchez, 2008; Kessler et al., 2005). Construct validity for the BDEFS derived factor scores were examined using continuous scales of predominantly inattentive and predominantly impulsive dimensions’ scores in addition to the overall ADHD score. Alpha reliability indices for scores of the ASRS version used in this study were 0.91 for the total ADHD score, 0.88 for inattention (I/A) and 0.86 for hyperactivity/impulsivity (H/I) scores (detailed psychometric properties including results from CFA are available from authors). Other researchers have reported on the psychometric characteristics of the Spanish version of the ASRS, including (Pedrero-Pérez & Puerta-García, 2007; Ramos-Quiroga et al., 2009). Pedrero-Perez & Puerta-Garcia (2007) report Cronbach’s alpha reliability coefficients of 0.92 for scores on the 18 items of the ASRS.

Procedures

The BDEFS was translated into Spanish. We used a back-translation procedure following guidelines for the translation and cultural adaptation process (Wild et al., 2005) and observing applicable standards for educational and psychological testing (American Educational Research Association et al., 2014). Procedures included two independent professional translators for the forward and backward versions, as well as reconciliation and comparisons of versions. Doctoral level psychology students administered the final Spanish language version of the BDEFS produced by this method to a community and university sample in paper and pencil self-report form. Three separate Institutional Review Boards (IRB), of the Carlos Albizu University, the University of Puerto Rico at Cayey and University of Puerto Rico at Carolina campuses approved all study procedures prior to data collection. Procedures were in accord with ethical standards for research, including engaging potential participants in a process of informed consent prior to their participation, and documenting said consent in writing.

Analytic strategy

Missing data

Patterns of missing data were analyzed for both the ASRS and the BDEFS. There were no apparent patterns of missing data for the instruments’ items, with all items completed by far the most common pattern in the ASRS (90%) and in the BDEFS (80%). Nevertheless, we decided to only allow observations to enter further analyses if the percentage of missing values did not exceed 15% of all items within an observation (Dong & Peng, 2013). This tolerance level amounts to accepting no more than two missing values on the ASRS, and no more than 13 items missing on the BDEFS. Nine observations were excluded from analyses using ASRS; two observations were discarded from the study with missing values on 17 and 24 items on the BDEFS. The total EFA/CFA sample was 452. The sample for models with ASRS scores as covariates was 443.

The ASRS total symptom score was calculated to use as a covariate on CFA MIMIC models. Prior to this, the ASRS questionnaires with between one and two missing items were included after correcting the final scores as follows: corrected score = raw score x (18/(18 – missing items)). This means that the corrected scores are equivalent to the ratio between the raw score and the maximum possible score, given the number of completed items. This is akin to imputing missing responses as the mean of all completed responses.

Full Information Maximum Likelihood is used by default in Mplus software models. This was the method of imputing 20% of data ranging from 1 to 12 items in the BDEFS.

Statistical Analyses

Absolute (n) and relative frequencies (%) were reported for all dichotomous/polytomous categorical variables, and means and standard deviation for continuous descriptive variables.

An exploratory factor analysis (EFA) of principal components with Geomin oblique rotation was performed to examine the structure of the BDEFS. Exploratory analyses were performed in solutions from 1 to 6 factors with the aim of selecting the solution that best fit the observations, based on item factor loadings, fit indices and a parsimonious structure consistent with theory. We also estimated exploratory factors’ scores alpha reliability indices and cross-correlations.

In order to validate both the BDEFS original English language factor structure (Barkley, 2011), and a modified version based only on the common items to both ours and the original author’s version, we performed confirmatory factor analysis (CFA). Because the BDEFS items range from 1–4, a Weighted Least Squares Mean and Variance (WLSMV robust) estimator was used to account for bias in estimates due to non-linearity of ordered categorical indicators. For identification of the CFA model, item variances were allowed to be estimated freely and the model was standardized by fixing factor variances at one. Model fit was assessed by the root mean square of approximation (RMSEA) and the Comparative-fit (CFI) and Tucker-Lewis (TLI) indices. Values of 0.06 or lower for RMSEA, and optimally above 0.95 for CFI and TLI indicate a very good model fit (Hu & Bentler, 1999; Yu & Muthén, 2002). The χ2 test is usually not a reliable measure of model fit with large sample sizes (Bentler & Bonett, 1980).

Construct validity was further assessed using a Multiple Indicators Multiple Causes CFA (MIMIC) model. In this procedure, the effects of a covariate (e.g., grouping variable, continuous score) on a measurement model such as CFA are tested in order to establish whether groups based on the covariate have differential scores on the latent factors and/or items of a scale, providing an estimate of measurement invariance (Marsh, Morin, Parker, & Kaur, 2013). This procedure is akin to mean difference tests (e.g., ANOVA, t-test) but with the advantage that it can be computed directly from the estimation of the CFA and its factor scores, therefore retaining correct standard errors and accounting for a portion of error variance. Covariates used to correlate with CFA-derived factors were age, gender, and inattention (I/A), hyperactivity/impulsivity (H/I) and overall ADHD symptom scores. CFA model-based post-hoc analyses allowed establishing whether the degree of associations between BDFES factor scores with I/A and H/I symptom dimensions differed from each other.

Stata version 13 (StataCorp., 2013) was used for data management and descriptive analyses. Mplus version 7.1 (Muthén & Muthén, 2013) was used for EFA, CFA, CFA with covariates and post-hoc analyses.

Results

Exploratory Factor Analysis (EFA)

We compared the results of 1 to 6 factor EFA solutions using oblique Geomin rotation. The decision for the solution and number of factors was based on a number of criteria: number of eigenvalues > 1, fit indices (i.e., RMSEA and CFI), and item loadings (Table 2). The fit indices favored solutions with 5 (RMSEA 0.029[CI 90% 0.027, 0.031], CFI 0.954, TLI 0.948) and 6 (RMSEA 0.025[CI 90% 0.023, 0.027], CFI 0.966, TLI 0.960) factors. The 5 factor solution was more parsimonious and more in accord with the original conceptual model for the scale, whereas the sixth factor of the 6 factor solution did not have salient loadings in it, which suggests over-fitting of the data for that model.

Table 2.

Fit indices of Exploratory Factor Analyses for 1 to 6 solutions

N° factors	Eigenvalue	RMSEA	RMSEA 90%CI	CFI	TLI
1	29.7	0.063	0.062, 0.065	0.749	0.743
2	8.2	0.048	0.046, 0.049	0.861	0.855
3	5.7	0.037	0.035, 0.038	0.920	0.915
4	3.2	0.033	0.031, 0.034	0.939	0.933
5	3.0	0.029	0.027, 0.031	0.954	0.948
6	2.2	0.025	0.023, 0.027	0.966	0.960

Note: RMSEA- Root Mean Square Approximation, CFI- Comparative Fit Index, TFI- Tucker-Lewis Index.

Geomin oblique rotation, Weighted Least Square-Mean and Variance (WLSMV) estimator

Overall, there were discrepancies in 11 out of the 89 items of our Spanish version compared to the original version of the BDEFS (Table 3). The item “Find it hard to focus on what is important from what is not important when I do things” from the original version Self-organization/problem solving factor loaded on our Self-management to time factor. Furthermore, 2 items from the original Self-organization/problem solving factor did not have significant loadings (i.e., >.40) on any of the factors in the Spanish version, although they loaded highest on the “correct” factor. Four items from the original version’s Self-restraint factor (i.e., “Find it difficult to tolerate waiting; impatient”, “Have difficulty being objective about things that affect me”, “Have a low tolerance for frustrating situations” and “Cannot inhibit my emotions as well as others”) loaded on our Self-regulation of emotion factor. The “excessive speeding” item from the Self-restraint factor loaded on our Self-motivation factor. Another 2 items originally from the Self-restraint and the Self-motivation factors did not load in any of our factors.

Table 3.

Exploratory Factor Analysis for the Barkley Deficits in Executive Functioning Scale (BDEFS) Spanish Language version: 5 factor solution

Items with sequence number identification	Factor 1	Factor 2	Factor 3	Factor 4	Factor 5	α
Factor 1: Self-management to time (factor correlations)	-	(0.516)	(0.323)	(0.412)	(0.601)	0.936
1 Procrastinate or put off doing things until the last minute.	0.737
2 Poor sense of time.	0.798
3 Waste or mismanage my time.	0.851
4 Not prepared on time for work or assigned tasks	0.748
5 Fail to meet deadlines for assignments	0.749
6 Have trouble planning ahead or preparing for upcoming events.	0.746
7 Forget to do things I am supposed to do.	0.763
8 Can’t seem to accomplish the goals I set for myself.	0.511
9 Late for work or scheduled appointments.	0.588
10 Can’t seem to hold in mind things I need to remember to do.	0.735
11 Can’t seem to get things done unless there is an immediate deadline.	0.783
12 Have difficulty judging how much time it will take to do something or get somewhere.	0.538
13 Have trouble motivating myself to start work.	0.541
14 Have difficulty motivating myself to stick with my work and get it done	0.526
15 Not motivated to prepare in advance for things I know I am supposed to do.	0.635
16 Have trouble completing one activity before starting into a new one.	0.494
17 Have trouble doing what I tell myself to do.	0.607
18 Difficulties following through on promises or commitments I may make to others.	0.482
19 Lacks self-discipline.	0.762
20 Have difficulty arranging or doing my work by its priority or importance; can’t “prioritize” well.	0.695
21 Find it hard to get started or get going on things I need to get done.	0.715
44 Find it hard to focus on what is important from what is not important when I do things	0.453§
Factor 2: Self-Organization/Problem Solving (factor correlations)	-	-	(0.449)	(0.463)	(0.479)	0.928
22 I do not seem to anticipate the future as much or as well as others.		0.437
23 Can’t seem to remember what I previously heard or read about.		0.588
24 I have trouble organizing my thoughts.		0.549
25 When I am shown something complicated to do, I cannot keep the information in mind		0.605
26 I have trouble considering various options for doing things and weighing their consequences		0.487
27 Have difficulties saying what I want to say.		0.820
28 Unable to come up with or invent as many solutions to problems as others seem to do.		0.731
29 Find myself at a loss for words when I want to explain something to others.		0.796
30 Have trouble putting my thoughts down in writing as well or as quickly as others.		0.661
31 Feel I am not as creative or inventive as others of my level of intelligence.		0.699
32 In trying to accomplish goals or assignments, I find I am not able to think of as many ways of doing things as others.		0.744
33 Have trouble learning new or complex activities as well as others.		0.765
34 Have difficulty explaining things in their proper order or sequence.		0.645
35 Can’t seem to get to the point of my explanations as quickly as others.		0.665
36 Have trouble doing things in their proper order or sequence.		0.656
37 Unable to “think on my feet” or respond as effectively as others to unexpected events		0.760
38 I am slower than others at solving problems I encounter in my daily life.		0.617
39 Easily distracted by irrelevant events or thoughts when I must concentrate on something		0.419
40 Not able to comprehend what I read as well as I should be able to do; have to reread material to get its meaning.		0.614
42 Easily confused.		0.594
45 I don’t seem to process information as quickly or as accurately as others.		0.711
Factor 3: Self-Regulation of Emotion (factor correlations)	-	-	-	(0.649)	(0.444)	0.923
46 Find it difficult to tolerate waiting; impatient			0.467¶
56 Have difficulty being objective about things that affect me.			0.406¶
61 Have a low tolerance for frustrating situations.			0.529¶
62 Cannot inhibit my emotions as well as others.			0.474¶
77 Quick to get angry or become upset.			0.745
78 Overreact emotionally.			0.805
79 Easily excitable.			0.705
80 Unable to inhibit showing strong negative or positive emotions.			0.795
81 Have trouble calming myself down once I am emotionally upset			0.895
82 Cannot seem to regain emotional control and become more reasonable once I am emotional.			0.923
83 Cannot seem to distract myself away from whatever is upsetting me emotionally to help calm me down. I can’t refocus my mind to a more positive framework.			0.782
84 Unable to manage my emotions in order to accomplish my goals successfully or get along well with others.			0.590
85 I remain emotional or upset longer than others.			0.655
86 I find it difficult to walk away from emotionally upsetting encounters with others or leave situations in which I have become very emotional.			0.660
87 I cannot rechannel or redirect my emotions into more positive ways or outlets when I get upset.			0.815
88 I am not able to evaluate an emotionally upsetting event more objectively.			0.760
89 I cannot redefine negative events into more positive viewpoints when I feel strong emotions.			0.828
Factor 4: Self-Restraint (factor correlations)	-	-	-	-	(0.475)	0.871
47 Make decisions impulsively.				0.643
48 Unable to inhibit my reactions or responses to events or others.				0.570
49 Have difficulty stopping my activities or behavior when I should do so.				0.480
50 Have difficulty changing my behavior when I am given feedback about my mistakes.				0.490
52 Likely to do things without considering the consequences for doing them.				0.686
53 Change my plans at the last minute on a whim or last minute impulse.				0.432
54 Fail to consider past relevant events or past personal experiences before responding to situations (I act without thinking).				0.609
55 Not aware of things I say or do.				0.499
57 Find it hard to take other people’s perspectives about a problem or situation.				0.432
58 Don’t think about or talk things over with myself before doing something.				0.594
59 Trouble following the rules in a situation.				0.466
63 I don’t look ahead and think about what the future outcomes will be before I do something				0.552
Factor 5: Self-motivation	-	-	-	-	-	.846
60 More likely to drive a motor vehicle much faster than others (excessive speeding)					0.443¶
65 Likely to take shortcuts in my work and not do all that I am supposed to do.					0.455
66 Likely to skip out on work early if my work is boring to do.					0.418
67 Do not put as much effort into my work as I should or than others are able to do.					0.612
68 Others tell me I am lazy or unmotivated.					0.593
70 Things must have an immediate payoff for me or I do not seem to get them done.					0.568
71 Have difficulty resisting the urge to do something fun or more interesting when I am supposed to be working.					0.571
72 Inconsistent in the quality or quantity of my work performance.					0.583
73 Unable to work as well as others without supervision or frequent instruction.					0.486
74 I do not have the willpower or determination that others seem to have.					0.474
75 I am not able to work toward longer term or delayed rewards as well as others					0.538
76 I cannot resist doing things that produce immediate rewards even if they are not good for me in the long run.					0.581

Note: Items 41, 43, 51, 64 and 69 were removed due to low or double loadings. All correlations between factors p < 0.001.

^§Item loaded on Barkley’s original scale on Self-Organization/Problem Solving factor.

^¶Items loaded on Barkley’s original scale on Self-Restraint factor.

From Barkley Deficits in Executive Functioning Scale (BDEFS) by Russell A. Barkley. Copyright 2011 by The Guilford Press. Reprinted with permission of The Guilford Press.

Reliability and cross-correlations

Cronbach’s alpha estimates based on the original structure with 89 items for each of the BDEFS factor subscale scores were: Self-management to time- 0.933, Self-organization/problem solving- 0.936, Self-regulation of emotion- 0.917, Self-restraint- 0.886 and Self-motivation- 0.862. The estimates for the factors based on our EFA 5 factor solution were 0.936, 0.928, 0.923, 0.871 and 0.846 respectively (Table 2). Factor correlations ranged from 0.323 (Self-management to time and Self-regulation of emotion), to 0.649 (Self-restraint and Self-regulation of emotion). Correlations were all significant at the p <0.001 level (Table 3).

Confirmatory Factor Analysis (CFA)

With the aim of further examining the 5-factor structure originally derived by Barkley (2011) we performed confirmatory factor analyses (CFA) on the English (“original”) language structure. In addition, we performed CFA on a modified version that included 78 items that loaded in factors consistent in our present version in Spanish, and the English original structure. Comparative CFA results are in Table 4. The fit indices were good in terms of RMSEA, the CFI and TLI for the modified version of the BDEFS. Fit indices for the modified version were rather similar to the original English language structure of the BDEFS, but with very slightly improved model fit. Table 5 shows that all indicators on both CFAs had significant loadings above the standard threshold (Stevens, 1992) of ≥ 0.40, and several above 0.60 (Guadagnoli & Velicer, 1988), which are all adequate.

Table 4.

Confirmatory Factor Analysis fit indices by solutions based on the original structure from the English language BDEFS, and the modified^a version

Version	Chi sq. (df)	RMSEA	RMSEA 90%CI	CFI	TLI
Original version (89/89 items)	5606.4 (3810)*	0.032	0.030, 0.034	0.935	0.934
Modified version (78/89 items)	4192.7 (2906)*	0.031	0.029, 0.033	0.947	0.945

Note: RMSEA- Root Mean Square Approximation, CFI- Comparative Fit Index, TFI- Tucker-Lewis Index.

^aModified is BDEFS administered in Spanish language, using only 78 items consistent with the original scale structure

^*p < 0.001

Table 5.

Confirmatory Factor Analysis for the modified^a BDEFS administered in Spanish language

Items with sequence number identification	Loadinga	R2
Factor 1: Self-management to time (alpha 0.933)
1 Procrastinate or put off doing things until the last minute.	0.691	0.478
2 Poor sense of time.	0.703	0.495
3 Waste or mismanage my time.	0.762	0.580
4 Not prepared on time for work or assigned tasks	0.693	0.480
5 Fail to meet deadlines for assignments	0.658	0.433
6 Have trouble planning ahead or preparing for upcoming events.	0.688	0.474
7 Forget to do things I am supposed to do.	0.575	0.331
8 Can’t seem to accomplish the goals I set for myself.	0.639	0.408
9 Late for work or scheduled appointments.	0.484	0.234
10 Can’t seem to hold in mind things I need to remember to do.	0.496	0.246
11 Can’t seem to get things done unless there is an immediate deadline.	0.748	0.559
12 Have difficulty judging how much time it will take to do something or get somewhere.	0.655	0.428
13 Have trouble motivating myself to start work.	0.744	0.554
14 Have difficulty motivating myself to stick with my work and get it done	0.768	0.590
15 Not motivated to prepare in advance for things I know I am supposed to do.	0.776	0.603
16 Have trouble completing one activity before starting into a new one.	0.764	0.583
17 Have trouble doing what I tell myself to do.	0.800	0.641
18 Difficulties following through on promises or commitments I may make to others.	0.644	0.415
19 Lacks self-discipline.	0.771	0.595
20 Have difficulty arranging or doing my work by its priority or importance; can’t “prioritize”.	0.790	0.625
21 Find it hard to get started or get going on things I need to get done.	0.850	0.722
Factor 2: Self-Organization/Problem Solving (alpha 0.928)
22 I do not seem to anticipate the future as much or as well as others.	0.586	0.344
23 Can’t seem to remember what I previously heard or read about.	0.699	0.489
24 I have trouble organizing my thoughts.	0.818	0.670
25 When I am shown something complicated to do, I cannot keep the information in mind so as to imitate or do it correctly.	0.669	0.447
26 I have trouble considering various options for doing things and weighing their consequences	0.733	0.537
27 Have difficulties saying what I want to say.	0.630	0.397
28 Unable to come up with or invent as many solutions to problems as others seem to do.	0.696	0.485
29 Find myself at a loss for words when I want to explain something to others.	0.614	0.377
30 Have trouble putting my thoughts down in writing as well or as quickly as others.	0.647	0.418
31 Feel I am not as creative or inventive as others of my level of intelligence.	0.588	0.346
32 In trying to accomplish goals or assignments, I find I am not able to think of as many ways of doing things as others.	0.677	0.458
33 Have trouble learning new or complex activities as well as others.	0.716	0.513
34 Have difficulty explaining things in their proper order or sequence.	0.673	0.455
35 Can’t seem to get to the point of my explanations as quickly as others.	0.730	0.533
36 Have trouble doing things in their proper order or sequence.	0.778	0.605
37 Unable to “think on my feet” or respond as effectively as others to unexpected events	0.660	0.436
38 I am slower than others at solving problems I encounter in my daily life.	0.763	0.583
39 Easily distracted by irrelevant events or thoughts when I must concentrate on something	0.750	0.562
40 Not able to comprehend what I read as well as I should be able to do; have to reread material to get its meaning.	0.721	0.520
42 Easily confused.	0.773	0.598
45 I don’t seem to process information as quickly or as accurately as others.	0.813	0.661
Factor 3: Self-Regulation of Emotion (alpha 0.917)
77 Quick to get angry or become upset.	0.682	0.465
78 Overreact emotionally.	0.775	0.600
79 Easily excitable.	0.537	0.289
80 Unable to inhibit showing strong negative or positive emotions.	0.629	0.396
81 Have trouble calming myself down once I am emotionally upset	0.767	0.588
82 Cannot seem to regain emotional control and become more reasonable once I am emotional.	0.847	0.717
83 Cannot seem to distract myself away from whatever is upsetting me emotionally to help calm me down. I can’t refocus my mind to a more positive framework.	0.810	0.656
84 Unable to manage my emotions in order to accomplish my goals successfully or get along well with others.	0.896	0.803
85 I remain emotional or upset longer than others.	0.761	0.579
86 I find it difficult to walk away from emotionally upsetting encounters with others or leave situations in which I have become very emotional.	0.821	0.675
87 I cannot rechannel or redirect my emotions into more positive ways or outlets when I get upset.	0.866	0.750
88 I am not able to evaluate an emotionally upsetting event more objectively.	0.866	0.749
89 I cannot redefine negative events into more positive viewpoints when I feel strong emotions.	0.802	0.643
Factor 4: Self-Restraint (alpha 0.871)
47 Make decisions impulsively.	0.676	0.457
48 Unable to inhibit my reactions or responses to events or others.	0.714	0.510
49 Have difficulty stopping my activities or behavior when I should do so.	0.787	0.620
50 Have difficulty changing my behavior when I am given feedback about my mistakes.	0.747	0.558
52 Likely to do things without considering the consequences for doing them.	0.741	0.550
53 Change my plans at the last minute on a whim or last minute impulse.	0.624	0.389
54 Fail to consider past relevant events or past personal experiences before responding to situations (I act without thinking).	0.750	0.562
55 Not aware of things I say or do.	0.796	0.634
57 Find it hard to take other people’s perspectives about a problem or situation.	0.660	0.436
58 Don’t think about or talk things over with myself before doing something.	0.663	0.440
59 Trouble following the rules in a situation.	0.639	0.408
63 I don’t look ahead and think about what the future outcomes will be before I do something	0.816	0.665
Factor 5: Self-motivation (alpha 0.856)
65 Likely to take shortcuts in my work and not do all that I am supposed to do.	0.563	0.317
66 Likely to skip out on work early if my work is boring to do.	0.550	0.303
67 Do not put as much effort into my work as I should or than others are able to do.	0.797	0.635
68 Others tell me I am lazy or unmotivated.	0.690	0.476
70 Things must have an immediate payoff for me or I do not seem to get them done.	0.633	0.400
71 Have difficulty resisting the urge to do something fun or more interesting when I am supposed to be working.	0.647	0.419
72 Inconsistent in the quality or quantity of my work performance.	0.850	0.722
73 Unable to work as well as others without supervision or frequent instruction.	0.798	0.637
74 I do not have the willpower or determination that others seem to have.	0.826	0.682
75 I am not able to work toward longer term or delayed rewards as well as others	0.888	0.789
76 I cannot resist doing things that produce immediate rewards even if they are not good for me	0.711	0.506

Note: Factors numbered in different order from the original English language scale for consistency with EFA solution

^aModified is BDEFS administered in Spanish language, using only 78 items consistent with the original scale structure PCA.

From Barkley Deficits in Executive Functioning Scale (BDEFS) by Russell A. Barkley. Copyright 2011 by The Guilford Press. Reprinted with permission of The Guilford Press.

CFA with covariates

Measurement invariance related to the factor means was assessed using a Multiple Indicators Multiple Causes CFA (MIMIC) model. A MIMIC model was tested for each of the dimensional scales of Inattention (I/A), Hyperactivity (H/I) and overall ADHD symptoms in association with the 5 factor scores. We also estimated the effects of age and gender on factor means. The results from these models are shown in Table 6.

Table 6.

CFA Multiple indicator multiple causes (MIMIC) model coefficients for BDEFS factor differences by ADHD dimensions (n = 443)

Covariates	BDEFS factor scores
	Time β coef.	Organization β coef.	Regulation β coef.	Restraint β coef.	Motivation β coef.
Demographics
Age	−0.015**	−0.001	−0.005	−0.001	−0.012*
Gender	−0.022	0.036	0.142	−0.120	−0.436***
ADHD dimensions
I/A	0.400***	0.288***	0.141***	0.186***	0.288***
H/I	0.214***	0.195***	0.203***	0.280***	0.224***
ADHD	0.208***	0.164***	0.115***	0.158***	0.177***
Contrastsa
I/A	0.371***	0.250***	0.060*	0.081**	0.237***
H/I	0.064*	0.082**	0.172***	0.237***	0.114***
I/A – H/I	0.307***	0.168**	−0.113*	−0.156**	0.124*

^aThese regression coefficients were adjusted for each other ADHD symptom dimension to evaluate direct associations, and to obtain model-based contrasts for differences.

Note: I/A = Inattention; H/I = Hyperactivity/Impulsivity; ADHD = ASRS total score

^*p < 0.05,

^**p < 0.01,

^***p < 0.001

Age was inversely associated with the Self-management to time (p <0.01) and Self-motivation (p <0.01) factors. Women scored significantly lower than males on Self-motivation (p <0.001), indicating less dysfunction on this factor.

Continuous ADHD symptom scores of I/A, H/I and overall ADHD were significantly associated with all BDEFS factor scores, before and after fitting I/A and H/I simultaneously (Table 6). Model-based post-hoc analyses to estimate differences between the I/A and H/I factors in terms of their effects on the BDEFS factor scores showed that associations between Self-management to time (p<0.001), Self-organization/problem solving (p <0.01) and Self-motivation (p <0.05) with I/A were significantly higher than with H/I. Contrariwise, associations of Self-regulation of emotion (p <0.05) and Self-restraint (p <0.01) with H/I were significantly higher than with I/A.

Discussion

Our exploratory and confirmatory factor analyses results on the 89 items of the Spanish-language BDEFS demonstrate robust reliability of BDEFS scores and a factor structure that is relatively consistent with the original English-language BDEFS. These results indicate that the Spanish-language version of the BDEFS and its English counterpart seemingly measure the same construct, and lend support to the theory of EF that underlies the instrument. Results also support an association between EF deficits and ADHD symptoms, consistent with the conceptual link between these constructs.

Factor structures (EFA and CFA)

After exploratory factor analysis on the 89 BDEFS items, the five-factor solution was found to be the most parsimonious and to fit the data best. Specifically, 78 of the 89 BDEFS items had salient loadings on the expected factor (i.e., similar to the original model in English). Five of the remaining items loaded on the expected factor, but had slightly lower loadings than the established threshold. The remaining 6 divergent items loaded on a different factor in our Spanish-language version. These were Self-restraint items representing low frustration tolerance and emotional inhibition, and they interestingly loaded on the Self-regulation of emotion factor. This may represent cultural factors related to how Latinos attribute emotional meaning in instances when having to exert self-control. Over-emotionality has been identified in Latinos, including Puerto Ricans, in culture-bound syndromes research. For instance, “Ataque de Nervios”, which has been defined among Latino samples (Guarnaccia et al., 2010), is characterized by intense emotions and anxiety, and has been related to emotional dysregulation (Suveg, Morelen, Brewer, & Thomassin, 2010). Additionally, the item “find it hard to focus on what is important” was originally part of the Self-Organization subscale, but in our sample loaded on Self-management to time factor, most likely related to prioritizing abilities, which pertain to both conceptual factors.

The 5 BDEFS factor subscales that we derived through EFA were consistent with the original subscales of the English-language version, and their scores’ Cronbach’s alpha internal reliability coefficients ranged from 0.85 to 0.94. These reliability coefficients are comparable to those reported for the English-language BDEFS subscale scores, and indicate optimal internal consistency for the Spanish-language version. Correlations among the subscale scores were all significant and in the moderate to high-moderate range, suggesting that the five dimensions are interrelated. Future studies should aim to confirm whether these empirically relate to a higher-order latent construct of EF deficits through hierarchical confirmatory models (e.g., general/specific CFA).

The modified version CFA supported the BDEFS 5-factor structure originally reported by Barkley (2011), albeit this was adapted to include only items that had loaded significantly, and similarly on both versions. In other words, this constitutes a preliminary cross-cultural test of the EF construct, as measured by the BDEFS. It is important to stress that although we used a slightly revised set of items to conduct this CFA, further investigation should aim to replicate these findings by confirmation of the factor structure in a different and larger sample. Nevertheless, that the 5-factor structure that we obtained is remarkably similar to that which Barkley identified is an important finding. This is particularly salient, as we are comparing our Spanish version with an instrument that was originally developed in English, and based on a majority white-American U.S. sample. The current study is an important step in the direction of understanding the factor structure of the BDEFS. We must clarify that Barkley (2011) did not conduct a CFA, and to date this has not been done. The manual reports PCA used to identify 5 components from 89 BDEFS items, which in itself carries its limitations when compared with EFA/CFA. Therefore, we used EFA on first instance to try to replicate this component structure preliminarily reported in the BDEFS manual.

The current study had the objective of examining the factor structure of the Spanish BDEFS, and attempting to confirm the previously published structure of the English language version that was based on PCA. Results are consistent with the preliminary original structure reported for the scale, which contrasts with what has been often reported with several other EF rating scales. Studies on self-report measures of EF tend to report inconstant factor structures, even when the same-language version is compared. For instance, this has been the case with the DEX, (Burgess et al., 1998; Chan, 2001; Chan et al., 2008; Gerstorf et al., 2008; Mooney et al., 2006; Pedrero-Pérez et al., 2011; Shinagawa et al., 2007; Simblett & Bateman, 2011). Similarly, the BRIEF-A (Roth et al., 2005) has yielded different factor solutions (Roth, Isquith, & Gioia, 2014), while the EFI (Spinella, 2005) has subscales that were not replicated in a Dutch version (Janssen et al., 2009). We consider our findings as preliminary evidence that indicates the factor structure of the BDEFS is consistent. This needs to be further explored, perhaps with translations into other languages and replication studies.

Reliability and validity

Experts recommend that clinicians and researchers assessing behaviors related to EF use tests whose scores’ reliability coefficients are at least 0.80, and 0.90 for composite scores (Naglieri & Goldstein, 2014). The internal consistency reliability coefficients obtained for the Spanish BDEFS subscale scores ranged from 0.94 to 0.85, well within these guidelines and comparable to those reported in the manual for the English BDEFS (0.96 to 0.91; Barkley, 2011).

Results provide preliminary evidence of the validity of the BDEFS scores as a measure of EF deficits, according to the EF model underlying the scale construction. They can be interpreted as supporting the theory of EF as goal directed self-regulation underlying the instrument, which in turn is related to symptoms of ADHD in adults, in accordance with the formulation of ADHD as impairment in EF (Barkley, 1997; Brown, 2009). Our findings indicated that the five BDEFS subscale factor scores are generally correlated with the symptoms of ADHD, corresponding with a preponderant view in the scientific literature that conceives ADHD as a developmental impairment of executive functions (Barkley, 1997; Barkley, 2012; Biederman et al., 2008; Biederman et al., 2007; Brown, 2006; Brown, 2009; Seidman, 2006). It has been argued that impaired EF involves “a pattern of chronic difficulties in executing a wide variety of daily tasks”, which is consistent with the finding that persons with ADHD exhibit or report EF deficits that are evident in daily living and occupational functioning (Barkley & Fischer, 2011; Barkley & Murphy, 2010), but not consistently evidenced on performance in EF tasks typically used in neuropsychological evaluation (Boonstra, Oosterlaan, Sergeant, & Buitelaar, 2005).

The CFA model-based comparisons of ADHD symptoms with the latent dimensions of EF provided evidence of differential patterns of associations. The EF factor scores suggesting deficits in time-management, self-motivation and self-organization/problem solving were more strongly associated with the inattention dimension of ADHD symptoms. This is consistent with studies documenting difficulties with time estimation in adults with ADHD (Carelli & Wiberg, 2012; Prevatt, Proctor, Baker, Garrett, & Yelland, 2011; Suarez, Lopera, Pineda, & Casini, 2013), reporting impaired timing processing in children and adolescents with ADHD (Hwang, Gau, Hsu, & Wu, 2010; McGee, Brodeur, Symons, Andrade, & Fahie, 2004; Smith, Taylor, Rogers, Newman, & Rubia, 2002) and deficiencies on the utilization of temporal information in children with ADHD (Radonovich, 2004).

Meanwhile, symptoms of hyperactivity/impulsivity were more closely related to EF deficits in terms of emotion regulation and self-restraint. The dimension of emotion regulation assessed by Self-Regulation of Emotion scale of the BDEFS is in line with Gross’ (Gross, 2014; Gross & Thompson, 2007) process model of emotional self-control. This model describes emotion regulation as a process that includes attention to person-situation transactions in the context of an individual’s goals, and produces coordinated actions. Recent research on the emotional regulation aspect of EF and ADHD tends to relate the symptoms of hyperactivity/impulsivity more strongly to emotional dysregulation, and seems to suggest that difficulties in emotion regulation are present in ADHD across the lifespan and in different contexts (Gonzalez, Gudjonsson, Wells, & Young, 2013; Reimherr et al., 2005; Shaw, Stringaris, Nigg, & Leibenluft, 2014; Sobanski et al., 2010). Importantly, this is also consistent with Paul Wender’s once controversial viewpoint on adult ADHD, which ascribes importance to emotional regulation deficits. In his pioneering work on adult ADHD, Wender proposed the Utah criteria, which included affective lability and emotional overreactivity as important characteristics, to be considered among the more “traditional” symptoms, such as disorganization and impulsivity (Wender, 1995). More recently, some have suggested that symptoms of emotional dysregulation seem to be distinct factors of the psychopathology of adult ADHD, and point to a growing body of evidence indicating that emotional dysregulation might be a core feature of ADHD, as Wender had originally proposed (Corbisiero, Stieglitz, Retz, & Rosler, 2013; Retz, Stieglitz, Corbisiero, Retz-Junginger, & Rösler, 2012). Additional studies show that in adolescents ADHD hyperactive-impulsive symptoms are more strongly associated with behavioral regulation EF deficits, while ADHD inattentive and sluggish cognitive tempo symptoms are not (Becker & Langberg, 2014). Similarly, among children with ADHD, disinhibition was positively related to hyperactivity/impulsivity (Brocki & Bohlin, 2006).

Limitations

Our study had several limitations, primarily related to sample size and characteristics. We used a relatively small community sample in which females were overrepresented (65%). This sample was also obtained by convenience, hindering generalizability of our findings. ADHD was based on a reliable, but nevertheless a self-reporting instrument that may have potentially added bias to the estimates. Confirmation of ADHD diagnosis by clinical interview may have yielded valuable information regarding severity of symptoms and psychosocial impairment. In addition, the age range differed slightly from the BDEFS normative sample (21–76 years vs. 18–81 years). Our EFA and CFA were conducted in the same sample, which is not generally advised. However, our CFA models were not based on the results from our EFA, but were aimed at replicating Barkley’s original structure based on PCA, and a preliminary modified and reduced version based only on common items to versions in both languages. Because both ADHD and EF were measured by self-report the issue of shared method variance must always be considered when making interpretations of their presumed associations. There is also further limitation of using rating scales associated with the subjectivity, reliability and accuracy of self- and other-reports, as well as the psychometric construction of the scales themselves (Naglieri & Goldstein, 2014). Finally, our cross-sectional design sets limits on any causal interpretation of the association between covariates and factor scores.

While results must be interpreted in light of these limitations, the study nevertheless contributes a new Spanish-language version of an EF measure, with potential for clinical and research use. Further studies may potentially confirm this structure and reliability estimates with larger samples, and in samples that capture the varying patterns of symptoms and the high rates of coexisting disorders associated with ADHD (Young et al., in press) and executive functioning. Further validity may be assessed with other non-ADHD populations that may be executively compromised, such as brain injured or the substance dependent population.

Conclusions

Our findings suggest that the Spanish BDEFS has adequate psychometric properties. Its subscale scores evidenced adequate reliability, a factor structure generally consistent with the English version, and we found associations between symptoms of ADHD with the BDEFS latent factors. These results may be cautiously interpreted as providing support to the theory of executive functioning proposed by Barkley (2011), inasmuch as the scale scores seem to assess EF in a manner consistent with this model. That is, it assesses EF at the Strategic-Cooperative level, and the 5 subscales are related to the self-regulatory activities described in the model. We provided evidence of the validity of scores on the Spanish version of the BDEFS as a measure of EF deficits in adults, and documented the relation of EF deficits to ADHD symptom clusters in community adults screened for ADHD symptoms.

ADHD causes significant impairment and morbidity in children and adults (Dalsgaard, Ostergaard, Leckman, Mortensen, & Pedersen, 2015; Gonzalez, Velez-Pastrana, Ruiz Varcarcel, Levin, & Albizu-Garcia, 2015; Shaw et al., 2012), and its relation to EF is receiving considerable attention in research, with sometimes-conflicting findings and views, underscoring the need to better understand the association between the two constructs.

This study makes an important contribution to the field, by translating and examining the psychometric properties of the BDEFS, thus providing an alternative for assessing EF in Spanish-speaking populations. Typically, research is scarce in often under-served and under-represented populations such as Spanish-speaking Hispanics/Latinos in the United States, despite being a fast growing segment of the population. Too often, clinical and research measures that are culturally sensitive and in an appropriate language are unavailable. Moreover, from a more global perspective, international collaborations would benefit greatly from having available measures of constructs such as ADHD and EF in the relevant languages, with necessary psychometric documentation. Both are issues that the current study addresses and in that respect serves to fill a significant gap.