Sluggish Cognitive Tempo in Adults: Psychometric Validation of the Adult Concentration Inventory

Abstract

As interest in sluggish cognitive tempo (SCT) increases, a primary limitation for the field is the lack of a unified set of symptoms for assessing SCT. No existing SCT measure includes all items identified in a recent meta-analysis as optimal for distinguishing between SCT and attention-deficit/hyperactivity disorder (ADHD). This study evaluates a new self-report measure for assessing SCT in adulthood, the Adult Concentration Inventory (ACI), which was developed in response to the meta-analytic findings for assessing SCT. Using a large, multi-university sample (N = 3,172), we evaluated the convergent and discriminant validity and reliability of the ACI. We also evaluated the ACI measure of SCT in relation to self-reported demographic characteristics, daily life executive functioning, socio-emotional adjustment (i.e., anxiety/depression, loneliness, emotion dysregulation, self-esteem), and functional impairment. Exploratory confirmatory factor analyses resulted in 10 ACI items demonstrating strong convergent and discriminant validity from both anxiety/depressive and ADHD inattentive symptom dimensions. SCT was moderately-to-strongly correlated with daily life EF deficits, poorer socio-emotional adjustment, and greater global functional impairment. Moreover, SCT remained uniquely associated in structural regression analyses with most of these external criterion domains above and beyond ADHD. Finally, when internalizing symptoms were also covaried, SCT, but not ADHD inattention, remained significantly associated with poorer socio-emotional adjustment. These findings support the use of the ACI in future studies examining SCT in adulthood and make a major contribution in moving the field toward a unified set of SCT items that can be used across studies.

It is increasingly clear that attention problems are heterogeneous in nature and are not wholly captured by the inattention symptoms used to assess, diagnose, and treat attention-deficit/hyperactivity disorder (ADHD). A separate set of attentional symptoms that includes daydreaming, staring, mental fogginess/confusion, and slowed behavior/thinking have been termed sluggish cognitive tempo (SCT) (Becker, Marshall, & McBurnett, 2014). Although it was initially unclear as to whether SCT symptoms are distinct from ADHD inattentive (ADHD-IN) symptoms, a recent meta-analysis found clear support for a set of SCT symptoms that are consistently distinct from ADHD-IN symptoms across studies that included over 19,000 individuals (Becker, Leopold et al., 2016). Moreover, a rapidly accumulating body of research indicates that SCT symptoms are associated with a range of adjustment difficulties and impairments (Barkley, 2014; Becker, 2017; Becker & Barkley, in press). For instance, recent research demonstrates that, above and beyond ADHD symptoms, SCT symptoms are significantly associated with increased loneliness and social withdrawal, greater anxiety/depression, lower self-esteem, academic difficulties, poorer emotion regulation, and greater suicide risk (Becker, Leopold et al., 2016). Given these findings, it has been suggested that SCT may be a psychiatric disorder in its own right or be a construct of cross-diagnostic importance (Barkley, 2014, 2016; Becker & Barkley, in press; Becker, Leopold et al., 2016).

Toward a Unified Set of SCT Symptoms

For the study of SCT to continue to advance, it is critical to develop measures that adequately capture the SCT construct and allow for comparing findings across studies. Although multiple SCT rating scales have been developed within the last decade (Barkley, 2012, 2013; Becker, Luebbe, & Joyce, 2015; Lee, Burns, Snell, & McBurnett, 2014; McBurnett et al., 2014; Penny, Waschbusch, Klein, Corkum, & Eskes, 2009), they vary widely in the number and type of SCT symptoms included. In fact, a recent SCT meta-analysis found that previous studies had used over 150 different items for assessing SCT (Becker, Leopold et al., 2016). This multitude of SCT items measured 18 core features of SCT, and 13 of these 18 core features were subsequently identified as optimal, consistently loading on an SCT factor that was distinct from the ADHD-IN factor in exploratory factor analyses (EFAs) (Becker, Leopold et al., 2016). However, none of the existing SCT measures included all 13 of these optimal SCT items. Moreover, a recent study by McBurnett and colleagues (McBurnett et al., 2014) reported that a set of “working memory” items assessing mental confusion were specific to SCT. The factor identified by McBurnett and colleagues is consistent with early studies of SCT (Carlson & Mann, 2002; Lahey et al., 1988) which included items reflecting difficulties with mental confusion. Thus, a new self-report measure for assessing SCT in adulthood, the Adult Concentration Inventory (ACI), was developed to include all 13 of the optimal SCT items identified in the meta-analysis (Becker, Leopold et al., 2016) as well as mental confusion items. Table 1 shows the 16 items on the ACI. Psychometric validation of meta-analytic-based measures such as the ACI are needed in order to move the field closer to a unified set of SCT items that can be examined across studies.

Table 1.

Adult Concentration Inventory (ACI) Items and Support from Meta-Analysis and Validation in Adults

ACI items	Support from SCT meta-analysisa	Support from validation in adults b
1. I am slow at doing things.	✓	–
2. My mind feels like it is in a fog.	✓	–
3. I stare off into space.	✓	✓
4. I feel sleepy or drowsy during the day.	✓	✓
5. I daydream.	✓	✓
6. I lose my train of thought.	✓	–
7. I am not very active.	✓	✓
8. I get lost in my own thoughts.	✓	✓
9. I get tired easily.	✓	✓
10. I forget what I was going to say.	–	✓
11. I feel confused.	✓	–
12. I am not motivated to do things.	✓	✓
13. I zone out or space out.	✓	✓
14. My mind gets mixed up.	–	–
15. My thinking seems slow or slowed down.	✓	✓
16. I have a hard time putting my thoughts into words.	–	–

Note. ACI = adult concentration inventory. SCT = sluggish cognitive tempo.

^aSupport from meta-analysis indicates that the ACI item was shown to be an optimal item for assessing in a recent meta-analysis examining SCT as distinguishable from attention-deficit/hyperactivity disorder inattentive (ADHD-IN) symptoms (Becker, Leopold et al., 2016).

^bSupport from validation in adults indicates that the ACI item demonstrated both convergent and discriminant validity in the exploratory confirmatory factor analyses (CFAs) conducted as part of this validation study (see Table 2 for specific exploratory CFA results).

Studies to date that have used EFA to examine the convergent and discriminant validity of SCT have focused solely on ADHD, and ADHD-IN symptoms in particular. That is, studies have examined whether SCT items load together on an SCT factor (i.e., convergent validity) while also loading weakly on the ADHD-IN factor (i.e., discriminant validity). This approach makes theoretical and empirical sense given the historical link between SCT and ADHD-IN, two constructs of attention that are strongly correlated and were at times believed to be one and the same (Becker, Marshall, et al., 2014). Yet SCT is also strongly associated with internalizing symptoms of anxiety and depression (Becker, Leopold et al., 2016). Although several recent studies have found SCT to be distinct from these internalizing domains using confirmatory factor analytic (CFA) techniques (Becker, Luebbe, Fite, Stoppelbein, & Greening, 2014; Becker, Luebbe, et al., 2015; Lee, Burns, & Becker, 2016; Lee et al., 2014; Willcutt et al., 2014), it is important to identify the best SCT items that show strong convergent and discriminant validity relative to anxiety and depression symptom dimensions. CFA procedures, in which cross-loadings are usually fixed at zero, do not allow us to know the magnitude that SCT symptoms cross-load on depression and anxiety factors. In contrast, EFA procedures do provide this information (i.e., the size of the SCT symptom loadings on anxiety and depression factors). The ideal set of SCT symptoms thus needs to show substantial loadings on the SCT factor (convergent validity) along with higher loadings on the SCT factor than ADHD-IN, depression, and anxiety factors (discriminant validity). The identification of individual SCT symptoms with convergent and discriminant validity is also a prerequisite condition for research to determine if SCT has unique and different external correlates relative to these other dimensions (e.g., the failure of the SCT dimension to have unique and different external correlates relative to other dimensions would not be due to failure of the individual SCT symptoms to have discriminant validity with the other dimensions).

SCT in Adults

Although a growing body of research shows SCT to be separable from ADHD-IN and associated with poorer adjustment across a range of functioning domains, all of the studies cited above were conducted in children. Far fewer studies have examined SCT in adults. Specifically, the aforementioned meta-analysis identified 61 studies examining SCT in children and only 12 studies examining SCT in adults. Only three studies have evaluated whether SCT symptoms are distinct from ADHD-IN symptoms in adulthood, with all three finding support for SCT being distinct from ADHD (Barkley, 2012; Becker, Langberg, Luebbe, Dvorsky, & Flannery, 2014; Leopold, Bryan, Pennington, & Willcutt, 2015). No study to date has examined whether SCT symptoms are statistically distinct from internalizing symptoms of depression and anxiety in adulthood. Establishing the convergent and discriminant validity of the individual SCT symptoms in relation to internalizing symptoms is a necessary condition for research to determine if the SCT dimension has unique and different correlates from the internalizing dimension. It is especially important to evaluate the internal validity of SCT in young adulthood, since it appears that both SCT and depression increase after childhood (Leopold et al., 2016; Merikangas et al., 2010), which may in part explain why higher rates of SCT have been found in college students as compared to children or general adult samples (Flannery, Becker, & Luebbe, 2016; Jarrett, Rapport, Rondon, & Becker, 2017; Wood, Lewandowski, Lovett, & Antshel, 2017). Furthermore, three of the features of SCT that were identified as optimal in the SCT meta-analysis (i.e., loses train of thought, lost in thoughts, stares blankly) remain unexamined in any adult study (Becker, Leopold et al., 2016).

Additional studies are likewise needed to examine the external correlates of SCT in young adulthood. The extant evidence suggests that SCT is associated with poorer functioning in adulthood. Specifically, consistent with findings of studies conducted with children, studies conducted with adults have found SCT in adults to be associated with greater internalizing symptoms (Becker, Langberg, et al., 2014; Leikauf & Solanto, 2017; Wood et al., 2017) and emotion regulation difficulties (Barkley, 2012; Flannery, Becker, et al., 2016; Jarrett et al., 2017; Wood et al., 2017). It thus appears that SCT is uniquely associated with poorer socio-emotional adjustment, although replication in a young adult population and extension to other domains of socio-emotional functioning (e.g., self-esteem, loneliness) is needed to more fully understand the impact of SCT on adjustment and to guide intervention development.

Daily life executive functioning (EF) has also been examined as a potential external correlate of SCT, with mixed evidence reported in the few studies to date. The mixed findings reported to date may be due to different measures used across studies or differences in associations between SCT and EF across development. Studies conducted with children generally demonstrate ADHD-IN symptoms to be more consistently and strongly associated than SCT symptoms with ratings of daily life EF, though SCT has been measured differently across these studies (Araujo Jiménez et al., 2015; Barkley, 2013; Becker & Langberg, 2014; Tamm, Brenner, Bamberger, & Becker, 2016). In contrast, studies with adults that have all used Barkley’s nine-item measure of SCT indicate that SCT and ADHD-IN may be similarly related to overall deficits in daily life EF (Flannery, Luebbe, & Becker, 2016; Leikauf & Solanto, 2017; Wood et al., 2017), with some differences emerging for specific EF domains. Three studies found ADHD-IN symptoms to be more strongly associated than SCT with motivation and time-management domains of EF, whereas SCT symptoms were found to be more strongly associated than ADHD-IN with self-regulation of emotion (Barkley, 2012; Jarrett et al., 2017; Leikauf & Solanto, 2017). These three studies found mixed evidence for SCT and ADHD-IN being similarly or differentially related the self-organization/problem-solving and self-restraint domains (Barkley, 2012; Jarrett et al., 2017; Leikauf & Solanto, 2017).

Similar to daily life EF, there also appears to be an association between SCT and overall self-reported functional impairment in adulthood (Barkley, 2012; Flannery, Luebbe et al., 2016; Jarrett et al., 2017; Wood et al., 2017). In a nationally representative sample of adults, Barkley (Barkley, 2012) found ADHD symptoms to be a much stronger predictor than SCT symptoms of overall impairment (measured by taking the mean self-report rating across 15 domains of functioning such as home life, work/occupation, romantic relationships, money management, and daily self-care). In contrast, using the same measure as Barkley, SCT and ADHD symptoms appear to be similarly related to self-reported impairment in college students specifically (Flannery, Luebbe et al., 2016; Wood et al., 2017). Additional studies are needed to shed light on the degree to which SCT symptoms are uniquely associated with impairment above and beyond ADHD symptoms, though extant studies provide support for the hypothesis that SCT will be uniquely associated with self-reported impairment in a college student sample.

The Present Study

The present study was designed to advance the field of SCT in several ways. First, this study aims to move the field closer to a unified set of SCT items by being the first study to include all 13 of the optimal SCT items identified in a recent meta-analytic review (Becker, Leopold et al., 2016) in addition to SCT-specific mental confusion items that have preliminary evidence (McBurnett et al., 2014) and have long been conceptualized as part of SCT (Carlson & Mann, 2002; Lahey et al., 1988). Second, this is the first study to use exploratory CFA (also referred to as exploratory structural equation modeling; ESEM) (Asparouhov & Muthén, 2009) to identify SCT items that are clearly distinct from not only ADHD-IN symptoms but also internalizing symptoms of anxiety/depression. In exploratory CFA, the number of factors is set a priori (i.e., SCT, ADHD-IN, anxiety/depression factors), but cross-loadings are allowed rather than being set to zero. Thus, exploratory CFA allows for the assessment of both convergent validity (i.e., SCT symptoms have strong primary loadings on the SCT factor) and discriminant validity (i.e., cross loadings of SCT symptoms on ADHD-IN and anxiety/depression are lower than their primary loadings on the SCT factor) of SCT symptoms. Third, this study contributes to the very small body of research examining SCT beyond childhood. Aside from a nationally representative study by Barkley (Barkley, 2012), previous studies examining SCT in adults consisted of adults referred to an ADHD specialty clinic (Leikauf & Solanto, 2017) or college students from one or two universities (Becker, Langberg, et al., 2014; Becker, Luebbe, & Langberg, 2014; Flannery, Becker et al., 2016; Langberg, Becker, Dvorsky, & Luebbe, 2014; Leopold et al., 2015; Wood et al., 2017). Given preliminary evidence that rates of SCT may be higher in college students specifically (Flannery, Becker, et al., 2016; Jarrett et al., 2017; Wood et al., 2017), coupled with the developmental and environmental demands encountered in emerging adulthood generally (Arnett, 2000; Arnett, Zukauskiene, & Sugimura, 2014), it is important to examine SCT in the college student population. The current study includes data collected from five universities across the United States in order to increase representativeness of the college student population. As such, this study also has the largest sample size of any SCT study conducted to date.

Using this large, multi-university sample, the current study had four objectives:

1. The first objective was to identify the SCT items that showed strong convergent and discriminant validity relative to the ADHD-IN and anxiety/depression symptom dimensions. Since the ACI SCT items were primarily based on items identified in a recent meta-analysis as optimal for assessing SCT (Becker, Leopold et al., 2016), we expected most, if not all, of the ACI SCT items to demonstrate strong convergent and discriminant validity.

2. The second objective was to evaluate the reliability of the SCT factor and the fit of an SCT, ADHD-IN, ADHD-hyperactivity/impulsivity (ADHD-HI), and internalizing CFA model (i.e., a determination of the true score variance for the total scores on the SCT dimension and the discriminant validity of the SCT factor with the other symptom factors). As in previous research (Becker, Leopold et al., 2016), we expected the SCT dimension to demonstrate excellent reliability as well as strong discriminant validity from the ADHD and internalizing dimensions.

3. The third objective was to evaluate SCT in relation to external correlates. Specifically, correlations were conducted to examine SCT and ADHD-IN in relation to self-reported demographic characteristics, socio-emotional functioning (i.e., anxiety/depression, loneliness, emotion dysregulation, self-esteem), daily life EF domains (i.e., self-management to time, self-organization and problem-solving, self-restraint, self-motivation, and self-regulation of emotion), and functional impairment. We expected SCT to be significantly correlated with each of the external correlates with the exception of age and race (Becker, Leopold et al., 2016).

4. The fourth objective was to examine whether SCT was uniquely associated with self-reported socio-emotional adjustment, daily life EF, and functional impairment above and beyond other psychopathology dimensions. We first controlled for the ADHD-IN and ADHD-HI dimensions in order to clearly establish the degree to which SCT has unique external correlates relative to ADHD and for our findings to be comparable to the Barkley’s (2012) nationally representative sample of adults. Since SCT is increasingly conceptualized as falling under the internalizing umbrella of psychopathology (Becker et al., 2013; Lee, Burns, Beauchaine, & Becker, 2016), we expected that SCT would be independently associated with poorer socio-emotional adjustment in adults (Becker, Langberg, et al., 2014; Jarrett et al., 2017; Leikauf & Solanto, 2017; Wood et al., 2017). We also expected to replicate findings from smaller college student studies that found SCT to be uniquely related to both daily life EF deficits and functional impairment above and beyond ADHD symptoms (Flannery, Luebbe et al., 2016; Wood et al., 2017). In terms of specific EF domains, we tentatively hypothesized that ADHD-IN would be more strongly associated than SCT with the self-motivation and time-management EF domains whereas SCT would be more strongly associated than ADHD with the self-regulation of emotion domain (Barkley, 2012; Jarrett et al., 2017; Leikauf & Solanto, 2017). Finally, we conducted additional analyses that controlled for the ADHD dimensions as well as internalizing symptoms in order to make our findings comparable to other studies (Jarrett et al., 2017; Leikauf & Solanto, 2017; Wood et al., 2017) in evaluating whether SCT remained uniquely associated with socio-emotional adjustment, EF functioning and impairment above and beyond not only ADHD symptoms but also anxiety/depression. We hypothesized that SCT would remain significantly associated with poorer socio-emotional adjustment, greater EF difficulties, and more functional impairment when controlling for ADHD and internalizing symptom dimensions (Becker, Langberg et al., 2014; Jarrett et al., 2017; Leikauf & Solanto, 2017; Wood et al., 2017).

Methods

Participants

Participants were 3,172 undergraduate students enrolled in five universities in the United States (between 431 and 826 students participated at each university). Four of the five universities are public universities, and the universities are located in the Midwestern, Southeast, and Northwest regions of the United States. Participants ranged in age from 18 to 29 years (M = 19.24, SD = 1.52) and approximately two thirds were female (69.8%, n = 2,214). The majority (80.4%) of participants self-identified as White; the remaining participants self-identified either as Asian/Asian American (6.4%), Black/African American (6.3%), Native Hawaiian/Other Pacific Islander (0.7%), American Indian/Alaska Native (0.6), or Biracial/Multiracial (5.6%). 6.7% of participants self-identified as Hispanic or Latino. Most participants (59.5%) were in their first year of college; the remaining participants were in their second (20.4%), third (12.1%), fourth (7.5%), or other (0.4%) year of college. Although a formal psychiatric assessment/diagnosis was not conducted in the context of this multi-site study, participants were asked if they had ever received a professional mental health diagnosis. Three-quarters of participants (n = 2,390) reported never receiving a mental health diagnosis; the remaining participants reported having been diagnosed with an anxiety disorder (12.6%, n = 401), depression (12.3%, n = 12.3%), ADD/ADHD (9.4%, n = 298), a learning disorder (2.5%, n = 78), a personality disorder (0.4%, n = 12), bipolar disorder (0.8%, n = 25), alcohol abuse/disorder (0.3%, n = 9), autism/Asperger’s (0.2%, n = 5), or schizophrenia (0.1%, n = 2).

Procedures

This study was approved by the local Institutional Review Board (IRB) at each university, with the individual study protocols specifying that data would be merged across sites for analysis and dissemination. Procedures varied slightly based on normative practices at each institution. At four of the sites, this study was an anonymous online survey. Specifically, after signing up for the study in Sona, participants were directed to the survey in Qualtrics where they first read an information sheet describing the study and providing contact information of the local investigator, IRB, and student counseling center. If the participant chose to continue, they were then directed to the survey, and after completing the survey, automatically received course credit in Sona for their participation. At the fifth university, participants were given an individual time-slot for coming to the investigator’s laboratory, and after providing informed consent in-person, completed the same Qualtrics survey as participants at the other four universities on their own time. They were similarly granted course credit for participation.

Measures

SCT symptoms

As described above, the Adult Concentration Inventory (ACI) (Becker, Burns, & Willcutt, 2015) was developed for this study as a new adult self-report measure of SCT that included all 13 SCT items identified in a recent meta-analysis as optimal for distinguishing between SCT and ADHD-IN symptoms (Becker, Leopold et al., 2016). Three additional mental confusion items were added since preliminary evidence found these items to be important for assessing SCT (McBurnett et al., 2014), resulting in a total of 16 SCT symptom items (see Table 1). These 16 items are rated on a four-point scale (0 = not at all, 1 = sometimes, 2 = often, 3 = very often) in reference to the past six months. The ACI also includes 8 items assessing current impairment due to the presence of any SCT behaviors (0 = no difficulty, 4 = severe difficulty), but these SCT-specific impairment items were not examined in the current study. The full ACI scale is available from the corresponding author. In the present study, Cronbach’s ɑ was .89 for the 10-item ACI scale identified below in the exploratory CFAs.

ADHD symptoms

The Barkley Adult ADHD Rating Scale-IV (BAARS-IV) (Barkley, 2011a) was used to assess ADHD symptoms. The BAARS-IV includes 18 items that are consistent with the Diagnostic and Statistical Manual of Mental Disorders (4^th ed.; DSM-IV) (American Psychiatric Association, 1994) symptoms of ADHD that have been updated in their wording to also reflect modifications made in DSM-5 (American Psychiatric Association, 2013). Using a four-point scale (in this study, 0 = not at all, 3 = very often), participants respond to each item with reference to how often each statement best describes their behavior over the past six months. The ADHD-IN (e.g., “difficulty sustaining my attention in tasks or fun activities”) and ADHD-HI (e.g., “fidget with hands and feet or squirm in seat”) subscales of the BAARS-IV have demonstrated satisfactory internal consistency and test-retest reliability over a 2- to 3-week time period (Barkley, 2011a). In the present study, Cronbach’s ɑs were .89 and .83 for the ADHD-IN and ADHD-HI dimensions, respectively.

Internalizing symptoms

The anxiety and depression subscales of the Depression Anxiety Stress Scales-21 (DASS-21) (Antony, Bieling, Cox, Enns, & Swinson, 1998; Lovibond & Lovibond, 1995) were used to assess internalizing symptoms. Each subscale has seven items (e.g., “I was worried about situations in which I might panic and make a fool of myself”, “I felt downhearted and blue”). Participants respond to each item in reference to the past week using a four-point scale (0 = did not apply to me at all, 3 = applied to me very much or most of the time). The DASS-21 demonstrates high reliability and is widely accepted as being valid for use with college-aged participants (Antony et al., 1998; Sinclair et al., 2012). In the present study, Cronbach’s ɑ was .89 for the internalizing scale identified below in the exploratory CFAs.

Executive functioning

The Barkley Deficits in Executive Functioning Scale (BDEFS) (Barkley, 2011b) was used to assess daily life EF. The BDEFS includes 89 items that assess multiple EF domains, including Self-Management to Time (e.g., “have trouble doing what I tell myself to do”; Cronbach’s ɑ = .96), Self-Organization/Problem Solving (e.g., “have trouble doing things in their proper order or sequence”; ɑ = .96), Self-Restraint (e.g., “likely to do things without considering the consequences for doing them”; ɑ = .93), Self-Motivation (e.g., “I do not have the willpower or determination that others seem to have”; ɑ = .93), and Self-Regulation of Emotion (e.g., “overreact emotionally”; ɑ = .93). Participants rate each item on a four-point scale (1 = never or rarely, 4 = very often) in reference to the past six months, with higher scores indicating greater EF deficits. The BDEFS has demonstrated satisfactory 2 to 3 week test-retest reliability and satisfactory validity (Barkley, 2011b).

Emotion dysregulation

The Difficulties in Emotion Regulation Scale (DERS) (Gratz & Roemer, 2004) was used to measure participants’ emotion dysregulation. The DERS is a 36-item scale that measures difficulties in one’s awareness and understanding of emotions, acceptance of emotions, and the ability to engage in goal-directed behavior (and refrain from impulsive behavior) when experiencing negative emotions (e.g., “When I’m upset, I believe that I will remain that way for a long time”, “When I’m upset, I become irritated at myself for feeling that way”). Respondents rate each item on a five-point scale (1 = almost never; 5 = almost always), with items coded so that higher scores indicate greater emotion regulation difficulties. The DERS is a widely accepted scale and has shown high internal consistency and good test–retest reliability over a period ranging from 4 to 8 weeks (Gratz & Roemer, 2004). As in previous SCT research (Flannery, Becker, et al., 2016), the total DERS score was used for analyses (Cronbach’s ɑ = .94), with higher scores indicating greater emotion dysregulation.

Self-esteem

The Rosenberg Self-Esteem (RSE) Scale (Rosenberg, 1965) is a well-validated and commonly-used measure of self-esteem in adults. The 10 items of the RSE assess a person’s overall evaluation of his or her worthiness as a human being (e.g., “I am able to do things as well as most other people”). Responses are coded on a 4-point scale ranging from 1 (strongly disagree) to 4 (strongly agree). Consistent with previous research (Schmitt & Allik, 2005; Tafarodi & Milne, 2002), a total mean score was used in this study (Cronbach’s ɑ = .90). Higher scores indicate higher self-esteem.

Loneliness

The UCLA Loneliness Scale (Version 3) is a well-validated and frequently used measure of loneliness in adulthood (Russell, 1996). The UCLA Loneliness Scale consists of 20 items (e.g., “How often do you feel alone?”) rated on a four-point scale (1 = never, 2 = rarely, 3 = sometimes, or 4 = often). A summary score is calculated in which higher scores indicated greater loneliness (Cronbach’s ɑ = .93).

Functional impairment

The Barkley Functional Impairment Scale (BFIS) (Barkley, 2011c) is a 15-item self-report measure of functional impairment in adults. Using a 10-point scale (0 = not at all, 9 = severe), as well as a “does not apply” option, participants indicate how much difficulty they have effectively functioning in 15 major life activities. The domains of impairment included on the BFIS are: home life, work or occupation, social interactions with strangers and acquaintances, relationships with friends, completing chores and household management, in community activities, educational activities, marital or dating relationships, sexual activities, management of finances, operating a motor vehicle, organizing daily responsibilities, taking care of and raising children, maintaining health (e.g., exercise, nutrition, preventative medical and dental care), and daily care (e.g., dressing, bathing, hygienic care). The present study included each of these impairment items with the exception of the “taking care of and raising children” item since this did not apply to the vast majority of students at the universities where the research was conducted. Consistent with previous research (Barkley, 2012; Wood et al., 2017), we calculated a mean score of the impairment items with higher scores indicated greater overall functional impairment (Cronbach’s ɑ = .95).

Analytic Strategy

Survey validity checks

Prior to running primary analyses, data were screened for invalid responses. In order to improve the quality of participant responses, we utilized an instructional manipulation check (IMC) (Oppenheimer, Meyvis, & Davidenko, 2009). The IMC is a question given at the start of a survey that measures whether a participant is reading the instructions carefully. The IMC consists of a set of instructions and a question that is similar to other survey questions in length and response format. In contrast to other questions, though, the instructions indicate that the respondent should ignore the question and click on a specific answer. The use of such questions has been shown to improve subsequent responses, since respondents are primed to focus on the importance of reading the instructions (see Oppenheimer et al., 2009). It should be noted that such a manipulation has been shown to be effective even when participants respond incorrectly to the initial IMC (i.e., they did not read the instructions carefully). In turn, we chose to use the IMC as a prompt for respondents to recognize the importance of attending to instructions but did not exclude participants with an initially incorrect IMC response. Respondents were required to answer this question correctly, though, to be able to proceed with the survey. We modeled our question after the example used in Oppenheimer et al. (2009). Please see Figure 1 in that paper for details.

In addition to the IMC, we also utilized “trap” questions that were designed to detect individuals who were quickly responding to survey questions without sufficient attention to item content. Within each measure of our battery, we included a question that stated something like “If you are paying attention, please click on the response ‘sometimes’.” In addition to these questions, we had one question at the end of the full survey that asked participants the following: “How much effort did you put into this study from 0 to 10 (0 = not much effort at all, 5 = moderate effort, 10 = my best effort)?”

To ensure the validity of responding, we set a threshold of 50% accuracy or higher for the “trap questions” and a self-reported effort rating of 5 or higher. We chose this threshold since we wanted to ensure that participants were putting forth sufficient effort while also not excluding participants who might have responded inaccurately due to attention lapses or impulsive responding (e.g., those with elevated ADHD or SCT symptoms). Of the 3,307 participants in our initial sample, 3,243 (98.1%) reported an overall effort of at least 5 out of 10. Further, of the 3,307 participants, 3,172 (95.9%) answered at least 50% of the “trap” questions correctly AND reported an overall effort of at least 5. The primary analyses included these 3,172 participants.

Primary analyses

All primary analyses were conducted in Mplus v7.4 (Muthén & Muthén, 1998–2012). First, a three-factor exploratory CFA model was applied to the 16 SCT items, the 9 ADHD-IN items, and the 14 anxiety/depression items¹ to determine the convergent and discriminant validity of the ACI SCT items (i.e., the number of factors were a priori set to three with items allowed to cross-load; see (Asparouhov & Muthén, 2009). To ensure a stringent test of convergent and discriminant validity, the full sample was randomly split into two samples (Sample A and Sample B) (Osborne & Fitzpatrick, 2012). Similar to previous studies (Willcutt et al., 2014), an item was determined to load onto a factor if there was a loading of 0.60 in at least one of the samples (e.g., Sample A) and a loading on the same factor of at least 0.50 in the other sample (e.g., Sample B) and loadings of less than 0.30 on the other two factors for both Samples A and B. These thresholds were used to establish convergent and discriminant validity, and this approach ensures that the items identified for each factor demonstrates structural replicability (Osborne & Fitzpatrick, 2012). The SCT, ADHD-IN, and anxiety/depression items that showed convergent and discriminant validity in the exploratory CFAs were used in the subsequent analyses. Item ratings were treated as ordered-categories. Analyses used the robust weighted least squares (WLSMV) estimator. There was little missing information (covariance coverage was approximately 99%; the WLSMV uses a pairwise approach to missing information).

Second, a four-factor CFA was conducted with the SCT, ADHD-IN, ADHD-HI, and anxiety/depression factors in the full sample. The primary purposes of this analysis were to calculate the reliability coefficient (i.e., true score variance) (Brown, 2015) in the total SCT and other symptom dimension scores and to evaluate the correlations among the psychopathology factors, and so ADHD-HI symptoms were also added to the model. As noted by Brown (Brown, 2015), “in applied research, a factor correlation that exceeds .80 or .85 is often used as the criterion to define poor discriminant validity” (p. 116). Using this convention in the field, correlations less than 0.80 between SCT and the other psychopathology factors would provide further support for the discriminant validity of the ACI SCT measure in a classical CFA framework (i.e., when factor loadings are set to zero).

Third, correlations were conducted to examine SCT, ADHD-IN and ADHD-HI in relation to demographic characteristics, daily life EF domains (i.e., self-management to time, self-organization and problem-solving, self-restraint, self-motivation, and self-regulation of emotion), socio-emotional functioning (i.e., anxiety/depression, loneliness, emotion dysregulation, self-esteem), and functional impairment. Fourth, structural regression analyses were conducted to determine the ability of SCT, ADHD-IN, and ADHD-HI to uniquely predict daily life socio-emotional functioning, EF, and functional impairment. The Mplus model constraint procedure was used to test factor correlations and the partial standardized regression coefficients for significant differences. Lastly, the structural regression analyses were repeated with the internalizing symptom dimension added as a predictor along with the SCT, ADHD-IN, and ADHD-HI dimensions in order to evaluate whether SCT remained uniquely associated with the outcomes when controlling for ADHD as well as anxiety/depression symptoms (this approach is similar to a traditional hierarchical regression analysis, which is not possible in Mplus).

Results

Exploratory Confirmatory Factor Analyses

As summarized in Table 2, 10 of the 16 SCT symptoms (ACI items 3, 4, 5, 7, 8, 9, 10, 12, 13, and 15) demonstrated convergent and discriminant validity in the factor analyses across both Samples A and B. All 9 ADHD-IN items demonstrated convergent and discriminant validity in both Samples A and B, and 9 of the 14 anxiety/depression items demonstrated convergent and discriminant validity across Samples A and B. As shown in Table 2 (and summarized in Table 1), four SCT symptoms (ACI items 1, 2, 6, and 11) were found to have strong convergent and discriminant validity in the SCT met-analysis but did not demonstrate adequate convergent and discriminant validity in this study. Two other SCT symptoms (ACI items 14 and 16) that were included in this study due to few studies including these aspects of SCT in previous studies also failed to demonstrate convergent and discriminant validity across both factor analytic models.

Table 2.

Exploratory Factor Analyses of ADHD-Inattention, Anxiety/Depression, and Potential Sluggish Cognitive Tempo (SCT) Symptoms from the Adult Concentration Inventory (ACI)

Item	Sample A			Sample B
	Factor 1: SCT	Factor 2: ADHD-IN	Factor 3: ANX/DEP	Factor 1: SCT	Factor 2: ADHD-IN	Factor 3: ANX/DEP
SCT 1	0.32*	0.33*	0.07*	0.34*	0.31*	0.03
SCT 2	0.44*	0.24*	0.21*	0.39*	0.22*	0.26*
SCT 3	0.83*	−0.001	−0.16*	0.84*	−0.003	−0.09*
SCT 4	0.64*	−0.04	0.14*	0.54*	−0.02	0.21*
SCT 5	0.78*	0.07*	−0.10*	0.73*	0.17*	−0.12*
SCT 6	0.31*	0.01	0.16*	0.19*	0.14*	0.19*
SCT 7	0.69*	0.06	0.002	0.66*	0.01	0.10*
SCT 8	0.66*	−0.10*	0.16*	0.55*	−0.02	0.23*
SCT 9	0.76*	0.02	−0.10*	0.65*	0.12*	−0.06
SCT 10	0.71*	0.02	0.09*	0.64*	0.11*	0.09*
SCT 11	0.30*	0.33*	0.21*	0.29*	0.28*	0.22*
SCT 12	0.84*	0.07*	−0.11*	0.89*	−0.02	−0.06
SCT 13	0.81*	0.02	0.01	0.68*	0.17*	0.02
SCT 14	0.61*	0.15*	0.07*	0.48*	0.27*	0.11*
SCT 15	0.76*	−0.05	−0.09*	0.74*	−0.14*	−0.004
SCT 16	0.56*	0.14*	0.03	0.46*	0.15*	0.12*
ADHD-IN 1	0.04	0.75*	0.06	0.02	0.78*	−0.03
ADHD-IN 2	0.12*	0.67*	0.10*	0.03	0.68*	0.14*
ADHD-IN 3	0.05	0.63*	−0.03	0.05	0.67*	−0.05
ADHD-IN 4	−0.13*	0.89*	0.01	−0.15*	0.92*	−0.01
ADHD-IN 5	0.01	0.75*	0.05	−0.09*	0.82*	0.06
ADHD-IN 6	−0.01	0.74*	0.10*	−0.02	0.74*	0.07*
ADHD-IN 7	−0.02	0.73*	−0.05	−0.07	0.84*	−0.06
ADHD-IN 8	0.24*	0.63*	−0.03	0.17*	0.62*	0.02
ADHD-IN 9	0.19*	0.71*	−0.07*	0.12*	0.74*	−0.01
ANX/DEP 1	0.15*	0.11*	0.24*	0.05	0.22*	0.24*
ANX/DEP 2	0.001	0.22*	0.77*	−0.05	0.02	0.89*
ANX/DEP 3	0.29*	−0.01	0.48*	0.01	0.20*	0.50*
ANX/DEP 4	0.08*	0.44*	0.39*	0.07	0.37*	0.41*
ANX/DEP 5	0.34*	−0.02	0.44*	0.06	0.22*	0.42*
ANX/DEP 6	0.30*	0.06	0.50*	0.01	0.16*	0.60*
ANX/DEP 7	−0.04	0.25*	0.79*	−0.08*	0.02	0.91*
ANX/DEP 8	0.07*	0.15*	0.74*	0.000	−0.03	0.86*
ANX/DEP 9	0.33*	−0.02	0.63*	0.05	0.07	0.72*
ANX/DEP 10	−0.02	0.23*	0.78*	0.03	−0.004	0.88*
ANX/DEP 11	0.06	0.17*	0.78*	−0.001	−0.09*	0.94*
ANX/DEP 12	0.30*	−0.04	0.46*	−0.02	0.20*	0.47*
ANX/DEP 13	0.32*	0.01	0.58*	0.02	0.14*	0.65*
ANX/DEP 14	−0.01	0.23*	0.78*	−0.05	−0.09	0.97*

Note. Items with a hypothesized primary loading on each factor are in bold. ADHD-IN = attention-deficit/hyperactivity disorder inattention; ANX/DEP = anxiety/depression. SCT = sluggish cognitive tempo.

^*p<.05

Confirmatory Factor Analysis

The four-factor SCT (10 items), ADHD-IN (9 items), ADHD-HI (9 items), and anxiety/depression (9 items) model provided a good global fit, χ²(623)=8233, CFI=.94, TLI=.93, RMSEA=.062 (90% CI: .061,.063). SCT items had substantial loadings on the SCT factor (.64–.83), with similar results for ADHD-IN (.71–.84), ADHD-HI (.50–.76), and anxiety/depression (.76–.88). Reliability coefficients for SCT, ADHD-IN, ADHD-HI, and anxiety/depression were .89, .90, .83, and .92, respectively. Table 3 shows the correlations among the psychopathology factors. These correlations provide support for the discriminant validity of the ACI SCT factor from the ADHD-IN, ADHD-HI, and internalizing factors (i.e., all correlations were less than 0.80, with the highest factor correlation being 0.73).

Table 3.

Correlations among Psychopathology Factors and Descriptive Statistics

Variable	1	2	3	4
1. SCT	–
2. ADHD-IN	0.73	–
3. ADHD-HI	0.56	0.71	–
4. ANX/DEP	0.57	0.61	0.42	–
Mean	1.16	0.64	0.54	0.47
SD	0.58	0.56	0.48	0.60
Range	0.00 – 3.00	0.00 – 3.00	0.00 – 3.00	0.00 – 3.00
Skew	0.62	1.07	0.30	1.69
Kurtosis	0.30	0.98	1.88	2.62

Note. All correlations were significant at p < .001. The typical 1 to 4 range of the ADHD-IN and ADHD-HI subscales on the Barkley Adult ADHD Rating Scale (BAARS-IV) were shifted to be 0 to 3 for this study to be consistent with the SCT and ANX/DEP scales and make the mean scores more easily compared. ADHD-HI = attention-deficit/hyperactivity disorder hyperactivity-impulsivity. ADHD-IN = attention-deficit/hyperactivity disorder inattention. ANX/DEP = anxiety/depression. SCT = sluggish cognitive tempo.

Descriptive Statistics

Table 3 also summarizes the descriptive statistics of the SCT, ADHD-IN, ADHD-HI, and internalizing scores. As shown, the maximum range for each psychopathology dimension was 4, which is the maximum range for the measures. The mean score on the ACI SCT measure was 1.16, which corresponds to a response of “sometimes”, and paired-samples t-tests indicated a pattern of SCT > ADHD-IN > ADHD-HI > internalizing symptom scores (all ps < .001; see Table 3 for mean scores for all dimensions).

SCT and ADHD Symptoms in Relation to Demographic Characteristics

The correlations between age and the ADHD-IN, ADHD-HI, and SCT factors were 0.004 (SE = .02, p = .83), −0.02 (SE = .02, p = .39), and −0.06 (SE = .02, p < .001), respectively. The correlations between sex (coded 0 = males, 1 = females) and the ADHD-IN, ADHD-HI, and SCT factors were 0.01 (SE = .02, p = .64), 0.01 (SE = .02, p = .61), and 0.16 (SE = .02, p < .001), respectively. The correlations between race (coded 0 = non-White, 1 = White) and the ADHD-IN, ADHD-HI, and SCT factors were −0.03 (SE = .02, p = .11), 0.04 (SE = .02, p = .04), and −0.02 (SE = .02, p = .31), respectively.

Correlations of SCT, ADHD-IN and ADHD-HI with Adjustment Domains

Table 4 shows the factor correlations of SCT, ADHD-IN, and ADHD-HI with self-reported socio-emotional functioning, daily life EF, and functional impairment. All three of the psychopathology domains were associated with significantly (ps < .001) poorer adjustment and greater functional impairment, with most correlations having a medium-to-large effect size.

Table 4.

Correlations (Standard Errors) of Sluggish Cognitive Tempo and ADHD Symptoms with Executive Functioning, Socio-Emotional Functioning, and Functional Impairment

Variable	Executive Functioning
	Self-Organize	Time Management	Self-Restraint	Emotion	Self-Motivation
SCT	0.63 (0.01)b	0.58 (0.01)b	0.48 (0.01)c	0.48 (0.01)b	0.47 (0.02)b
ADHD-IN	0.70 (0.01)a	0.73 (0.01)a	0.59 (0.01)a	0.51 (0.01)a	0.62 (0.01)a
ADHD-HI	0.50 (0.02)c	0.45 (0.02)c	0.57 (0.01)b	0.46 (0.02)b	0.44 (0.02)b
	Socio-Emotional Functioning				Functional Impairment
	Anxiety/Depression	Emotion Dysregulation	Self-Esteem	Loneliness	Mean Impairment
SCT	0.49 (0.01)a	0.51 (0.01)a	−0.44 (0.02)a	0.40 (0.02)a	0.43 (0.02)b
ADHD-IN	0.51 (0.01)a	0.52 (0.01)a	−0.45 (0.02)a	0.41 (0.02)a	0.53 (0.01)a
ADHD-HI	0.37 (0.02)b	0.42 (0.02)b	−0.30 (0.02)b	0.26 (0.02)b	0.40 (0.02)c

Note. ADHD-IN = attention-deficit/hyperactivity disorder. HI = hyperactivity/impulsivity. IN = inattention. SCT = sluggish cognitive tempo.

All correlations significant, p < .001. Correlation coefficients with different superscripts differ significantly in relation to the outcome variable at p < .05.

The correlations between ADHD-IN and SCT with socio-emotional functioning domains did not significantly differ (ps > .05). Both ADHD-IN and SCT were both more strongly associated than ADHD-HI with higher anxiety/depression, more loneliness, greater emotion dysregulation, and lower self-esteem (ps < .001; see Table 4).

ADHD-IN had significantly (ps < .05) higher correlations than SCT and ADHD-HI with each of the five daily life EF domains, though the magnitude of difference was small (difference of < .10) for the self-organization/problem-solving and self-regulation of emotion domains. SCT had significantly (ps < .001) higher correlations than ADHD-HI with the daily EF domains of self-organization/problem-solving and self-management to time, whereas ADHD-HI had a significantly (p < .001) higher correlation than SCT with the self-restraint domain of EF (see Table 4).

ADHD-IN had significantly (ps < .001) higher correlations than SCT and ADHD-HI with functional impairment. SCT had a significantly (p = .02) higher correlation than ADHD-HI with functional impairment, though the magnitude of the difference was small (see Table 4).

Unique Effects of SCT on Adjustment Domains

Regression analyses controlling for ADHD-IN and ADHD-HI

Table 5 shows the unique effects (standardized partial regression coefficients) of the three models examining SCT, ADHD-IN, and ADHD-HI on self-reported daily life socio-emotional adjustment, EF domains, and functional impairment².

Table 5.

Standardized Unique Effects (Standard Errors) of Sluggish Cognitive Tempo and ADHD Symptoms on Executive Functioning, Socio-Emotional Functioning, and Functional Impairment

Variable	Executive Functioning
	Self-Organize	Time Management	Self-Restraint	Emotion	Self-Motivation
SCT	0.25 (0.02)***b	0.12 (0.02)***b	0.07 (0.02)**b	0.22 (0.02)***a	0.03 (0.02)b
ADHD-IN	0.52 (0.03)***a	0.74 (0.02)***a	0.34 (0.03)***a	0.22 (0.03)***a	0.61 (0.03)***a
ADHD-HI	−0.004 (0.02)c	−0.15 (0.02)***c	0.29 (0.02)***a	0.19 (0.02)***a	−0.01 (0.02)b
	Socio-Emotional Functioning				Functional Impairment
	Anxiety/Depression	Emotion Dysregulation	Self-Esteem	Loneliness	Mean Impairment
SCT	0.25 (0.03)***a	0.26 (0.03)***a	−0.24 (0.03)***a	0.23 (0.03)***a	0.10 (0.02)***b
ADHD-IN	0.32 (0.03)***a	0.27 (0.03)***a	−0.31 (0.03)***a	0.30 (0.04)***a	0.44 (0.03)***a
ADHD-HI	0.004 (0.03)b	0.09 (0.03)***b	0.05 (0.03)*b	−0.08 (0.03)**b	0.03 (0.02)b

Note. ADHD-IN = attention-deficit/hyperactivity disorder. HI = hyperactivity/impulsivity. IN = inattention. SCT = sluggish cognitive tempo.

^*p<.05.

^**p<.01.

^***p<.001.

Regression coefficients with different superscripts differ significantly in relation to the outcome variable at p < .05.

In the model examining socio-emotional functioning domains, higher levels of ADHD-IN and SCT each uniquely predicted significantly more anxiety/depression, emotion dysregulation, and loneliness as well as lower self-esteem, and the strength of these associations did not differ between ADHD-IN and SCT. Higher levels of ADHD-HI predicted significantly greater emotion dysregulation, lower self-esteem, and less loneliness, though the Mplus model constraint procedure indicated that both ADHD-IN and SCT were significantly (ps < .01) more strongly associated than ADHD-HI with poorer socio-emotional adjustment (Table 5).

In the regression model examining daily life EF domains, higher levels of SCT, ADHD-IN, and ADHD-HI each uniquely predicted significantly poorer EF in the domains of self-management to time, self-restraint, and self-regulation of emotion. Higher levels of SCT and ADHD-IN were also both uniquely associated with poorer EF in self-organization/problem-solving. Only higher levels of ADHD-IN predicted poorer EF in self-motivation in the regression model. Consistent with the bivariate correlations, ADHD-IN was significantly (ps < .05) more strongly associated than SCT and ADHD-HI with all of the daily life EF domains with the exception of self-regulation of emotion (for which SCT, ADHD-IN, and ADHD-HI were all significantly associated and did not differ from each other in their magnitude in predicting self-regulation of emotion). In addition, SCT was significantly (ps < .001) more strongly associated than ADHD-HI with the self-organization/problem-solving and self-management to time domains, whereas ADHD-HI was significantly (p < .001) more strongly associated than SCT with the self-restraint domain of EF (see Table 5).

In the regression model examining functional impairment, higher levels of both SCT and ADHD-IN predicted significantly greater functional impairment. ADHD-HI did not significantly predict functional impairment above and beyond ADHD-IN and SCT. ADHD-IN was significantly (ps < .001) more strongly associated with greater functional impairment than both SCT and ADHD-HI (see Table 5).

Regression analyses controlling for ADHD and internalizing symptoms

When internalizing symptoms were included as a predictor in the regression models along with SCT, ADHD-IN, and ADHD-HI, SCT symptoms remained significantly associated with more loneliness (p = .01), lower self-esteem (p = .005), and greater emotion dysregulation (p < .001). In contrast, ADHD-IN was no longer associated with the socio-emotional adjustment domains when internalizing symptoms were added to the model (ps > .05). SCT also remained associated with poorer functioning in the daily life EF domains of self-organization/problem-solving, self-management to time, and self-regulation of emotion (ps < .001) but were no longer significantly associated with self-restraint (p = .07). Finally, when internalizing symptoms were added to the regression model, ADHD-IN symptoms remained significantly associated with functional impairment (p < .001) whereas SCT did not (p > .05). Internalizing symptoms were significantly associated with poorer functioning across all outcome measures (i.e., daily life EF, socio-emotional adjustment, impairment) above and beyond SCT, ADHD-IN, and ADHD-HI (all ps < .001). Full results for these analyses are available from the corresponding author.

Discussion

As interest in SCT increases (Barkley, 2014; Becker, 2017; Becker, Marshall et al., 2014), a primary limitation for the field has been the lack of a unified set of symptoms for assessing SCT. The current study makes a significant contribution toward addressing this limitation by including in a single measure each of the SCT items identified as optimal in distinguishing SCT from ADHD in a recent meta-analysis (Becker, Leopold et al., 2016) as well as several items related to mental confusion that may further help characterize the SCT construct (McBurnett et al., 2014). Further, this is the first study to identify SCT items that at the item-level demonstrate strong convergent and discriminant validity from anxiety/depressive symptoms as well as ADHD-IN symptoms. In addition, this study adds to the very small body of research showing SCT to be associated with poorer functioning in adulthood and does so in a large, multi-site study of college students.

Development, Convergent Validity, and Discriminant Validity of the ACI

The challenges in measuring SCT were perhaps most clearly evident in a recent meta-analysis on the topic, which found that over 150 different items had been used to assess SCT, with rating scales including anywhere from 2 to 44 items (Becker, Leopold et al., 2016). With such heterogeneity used in assessing SCT across studies, it is incredibly difficult to compare findings across studies. Are inconsistent findings due to sample characteristics, outcome measures, or SCT measures themselves? As a first step in moving towards a unified set of SCT symptoms that can be used across studies, the ACI was developed and based directly on the meta-analytic findings of items identified as optimal for distinguishing between SCT and ADHD-IN symptoms. In an effort to increase clarity rather than add to the confusion surrounding SCT measurement, we took a rigorous approach in this initial validation study of the ACI. First, given SCT’s strong relation to both ADHD-IN and internalizing symptoms (Becker, Leopold et al., 2016), we required each SCT item to demonstrate divergence from both ADHD-IN and internalizing symptoms. This is the first time any study has required SCT symptoms to demonstrate convergent and discriminant validity from both ADHD-IN and internalizing symptoms at the item level. Second, we split the sample in half and only retained SCT items that met convergent and discriminant validity thresholds across both subsamples. By doing so, we not only identified the SCT items with the strongest evidence in our sample, but perhaps more importantly, identified items with likelihood of replication in subsequent studies.

Ten of the original 16 ACI items met our stringent threshold. These 10 items include 9 of the 13 items identified in the recent SCT meta-analysis as optimal for identifying SCT in addition to one mental confusion item (“I forget what I was going to say”). The pattern of loading of the other two mental confusion items (“My mind gets mixed up”, “I have a hard time putting my thoughts into words”) highlights the importance of using a rigorous approach for retaining SCT items like the one employed in this study. Specifically, each of these items showed adequate convergent validity in one of the subsamples but not in both. In addition, four items identified in the SCT meta-analysis as useful for identifying SCT separately from ADHD-IN did not meet our validity criteria (see Table 1). Although initially surprising given the robust support for these items in the meta-analytic review (Becker, Leopold et al., 2016), two issues are important to keep in mind when considering this finding. First, the meta-analysis only evaluated whether SCT items were consistently distinct from ADHD-IN, whereas we required each item to demonstrate divergence from both ADHD-IN and internalizing symptoms. Second, while the meta-analysis included studies conducted with children and adults, far more studies with children were available. In fact, none of the SCT items examined in the present study had been evaluated in more than three previous studies with adults, with three items unexamined entirely (see Table 1). Additional studies will be needed to clarify if the items that failed to meet our validity benchmarks do so in other studies when internalizing symptoms are considered in tandem with ADHD or whether the optimal items for assessing SCT vary across development. In either case, the current study provides strong support for the 10 ACI items identified in this study as having convergent and discriminant validity, as well as excellent reliability, among young adult college students. Additional studies with the full ACI (16 items) will be needed to evaluate whether this 10-item symptom set is specific to college students or also ideal for use in general adult samples, clinical samples, and samples of children and adolescents. In addition, it is important to note that, as expected, the SCT factor was strongly correlated with ADHD-IN. Indeed, the magnitude of the association between SCT and ADHD-IN in this study (0.73) was almost exactly the same as the magnitude of the correlation between SCT and ADHD-IN in adults reported in the recent SCT meta-analysis (0.72) (Becker, Leopold et al., 2016).

External Validity of the ACI

This study also provides preliminary support for the criterion (concurrent) validity of the 10-item ACI measure of SCT. Specifically, SCT was moderately-to-strongly correlated with other self-report measures in expected ways, including poorer socio-emotional adjustment, greater daily life EF deficits, and higher global functional impairment. Moreover, SCT symptoms remained uniquely associated in the regression analyses with most of these external criterion domains above and beyond ADHD symptoms.

Socio-emotional adjustment

Clear support was found for SCT symptoms being associated with poorer socio-emotional adjustment in adulthood. We replicated findings from previous studies in the domains of internalizing symptoms and emotion dysregulation (Becker, Langberg et al., 2014; Flannery, Becker et al., 2016) and also extend the current research to the related domains of loneliness and self-esteem. Not only did SCT remain associated with these domains above and beyond ADHD, but when internalizing symptoms were also covaried, ADHD-IN was no longer significantly associated with socio-emotional functioning. This is an important dissociation of SCT and ADHD-IN symptoms in predicting key clinically-relevant outcomes (Hawkley & Cacioppo, 2010; Swann, Chang-Schneider, & Larsen McClarty, 2007). In considering our findings alongside studies conducted with youth that show SCT to be linked to greater withdrawal/isolation, loneliness, and suicide risk (Becker, Garner, Tamm, Antonini, & Epstein, 2017; Becker, Luebbe, et al., 2015; Becker, Withrow et al., 2016; Carlson & Mann, 2002; Marshall, Evans, Eiraldi, Becker, & Power, 2014; Willcutt et al., 2014), a picture is emerging of SCT being related to a number of functioning domains that have clear relevance for overall adjustment, long-term outcomes, and even personal health and safety. These findings underscore the importance of including SCT in research seeking to understand the developmental trajectory and treatment of socio-emotional problems.

Daily life EF

In contrast to the findings for socio-emotional functioning, we found that ADHD-IN was a stronger predictor than either SCT or ADHD-HI of daily life EF deficits. The only exception to this pattern was for the domain of self-regulation of emotion, for which SCT, ADHD-IN, and ADHD-HI were each similarly related to greater deficits. The degree to which our findings are consistent with the four previous studies that have examined SCT and ADHD dimensions in relation to adult daily life EF domains (Barkley, 2012; Jarrett et al., 2017; Leikauf & Solanto, 2017; Wood et al., 2017) varies based on the EF domain considered. As in previous studies (Barkley, 2012; Jarrett et al., 2017; Leikauf & Solanto, 2017), we found ADHD-IN to be a stronger predictor than SCT in the domains of self-management to time and self-motivation. Still, SCT remained uniquely associated with poorer functioning in these EF domains, with SCT contributing more variance than ADHD-HI in predicting self-management to time, again consistent with previous studies in this area (Barkley, 2012; Jarrett et al., 2017; Wood et al., 2017). Also, similar to the two previous college studies (Jarrett et al., 2017; Wood et al., 2017) and the study of adults evaluated at an ADHD specialty clinic (Leikauf & Solanto, 2017), but differing from Barkley (2012), we found SCT to be a weaker predictor of self-restraint difficulties than either ADHD-HI or ADHD-IN. This finding makes sense when considering the strong association between impulsivity and self-restraint difficulties (Barkley, 2012; Becker & Langberg, 2014; Chamberlain & Sahakian, 2007), particularly in light of studies showing SCT to be linked to lower rates of impulsive behaviors when controlling for ADHD (Becker, Luebbe, Fite, et al., 2014; Burns, Becker, Servera, Bernad, & García-Banda, 2017; Marshall et al., 2014). Our supplemental regression analyses that controlled for internalizing symptoms also supported this finding. In these analyses, SCT was no longer significantly associated with self-restraint EF difficulties but remained significantly associated with EF difficulties in the domains of self-organization/problem-solving, self-management to time, and self-regulation of emotion.

In contrast to previous studies (Barkley, 2013; Jarrett et al., 2017; Wood et al., 2017), we found ADHD-IN to be more strongly associated than SCT with self-organization/problem solving deficits, and SCT was in turn more strongly associated than ADHD-HI with self-organization/problem solving deficits. It is not entirely surprising that ADHD-IN would contribute the most variance in predicting self-organization EF deficits since the ADHD-IN dimension itself is often conceptualized as containing both inattention and disorganization (Hinshaw, Arnold, & MTA Cooperative Group, 2015; Wilens, Biederman, & Spencer, 2002). In terms of self-regulation of emotion, we found SCT, ADHD-IN, and ADHD-HI to each contribute similarly in predicting poorer functioning in this domain. Previous studies have reported mixed findings in terms of whether SCT is similarly or more strongly associated than ADHD-HI or ADHD-IN with self- regulation of emotion deficits (Barkley, 2012; Jarrett et al., 2017; Wood et al., 2017). As noted by Barkley (2012), self-regulation EF deficits are multifaceted and include difficulties associated with impulsiveness as well as depression and withdrawal, and for this reason multiple psychopathology dimensions may uniquely and similarly contribute to broad measures of self-regulation EF deficits. In any event, though the magnitude of effects have varied across studies, in each study SCT has uniquely been associated with more deficits in the self-regulation of emotion.

Functional impairment

As in Barkley (2012), ADHD-IN was the strongest predictor of global self-reported functional impairment, though SCT also contributed unique variance above and beyond ADHD symptoms. It therefore appears that ADHD-IN symptoms may have a particularly deleterious effect on self-reported global functioning, with SCT somewhat less impactful in terms of overall functioning. Nevertheless, other findings of this study and the findings related to socio-emotional adjustment in particular point to the importance of considering specific, nuanced domains of adjustment when evaluating external correlates of SCT and other psychopathology dimensions.

Demographic characteristics

SCT was generally unassociated with demographic characteristics in this study. Similar to Barkley (2012), we found no association between SCT and either age or race/ethnicity. In contrast to Barkley (2012), we found a small but significant positive association between SCT and female (vs. male) sex. In a community-based study of adults, Combs and colleagues (Combs, Canu, Broman-Fulks, & Nieman, 2014) also found women to have higher SCT scores than men, though other studies have reported the reverse and the overall evidence to date points to no association between SCT and sex in adulthood (Becker, Leopold et al., 2016). This is another way in which SCT and ADHD appear to part ways, as ADHD in adulthood remains more common in males than in females (Willcutt, 2012).

Limitations and Future Directions

This study should be evaluated in light of several limitations. First, like Barkley (2012), this was a cross-sectional study that consisted entirely of self-report rating scales. As such, we were unable to examine predictive validity, which is a critically important area for future research, and it will likewise be important for future research to incorporate a multi-method, multi-informant approach in order to bolster confidence in the findings obtained. For instance, studies are also needed to evaluate other domains of functioning and more specific domains of functional impairment that are relevant to college students (e.g., study skills). In addition, we were not able to include collateral reports of symptoms or evaluate SCT in relation to other key criterion domains, including clinician-established diagnoses, grades, or performance-based EF tests. Thus, our findings related to the criterion validity of the ACI should be considered preliminary. This is a particularly important limitation and area for future research since adults may either under-report or over-report their own ADHD symptoms, making the collection of both self and informant ratings optimal when assessing ADHD in adulthood (Barkley, Murphy, & Fischer, 2008; Sibley, Mitchell, & Becker, 2016). This may be especially true in college student samples who may feign ADHD for secondary gain (e.g., educational accommodations, psychostimulant medication prescription) (Quinn, 2003; Sollman, Ranseen, & Berry, 2010). Although our study did not recruit a sample seeking treatment or educational services, nor did we diagnose individuals with ADHD, the limitations of self-reported ADHD symptoms should nevertheless be noted. Thankfully, there is ongoing research examining how to best assess non-credible ADHD reporting in adults (Cook, Bolinger, & Suhr, 2016; Sollman et al., 2010), and future studies would benefit from including measures that aim to reduce possible biases associated with self-report. Informant ratings may also be important when assessing SCT among college students (Leopold et al., 2015), though research is needed to evaluate whether SCT ratings are subject to the same self-report biases as ADHD ratings.

Second, while we sought to increase generalizability by recruiting a large sample of participants across five sites, our sample consisted entirely of college students and was predominantly White and female and so cannot be assumed to generalize to adults (or even young adults) broadly. Still, examining SCT in college students specifically seems particularly important given higher rates of elevated SCT in other college students compared to the general adult population (Flannery, Becker et al., 2016; Jarrett et al., 2017; Wood et al., 2017). In line with these studies, we found the mean SCT score in our large college student sample to be significantly higher than the mean scores for the ADHD and internalizing domains. Future studies are needed to understand why college students might endorse particularly high levels of SCT symptomatology. Moreover, it will be important for future studies to use a broader assessment and sampling approach including clinical samples in order to more thoroughly evaluate that SCT is not only distinct from ADHD and internalizing symptoms but also uniquely associated with poorer adjustment above and beyond clinical symptom presentation (e.g., depression, anxiety, or sleep problem severity; for a rare example, see Becker, Garner, & Byars, 2016). The use of consistent SCT measures across studies is of great research and clinical importance. Without the use of consistent SCT methodology across studies, it will remain difficult to compare findings across studies or know the extent to which SCT is related to functioning and impairment across clinical samples, and this issue will only become more complicated as the study of SCT moves outside the confines of ADHD. Furthermore, until a body of research using comparable measures of SCT amasses, whether SCT is best thought of as a discrete psychiatric diagnosis or a construct of transdiagnostic import will remain unclear (Barkley, 2014; Becker & Barkley, in press; Becker, Leopold et al., 2016), research investigating the etiology and life course of SCT will be limited, and the role of SCT in clinical assessment and treatment planning/monitoring will be uncertain.

Conclusion

The ACI, developed in response to meta-analytic results that identified optimal items for distinguishing SCT from ADHD inattention, is a new adult self-report measure that demonstrates strong convergent and discriminant validity from ADHD-IN and internalizing symptoms. The 10-item ACI validated in this multi-university study also demonstrated preliminary evidence of criterion validity, with SCT as measured on the ACI significantly associated with poorer socio-emotional functioning, increased daily life EF deficits, and greater functional impairment. Though additional validation is surely needed, it is our hope that the ACI will make a significant advance toward establishing a unified set of SCT symptoms that can be used in both research and clinical settings.

Public Significance Statement.

This study provides initial validation of a new self-report measure for assessing sluggish cognitive tempo (SCT) symptoms in adults, the Adult Concentration Inventory (ACI), and moves the field toward a unified set of SCT items that can be used in both research and clinical settings.