Development and Evaluation of the ADHD Cognitions Scale for Adults

Abstract

The clinical literature on ADHD in adults suggests that “overly positive” or optimistic cognitions may contribute to impairment and failure to use self-regulation skills in this population, yet the research literature on this topic is limited. We developed the ADHD Cognitions Scale (ACS), a brief self-report measure of ADHD-related thoughts, and evaluated its psychometric properties in two large community samples (Ns = 262, 304). The measure demonstrated a one-factor solution that replicated in the second sample. Evidence of good internal consistency and also convergent and divergent validity was obtained in both samples. Scores on the ACS correlated with functional impairment, time management problems, and avoidant coping strategies. With additional study, the ACS may be useful to identify and track maladaptive ADHD-related cognitions during cognitive-behavioral treatment and to further study the role of these thoughts in ADHD-related impairment.

ADHD in adults is characterized by developmentally inappropriate and impairing symptoms of inattention and/or hyperactivity-impulsivity. In about two-thirds of cases, the disorder and its related impairments persist from childhood into adulthood, leading to new impairments in adult domains of functioning (Barkley, Murphy, & Fischer, 2008). Cognitive-behavioral therapy (CBT) is an efficacious treatment for adults with the disorder that teaches compensatory strategies to reduce functional impairment associated with symptoms (Safren et al., 2010; Solanto et al., 2010). In addition to behavioral self-management strategies, many of these approaches also involve management of depressive and anxious cognitions that may impact clients’ functioning (Ramsay & Rostain, 2008; Safren, Perlman, Sprich, & Otto, 2005; Solanto, 2011). As is the case with CBT in general, the main focus has thus far been on the effects of overly negative maladaptive thoughts associated with internalizing disorders.

Targeting such patterns of negative maladaptive thoughts is important since there is strong evidence that ADHD in adults is associated with increased risk for depression and anxiety disorders and, more specifically, that ADHD in adults is also associated with the patterns of negative and distorted thinking associated with these disorders (Abramovitch & Schweiger, 2009; Knouse, Zvorsky, & Safren, 2013; Mitchell, Benson, Knouse, Kimbrel, & Anastopoulos, 2013; Strohmeier, Rosenfield, DiTomasso, & Ramsay, 2016). However, there is growing recognition that maladaptive patterns of thinking are not restricted to these “negative” thinking styles in adult ADHD (Knouse & Mitchell, 2015). On the contrary, recurrent problematic thoughts in adults with ADHD may appear quite optimistic on the surface—for example, “I’m the kind of person who does my best work at the last minute.” Such cognitions are topographically different from typical depressive and anxious cognitions targeted in most manualized CBT approaches for adults diagnosed with ADHD, but are functionally thought to have a similar outcome: failure to engage in self-management strategies. This issue has been repeatedly described in the clinical literature on adult ADHD (Ramsay & Rostain, 2008; Sprich, Knouse, Cooper-Vince, Burbridge, & Safren, 2010; Zylowska, 2012) and the issue of bias in the cognitions of some children with ADHD—the positive illusory bias—has also been studied extensively (see Owens, Goldfine, Evangelista, Hoza, & Kaiser, 2007 for a review). Evidence for these biases and their effects in adults with the disorder is compelling, but more limited (Golden, Owens, Evangelista, & Micheli, 2006; Knouse, Bagwell, Barkley, & Murphy, 2005; Lui, Johnston, Lee, & Lee-Flynn, 2013; Prevatt et al., 2012).

In seeking to further investigate maladaptive thoughts in adult ADHD, at least two distinct approaches are possible. As in the research on the positive illusory bias, researchers could focus on the accuracy of participants’ domain-specific self-assessments relative to some standard (e.g., Knouse et al., 2005; Lui et al., 2013) and then examine whether individuals with miscalibrated opinions are more likely to experience negative outcomes. An alternative approach and one that is more consistent with the way that maladaptive cognitions are approached in CBT is to measure the frequency or severity with which overly positive thoughts occur in the daily lives of people with ADHD and to evaluate whether these thoughts are associated with negative outcomes. This approach has been crucial to understanding the role of negative automatic thoughts in internalizing disorders (R. Beck & Perkins, 2001) and self-report measures such as the Automatic Thoughts Questionnaire (Hollon & Kendall, 1980) have been an important tool in these investigations. A similar instrument for studying maladaptive cognitions associated with ADHD and its impairments would aid further investigation in this area and the lack thereof is currently a barrier for researchers. Furthermore, alteration of problematic ADHD-related cognitions may be a mechanism of change in CBT for ADHD (Ramsay, 2010) and the ability to measure the frequency of these thoughts as treatment progresses could benefit both clinical research and clinical practice.

Study Objectives

The objective of the present study was to develop and evaluate a brief measure of ADHD-related automatic thoughts, which we refer to as the ADHD Cognitions Scale (ACS). The ACS was originally developed by the study authors as a clinical measure used with adult clients who were being assessed and treated for ADHD in an ADHD specialty clinic. The earliest version of the measure contained 33 items that were generated by the study authors in a rational-intuitive manner. Over time, the number of items that we administered to clients was reduced from 33 items to 20 items based on item performance in the clinic (e.g., associations with ADHD symptoms; item-total correlations; Anastopoulos et al., 2012). The objective of the present research was to further refine the ACS by developing a brief, reliable, valid, and unidimensional version of the ACS that could be used by clinicians and researchers alike in a wide range of settings. Consistent with the cognitive-behavioral model of ADHD, we expected that ACS scores would be positively associated with ADHD symptoms and functional impairment. Furthermore, consistent with the view that ADHD-related cognitions may function to avoid negative affect (Knouse & Mitchell, 2015), we also predicted that scores on the ACS would be positively associated with disengagement coping strategies (cognitive and behavioral avoidance, self-distraction, denial) and negatively associated with engagement coping (active coping, positive reframing, acceptance, planning).

Method

Participants

Study 1

Participants were 262 adults completing online self-report measures recruited through Amazon’s Mechanical Turk (MTurk; see Procedure). The mean age of the sample was 35.22 years (SD = 11.56) and ranged from 18 to 74 years with a median of 32. See Table 1 for additional sample demographics and descriptive statistics.

Table 1.

Sample Demographics

	Sample 1	Sample 2
N	262	304
Age (SD)	35.22 (11.56)	33.99 (9.80)
Gender (%)
Men	110 (42%)	146 (48%)
Women	156 (58%)	158 (52%)
Race (%)
White/Caucasian	217 (83%)	258 (84.9%)
Black/African American	19 (7.3%)	23 (7.6%)
Asian/Pac. Isl./Asian American	22 (8.4%)	24 (7.9%)
Native American	8 (3.1%)	7 (2.3%)
Other	4 (1.5%)	4 (1.3%)
Ethnicity (%)
Hispanic	12 (4.6%)	27 (8.9%)
Non-Hispanic	250 (95.4%)	277 (91.1%)
Highest Level of Education (%)
No high school diploma	1 (0.4%)	2 (0.7%)
High school	38 (14.5%)	46 (15.1%)
Some College	65 (24.8%)	71 (23.4%)
Associate’s or Technical degree	30 (11.4%)	36 (11.8%)
Bachelor’s degree	102 (38.9%)	118 (38.8%)
Master’s degree	22 (8.4%)	26 (8.6%)
Doctorate/medical/law degree	3 (1.1%)	5 (1.6%)
Employment Status (%)
Full-time	133 (50.8%)	202 (66.4%)
Part-time	50 (19.1%)	53 (17.4%)
Full-time student	23 (8.8%)	10 (3.3%)
Part-time student	10 (3.8%)	6 (2.0%)
Stay-at-home parent or homemaker	27 (10.3%)	26 (8.6%)
Unemployed	35 (13.4%)	19 (6.3%)
Receiving disability benefits	7 (2.7%)	4 (1.3%)
Receiving welfare benefits	2 (0.8%)	1 (0.3%)
Retired	8 (3.1%)	4 (1.3%)
Marital Status (%)
Single	103 (39.3%)	138 (45.4%)
Living with Partner	44 (16.8%)	47 (15.5%)
Married	91 (34.7%)	99 (32.6%)
Divorced	22 (8.4%)	17 (5.6%)
Widowed	2 (0.8%)	3 (1.0%)
With Children (%)	109 (41.6%)	112 (36.8%)

Note. Participants were able to check more than one race category and employment descriptor.

With respect to ADHD status, 7.3% of the sample reported receiving an ADHD diagnosis at some point in their lives and 5.7% endorsed a current diagnosis. Among those ever diagnosed with ADHD, 95% had taken medications for ADHD at some point in their lives and 61% were currently taking medications for the disorder. 16% of the sample reported current non-medication treatment for ADHD. Using responses on the Barkley Adult ADHD Rating Scale and applying analogue DSM-5 criteria of 5 or more symptoms on either list and related impairment, 13% of the sample met these criteria as follows: 5.7% of the sample met criteria for the Predominantly Inattentive presentation, 3.4% the Predominantly Hyperactive-Impulsive presentation, and 3.5% the Combined presentation. With the addition of the requirement that participants endorse of “most of the problems” starting before age 12, the rate fell to 4.2% (i.e., 1.1% Inattentive, 2.3% Hyperactive-Impulsive, and 0.8% Combined). In this sample, participants reporting that they were ever diagnosed with ADHD were more likely to be male (p = .05), whereas participants currently meeting proxy DSM-5 criteria without age of onset were more likely to be female (p = .05). A greater proportion of adults meeting proxy DSM-5 criteria that included the age of onset were Hispanic/Latino (p < .001). No other differences in gender, education level, race, or ethnicity were detected for ADHD groups.

Study 2

Participants were 304 adults completing online self-report measures recruited through MTurk using similar procedures to Study 1. The mean age of the sample was 33.99 years (SD = 9.80) and ranged from 19 to 69 years with a median of 32. See Table 1 for additional sample demographics and descriptive statistics.

With respect to ADHD status, 7.6% of the sample reported ever receiving an ADHD diagnosis and 4.9% endorsed a current diagnosis. Among those ever diagnosed with ADHD, 78% had taken medications for ADHD at some point in their lives and 44% were currently taking medications for the disorder. 17% of those ever diagnosed reported currently receiving non-medication treatment for the disorder. Using responses on the Barkley Adult ADHD Rating Scale and applying analogue DSM-5 criteria of 5 or more symptoms on either list and related impairment 10.2% of the sample met these criteria as follows: 4.9% of the sample met criteria for the Predominantly Inattentive presentation, 1.6% the Predominantly Hyperactive-Impulsive presentation, and 3.6% the Combined presentation. When endorsement of “most of the problems” starting before age 12 was added to these criteria, the rate fell to 4.6% (i.e., 2.0% Inattentive, 0.7% Hyperactive-Impulsive, and 2.0% Combined). In this sample, participants reporting that they had ever been diagnosed with ADHD were more likely to be female (p = .08). No other differences in gender, education level, race, or ethnicity were detected for ADHD groups.

Measures

Study 1

ADHD Cognitions Scale (ACS)

The ACS was modeled on the Automatic Thoughts Questionnaire (Hollon & Kendall, 1980), a measure of negative automatic thoughts that were associated with depression. As noted above, ACS items were originally generated on a rational-intuitive basis by the study authors through their clinical work with patients with ADHD. Examples of items are: “I’ll just do this one thing first,” and “I do better waiting until the last minute.” Participants are instructed to rate how often each of these automatic thoughts occurs to them using the following Likert scale: 1 = Not at all; 2 = Sometimes; 3 = Often; 4 = Very Often; 5 = All the Time. The psychometric properties of the ACS are detailed below and the final version of the scale can be found in the Appendix.

Barkley Adult ADHD Rating Scale-IV (BAARS; Barkley, 2011a)

The BAARS-IV is a self-report scale in which participants report on the frequency with which they have experienced ADHD symptoms over the past six months on a four-point Likert scale (Never or rarely, Sometimes, Often, Very often). The scale includes the 18 DSM-IV ADHD items. The scale also asks participants to report on the age of onset of endorsed symptoms as well as domains in which the symptoms impaired their functioning. Evidence of reliability and validity comes from a normative sample of 1,249 adults (test-retest reliability .75 at 2–3 weeks; internal consistency .91). Internal consistency for the 18 DSM-IV ADHD items in this sample was also excellent (α = .92).

Barkley Functional Impairment Scale (BFIS; Barkley, 2011c)

The Barkley Functional Impairment Scale is a 15-item measure that asks participants to rate how impaired they are (i.e., how much difficulty they have in functioning) in a variety of domains of life including home, work, romantic relationships, driving, and parenting. Participants rate their impairment on a scale from 0 (not at all) to 9 (severe). Evidence of reliability and validity come from the same normative sample as the BAARS-IV above and includes good internal consistency (α = .969) and test-retest reliability at 2–3 weeks (.72) for the total impairment score. In our survey, we observed a higher-than-expected number of missing values on this measure, which we believe this was due to its formatting in the online survey. Specifically, participants responded t by clicking and moving a slider to indicate their level of impairment. If participants wished to indicate no impairment (score of zero), they had to click (but not move) the slider, as zero was its default position. Thus, some participants who intended to indicate no impairment may have had missing data. In addition, we did not include a “does not apply” option and so participants may have left items blank for this reason. Despite these missing values, the internal consistency for the scale was strong (α = .93) and use of mean scores instead of totals attenuated the impact of missing data on this measure. Note that formatting issues were corrected in Study 2.

Automatic Thoughts Questionnaire (ATQ; Hollon & Kendall, 1980)

The 30-item ATQ was designed to measure the occurrence of automatic negative thoughts associated with depression. Respondents rate how frequently a thought occurred over the last week on a scale ranging from 1 (not at all) to 5 (all the time). The measure has good reliability and distinguishes between depressed and non-depressed groups (Hollon & Kendall 1980). Internal consistency in Study 1 was excellent (α = .99).

Center for Epidemiologic Studies – Depression Scale (CES-D; Radloff, 1977)

The frequency of depressive symptoms. Participants rate the frequency of each item over the past week on a four-point scale ranging from “Rarely or none of the time (Less than 1 day)” to “Most or all of the time (5–7 days).” Internal consistency of the CES-D in Study 1 was excellent (α = .94).

GAD-7 (Spitzer, Kroenke, Williams, & Löwe, 2006)

The GAD-7 is a short scale designed to screen for and measure symptoms of generalized anxiety over the past two weeks. Participants respond on a four-point scale from 0 (Not at all) to 3 (Nearly every day). The GAD-7 correlates with functional impairment and measure anxiety that is related to but distinct from depression (Spitzer et al., 2006). Internal consistency of this measure in Study 1 was excellent (α = .93).

Barkley Deficits in Executive Functioning Scale – Short Form (BDEFS; Barkley, 2011b)

The 20-item BDEFS measures problems with executive functioning in daily life. Participants rate the frequency with which they have experienced each problem over the past six months on a four-point scale from 1 (“Never or Rarely”) to 4 (“Very Often”). The BDEFS has five subscales measuring deficits in self-management to time, self-organization/problem solving (information processing difficulties/cognitive inflexibility), self-restraint, self-motivation, and self-regulation of emotions. The BDEFS was normed on a large nationally representative sample of adults (n = 1240) and demonstrated strong reliability and validity. Internal consistency of the BDEFS in Study 1 was excellent (α = .94).

Cognitive-Behavioral Avoidance Scale (CBAS; Ottenbreit & Dobson, 2004)

On the CBAS, participants rate 31 statements that describe how they might deal with situations and problems in their lives on a scale from 1 (Not at all) to 5 (Extremely true for me). The CBAS has subscales for both cognitive and behavioral avoidance. CBAS scores showed good psychometric properties including high internal consistency and test-retest reliability (Ottenbreit & Dobson, 2004) and total CBAS scores showed excellent internal consistency in Study 1 (α = .97).

Study 2

Measures in Study 2 included the following measures described above for Study 1: ADHD Cognitions Scale (ACS), Barkley Adult ADHD Rating Scale-IV (BAARS-IV; Barkley, 2011; Study 2 α = .94); Barkley Functional Impairment Scale (BFIS; Barkley, 2011; α = .94); Automatic Thoughts Questionnaire (ATQ; Hollon & Kendall, 1980; α = .99), Center for Epidemiologic Studies – Depression Scale (CES-D; Radloff, 1977; α = .95), GAD-7 (Spitzer, Kroenke, William, & Löwe, 2006; α = .93), Barkley Deficits in Executive Functioning Scale – Short Form (BDEFS-S; Barkley, 2011; α = .95), and the Cognitive-Behavioral Avoidance Scale (CBAS; Ottenbreit & Dobson, 2004; α = .97).

In addition, the Study 2 self-report battery also included items from the Weiss Functional Impairment Rating Scale (WFIRS; Weiss, 2000). We included the WFIRS as an additional, more fine-grained index of functional impairment, including items from the following subscales: Family (α = .88), Work (α = .88), School (α = .91), Life Skills (α = .90), Self-Concept (α = .96), and Social (α = .89). We omitted items from the Risk subscale due to concerns about asking participants to report on potentially illegal activities. For Study 2, we also included the brief form of the COPE Inventory (Carver, 1997), which provides an efficient way to measure a broad array of coping strategies. Each subscale of the brief COPE contains two items and internal consistencies ranged from poor to excellent as follows: Self-Distraction (α = .46), Active Coping (α = .80), Denial (α = .75), Substance use (α = .97), Emotional Support (α = .91), Instrumental Support (α = .93), Behavioral Disengagement (α = .73), Venting (α = .64), Positive Reframing (α = .82), Planning (α = .84), Humor (α = .91), Acceptance (α = .53), Religion (α = .94), Self-Blame (α = .86).

Procedure

Study 1

Data were collected using Amazon’s Mechanical Turk between May 30^th and June 2^nd, 2014. Mechanical Turk (MTurk; http://www.mturk.com) is an online crowdsourcing platform where requesters recruit workers to complete tasks and pay those workers for successful task completion (see Mason & Suri, 2012 for an introduction). MTurk samples are often more diverse than typical internet and college student samples and data reliability appears to be generally comparable to traditional methods (Buhrmester, Kwang, & Gosling, 2011). Relative to the U.S. population, MTurk samples, however, tend to be younger, better educated and less racially and ethnically diverse, although they also tend to contain more participants who are unemployed or underemployed (Chandler, & Shapiro, 2016). Mechanical Turk has been used to recruit clinical samples (Shapiro, Chandler, & Mueller, 2013) and to investigate ADHD in adults (Wymbs & Dawson, 2015). Specifically, Wymbs and Dawson (2015) found that the proportion of adults who self-reported an ADHD diagnosis in an MTurk sample was similar to other non-internet community samples of adults.

We used selection parameters for our MTurk sample and a validity scale to increase the likelihood of obtaining reliability and high-quality data in the current study. The study was only available to MTurk workers with U.S. residency and who had at least 95% of their prior work on MTurk accepted by the requester. These selection parameters were intended to increase the likelihood that participants would complete the survey in a valid way. To further enhance data quality, we included a 13-item infrequency scale that was placed randomly within the survey battery. This scale consisted of the 13 infrequency items developed by Chapman and Chapman (1983) randomly intermixed with another measure that is not reported on in this study. Consistent with the authors’ recommendations, participants who endorsed more than 2 infrequency scale items were dropped from further study (n = 43; 14% of the original sample). Thus, our analyzed sample of 262 includes only those participants who passed the validity screen.

The survey battery was administered using Qualtrics online survey software (www.qualtrics.com). Participants read a consent form and clicked to indicate their consent and to attest that they were 18 years or older. They then completed a demographics questionnaire, the ADHD Cognitions Scale, questions about ADHD diagnosis and treatment history, the BAARS, and the BFIS. Next, participants completed the other measures in a randomized order. Participants were paid $2.00 through the MTurk platform.

Study 2

Data for Study 2 were collected using Amazon’s Mechanical Turk between January 12^th and 14^th 2016. We employed procedures nearly identical to Study 1, including selection parameters of U.S. residency and at least 95% of prior work on MTurk accepted by the requester. We again included the 13-item infrequency scale and participants who endorsed more than 2 items were dropped from further study (n = 49; 14% of the original sample). Study 2 participation was unavailable to Study 1 participants to prevent duplicate survey-takers in the samples. Participants were paid $2.00.

Results

Factor Analysis

Sample 1

An exploratory principal axis factor (PAF) analysis was conducted in Sample 1 in order to examine the initial factor structure of the full 20-item version of the ACS. As expected, examination of the scree plot revealed that a 1-factor solution was most appropriate. Accordingly, the PAF was re-run and a single factor was extracted. A total of 9 items with factor loadings of 0.50 or higher were retained from the PAF for further analysis.

Exploratory/confirmatory factor analysis (E/CFA) was used to further evaluate the factor structure of the ACS, as use of E/CFA as an intermediate step between PAF and CFA frequently identifies additional sources of model misspecification that are not able to be identified with PAF alone (Brown, 2006; Jöreskog, 1969; Kimbrel et al., 2011). Root mean square error of approximation (RMSEA), comparative fit index (CFI), Tucker-Lewis index (TLI), and standardized root mean square residual (SRMR) were used to assess model fit. RMSEA values near or below .06, CFI and TLI values near or above .95, and SRMR values near or below .10 (Kline, 2011; Hu & Bentler, 1999) are generally considered to be indicative of close fit. A 1-factor model that included the 9 items with the highest factor loadings from the PAF was evaluated first. Examination of the fit indices from the E/CFA revealed that the proposed one-factor solution fit the data poorly, χ² (27) = 182.072, p < 0.001, RMSEA = .15, CFI = 0.823, TLI = 0.764, SRMR = .068. Examination of modification indices indicated that nearly all of the model misfit was due to covariance between the error terms of two of the items with the error terms of the remaining items. Accordingly, in order to further increase scale brevity, the two problematic items were removed from the scale and the 1-factor E/CFA was repeated on the remaining seven items. The resulting 1-factor solution fit the data well, falling within the range of values generally accepted to indicate close model fit, χ² (14) = 27.410, p = 0.017, RMSEA = .060, CFI = .975, TLI = .963, SRMR = .032. Factor loadings were also high, ranging from 0.51 to 0.80, with an average factor loading of 0.63 (Table 2)

Table 2.

Factor Loadings for the ADHD Cognitions Scale (ACS) items in Samples 1 and 2

Items	Sample 1	Sample 2
I’ll just do this one thing first.	0.58	0.62
I do better waiting until the last minute.	0.57	0.51
I can’t stop right now.	0.56	0.48
Though this usually “sucks me in,” I’ll just do it for a minute.	0.78	0.71
I have plenty of time---I’ll just do one more thing before I go.	0.64	0.71
Being impulsive is a big part of who I am.	0.51	0.62
I know I’m supposed to be doing something else but I want to do this now.	0.80	0.78
Mean	0.63	0.63

Sample 2

Maximum-likelihood based confirmatory factor analysis (CFA) was used to further evaluate the factor structure of the 7-item version of the ACS in Sample 2. Examination of the fit indices revealed that a 1-factor solution continued to provide good fit to the data for the 7-item version of the ACS in Sample 2, χ² (21) = 637.771, p < 0.001, RMSEA = .065, CFI = .970, TLI = .956, SRMR = .033. Factor loadings also continued to be high in Sample 2 (Table 2), ranging from .48 to .78, with an average factor loading of 0.63. Notably, the item “I know I’m supposed to be doing something else, but I want to do this now” had the highest factor loading across both samples. Given that the 7-item version of the ACS was found to demonstrate good factor structure across samples, all remaining analyses were limited to this version of the measure, which is provided in the Appendix.

Reliability

The 7-item version of the ACS demonstrated good internal consistency (α = .82) and good split-half reliability (Spearman-Brown Coefficient for Unequal Length = .84) in Sample 1. Internal consistency (α = .82) and split-half reliability (Spearman-Brown Coefficient for Unequal Length = .84) were identical for the ACS in Sample 2, suggesting that internal consistency reliability for the ACS was stable across samples.

Validity

Sample 1

To examine validity in terms of the ACS’s relationship to ADHD, we used both a categorical and a dimensional approach. We examined scores on the measure among participants with and without ADHD based on several different methods of operationalizing ADHD (Table 3). Participants who self-reported ever having been diagnosed with ADHD or having a current diagnosis did not score significantly higher on the ACS than other participants, although the means were in the hypothesized direction, albeit with a small effect size. However, when participants were placed in diagnosis proxy groups based upon their BAARS scores (see Participants section), participants who met analogue DSM-5 criteria both with and without the age of onset criterion applied showed higher mean ACS scores than participants who did not with a very large effect size (Table 3). When combining proxy diagnosis methods to create a group who reported being diagnosed with ADHD and meeting DSM-5 criteria on the BAARS without age of onset (n = 7), this group also had a significantly higher mean ACS score with a large effect size. When examining associations with ADHD symptoms dimensionally, as shown in Table 5, ACS scores showed significant and moderate correlations with the total and symptom subscales of the BAARS.

Table 3.

Mean ACS Scores by ADHD Status – Study 1

Basis of ADHD Grouping	ADHD			Comparison
	N	M	SD	N	M	SD	p	d
Self-Report: Ever Diagnosed	19	2.46	0.85	243	2.24	0.69	.187	0.28
Self-Report: Currently Diagnosed	15	2.45	0.77	247	2.24	0.70	.272	0.29
BAARS DSM-5 Analogue, No Age of Onset	34	3.11	0.65	228	2.12	0.62	<.001	1.56
BAARS DSM-5 Analogue with Age of Onset	11	2.94	0.60	251	2.22	0.69	.001	1.14
Ever Diagnosed + BAARS DSM-5, No Age of Onset	7	3.00	0.86	255	2.23	0.69	.004	0.99

Note: BAARS = Barkley Adult ADHD Rating Scale

Table 5.

Correlations between ACS and Constructs Related to Convergent Validity, Divergent Validity, and Functional Impairment

Convergent Validity	Study 1		Study 2
	r	p	r	p
ADHD Symptoms (BAARS Total)	.61	<.001	.56	<.001
Inattentive Symptoms (BAARS INATT)	.60	<.001	.55	<.001
Hyperactive-Impulsive Symptoms (BAARS HY-IM)	.50	<.001	.49	<.001
Self-Management to Time (BDEFS)	.64	<.001	.60	<.001
Self-Organization/Problem-Solving (BDEFS)	.36	<.001	.36	<.001
Impulsivity (BDEFS)	.53	<.001	.49	<.001
Self-Motivation (BDEFS)	.48	<.001	.44	<.001
Emotion Regulation (BDEFS)	.44	<.001	.37	<.001
Cognitive-Behavioral Avoidance (CBAS)	.49	<.001	.37	<.001
Cognitive Avoidance (CBAS – Cognitive)	.51	<.001	.40	<.001
Behavioral Avoidance (CBAS – Behavioral)	.42	<.001	.31	<.001
Self-Distraction (COPE)	---	---	.19	.001
Active coping (COPE)	---	---	-.25	<.001
Denial (COPE)	---	---	.31	<.001
Substance use (COPE)	---	---	.32	<.001
Use of emotional support (COPE)	---	---	.04	.473
Use of instrumental support (COPE)	---	---	.05	.393
Behavioral disengagement (COPE)	---	---	.30	<.001
Venting (COPE)	---	---	.14	.018
Positive reframing (COPE)	---	---	-.11	.047
Planning (COPE)	---	---	-.21	<.001
Humor (COPE)	---	---	.13	.021
Acceptance (COPE)	---	---	-.13	.023
Religion (COPE)	---	---	-.07	.216
Self-blame (COPE)	---	---	.35	<.001
Divergent Validity	r	p	r	p
Negative Automatic Thoughts (ATQ)	.40	<.001	.39	<.001
Depressive Symptoms (CES-D)	.41	<.001	.38	<.001
Generalized Anxiety Symptoms (GAD-7)	.45	<.001	.37	<.001
Correlation with Functional Impairment	r	p	r	p
Overall Functional Impairment (BFIS)	.48	<.001	.40	<.001
Family Impairment (WFIRS Family)	---	---	.32	<.001
Work Impairment (WFIRS Work)	---	---	.37	<.001
School Impairment (WFIRS School)	---	---	.44	<.001
Life Skills Impairment (WFIRS Life Skills)	---	---	.43	<.001
Self-Concept Impairment (WFIRS Self-Concept)	---	---	.31	<.001
Social Impairment (WFIRS Social)	---	---	.27	<.001

BAARS = Barkley Adult ADHD Rating Scale; BDEFS = Barkley Deficits in Executive Functioning Scale; CBAS = Cognitive Behavioral Avoidance Scale; COPE = Brief COPE; ATQ = Automatic Thoughts Questionnaire; CES-D = Center for Epidemologic Studies - Depression scale; BFIS = Barkley Functional Impairment Scale; WFIRS = Weiss Functional Impairment Rating Scale

In Sample 1 (Table 5), the ACS also correlated significantly with self-reported executive functioning problems in daily life—particularly with problems related to Self-Management to Time (r = .64). ACS also correlated moderately with cognitive and behavioral avoidance. Finally, ACS scores correlated to a moderate degree with overall functional impairment as measured by the BFIRS.

In terms of divergent validity, we examined correlations of ACS with related but distinct constructs and found significant but not overly inflated correlations with negative automatic thoughts depressive symptoms, and anxiety symptoms (Table 5), suggesting that the construct measured by ACS is distinct from these other constructs.

Sample 2

In this sample, participants who self-reported ever receiving an ADHD diagnosis or currently being diagnosed with ADHD scored significantly higher on the ACS than other participants (Table 4). Compared to Sample 1, the between-groups effect size in Sample 2 was larger. As in Sample 1, when participants were placed in diagnosis proxy groups based upon their BAARS scores, participants who met analogue DSM-5 criteria both with and without the age of onset criterion applied showed higher mean ACS scores than participants who did not with large to very large effect sizes (Table 4). When combining proxy diagnosis methods to create a group who reported being diagnosed with ADHD and meeting DSM-5 criteria on the BAARS without age of onset (n = 9), this group also had a significantly higher mean ACS score with a large effect size. Also as in Sample 2, ACS scores showed significant and moderate correlations with the total and symptom subscales of the BAARS and the ACS continued to correlate significantly with self-reported executive functioning problems in daily life. In this sample, correlations with cognitive and behavioral avoidance were weaker, but still significant.

Table 4.

Mean ACS Scores by ADHD Status – Study 2

Basis of ADHD Grouping	ADHD			Comparison
	N	M	SD	N	M	SD	p	d
Self-Report: Ever Diagnosed	23	2.61	0.94	281	2.23	0.68	.012	0.46
Self-Report: Currently Diagnosed	15	2.68	0.92	289	2.23	0.69	.018	0.55
BAARS DSM-5 Analogue: No Age of Onset	31	3.08	0.72	273	2.16	0.64	<.001	1.35
BAARS DSM-5 Analogue with Age of Onset	14	2.92	0.91	290	2.22	0.68	<.001	0.87
Ever Diagnosed + BAARS DSM-5, No Age of Onset	9	3.12	1.08	295	2.23	0.68	<.001	0.99

Note: BAARS = Barkley Adult ADHD Rating Scale

In Sample 2, ACS scores correlated with overall functional impairment as measured by the BFIS and with specific dimensions of impairment as measured by the subscales of the WFIRS. Correlations were strongest with school and life skills impairment and weaker with social impairment. As in Sample 1, significant but not excessive correlations with negative automatic thoughts, depressive symptoms, and anxiety symptoms provided evidence of divergent validity. As predicted, ACS scores correlated positively with coping strategies characterized by disengagement including cognitive (r = .40) and behavioral avoidance (r = .31) and behavioral disengagement (r = .30), denial (r = .31), and self-distraction (r = .19). As predicted, ACS scores also correlated negatively with engagement-oriented coping strategies including active coping (r = -.25) and planning (r = -.21), while correlations were weaker but still negative with positive reframing (r = -.11) and acceptance (r = -.13). It should be noted that correlations for self-distraction and acceptance may be attenuated due to poor scale internal consistency, as disattenuated correlations were .34 and .22, respectively. Unexpectedly, some of the strongest scores between ACS and coping were for substance use (r = .32) and self-blame (r = .35).

Discussion

The present research evaluated a brief measure of ADHD-related cognitions. This measure, known as the ACS, demonstrated good reliability, factor structure, and validity across two large community samples. As expected, scores on the ACS correlated with ADHD symptoms. ACS scores were significantly higher in adults with DSM-5 proxy diagnoses based upon self-report and, in one sample, were higher for people ever diagnosed and currently diagnosed with ADHD. Higher ACS scores were also associated with greater global and domain-specific functional impairment. In both samples, ACS scores correlated with executive functioning problems in daily life—in particular with managing behavior with respect to time. Indeed, on their face, many of the ACS items relate to difficulties disengaging from preferred tasks and shifting to higher-priority or more urgent but less preferred tasks. Relatedly, exploratory analyses of the relationship between the ACS and coping strategies suggests that the thoughts tapped by the measure may be associated with avoidant or disengagement coping as opposed to active strategies.

In considering the possible function of cognitions tapped by the ACS, there is considerable similarity of ACS items to permission-giving thoughts as described in the literature on cognitive therapy for substance abuse (A. T. Beck, Wright, Newman, & Liese, 2001) and trichotillomania (Rehm, Nedeljkovic, Thomas, & Moulding, 2015). In their model of cognitions in trichotillomania, Rehm and colleagues (2015) describe a process by which a “permission giving narrative” precedes hair pulling and justifies the behavior to the client (e.g., “I can just pull one hair and then stop.”), whereupon the act of pulling itself serves an emotion regulation function.

Similarly, Knouse and Mitchell (2015) have proposed that “overly positive” thoughts associated with ADHD often constitute avoidance behavior—that is, the act of thinking these thoughts may be reinforced by escape from negative affect that is triggered when clients are reminded of the need to engage in non-preferred tasks or similar anxiety-provoking situations. For example, a thought such as, “I’ll just do this one thing first,” gives the client permission to avoid a task that may begin to evoke negative affect and the act of thinking the permission-giving thought is subsequently reinforced by the temporary removal of negative affect. One implication of this view is that the types of thoughts measured by the ACS are by no means specific to people with ADHD, although they may represent an important link in the behavioral chain connecting ADHD symptoms to impairment for this population. Another implication of this view is that topographically “positive” and “negative” cognitions may often have the same avoidance function and so they may be positively correlated, as was observed in the current study (r = .39 with ATQ in the Study 1 sample, .40 in the Study 2 sample). Future studies will need to more rigorously evaluate these possibilities, as suggested below.

Our findings regarding the ACS should be considered in light of the limitations of the data presented here. First, we have presented results from two community samples of adults and, although the samples contained some adults that met our research criteria for ADHD, participants with the disorder were not specifically recruited for this study. As a result, our sample size of adults with ADHD (previous diagnosis or diagnosis proxy based on self-report) is relatively small. Furthermore, information on ADHD diagnoses, symptoms, impairment, and onset were all based upon self-report. The psychometrics of the ACS and evidence of clinical utility and sensitivity of the measure to treatment-related change will need to be evaluated in larger clinical samples and treatment studies that use multiple methods of assessment. Second, we did not use a prospective design and thus we cannot provide evidence of the stability of ACS scores (test-retest reliability) nor of the predictive validity of scores across time. Finally, our results cannot shed light on the degree to which the cognitions tapped by the ACS are actually associated with avoidance behavior in daily life or the degree to which people’s perceptions of the frequency of these thoughts are accurate. Importantly, this is a general limitation of using any self-report rating scale to assess cognitive and behavioral processes that may be fleeting and dependent upon the respondent’s perception and recollection. In future studies, this limitation might be addressed by using experience sampling methodology to evaluate the frequency and consequences of thoughts tapped by the ACS in daily life.

In addition to studies further evaluating the validity of the ACS, the measure may also potentiate additional research into the role of cognitions in the manifestation of ADHD symptoms in daily life and the difficulties adults with the disorder experience in implementing the self-regulation skills they “know” but may not consistently “show.” For example, it would be interesting to investigate whether frequency of thoughts as measured by the ACS is related to the accuracy of more global self-ratings, as evaluated in studies on Positive Bias. These measures may tap two aspects of the same “overly optimistic” cognitive biases or may represent two distinct cognitive phenomena associated with ADHD. The ACS may also be useful to clinicians conducting CBT with adults with the disorder. For example, the measure could be used to identify problematic thoughts associated with failure to use self-regulation skills and awareness of these thoughts could then serve as a cue for use of specific skills (see Knouse & Mitchell, 2015 for a detailed description). Completing the measure and discussing it may also help clients to identify their own idiosyncratic “permission-giving thoughts” that tend to precede problematic avoidance.

In sum, we have presented data from two large community samples that lend preliminary support for the reliability and validity of a short self-report measure of ADHD-related cognitions. Scores on the measure are associated with self-reported ADHD symptoms and related impairment, as well as time management problems and the tendency to engage in problematic avoidance and disengagement coping. We hope that the measure will be the subject of future study in well-defined ADHD samples and that it may prove useful in studying the contribution of maladaptive thoughts to ADHD-related impairment.

Appendix Thoughts Questionnaire

Directions

Listed below are different kinds of thoughts that people may have about themselves. Using the scale below, please indicate how often each thought occurs to you.

• 1 = Not at all

• 2 = Sometimes

• 3 = Often

• 4 = Very Often

• 5 = All the time

• _______ 1. I’ll just do this one thing first.

• _______ 2. I do better waiting until the last minute.

• _______ 3. I can’t stop right now.

• _______ 4. Though this usually “sucks me in,” I’ll just do it for a minute.

• _______ 5. I have plenty of time—I’ll just do one more thing before I go.

• _______ 6. Being impulsive is a big part of who I am.

• _______ 7. I know I’m supposed to be doing something else but I want to do this now.