Self-reported Cognitive Scales in a U.S. National Survey: Reliability, Validity, and Preliminary Evidence for Associations with Alcohol and Drug Use

Abstract

Aims

To evaluate relationships between measures of cognitive functioning and alcohol or drug use among adults (≥18 years) in the U.S. general population.

Design

Two cognitive scales were created based on dimensionality and reliability of self-reported Executive Function Index items. Relationships between the two scales and validators were evaluated. Associations between the cognitive scales and past-year frequency of alcohol or drug use were estimated with adjusted odds ratios (aOR).

Setting

USA, using the 2012–2013 National Epidemiologic Survey on Alcohol and Related Conditions-III, a nationally representative adult sample selected by multistage probability sampling.

Participants

36,085 respondents.

Measurements

Past-year substance use outcome variables categorized binge drinking, marijuana, cocaine, opioid, sedative/tranquilizer, and stimulant use as frequent (at least weekly to daily), infrequent (any to 2–3 times/month), or no use, assessed by the Alcohol Use Disorder and Associated Disabilities Interview Schedule-V. Key predictors were the two cognitive scales. Construct validators included education and functional impairment. Covariates included age, gender, income, and race/ethnicity.

Findings

Nine cognitive items fit a two-factor model (Comparative Fit Index=.973): attention (5 items) and executive functioning (4 items). Both scales were positively associated with higher education (ps<.001) and negatively associated with functional impairment (ps<.001), demonstrating construct validity. Poorer attention was associated with frequent and infrequent binge drinking and use of drugs (aOR range=1.07 [binge drinking] to 1.72 [stimulants], ps≤0.01). Poorer executive functioning was associated with frequent binge drinking and use of drugs (aOR range=1.22 [binge drinking] to 2.03 [cocaine], ps<0.001), and infrequent use of all drugs (aOR range=1.19 [marijuana] to 1.63 [cocaine], ps<0.001).

Conclusions

Impairments in attention and executive functioning are positively correlated with substance use in the U.S. general population.

INTRODUCTION

Cognitive functioning refers to the mental processes that store, retrieve, transform, and use information. Such functioning involves domains such as memory, learning and attention, and higher executive functions, e.g., decision making, organization, planning, and control inhibition. Adequate functioning is necessary for important life activities, ranging from simple, e.g., grocery shopping, to complex, e.g., career planning.

Neuropsychological and neuroimaging evidence indicates impaired cognitive functioning in adult patients treated for substance use disorders (SUD) [1], potentially impacting their ability to utilize treatment [2,3] or function well in important interpersonal or occupational areas [4,5]. These cognitive impairments are associated with many substance disorders [6], including alcohol [7–10], cannabis [11–15], cocaine [16–24], and others [25–29]. In domains including attention and executive functioning, impairments range from mild to severe, as assessed with measures ranging from short tests to extensive neuropsychological batteries.

Whether cognitive functioning is also impaired in general population substance users is unknown. Extant surveys with measures of substance use and cognitive functioning included only middle-aged or older U.S. adults [30–35], omitting late adolescent and early adult participants, when prevalence of alcohol and drug use and related disorders is highest [36–38]. Thus, in large, national samples with participants at the peak ages of risk, information is lacking on the relationship of cognitive functioning to substance use, an increasing health problem with high morbidity and mortality among adults [36,37,39].

The National Epidemiologic Survey on Alcohol and Related Conditions-III (NESARC-III) provides nationally representative survey data on U.S. adults ages ≥18, including measures of cognition and drug and alcohol use, which can be used to fill the knowledge gap about cognition and substance use and related consequences in the general population. Objective (interviewer-rated) neuropsychological assessments were unfeasible in NESARC-III due to the extensive resources (e.g., interview time, specialized interviewer training) needed. Since self-report measures of cognition show moderate to strong relationships to objective measures [40–42], particularly when specific cognitive domains are assessed [43], a brief self-report assessment of cognitive functioning was used, the Executive Function Index (EFI). The EFI is designed to assess specific cognitive domains in adult community surveys, in which it has previously demonstrated good internal consistency and convergent validity [44].

In this study, we examined the relationships between cognitive functioning and alcohol and drug use in NESARC-III. Before investigating these relationships, information on the psychometric properties of the self-reported EFI items in NESARC-III was needed. Thus, the study was conducted in four steps. 1. We assessed the dimensionality of EFI items, which suggested two underlying factors. 2. We estimated the reliability of these factors. 3. We assessed the construct and discriminant validity of two scales based on these factors. 4. We estimated the association between these cognitive scales and past-year frequency of binge drinking and drug use.

METHODS

Design

To address the study aim, data from a large, general population U.S. sample were used. We created reliable measures of cognitive functioning, through modelling the dimensionality of the Executive Function Index (EFI) items and assessing the reliability of the resulting factors. Two cognitive scales were derived from the factors and validated by evaluating relationships with construct and discriminant validators. Associations between the frequency of alcohol (binge drinking) or drug use and the cognitive scales were estimated.

Data

The 2012–2013 NESARC-III is a nationally representative survey of the non-institutionalized, civilian, adult (≥18 years old) U.S. population, including residents of households and group quarters [37,45]. The multistage probability sampling scheme included primary sampling units (largely counties) from the entire USA, secondary sampling of groups of census-defined blocks, and tertiary sampling of households, with random selection of adults within households [46]. The response rate was 60.1%, comparable to other large-scale national studies [47,48]. Interviews were face-to-face, using computer-assisted personal interviewing [46]. Interview quality was assured by rigorous interviewer training, supervision and random callbacks to verify responses. Participants provided informed consent and received $90.00. The National Institutes of Health and Westat Institutional Review Boards approved the NESARC-III protocol. This study included 36,085 respondents, after excluding 224 respondents (0.6% of the sample) missing responses for ≥25% of the EFI items.

Measures

Outcomes: Past-year substance use

The Alcohol Use Disorder and Associated Disabilities Interview Schedule-5 (AUDADIS-5) assessed frequency of binge drinking and non-medical substance use, including marijuana, cocaine, opioids, sedatives/tranquilizers, and stimulants. AUDADIS-5 alcohol and drug use questions were identical to those in AUDADIS-IV, which showed good-to-excellent test-retest reliability [46,49]. Frequency of use was indicated by a three level variable (frequent use, infrequent use, no use), based on how often in the past year respondents engaged in drug use or binge drinking (≥5 drinks in a day for men, ≥4 for women). Frequent was defined as once a week to daily; infrequent as 1–2 times in the past year to 2–3 times a month.

Cognitive functioning

The EFI [44] is a self-reported measure of functioning on cognitive tasks encountered in daily life. NESARC-III included 12 items from the Strategic Planning and Organization EFI sub-scales to provide brief assessment of domains with widely-established impairments among patients in treatment for substance use [3,6,7]. These subscales previously evidenced good reliability (α=.70 and .75, respectively [44]), and correlated as predicted to demographic characteristics, other cognitive measures, and psychological attributes [44,50,51]. Items were rated on five-point scales (1=not at all; 2=a little; 3=somewhat; 4=a lot; 5=very much). Items that assessed difficulty with functioning (‘mix up sequence’; ‘trouble multi-tasking’; ‘lose track’; ‘trouble summing’; ‘lose interest’) were reverse-coded so that higher response values indicated better functioning, consistent with the other items. Items were summed into scales (Scale construction below), used as outcomes for validity analyses and then as predictors of substance use.

Validators: predictors of cognitive functioning

Two construct validators were posited. One was education: less than high school, completed high school, some college, college degree, and graduate study. Participants with higher education levels were predicted to score higher (positive association) on the cognitive function (EFI) scales. The other was functional impairment, based on the 12-Item Short Form Health Survey Mental Component Summary (MCS) score, which assesses performing activities less carefully, accomplishing less, or social problems due to mental health issues [37,52]. This score is a reliable and valid measure of current functioning widely used in population surveys, and was used previously for construct validation [53,54]. To facilitate interpretation, we defined impairment as MCS score in the bottom twenty-fifth percentile (≤45.8), similar to cut-offs suggested in general population studies [55,56]. Participants with MCS-based functional impairment were predicted to score lower (negative association) on the cognitive scales. As a sensitivity analysis, we included another functional impairment variable. Respondents were considered to have functional impairment if, when interviewed, they were not functioning in a major role, defined as: full time employment; full time student; full time homemaker; part time school and employment; or ≥65 year old retiree. This variable was related to the MCS-based variable (χ²₍₁₎ =482.53, p≤.0001).

Discriminant validators were hypothesized to have a null relationship to cognitive functioning. These included U.S. Census-defined geographical region (Northeast; Midwest; West; South) and height. Height was dichotomized at the median separately by gender.

Sociodemographic control variables

Covariates that could potentially confound the relationships of interest were included: gender (male, female); age (18–24, 25–44, 45–64, 65–74, ≥75); education; personal income ($0–$19,999, $20,000–$34,999, $35,000–$69,999, ≥$70,000); and race/ethnicity (Hispanic, or Non-Hispanic: White, Black, Native American, Asian/Pacific Islander).

Statistical analysis

Factor analysis

Factor analysis assessed the dimensionality and factor structure of the cognitive items. First, the NESARC-III sample was split into two random splits (SAS 9.4 [57]). In Split 1, exploratory factor analysis determined the number of factors (dimensions) based on the number of Eigenvalues >1 and standard model fit indices: comparative fit index (CFI), Tucker-Lewis Index (TLI), and root-mean-square error of approximation (RMSEA), using widely-accepted values of good fit (CFI, TLI ≥0.95, RMSEA ≤0.06) [58]. In Split 2, confirmatory factor analysis confirmed the factor structure. In confirmatory factor analysis, the latent factors were parameterized to have mean=0 and variance=1, freeing factor loadings for all items. Factor analysis was done iteratively; when model fit indices indicated poor fit, item loadings and substantive meaning were used to determine items to exclude, until nine items were included in the final item set. Last, confirmatory factor analysis was carried out for the whole dataset. Factor analyses were conducted with Mplus 7.11 [59], adjusting for the complex sample design and using a weighted least squares estimator appropriate for categorical items. Since 365 participants (1.0% of the sample) were missing one or two EFI items, correlation estimates for each pair of items were calculated among participants with responses for both items (pairwise present analysis), a standard method for low levels of missingness [60].

Internal consistency (Reliability)

Cronbach’s alpha was calculated using SAS 9.4, with α values ≥0.65 considered acceptable [61,62].

Scale construction

Given the evidence for two latent variables underlying the cognitive function items, two scales were created by summing item responses, after recoding responses to 0–4 (instead of 1–5), so that the lowest score=0 for interpretability. Higher values indicate better functioning. The attention scale included: ‘mix up sequence’, ‘trouble multi-tasking’, ‘lose track’, ‘trouble summing’ and ‘lose interest’. The executive functioning scale included: ‘future planning’, ‘learn from mistakes’, ‘monitor self’, and ‘consider consequences’. During summation, mean imputation was used for those missing responses to 1–2 items; scales were rounded to the nearest integer. Individuals missing ≥3 items were excluded.

Validation

To assess the relationships between the attention and executive scales (dependent outcomes) and the validators (education level, MCS-based functional impairment, region, height in men, and in women), ten linear regression models were analyzed (Table S1). In five models, attention was the outcome, with each model testing one validator (predictor); the other five models were the same, but with executive as the outcome. Following standard epidemiologic practice, sociodemographic covariates (age, gender, income, race/ethnicity, and education) were included as predictors in the models to control for their effects on the relationships of interest [63–65]. Due to multiple models, a Bonferroni-corrected p-value of .05/10=.005 was used to declare significance. SUDAAN 11.0.1 [66] was used, including sample weights to adjust for the complex sampling and non-response.

Finding that a construct validator showed a stronger relationship to one of the correlated outcomes (attention or executive functioning) would suggest that the scales assessed different domains, supporting two scales. We used Mplus 7.11 to perform regression of the two scales simultaneously as dependent outcomes, using each validator as a predictor, and controlling for demographic covariates (Table S1). Standardized regression coefficients were used so that effects for the two scales could be compared. For each validator subcategory (4 for education, 1 for MCS-based functional impairment), the difference between the standardized effect for the attention and executive scales was estimated; a difference significantly different from 0 indicated a stronger relationship to one of the scales. A Bonferroni-corrected p-value of .05/5=.01 was used.

Sensitivity analysis

To increase confidence in the MCS-based impairment results, validation analysis was done for the role-based functional impairment variable. Two linear regression models were analyzed: one for each cognitive scale (dependent outcome), predicted by role-based impairment, controlling for age, gender, income, education, and race/ethnicity (Table S1).

Associations of substance use and cognition

We explored the relationships between substance use and cognition and vice versa. First, using linear regression with the cognitive scales as the dependent outcomes, we tested whether the mean attention or executive scale scores differed significantly by frequency of use. We analyzed 12 models using SUDAAN 11.0.1: six models with attention as the outcome, predicted by each of 6 substance use frequency variables and the sociodemographic covariates; the other 6 models had the same predictors with executive as the outcome. In each model, the adjusted mean scale score was calculated for each level of the substance use variables (frequent use; infrequent use; no use [reference group]). A significant regression coefficient for frequent or infrequent use indicated that the mean scale score differed significantly from the mean score for no use.

Second, using generalized multinomial logistic regression with substance use frequency as the dependent outcomes, we estimated the unique effects of each cognitive scale on substance use frequency. Six models were analyzed using Mplus 7.11: one model for each substance use frequency outcome, predicted by both cognitive scales and the sociodemographic covariates, producing adjusted odds ratios (Table S2). Multinomial logistic regression is appropriate for dependent variables with more than two categories (here, frequency of use, with three levels) [67]. To facilitate interpretation, the cognitive scales were standardized (mean=0; standard deviation=1), as is common for continuous predictors. Thus, the odds of frequent use (vs. no use) or infrequent use (vs. no use) were assessed for a one standard deviation decrease in the cognitive scales, corresponding to cutoffs for clinically meaningful cognitive impairment [68,69].

RESULTS

Sample characteristics

About half the sample was female, age <45, income ≥$20,000; about two-thirds were Non-Hispanic White; and 60% completed at least some college (Table S3). Prevalence of substance use ranged from 33% for binge drinking to 1% for cocaine (Table 1).

Table 1.

Descriptives of past-year substance use variables and cognitive functioning in NESARC-III (N=36,085)

	n	Prevalence % (SE)a of substance use	Mean scale score (SE)a,b
			Attention	Executive
Binge drinkingc
Frequent use	4,142	10.9 (.28)	15.8 (.07)d	9.8 (.08)d
Infrequent use	7,416	21.9 (.37)	15.9 (.06)d	10.4 (.04)
No use	24,527	67.2 (.49)	16.1 (.04)	10.5 (.04)
Marijuana
Frequent use	1,892	04.6 (.17)	15.0 (.12)d	9.6 (.10)d
Infrequent use	1,777	04.9 (.18)	15.3 (.11)d	9.8 (.09)d
No use	32,416	90.5 (.27)	16.1 (.04)	10.5 (.04)
Cocaine
Frequent use	80	00.2 (.02)	13.8 (.54)d	7.6 (.34)d
Infrequent use	332	00.8 (.07)	14.8 (.25)d	8.7 (.28)d
No use	35,673	99.0 (.07)	16.0 (.03)	10.4 (.04)
Opioids
Frequent use	658	01.6 (.10)	15.1 (.24)d	9.3 (.17)d
Infrequent use	898	02.4 (.11)	15.2 (.12)d	9.7 (.11)d
No use	34,529	95.9 (.16)	16.1 (.03)	10.5 (.04)
Sedatives
Frequent use	356	00.9 (.07)	14.2 (.31)d	9.2 (.21)d
Infrequent use	461	01.4 (.09)	14.7 (.20)d	9.4 (.16)d
No use	35,268	97.7 (.12)	16.1 (.03)	10.4 (.04)
Stimulants
Frequent use	147	00.4 (.04)	13.5 (.41)d	9.0 (.28)d
Infrequent use	257	00.8 (.07)	14.5 (.26)d	9.1 (.23)d
No use	35,681	98.8 (.08)	16.1 (.03)	10.4 (.04)

^aadjusted for complex sampling using SUDAAN 11.0.1

^badjusted for age, gender, income, education, and race/ethnicity, using linear regression

^cbinge drinking defined as ≥4 drinks in a day for women, ≥5 drinks in a day for men

^dmean scale score is significantly lower than among those with no use, p≤.0001

Factor analysis

In Split 1 of the sample, exploratory factor analysis of the 12 EFI items suggested 2 factors (2 eigenvalues>1; Table S4), but the 2-factor model fit indices were below recommended values of .95 (CFI=.93; TLI=.89). To improve model fit, we removed items after considering their substantive meaning, including ‘memory strategies’ and ‘save money’, which may assess aspects of personality more than cognition. Exploratory factor analysis of the remaining 10 items led to acceptable 2-factor model fit (Table S4; CFI=.98; TLI=.97), but confirmatory factor analysis in Split 2 indicated reduced model fit (CFI=0.96, TLI=0.94). To further improve model fit, the ‘organized’ item, which cross-loaded on both factors in the exploratory analysis (Table S4), was removed. Exploratory factor analysis (Split 1) of the remaining 9 items supported a 2-factor model based on 2 eigenvalues >1 and model fit indices (Table 2). Factor 1, labeled ‘attention’, included ‘lose interest’, ‘mix up sequence’, ‘lose track’ ‘trouble multi-tasking’, and ‘trouble summing up’. Factor 2, labeled ‘executive’, included ‘future planning’, ‘learn from mistakes’, ‘monitor self’, and ‘consider consequences’. Confirmatory factor analysis in Split 2 confirmed the factor structure, based on model fit indices: CFI=0.98; TLI=0.97; RMSEA=0.03. Figure 1 shows the confirmatory factor analysis results in the entire sample.

Table 2.

Exploratory Factor Analysis of nine cognition items in NESARC-III (Split 1; n=18,042)

	1 factor model	2 factor model
ITEMS		Factor 1 (Attention)	Factor 2 (Executive)
MIX UP SEQUENCE: mix up the sequence when doing several things in a row	0.559	0.590	−0.023
TROUBLE MULTI-TASKING: have trouble doing two things at once, multi-tasking	0.627	0.649	0.015
LOSE TRACK: sometimes lose track of what I’m doing	0.671	0.746	−0.052
TROUBLE SUMMING UP: have trouble summing up information in order to make a decision	0.643	0.661	0.008
LOSE INTEREST: start things, but then lose interest and do something else	0.637	0.661	0.011
FUTURE PLANNING: try to plan for the future	0.380	0.005	0.539
LEARN FROM MISTAKES: only have to make a mistake once in order to learn from it	0.411	0.114	0.488
CONSIDER CONSEQUENCES: think about the consequences of an action before do it	0.416	0.003	0.710
MONITOR SELF: monitor myself so that can catch any mistakes	0.315	−0.108	0.660
Model fit indices
RMSEA (90% CI)	0.106 (0.103, 0.108)	0.026 (0.024, 0.029)
CFI	0.769	0.990
TLI	0.692	0.981
correlation	---	0.404

Notes: First four egeinvalues = 3.19, 1.64, 0.75, 0.67; RMSEA = root mean square error of approximation (lower values indicate better model fit); CI = confidence interval; CFI = comparative fit index, TLI = Tucker-Lewis index (higher values indicate better model fit)

Figure 1.

Confirmatory factor analysis of nine cognitive items in the NESARC (N=36,085)

Notes: For each item, factor loading is shown above the arrow. Correlation between the two factors is shown above the bi-directional arrow. CFI=comparative fit index; TLI= Tucker-Lewis index; RMSEA= root mean square error of approximation

Reliability

Both scales showed acceptable reliability: attention items (α =0.73); executive items (α =0.65).

Validity

Attention scale scores were associated as predicted with both construct validators (Table 3). Compared to participants who did not complete high school, those who completed high school or more had significantly higher mean attention scores (Table 3). Those with MCS–based functional impairment had significantly lower mean attention scores, as did those with role-based functional impairment. In contrast, attention scores were not significantly associated with the discriminant validators (Table 3)

Table 3.

Relationship of cognitive scales and validators, NESARC-III (N=36,085)

	Association with attention scale				Association with executive scale				Difference in relationship of construct validator category to each scaleb
	Wald F(degrees-of-freedom)	p-value	Mean scale score (SE)a	Effect size (95% CI)a	Wald F(degrees-of-freedom)	p-value	Mean scale score (SE)a	Effect size (95% CI)a
Construct validators
Education level	24.85(4)	≤.0001e			99.04(4)	≤.0001e
Less than high school			15.4 (.08)	reference			9.4 (.09)	reference
Completed high school			15.9 (.06)	.134 (.090,.178)			9.9 (.06)	.150 (.108,.191)	No
Some college			16.1 (.05)	.213 (.161,.265)			10.5 (.05)	.306 (.253,.359)	Yesh; executive
College degree			16.3 (.05)	.258 (.205,.312)			10.9 (.05)	.438 (.384,.491)	Yesi; executive
Post-graduate study			16.3 (.06)	.271 (.211,.331)			11.3 (.05)	.555 (.494,.615)	Yesi; executive
Functional impairment
MCS-basedc	1049.71(1)	≤.0001e			427.38(1)	≤.0001e
Yes			14.5 (.06)	−.596 (−.632,−.560)			9.6 (.05)	−.317 (−.347,−0.287)	Yesi; attention
No			16.5 (.03)	reference			10.7 (.04)	reference
Role-basedd	158.67(1)	≤.0001f			53.77(1)	≤.0001f
Yes			15.6 (.06)	−.201 (−.233,−.170)			10.1 (.05)	−.117 (−.149,−0.086)	Yesi; attention
No			16.2 (.04)	reference			10.5 (.04)	reference
Discriminant validatorsj
Region	2.16(4)	.10g			1.30(1)	.28g
Height
Men	2.19(1)	.14g			5.34(1)	.02g
Women	0.10(1)	.75g			0.15(1)	.70g

^aadjusted for sociodemographic covariates using linear regression in SUDAAN 11.0.1. For education level, these included: age (18–24, 25–44, 45–64, 65–74, 75+), gender (men, women), personal income ($0–$19,999, $20,000–$34,999, $35,000–$69,999, ≥$70,000), and race/ethnicity (White, Black, Native American, Asian/Pacific Islander, Hispanic). For functional impairment and region, those covariates were included as well as education (see categories in table above). For height, covariates included age, personal income, race/ethnicity and education.

“Effect size” refers to the “standardized difference”, the difference in the mean scale score for each group as compared to the reference, divided by the standard deviation of the scale in the sample (attention scale=3.34; executive scale=3.31).

^bthe regressions for each scale was carried out simultaneously (bivariate regression), to test if the difference in the effect sizes for the scales (attention minus executive) was significantly different from zero, using Mplus 7.11 “model constraint” ; if there is a significant difference, the scale with greater effect size is listed. A Bonferroni corrected p-value of 0.05/5=.01 was used to declare significance, since five differences were initially tested: each of 4 non-reference education levels and MCS-based functional impairment; as sensitivity analysis, role-based functional impairment was not included in the Bonferroni corrections.

^cthose with functional impairment had scores in the bottom twenty-fifth percentile of the Mental Component Summary (MCS) of the 12-Item Short Form Health Survey version 2

^dthose with functional impairment are respondents whose situation at the time of the interview did not indicate functioning in a major role (full time work, studying, or homemaking; part time work and study; or retired and 65 or older).

^esignificant, below the Bonferroni corrected p-value of 0.05/10=0.005. A denominator of 10 is used to account for the 5 initial association tests carried out for each of 2 scales: education, MCS-based functional impairment, region, height in men, and height in women.

^fas sensitivity analysis, not included in the Bonferroni corrections

^gnot significant, above the Bonferroni corrected p-value of 0.05/10=0.005

^hp-value =0.002

ⁱp-value <0.001

^jMean scale scores and effect sizes were not reported since the discriminant validators were not significantly associated with the cognitive scales

Executive scale scores were associated as predicted with both construct validators (Table 3). Compared to participants who did not complete high school, other participants had significantly higher mean executive scores. Those with MCS-based functional impairment had significantly lower mean executive scores, as did those with role-based functional impairment. Executive scores were not significantly associated with the discriminant validators (Table 3)

Significant differences in the magnitude of association between the construct validators and the two scales were observed (Table 3). The attention scale showed significantly stronger association with functional impairment (both forms), while the executive scale showed stronger association with education.

Association of cognitive scales and substance use

Linear regressions of substance use predicting the cognitive scales showed that mean scores for the attention and executive scales were significantly lower among those with frequent or infrequent past-year binge drinking or use of marijuana, cocaine, opioids, sedatives/tranquilizers, or stimulants, as compared to those with no use, except for the executive scale and infrequent binge drinking (Table 1).

Logistic regressions of cognition predicting substance use showed that for all substances, the attention and executive scales were independently and negatively associated with the frequency of past-year substance use, except for infrequent binge drinking and the executive scale (Table 4). The effects of cognition on the odds of substance use frequency were assessed for a one standard deviation decrease in the standardized scales, corresponding to common cutoffs for clinically important impairment [68,69]. For example, a one-unit decrease in the attention scale was associated with 1.5 times increased odds of frequent cocaine use and 1.3 times increased odds of infrequent use, independent of the executive scale; a one-unit decrease in the executive scale was associated with 2.0 and 1.6 times increased odds of frequent or infrequent cocaine use, respectively, independent of the attention scale.

Table 4.

Association of cognitive scales and frequency of past year substance use, NESARC-III (N=36,085)

	Binge drinkingb		Marijuana use		Cocaine use		Opioid use		Sedative/ tranquilizer use		Stimulant use
	aORa (95% CI)	p	aORa (95% CI)	p	aORa (95% CI)	p	aORa (95% CI)	p	aORa (95% CI)	p	aORa (95% CI)	p
Decreased Attention scale
Frequent use	1.07 (1.02–1.12)	.01	1.29 (1.21–1.38)	<.001	1.49 (1.26–1.77)	<.001	1.22 (1.09–1.36)	<.001	1.46 (1.30–1.64)	<.001	1.72 (1.48–2.00)	<.001
Infrequent use	1.08 (1.04–1.12)	<.001	1.23 (1.15–1.31)	<.001	1.29 (1.15–1.45)	<.001	1.22 (1.13–1.30)	<.001	1.37 (1.25–1.50)	<.001	1.45 (1.28–1.64)	<.001
Decreased Executive scale
Frequent use	1.22 (1.16–1.26)	<.001	1.26 (1.19–1.34)	<.001	2.03 (1.70–2.42)	<.001	1.34 (1.21–1.49)	<.001	1.34 (1.18–1.51)	<.001	1.41 (1.17–1.69)	<.001
Infrequent use	1.02 (0.98–1.07)	.24	1.19 (1.12–1.27)	<.001	1.63 (1.40–1.89)	<.001	1.24 (1.15–1.33)	<.001	1.31 (1.19–1.45)	<.001	1.45 (1.26–1.68)	<.001

Notes: aOR = adjusted odds ratio; CI = confidence interval; frequent use = use of substance (or binge drinking) at least weekly in the past year; infrequent use= use of substance from one time a year up to 3 times a month in the past year

^aaORs indicate the increased odds of frequent use (vs. no use) or infrequent use (vs. no use) for a one unit decrease in the standardized cognitive scale. AORs were estimated using logistic regression, including both scales in the model, adjusted for age (18–24, 25–44, 45–64, 65–74, 75+), gender (men, women), personal income ($0–$19,999, $20,000–$34,999, $35,000–$69,999, ≥$70,000), race/ethnicity (White, Black, Native American, Asian/Pacific Islander, Hispanic), and education (less than high school, completed high school, some college, college degree, at least some post-graduate study), using MPlus version 7.11.

^bbinge drinking defined as ≥4 drinks in a day for women, ≥5 drinks in a day for men

DISCUSSION

In a nationally representative survey of U.S. adults (NESARC-III), nine Executive Function Index (EFI) items fit a two-factor model representing attention and executive factors, each with acceptable internal consistency. Validity of the attention and executive scales was demonstrated through construct and discriminant validation. Importantly, poorer scores on both the attention and executive scales were associated with the frequency of past-year binge drinking and drug use.

This is the first study showing that impairments in attentional and executive aspects of cognitive functioning are associated with frequency of binge drinking and use of marijuana, cocaine, opioids, sedative/tranquilizers, and stimulants in the U.S. general population. Similar impairments are well documented in clinical samples of patients with alcohol [70–73] and drug use disorders [74–78].

This study supports using the attention and executive scales in NESARC-III data to further investigate the relationships of cognition to substance use and SUD. While we estimated the associations of cognitive scores with each substance, regardless of other substance use, future studies should determine if observed associations are due to poly-substance use, and whether associations unique to a specific substance are stronger for one aspect of cognition than another. Additional studies should examine the associations of cognition with SUD, course of disorders, and treatment utilization. Studies should also determine whether the associations mentioned above differ by sociodemographic characteristics, economic circumstances (e.g., income levels, employment) or psychiatric comorbidity. Results from such studies could generate hypotheses for research involving neuroimaging, such as the Adolescent Brain and Cognition Development (ABCD) study [79], which follows 10,000 children through young adulthood, using structural and functional imaging to investigate the effects of substance use on brain development.

In the cross-sectional NESARC-III survey, whether cognitive impairments precede substance use or vice versa cannot be determined. A reciprocal relationship may exist, in which impairments and substance use influence each other. The longitudinal ABCD study could determine the direction of effect between cognition and substance use/disorders, and if specific substances influence particular aspects of cognition by affecting specific areas of the developing brain. However, regardless of direction, poorer cognitive functioning negatively impacts daily life [4,5] and may cause lack of insight into one’s substance use as a source of problems, impeding treatment utilization [80,81] or decreasing the likelihood of effective treatment [1,3]. While abstinence or reduced substance use may partially improve cognition [69,82], future research should determine whether factors shown to protect against cognitive impairments in aging adults, e.g., healthy diet [83,84], physical activity [85], and intellectual activities [86], also protect against cognitive impairments in populations with difficulties in reducing substance use.

Limitations

First, NESARC-III assessed cognitive functioning (planning; temporal sequencing, monitoring self, sustaining attention) with self-reported measures [44]. Although such measures may be subject to inaccuracies due to impaired insight or efforts to conceal deficits, self-reported and objective measures of cognitive performance show moderate to strong relationships [40–42], with stronger relationships when specific cognitive domains are assessed [43], e.g., the attention and executive scales derived from the EFI. Additionally, the EFI assesses consequences of cognitive impairment (e.g., trouble with multi-tasking) rather than directly testing cognition. However, administering standard objective cognitive tests requires resources [87–89] that are unfeasible for surveys such as NESARC-III. Use of the valid, reliable attention and executive scales will allow investigation of the relationships of cognitive functions to substance-related factors in the general population, which can generate hypotheses for further examination with full-scale objective tests in targeted samples. Second, data were cross-sectional; longitudinal studies are warranted to unravel the interplay between cognitive impairments and substance use. Third, as in other national surveys, NESARC-III used self-report rather than biological testing for substance use [39]. While self-report might introduce bias, those with frequent use may report more reliably on use [90].

Strengths

The NESARC-III sample is nationally representative [46], large enough to conduct factor analysis in random sample halves, covers a broader age range than previous general population samples with cognition measures [31–35], and includes measures for a range of substances. The two self-reported scales, which offer the advantages of fast, efficient cognitive data collection [44], were supported by discriminant and construct validation. Two considerably different functional impairment variables worked the same way, strengthening confidence in the results. Therefore, this study contributes uniquely to the literature and provides reliable, valid attention and executive scales, assessing two related but distinct cognitive domains.

Conclusions

This study shows the validity of two brief scales assessing key aspects of cognition, attention and executive functioning, in a U.S. nationally representative sample. The study demonstrates the association of these cognitive measures with frequency of alcohol and drug use in the general population, indicating that cognitive impairments, previously shown to be associated with substance use in patient samples, constitute a broader problem with more general substance users. Using these scales in NESARC-III data will allow investigation into the relationships of cognitive functioning to poly-substance use, SUD, and treatment, advancing our knowledge of substance use, a major public health problem.

Supplementary Material

Supplementary Materials

Table S1: Linear regression models for association of cognitive scales with validators, NESARC-III (N=36,085).

Table S2: Generalized multinomial logistic regression models for association of frequency of substance use with cognitive scales, NESARC-III (N=36,085).

Table S3. Description of dataset from NESARC-III (N=36,085)

Table S4. Initial Exploratory Factor Analysis of cognition items in NESARC-III (Split 1, n=18,065)