Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2017 Nov 1.
Published in final edited form as: Exp Clin Psychopharmacol. 2017 Aug;25(4):322–332. doi: 10.1037/pha0000115

Neurocognitive Dysfunction in Addiction: Testing Hypotheses of Diffuse versus Selective Phenotypic Dysfunction with a Classification-based Approach

Warren K Bickel 1,2, Lara N Moody 1,2, Celia R Eddy 3, Christopher T Franck 1,3
PMCID: PMC5606154  NIHMSID: NIHMS870878  PMID: 28782983

Abstract

Neurocognitive dysfunctions are frequently identified in the addictions. These dysfunctions may indicate either diffuse dysfunction or may represent separate facets that have differential importance to the addiction phenotype. In a sample (n=260) of alcohol and/or stimulant users and controls we measured responses across seven diverse neurocognitive measures. These measures were Continuous Performance, Delay Discounting, Iowa Gambling, Stroop, Tower, Wisconsin Card Sorting, and Letter Number Sequencing. Comparisons were then made between the drug-dependent groups and controls using analysis of variance and also using a machine learning approach to classify participants based on task performance as substance-dependent or controls in one tree and as alcohol and/or stimulant users or controls in a second tree. The analysis of variance showed significant differences between groups on the Delay Discounting (p<0.001), Iowa Gambling (p<0.001), Letter Number Sequencing (p<0.001), and Wisconsin Card Sorting (p<0.05) tasks. The first classification tree correctly classified between substance-dependent or controls for 88.3% of participants and classified between alcohol and/or stimulant users or controls for 63.9% of participants. Delay discounting was the first split in both trees and in the substance-dependent and control tree. The analysis of variance results largely replicate previous findings. The machine learning classification tree analysis provides evidence to support the hypothesis that different measures of neurocognitive dysfunction represent different processes. Among them, delay discounting was the most robust in categorizing drug dependence.

Keywords: delay discounting, neurocognitive deficits, alcohol, stimulants, machine learning, classification tree

Introduction

Neurocognitive deficits have been frequently identified as a core component of addiction (Bechara, 2005; Bickel, Jarmolowicz, Mueller, Gatchalian, & McClure, 2012; Ersche, Clark, London, Robbins, & Sahakian, 2006; Verdejo-García, Rivas-Pérez, López-Torrecillas, & Pérez-García, 2006). Neurocognitive deficits are composed of multiple component constructs and measurement of those components generally show that they are not strongly correlated with one another (Bates, Bowden, & Barry, 2002; Bickel, Jarmolowicz, Mueller, Gatchalian, et al., 2012). Nonetheless, the general finding is that those who are drug dependent will exhibit greater evidence of neurocognitive dysfunction across multiple measures when compared to community controls.

The pervasiveness of neurocognitive deficits across diverse measures in the drug dependent and the lack of strong correlation among the measures suggests two distinct and divergent hypotheses. The first hypothesis was suggested by Stavo (2013) following a meta-analysis of neurocognitive deficits in chronic alcohol users in which verbal fluency/language, speed of processing, working memory, attention, problem solving, impulsivity, verbal learning, verbal memory, visual learning, visual memory and visuospatial abilities were assessed. Based on the meta-analysis, the authors concluded diffuse, as opposed to specific, neurocognitive dysfunction following chronic alcohol use. From this perspective, one might deduce that any measure of neurocognitive function is a reasonable proxy of another in terms of indicating the generalized deficit. An alternative hypothesis is that the different measures of neurocognitive function represent and measure divergent processes that are differentially important to the phenotype of drug dependence. From this perspective, some of the measures may have more centrality in the emergence and continuation of drug dependence, while others may be more peripheral or secondary. Evidence supportive of this view comes from studies showing some types of neurocognitive deficits are not present in some substance use disorders (Johnson et al., 2010) and that different pharmacological or neurocognitive training can selectively modulate certain measures while leaving others unchanged (Bickel, Yi, Landes, Hill, & Baxter, 2011; Winstanley, Dalley, Theobald, & Robbins, 2004).

One factor that strongly influences our understanding of neurocognitive function in drug dependence is the statistical and analytical tools we use to analyze and interpret our findings. Studies to date have largely compared multiple measures of neurocognitive function within or across groups using inferential statistical methods to make probabilistic statements about the compatibility between observed data and posited models (Madden, Petry, Badger, & Bickel, 1997). Historically, many of these techniques were devised to analyze agricultural data in the early 19th century. An alternative rarely employed approach, which we also use in this report, arises from the machine learning tradition. Machine learning employs algorithms that both learn from and make predictions based on data (Kohavi & Provost, 1998). One form of machine learning is binary recursive partitioning, including classification tree methodologies (Lewis, 2000). This approach, when applied to a variety of measures of neurocognitive function obtained from drug dependent and community control groups, seeks to correctly classify the participants to the drug dependent or control group based solely on the neurocognitive measures. This algorithmic process first identifies the most discriminating measure that would serve as the first node of the tree, establishing two branches based on the optimal threshold of that measure. Each branch is similarly split with another node into two more branches based on the measures that best classifies individuals within each branch. The process continues iteratively in this fashion until pre-specified criteria are met. When the classification method is applied to a subset of the data to learn the tree structure, and the resulting tree is applied to the portion of data not included in the construction of the tree, then the out-of-sample classification rate can be assessed. Also, the algorithm quantifies the proportion each measure contributes to predictive accuracy.

A classification tree analysis should result in differing outcomes depending if the data are consistent with the two hypotheses discussed above. If drug dependence is the result of diffuse dysfunction with no differential importance for any neurocognitive measure, then three observations should emerge from the tree. First, the tree derived with one set of data would not be expected to be highly efficacious on subsequent application to data not included in the initial formation of the tree; that is, since no particular measure is of more centrality to the phenotype of addiction, then individual differences would predominate. Second, the proportion that each measure contributes to predictive accuracy should not dramatically favor any one measure and the contribution of the various measures to correct classification should not be dramatically different across measures. This is related to the point addressed above that any measure of neurocognitive function is a reasonable proxy of another. Third, application of the classification tree to different forms of drug dependence would not identify a single measure as most important across the different forms.

Conversely, if different measures of neurocognitive function represent divergent processes that are differentially important to the phenotype of drug dependence, then three outcomes should be observed with the application of the classification tree methodology. First, the tree derived with one set of data would be expected to be highly efficacious on subsequent application to data not included in the formation of the tree. Second, the proportion that each measure contributes to the predictive accuracy should favor one or more measures that are differentially important to the phenotype of drug dependence. Third, if one or more measures are not specific to only one form of drug dependence and are in fact trans-disease processes (Bickel, Jarmolowicz, Mueller, Koffarnus, & Gatchalian, 2012) related to the drug dependence phenotype, then application of the classification tree to different forms of drug dependence should result in the same one or two measures being selected as most important. Moreover, if this third outcome holds, then we would also expect the classification tree would not be as effective in distinguishing between different forms of drug dependence since they would share the centrality of the same measures for the phenotype.

In the present study, we obtained seven task-based measures of neurocognitive function (i.e., Continuous Performance, Delay Discounting, Iowa Gambling, Stroop, Tower, Wisconsin Card Sorting, and Letter Number Sequencing) from alcohol, stimulant, or dual alcohol-stimulant dependent individuals and community control participants. Alcohol and stimulant dependence have been shown to be associated with a wide variety of neurocognitive measures (Bickel, Jarmolowicz, Mueller, Gatchalian, et al., 2012; Verdejo-García, Bechara, Recknor, & Perez-Garcia, 2006; Verdejo-García, Rivas-Pérez, et al., 2006). We first analyze the groups using standard statistical methods to determine whether these samples replicate previous work by indicating greater neurocognitive dysfunction across the measures in the drug dependent groups. Next, we report the correlations between the measures to see if we also replicate the general finding that most of the measures are only minimally to moderately correlated. Finally, we apply the classification tree analysis to test which of the two alternative hypotheses better describes the data and to inform if one, or a few, neurocognitive measures successfully distinguishes between substance using individuals and controls, thus reducing the burden of measures needed when studying substance using groups of individuals.

Methods and Materials

All participants (n = 260) provided written consent that was approved by either the IRB at University of Arkansas or Virginia Tech. Of the 260 participants who enrolled in the study, 249 completed the seven neurocognitive function tasks. The 11 individuals with missing values for at least one of the tasks were removed from the analysis. A subset of the data being reported on in this manuscript was reported previously in Moody et al. (2016); however, the scope and computational approach used in this manuscript provides unique insights into the substance use phenotype. Participants were recruited from the community of either the greater Little Rock metropolitan area in Arkansas or the greater Roanoke Valley region of Virginia. They were recruited via community outreach including flyers, postings on social media outlets including Facebook and Craigslist, and word of mouth. To participate, all participants had to be at least 18 years of age and completed a demographics questionnaire. Control participants had to be free from any form of drug dependence, recent drug use, or mental health disorder as screened with the Mini-International Neuropsychiatric Interview (Sheehan et al., 1998). Substance using participants had to either meet Diagnostic and Statistical Manual (DSM-IV) (American Psychiatric Association, 1994) criteria for stimulant dependence (either cocaine or methamphetamine dependence), for alcohol dependence, or for both stimulant and alcohol dependence. Thus, participants were selected into one of four groups: control, stimulant dependent, alcohol dependent, or both stimulant and alcohol dependent. The current study was collected using DSM-IV criteria for substance dependence. The more recent DSM-V criteria merge what was previously substance abuse and substance dependence into substance use disorder with mild, moderate, and severe classifiers based on number of symptoms endorsed (American Psychiatric Association, 2013). Individuals that previously met diagnostic criteria for substance dependence would meet current DSM-V criteria for substance use disorder. Participants completed the consent and assessment sessions on separate days within one week of each other and were compensated $10 for consent and $20–$30 for completion of assessments based on amount of time spent in the laboratory with a rate of compensation of $10 per hour spent in the research center.

Measures

The neurocognitive tests were selected to represent diverse measures related to the cortico-striatal and medio-temporal neural substrates associated with drug dependence (Ersche et al., 2012). An assessment battery was administered with the following neurocognitive function measures:

Continuous Performance

The Continuous Performance task is a computerized measure of attention and impulsivity (Conner, 1995) and has relatively high correlations with self-report measures of impulsivity compared to other neurocognitive function measures (Duckworth & Kern, 2011). During the Continuous Performance task, participants were required to press the appropriate key in response to any presented letter except for a target letter. Six blocks were presented with varying inter-stimulus intervals. Commission errors were recorded as the primary outcome measure. A commission error is a response when no target is presented. Elevated commission errors are indicative of high impulsivity.

Delay Discounting

A computer-based delay discounting task was used to assess participants’ impulsive choice compared to self-controlled behavioral strategies. Participants were presented with hypothetical scenarios in which the extent of their discounting of a delayed “reward” (i.e., $1000) relative to an immediate “reward” was determined at seven delays (i.e., 1 day, 1 week, 1 month, 6 months, 1 year, 5 years, and 25 years) presented in chronological order. The smaller sooner amount titrates until an indifference point where the subjective value of the immediate reward is approximately equal to the value of the delayed reward (Du, Green, & Myerson, 2002). The indifference points are then fit to the following equation:

V=A/(1+kD) (Eq. 1)

where V is the subjective value of the objective monetary amount A, to be delivered after some delay, D (Mazur, 1987). The parameter k describes the hyperbolic function and is used as an index of the extent to which participants discount the value of future rewards. Taken together, higher k values indicate a tendency to devalue future rewards at a higher rate and this, in turn, suggests greater impulsivity.

Iowa Gambling

The Iowa Gambling task is a computer-based test that measures decision-making, flexibility and sensitivity to punishment (Bechara, 2007). Four identical decks were presented that, when chosen, produce gains or loses in hypothetical money. Decks differ in the magnitude of loss and gain and the net gains were used as the outcome measure. Previous work has shown that stimulant users perform worse than controls on this same outcome measure (Verdejo-Garcia et al., 2007). Letter Number Sequencing

The Letter Number Sequencing task (Wechsler, 2008) is a measure of attention and working memory. It requires participants to sequence a combination of letters and numbers in a prescribed order. Participants are given a string of items, presented as a mixed combination of letters and numbers that they must then put in sequential order of numbers followed by alphabetical order of letters. The string presentation increases in difficulty until the participant is no longer able to correctly sequence two strings of equivalent difficulty. The longest string length completed before discontinuation due to failure to correctly respond to three strings of the same length was used as the outcome measure of Letter Number Sequencing performance.

Stroop

The Stroop task (Golden & Freshwater, 1978) measures response inhibition in addition to complex concentration, selective processing, and attention. The Color-Word Interference Test from Delis-Kaplan Executive Function System was used for this measure (Delis, Kaplan, & Kramer, 2001a). The task consists of three components. First, participants read aloud a series of color names printed in black (e.g., the color red printed in black ink). Second, participants are asked to say the color of the ink in a series of colors names printed in the same color ink (e.g., the color red printed in red ink). Third, the participants are asked to say the color of the ink in a series of color names printed in incongruent ink colors (e.g., the word red printed in blue ink). This third subtest required participants to inhibit their tendency to read the word and instead read the color of the ink. Completion time for the third subset, a measure of inhibition, was used as the primary outcome measure. Previous work has indicated poorer inhibition on the Stroop task compared to healthy controls (Albein-Urios, Martinez-González, Lozano, Clark, & Verdejo-García, 2012).

Tower

The Tower task (Delis, Kaplan, & Kramer, 2001b) assesses several domains including planning, impulse inhibition, rule learning, perseveration, and set maintenance. For each trial, the examiner arranges disks on the pegs in a predetermined starting position and presents a picture that illustrates a predetermined ending position. Number of moves to completion, item-completion time, and correctness of final tower were recorded. Once the score for all items were collected, the sum of scores was calculated to create the overall achievement score that summarizes the participants overall performance on the Tower task.

Wisconsin Card Sorting

The Wisconsin Card Sorting task (Heaton & Others, 1993) is largely an assessment of abstraction ability and the ability to shift and maintain cognitive strategies for categorization. Participants were given 128 computerized stimulus cards individually and were required to match each card to one of four key cards that varied on three perceptual dimensions (i.e., color, form, and number). At any given time, the matching rule was to match on one of these dimensions. After each match, the computer indicated whether they were correct. After ten correct matches, the matching rule changes without warning and the participant must learn the new matching rule for the next set. The test requires planning, organized searching, use of environmental feedback to inform decisions, and restriction of impulsive responding. For the below analysis, the total number of perseverative errors was used as a measure to encompass neurocognitive functions of cognitive flexibility and attention. Perseverative errors during the Wisconsin Card Sorting task are associated with several neurocognitive impairments and brain damage (Demakis, 2003).

Data Analysis

Participants’ demographics measures (age, gender, marital status, race, years of education, and income) were summarized. The suitability of individual measures was assessed for parametric analysis by examining normality of residuals for the models used and homogeneity of variance. Pearson’s correlation coefficient was used to quantify the linear dependency among the seven measures among the control subjects and separately for substance using participants. The association between dependency group and each individual neurocognitive function measure was assessed using one-way ANOVA. F-tests were performed to compare all four groups’ outcomes for each of the seven neurocognitive function tasks. The effect size R2 was used to measure strength of association between each measure and substance use group. R2 directly relates to a commonly used effect size index, f2 (f2 = R2/1-R2). Given this relationship, common conventions for small, medium, and large R2 are 0.02, 0.13, and 0.26, respectively (Cohen, 1992). These associations are also represented graphically to facilitate interpretability.

Since substance dependence may be partially attributable to a constellation of neurocognitive deficits, methods are needed to determine which deficits in which neurocognitive function best classify individuals into substance dependence groups. To address this need, we used classification tree methodology in combination with k-fold cross validation (Nordhausen, 2009) to use the set of available neurocognitive function measures to best classify dependence status. We did not include demographic variables as part of the classification tree; as our interest here was to understand the classification capabilities of select neurocognitive function measures alone.

Classification trees

The primary goal of this study was to determine which combination of the seven neurocognitive function tasks best classified participants into substance dependence groups to inform future work on what neurocognitive measure may be central and necessary in the characterization of substance dependent groups and controls. The analyses were conducted using the tree package in R (R Core Team, 2014). Classification trees are formed using binary recursive partitioning. All participants begin in a single group, or “parent node.” Then, the algorithm screens all variables under consideration (neurocognitive measures in this study), and identifies the value of a single variable which splits participants into two “child nodes” so as to maximize similarity of dependency profiles within each node. For example, a split would be chosen so as to best group substance users in one node and controls in the other. Then, each of the child nodes becomes a parent node, and is split again based on value of a potentially different measure until a stopping criterion is reached (Lewis, 2000). In this study, the minimum sizes for parent and child nodes are 10 and 5, respectively in order to continue the recursive partitioning. These are the default settings of the tree package. See Figure 1 for a conceptual schematic of the approach. Classification trees do not make distributional assumptions about the independent variables and can handle variables with distributions that are skewed or multimodal. Tree 1 (Figure 3) divides the sample into controls and substance users and Tree 2 (Figure 4) divides the sample into four groups based on specific usage (alcohol, stimulant, alcohol and stimulant, and control). The length of each branch is proportional to the contribution that decision makes to predictive accuracy. This approach first offers a molar “control versus dependence” analysis, and then provides a second classification scheme that illustrates both the potentials and difficulties of further classifying specific subgroups. Further, trees (not shown) that compared (1) controls to alcohol-only users, (2) controls to stimulant-only users, (3) controls to users of stimulants or joint stimulants- and alcohol users but not alcohol-only users, and (4) controls to joint users of alcohol and stimulants but not single users of either were examined to determine which single measure provides the most predictive accuracy to separate controls from substance users in a variety of scenarios. Finally, (5) a tree that compares alcohol-only, stimulants-only, or both (but no controls) was constructed to determine which single measure best differentiates specific substance use groups.

Figure 1.

Figure 1

Open in a new tab

Classification tree schematic. Data begin at the top node and move through the branches and other nodes according to the threshold rules until a terminal node is reached. Once data reach a terminal node they are classified as the most prevalent group in that node and successful classification rate is computed.

Figure 3.

Figure 3

Open in a new tab

Classification tree using the seven neurocognitive function measures to classify between substance users and controls.

Figure 4.

Figure 4

Open in a new tab

Classification tree using the seven neurocognitive function measures to classify between alcohol, stimulant, combined alcohol and stimulant and non-drug using controls.

Cross validation approach to assess predictive accuracy

Cross validation is a technique used to assess the predictive performance of a classifier for data that were not used to train the approach. In the overall tree, classification rates may be inflated because the classification rules used to classify “learned” from the data that were classified. By contrast, k-fold cross validation trains multiple trees, then assesses the performance on “holdout data” to determine whether the tree approach yields good predictive performance for new data. The k-fold cross-validation method has been used extensively in the machine learning field (Arlot & Celisse, 2010; Zhang, 1993) and is a standard procedure to defend against over-fitting of data. Conceptually, this moves the classification tree paradigm beyond a descriptive account of how well a classifier performs on available data towards the goal of describing that classifier’s performance on new data.

In this study, a five-fold cross validation approach was used to evaluate the performance of the classification tree approach for the two primary trees. To cross-validate, the original data set was randomly split into five subsets. For each of the five cross-validation exercises, one of the subsets was designated as a “holdout” dataset. The other four sections were combined and used as a “learning” dataset to create a new tree. This new tree was then used to predict the outcomes in the naive holdout dataset that was not used to train the classification tree, and classification performance was recorded. As stated above, this process was repeated five times, and in each case the performance of the new tree on the holdout set was reported. Misclassification rate on the holdout dataset was used as our metric for performance in both the overall trees and also in the k-fold trees.

Results

Of the 249 complete cases, 67 (27%) were alcohol dependent, 23 (9%) were stimulant dependent, 100 (40%) were alcohol and stimulant dependent, and 59 (24%) were controls. There were 190 (76%) males and 59 (24%) females. Demographic characteristics by group are shown in Table 1. Substance using groups were older (F(3,245) = 13.94, p < 0.001) and had lower income (X2 = 20.88, p < 0.001), though education was largely commensurate across groups (F(3,244) = 2.68, p = 0.05).

Table 1.

Demographic variables by group status.

Control Alcohol Stimulant Alcohol/Stimulant
Average Age 35.25 46.4 47.7 43.17
Average Years of Education 12.83 13.01 12.93 12.32
Median Monthly Income 836.66 200 200 200
Open in a new tab

Statistical Associations

As shown in Table 2, significant correlations (at the α = 0.05 level) were observed in controls between the Continuous Performance task and Wisconsin Card Sorting task (p < 0.001), the Stroop task and both the Letter Number Sequencing task (p < 0.001) and the Wisconsin Card Sorting task (p = 0.027), the Iowa Gambling task and the Tower task (p = 0.023), and the Letter Number Sequencing task with both the Tower task (p = 0.015) and the Wisconsin Card Sorting task (p = 0.001). Significant correlations (at the α = 0.05 level) were observed in substance using participants between the Continuous Performance task and Wisconsin Card Sorting task (p < 0.001), the Stroop task and both the Letter Number Sequencing task (p < 0.001) and the Tower task (p = 0.001), and the Letter Number Sequencing task with the Tower task (p < 0.001). The R2 values associating substance dependence with each neurocognitive function measure ranged from 0.01–0.10, which correspond with small to medium effect sizes. Statistically significant group differences were observed in the Delay Discounting task, Iowa Gambling task, Letter Number Sequencing task, and Wisconsin Card Sort task (see Table 3). Figure 2 depicts bar graphs and standard errors for each neurocognitive function task so as to visually display the relative performance across groups for each substance use group (e.g., alcohol, stimulant and combined alcohol and stimulant use) in addition to control group performance.

Table 2.

Correlations among neurocognitive functions measures. Controls are in the top pane (n=59) and substance users are in the bottom pane (n=189). Top entry in each cell is Pearson’s correlation coefficient (r). Bottom entry is a p-value for the two tailed test against H0: ρ =0. Observed p-values below α = 0.05 are boldfaced. Note that for n = 59 one has 80% power to detect correlations of ρ = 0.35 in magnitude, and with n = 189 one has 80% power to detect correlations of ρ = 0.20 in magnitude, i.e. smaller effects can be detected among substance users in this analysis.

1 2 3 4 5 6 7
1. Continuous Performance 0.17 0.13 −0.04 −0.2 −0.12 0.57
0.1928 0.3424 0.7881 0.1283 0.3718 <.0001
2. Stroop 0.07 −0.1 −0.52 −0.21 0.29
0.6151 0.455 <.0001 0.103 0.0273
3. Delay Discounting −0.03 −0.15 0.03 0.20
0.7975 0.2662 0.7966 0.1275
4. Gambling 0.24 0.30 −0.14
0.0633 0.0231 0.2759
5. Letter Number Sequencing 0.32 −0.41
0.0149 0.0011
6. Tower −0.14
0.2738
7. Card Sort
1 2 3 4 5 6 7
1. Continuous Performance 0.08 −0.07 0.05 0.00 −0.08 0.31
0.2607 0.3442 0.4804 0.9674 0.2727 <.0001
2. Stroop 0.05 −0.02 −0.31 −0.23 0.13
0.4593 0.7537 <.0001 0.0012 0.0668
3. Delay Discounting −0.04 −0.14 0.00 0.06
0.5844 0.0605 0.9603 0.4487
4. Gambling 0.11 0.01 −0.04
0.1181 0.8694 0.6054
5. Letter Number Sequencing 0.35 −0.10
<.0001 0.1708
6. Tower −0.12
0.0928
7. Card Sort
Open in a new tab

Table 3.

Association between task and substance use group. P-values reflect the overall F-test comparing all four substance use groups.

Measure R2 Prob > F
Delay Discounting 0.1 < 0.0001**
Gambling 0.09 < 0.0001**
Letter Number Sequencing 0.09 < 0.0001**
Card Sort 0.03 0.011*
Stroop 0.03 0.0639
Tower 0.02 0.1226
Continuous Performance 0.01 0.572
Open in a new tab
*

indicates significance at the 0.05 level, while

**

indicates significance at the 0.01 level.

Figure 2.

Figure 2

Open in a new tab

Bar graphs comparing substance use groups for each of the neurocognitive function tasks. Brackets indicate significant Tukey adjusted differences between group means.

Classification results

The overall classification rate for Tree 1 (which classified individuals as “substance dependent” or “control”) was 88.3%. This means that the tree incorrectly classified individuals as substance users or controls in 11.7% of cases. Classification rates for the five holdout cross validation trees ranged from 87.4%–89.9% within the training data sets, and 70.0%–90.0% in the holdout sets. Each tree (overall and five cross-validation trees) split on the Delay Discounting task first, with ln(k), the outcome variable from the Delay Discounting task, ranging from −8.95 to −6.74 on the first split. Consistent with the hypothesis that neurocognitive function measures represent divergent processes that are differentially important to the alcohol and stimulant dependence phenotype, delay discounting was the basis of the first split and delay discounting alone correctly classifies 79.52% of individuals as controls or substance users. Table 5 displays complete results for the six trees: one overall tree and five cross-validation trees. The full tree is displayed in Figure 3.

Table 5.

Cross-validation for Tree 2. Results of cross validation on Tree 2 (data were classified into four groups: stimulant, alcohol, stimulant and alcohol, and controls). Note every tree split on discounting first, and the threshold for ln(k) was very stable among all trees.

Tree 2 Training Set Misclassification Rate Holdout Set Misclassification Rate ln(k) Cutoff
Holdout 1 32.70% 52.00% −6.75
Holdout 2 33.20% 66.00% −6.95
Holdout 3 39.20% 56.00% −6.02
Holdout 4 34.00% 55.10% −6.95
Holdout 5 39.20% 46.00% −6.02
Average Holdout 35.70% 55.00% −6.54
Full Tree 36.10% −6.95
Open in a new tab

The overall classification rate for Tree 2 (which classified data into four groups based on specific substance use group of either alcohol, stimulant or both alcohol and stimulant dependence) was 63.9%. The misclassification rate for the five cross-validation trees ranged from 60.8% to 67.3% within training datasets and 34% to 54% on holdout datasets. The Delay Discounting task was the first split on all six trees, with the initial split criterion ranging from −6.95 to −6.02. This finding is consistent with the subcomponent hypothesis that trans-disease processes operate across addiction types and thus the classification tree should be less accurate when categorizing specific forms of drug dependence. Full results for Tree 2 are included in Table 5, and the tree created on the full dataset can be found in Figure 4. For the trees (1)–(4) that compare controls directly to various substance use profiles (not shown), discounting was the measure which split the tree first in each case. The tree that compared alcohol, stimulant, and joint use split first on the Tower task and achieved a classification rate of 72%.

As a note, when other variables are entered into the classification analysis, different results may be observed. To assess whether non-neurocognitive function measures affect the resulting classification results, we added age, education, and income alongside the neurocognitive measures. In these analyses (see Supplemental Materials Figure 1 and 2) age and income play important roles in classifying between individuals that use substances and those that do not. However, delay discounting remains as the neurocognitive measure that enters earliest and provides the greatest predictive contribution (visualized by branch length) among neurocognitive measures. Given that the purpose of the current study is to elucidate which neurocognitive measures, if any, are best at identify between individuals that use substances and those that do not, these trees indicate that delay discounting plays a major role compared to other neurocognitive measures even when demographic variables are used in the classification process. Moreover, classification trees with both demographic variables and neurocognitive measures are consistent with the classification trees of the neurocognitive measures alone in that the first split that includes a neurocognitive function measures continues to include the delay discounting measure.

Discussion

In this report we compared seven neurocognitive function measures in three drug dependent groups (i.e., alcohol, stimulant, alcohol-stimulant dependent) and community controls. We found that a subset of measures distinguished the drug-dependent from controls and that the measures were not strongly correlated. These observations generally support previously reported findings (de Wit, 2009; Verdejo-García, Rivas-Pérez, et al., 2006). Figure 2 and Table 3 quantify the extent to which various tasks excel at differentiating groups, with Delay Discounting, Iowa Gambling, and Letter Number Sequencing tasks providing the most evidence of an ability to discriminate between controls and substance users, but limited ability to differentiate among specific substance use disorders. Further, the machine learning classification tree analysis provides evidence of the hypothesis that different measures of neurocognitive deficits represent divergent processes that are differentially sensitive to drug dependence by demonstrating good predictive accuracy at the individual subject level. Specifically, a classification tree derived with one set of data was efficacious on subsequent application to data not included in the formation of the tree. We also found that delay discounting by itself was the best predictive neurocognitive measure across all trees when each drug dependent group was compared to controls suggesting that delay discounting may be differentially important to the phenotype of addiction. Additionally, neither the ANOVA nor the classification tree was able to efficaciously distinguish between different forms of drug dependence consistent with the notion of trans-disease processes where certain processes are important to the phenotype of more than one disorder. Also, important to note is that these analyses only considered neurocognitive measures and their ability to distinguish between alcohol, stimulant, combined alcohol/stimulant and community controls. Thus, the potential applications of this report are limited to identifying neurocognitive measures that differentiate between these groups and do not consider the impact of demographic variables that also differ between groups. There are four points that we would like to comment about this finding.

First, the importance of scientific tools to confirming or suggesting new theories have long been commented upon (Hanson, 1958; Kuhn, 1961); that is, the introduction of a novel device or tool has been associated with changes in scientific theory. Here we examined the same data seen through two different analytical tools, each of which revealed unique insights about the nature of neurocognitive deficits in relation to addiction. In this first case, we applied statistical approaches consistent with those widely employed in this field. The results of those analyses are generally consistent with prior neuroscientific observations conducted to date suggesting that drug dependent individuals exhibited dysfunction on several measures that were not strongly correlated with each other. Next we applied the machine learning classification tree technique. This approach quantifies the predictive capacity of neurocognitive measures to classify individuals at the participant level. This view is clarifying because it (a) supplements insights gained by the more common approach of differentiating groups on average (see e.g., Stavro et al., 2013), and (b) allows for performance-based selection of measures that classify individuals into usage groups. Further, different outcomes would be expected if the diffuse dysfunction or the differentiated subsystems hypotheses were operative, permitting greater resolution. The machine learning approach supported the differentiated subsystems hypothesis. However, only replication and extension of the classification trees to diverse areas of addiction science will determine whether this differentiated subsystem hypothesis is broadly supported.

Second, the classification tree analysis conducted here identified delay discounting as the first split of each tree for every condition that compared substance users and controls. Specifically, the first split based on discounting in Tree 1 correctly categorized 79.5% of the sample with the remaining measures only adding less than 10% correct classification among all remaining splits using all available measures. This finding is consistent with the hypothesis that excessive delay discounting is a subsystem that is important to the phenotype of alcohol, stimulant and dual alcohol/stimulant dependence (Moody et al., 2016). These findings fit with recent suggestions that delay discounting is a behavioral marker of addiction (Bickel, Koffarnus, Moody, & Wilson, 2014). Indeed, delay discounting in a small number of longitudinal studies is predictive of later drug use, excessive delay discounting is observed in nearly every form of addiction, the extent of delay discounting is proportional to amount of the drug used, delay discounting at baseline is predictive of treatment outcomes for moderately effective treatments, and delay discounting improves following a multimodal highly efficacious treatment. The data provided here support and contribute to further delay discounting as a behavioral marker of addiction.

Third, the classification tree analysis identified the same neurocognitive function measure at the first split and as the predominate classifier relative to other neurocognitive function measures across two forms of drug dependence and a poly-dependent combination of the two supporting the contention that delay discounting functions as a trans-disease process. While measures of neurocognitive deficits were included in a recent meta-analysis comparing short, intermediate, and long term alcohol users with controls, these measures did not include delay discounting (Stavro et al., 2013). Stavro et al. (2013) concluded that all assessed measures of neurocognitive function were similarly impaired indicating diffuse deficits, while the present report suggests delay discounting is among the most closely aligned with the phenotypes of addiction explored here.

Fourth, the weaknesses of this study include the tasks employed and the sample employed. The outcomes are dependent upon the tasks utilized. If other tasks were included, a different profile of results may have been obtained. However, what would be useful in subsequent extensions of these findings would be to include delay discounting along with other tasks to see if delay discounting would function similarly to the classification trees reported here. Additionally, this study examined alcohol, stimulant and dual alcohol-stimulant dependent individuals relative to controls. Importantly, some participants in this study were cigarette smokers and the influence of this substance use is not taken into account in the current analyses. Whether similar results would be obtained with other drug dependencies, such as nicotine dependence, remains to be determined and is beyond the scope of the current study. Finally, demographic variables are not included in these analyses despite differences in demographic variables between groups. As the current study aimed to identify neurocognitive measure that contribute to the phenotypes of substance dependence, the contributions of demographic variables to the analyses were not included here; however, these characteristics are likely integrally related to substance use.

Conclusion

This study is the first to utilize both statistical and machine learning approaches to evaluate several neurocognitive function measures in alcohol, stimulant, and alcohol-stimulant dependence relative to a control group. These analyses concurred that one measure in particular, Delay Discounting, is important to the phenotype of alcohol, stimulant and dual alcohol/stimulant dependence. The combined classification and cross validation approach quantifies classification rates at the individual level which is a potentially useful metric to consider alongside traditional statistical comparisons that differentiate group means. These findings are consistent with recent findings supporting delay discounting as a behavioral marker (Bickel et al., 2014) and endophenotype (Bickel, 2015; MacKillop, 2013).

Supplementary Material

1

Table 4.

Cross-validation for Tree 1. Results of cross validation on Tree 1 (Figure 3) where were classified into two groups: substance users and controls. In all instances, delay discounting was the first split and contributed the most to successful classification. The threshold for ln(k) was relatively stable among all trees.

Tree 1 Training Set Misclassification Rate Holdout Set Misclassification Rate ln(k) Cutoff
Holdout 1 12.60% 30.00% −6.74
Holdout 2 11.50% 24.50% −6.99
Holdout 3 10.10% 28.00% −8.95
Holdout 4 11.60% 20.00% −6.95
Holdout 5 10.10% 10.00% −8.95
Average Holdout 11.20% 22.50% −7.716
Full Tree 11.70% -- −6.95
Open in a new tab

Public Significance Statement.

Neurocognitive deficits are a core component of addiction. Understanding if neurocognitive deficits are diffuse or represent separate facets of differential importance to the addiction phenotype may inform diagnosis and treatment. Using a novel exploratory classification-based approach in conjunction with traditional confirmatory statistics, we report distinct, as opposed to diffuse, deficits in substance users.

Acknowledgments

We would like to acknowledge the participants that volunteered time to the completion of this study and the Addiction Recovery Research Center staff without whom this project would not have been successful. We would like to extend a special acknowledgement to Brian Brown for his help with graphics.

Footnotes

Funding and Disclosures

This project was funded by the National Institute of Drug Abuse (R01DA024080). LNM’s time was funded by the National Institute of Alcoholism and Alcohol Abuse (F31AA024368).

LNM, CRE, and CTF do not have any conflicts of interest to disclose. WKB discloses HealthSim LLC and NotifiUS LLC as organizations that he has interest as a principal.

References

  1. Albein-Urios N, Martinez-González JM, Lozano O, Clark L, Verdejo-García A. Comparison of impulsivity and working memory in cocaine addiction and pathological gambling: Implications for cocaine-induced neurotoxicity. Drug and Alcohol Dependence. 2012;126(1–2):1–6. doi: 10.1016/j.drugalcdep.2012.03.008. [DOI] [PubMed] [Google Scholar]
  2. American Psychiatric Association. Diagnostic and statistical manual of mental disorders. American Psychiatric Association; Washington, DC: 1994. p. 210. [Google Scholar]
  3. American Psychiatric Association. Diagnostic and statistical manual of mental disorders: DSM-5. Arlington: American Psychiatric Publishing; 2013. Retrieved from http://www.lswo2014.org/StanMcCrackenDSMV.pdf. [Google Scholar]
  4. Arlot S, Celisse A. A survey of cross-validation procedures for model selection. Statistics Surveys. 2010;4:40–79. [Google Scholar]
  5. Bates ME, Bowden SC, Barry D. Neurocognitive impairment associated with alcohol use disorders: implications for treatment. Experimental and Clinical Psychopharmacology. 2002;10(3):193–212. doi: 10.1037//1064-1297.10.3.193. [DOI] [PubMed] [Google Scholar]
  6. Bechara A. Decision making, impulse control and loss of willpower to resist drugs: a neurocognitive perspective. Nature Neuroscience. 2005;8(11):1458–1463. doi: 10.1038/nn1584. [DOI] [PubMed] [Google Scholar]
  7. Bechara A. Psychological Assessment Resources. 2007. Iowa gambling task. [Google Scholar]
  8. Bickel WK. Discounting of delayed rewards as an endophenotype. Biological Psychiatry. 2015;77(10):846–847. doi: 10.1016/j.biopsych.2015.03.003. [DOI] [PubMed] [Google Scholar]
  9. Bickel WK, Jarmolowicz DP, Mueller ET, Gatchalian KM, McClure SM. Are executive function and impulsivity antipodes? A conceptual reconstruction with special reference to addiction. Psychopharmacology. 2012;221(3):361–387. doi: 10.1007/s00213-012-2689-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Bickel WK, Jarmolowicz DP, Mueller ET, Koffarnus MN, Gatchalian KM. Excessive discounting of delayed reinforcers as a trans-disease process contributing to addiction and other disease-related vulnerabilities: emerging evidence. Pharmacology & Therapeutics. 2012;134(3):287–297. doi: 10.1016/j.pharmthera.2012.02.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Bickel WK, Koffarnus MN, Moody L, Wilson AG. The behavioral- and neuro-economic process of temporal discounting: A candidate behavioral marker of addiction. Neuropharmacology. 2014;76(Pt B):518–527. doi: 10.1016/j.neuropharm.2013.06.013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Bickel WK, Yi R, Landes RD, Hill PF, Baxter C. Remember the future: working memory training decreases delay discounting among stimulant addicts. Biological Psychiatry. 2011;69(3):260–265. doi: 10.1016/j.biopsych.2010.08.017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Cohen J. A power primer. Psychological Bulletin. 1992;112(1):155–159. doi: 10.1037//0033-2909.112.1.155. [DOI] [PubMed] [Google Scholar]
  14. Conner CK. Conner’s continuous performance test. North Tonawanda (NY): Multi-Health Systems; 1995. [Google Scholar]
  15. Delis DC, Kaplan E, Kramer JH. Delis-Kaplan executive function system (D-KEFS) Psychological Corporation; 2001a. [Google Scholar]
  16. Delis DC, Kaplan E, Kramer JH. Delis-Kaplan executive function system: Technical manual. Psychological Corporation; 2001b. [Google Scholar]
  17. Demakis GJ. A meta-analytic review of the sensitivity of the Wisconsin Card Sorting Test to frontal and lateralized frontal brain damage. Neuropsychology. 2003;17(2):255–264. doi: 10.1037/0894-4105.17.2.255. [DOI] [PubMed] [Google Scholar]
  18. de Wit H. Impulsivity as a determinant and consequence of drug use: a review of underlying processes. Addiction Biology. 2009;14(1):22–31. doi: 10.1111/j.1369-1600.2008.00129.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Duckworth AL, Kern ML. A Meta-Analysis of the Convergent Validity of Self-Control Measures. Journal of Research in Personality. 2011;45(3):259–268. doi: 10.1016/j.jrp.2011.02.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Du W, Green L, Myerson J. Cross-cultural comparisons of discounting delayed and probabilistic rewards. The Psychological Record. 2002;52(4):479–492. [Google Scholar]
  21. Ersche KD, Clark L, London M, Robbins TW, Sahakian BJ. Profile of executive and memory function associated with amphetamine and opiate dependence. Neuropsychopharmacology: Official Publication of the American College of Neuropsychopharmacology. 2006;31(5):1036–1047. doi: 10.1038/sj.npp.1300889. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Ersche KD, Turton AJ, Chamberlain SR, Müller U, Bullmore ET, Robbins TW. Cognitive dysfunction and anxious-impulsive personality traits are endophenotypes for drug dependence. The American Journal of Psychiatry. 2012;169(9):926–936. doi: 10.1176/appi.ajp.2012.11091421. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Golden CJ, Freshwater SM. Stroop color and word test. 1978 Retrieved from http://v-psyche.com/doc/Clinical%20Test/Stroop%20Color%20and%20Word%20Test.doc.
  24. Hanson NR. The Logic of Discovery. The Journal of Philosophy. 1958;55(25):1073–1089. [Google Scholar]
  25. Heaton RK, et al. Wisconsin card sorting test: computer version 2. Odessa: Psychological Assessment Resources; 1993. Retrieved from http://www.psykologiforlaget.se/Global/Exempelrapporter/WCSTCV4%20SAMPLE.pdf. [Google Scholar]
  26. Johnson MW, Bickel WK, Baker F, Moore BA, Badger GJ, Budney AJ. Delay discounting in current and former marijuana-dependent individuals. Experimental and Clinical Psychopharmacology. 2010;18(1):99–107. doi: 10.1037/a0018333. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Kohavi R, Provost F. Glossary of terms. Machine Learning. 1998;30(2–3):271–274. [Google Scholar]
  28. Kuhn TS. The Function of Measurement in Modern Physical Science. Isis; an International Review Devoted to the History of Science and Its Cultural Influences. 1961;52(2):161–193. [Google Scholar]
  29. Lewis RJ. An introduction to classification and regression tree (CART) analysis. Annual Meeting of the Society for Academic Emergency Medicine; San Francisco, California. 2000. pp. 1–14. [Google Scholar]
  30. MacKillop J. Integrating behavioral economics and behavioral genetics: delayed reward discounting as an endophenotype for addictive disorders. Journal of the Experimental Analysis of Behavior. 2013;99(1):14–31. doi: 10.1002/jeab.4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Madden GJ, Petry NM, Badger GJ, Bickel WK. Impulsive and self-control choices in opioid-dependent patients and non-drug-using control patients: Drug and monetary rewards. Experimental and Clinical Psychopharmacology. 1997;5(3):256. doi: 10.1037//1064-1297.5.3.256. [DOI] [PubMed] [Google Scholar]
  32. Mazur JE. An adjusting procedure for studying delayed reinforcement. In: Commons ML, Mazur JE, Nevin JA, Rachlin H, editors. The Effect of Delay and of Intervening Events on Reinforcement Value: Quantitative Analyses of Behavior. Vol. 5. Psychology Press; 1987. pp. 55–73. [Google Scholar]
  33. Moody L, Franck C, Hatz L, Bickel WK. Impulsivity and Polysubstance Use: A Systematic Comparison of Delay Discounting in Mono-, Dual-, and Trisubstance Use. Experimental and Clinical Psychopharmacology. 2016;24(1):30–37. doi: 10.1037/pha0000059. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Nordhausen K. The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Second Edition by Trevor Hastie, Robert Tibshirani, Jerome Friedman. International Statistical Review = Revue Internationale de Statistique. 2009;77(3):482–482. [Google Scholar]
  35. R Core Team. R: A language and environment for statistical computing. R Foundation for Statistical Computing; Vienna, Austria: 2014. 2013. [Google Scholar]
  36. Sheehan DV, Lecrubier Y, Sheehan KH, Amorim P, Janavs J, Weiller E, … Dunbar GC. The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. The Journal of Clinical Psychiatry. 1998;59(Suppl 20):22–33. quiz 34–57. [PubMed] [Google Scholar]
  37. Stavro K, Pelletier J, Potvin S. Widespread and sustained cognitive deficits in alcoholism: a meta-analysis. Addiction Biology. 2013;18(2):203–213. doi: 10.1111/j.1369-1600.2011.00418.x. [DOI] [PubMed] [Google Scholar]
  38. Verdejo-García A, Bechara A, Recknor EC, Perez-Garcia M. Executive dysfunction in substance dependent individuals during drug use and abstinence: an examination of the behavioral, cognitive and emotional correlates of addiction. Journal of the International Neuropsychological Society: JINS. 2006;12(03):405–415. doi: 10.1017/s1355617706060486. [DOI] [PubMed] [Google Scholar]
  39. Verdejo-Garcia A, Benbrook A, Funderburk F, David P, Cadet JL, Bolla KI. The differential relationship between cocaine use and marijuana use on decision-making performance over repeat testing with the Iowa Gambling Task. Drug and Alcohol Dependence. 2007;90(1):2–11. doi: 10.1016/j.drugalcdep.2007.02.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Verdejo-García A, Rivas-Pérez C, López-Torrecillas F, Pérez-García M. Differential impact of severity of drug use on frontal behavioral symptoms. Addictive Behaviors. 2006;31(8):1373–1382. doi: 10.1016/j.addbeh.2005.11.003. [DOI] [PubMed] [Google Scholar]
  41. Wechsler D. Wechsler adult intelligence scale-fourth. San Antonio: Pearson; 2008. [Google Scholar]
  42. Winstanley CA, Dalley JW, Theobald DEH, Robbins TW. Fractionating impulsivity: contrasting effects of central 5-HT depletion on different measures of impulsive behavior. Neuropsychopharmacology: Official Publication of the American College of Neuropsychopharmacology. 2004;29(7):1331–1343. doi: 10.1038/sj.npp.1300434. [DOI] [PubMed] [Google Scholar]
  43. Zhang P. Model Selection Via Multifold Cross Validation. Annals of Statistics. 1993;21(1):299–313. [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

1
NIHMS870878-supplement-1.docx (122.9KB, docx)

RESOURCES