Decision-Making as a Latent Construct and its Measurement Invariance in a Large Sample of Adolescent Cannabis Users

Abstract

Objective:

Relative to the vast literature that employs measures of decision-making (DM), rigorous examination of their psychometric properties is sparse. This study aims to determine whether three measures of DM assess the same construct, and to assess measurement invariance of this construct across relevant covariates.

Method:

Participants were 372 adolescents at risk for escalation in cannabis use. DM was assessed via four indices from the Cups Task, Game of Dice Task (GDT), and Iowa Gambling Task (IGT). We used confirmatory factor analysis to assess unidimensionality of the DM construct, and moderated nonlinear factor analysis (MNLFA) to examine its measurement invariance.

Results:

The unidimensional model of DM demonstrated good fit. MNLFA results revealed that sex influenced mean DM scores, such that boys had lower risk-taking. There was evidence of differential item functioning (DIF), such that IQ and age moderated the IGT intercept and GDT factor loading, respectively. Significant effects were retained in the final model, which produced participant-specific DM factor scores. These scores showed moderate stability over time.

Conclusions:

Indices from three DM tasks loaded significantly onto a single factor, suggesting that these DM tasks assess a single underlying construct. We suggest that this construct represents the ability to make optimal choices that maximize rewards in the presence of risk. Our final DM factor accounts for DIF caused by covariates, making it comparable across adolescents with different characteristics.

Introduction

Measures of decision-making (DM) have been commonly used with various populations. However, few studies have reported detailed psychometric properties of these tasks, including how they might relate to each other, particularly in adolescent samples. Some studies have found overlap in the constructs measured by these tasks (Buelow & Blaine, 2015; Monterosso, Ehrman, Napier, O’Brien, & Childress, 2001). Others suggest that between-task correlations may vary with population characteristics, or that different DM tasks may capture different aspects of this complex construct, (Brown et al., 2015; Buelow & Blaine, 2015). Neuroimaging studies support this conclusion, as performance of different DM tasks activates different brain networks (Labudda et al., 2008; Li, Lu, D’Argembeau, Ng, & Bechara, 2010; Xue et al., 2009). For example, DM during the Game of Dice Task has elicited activation of the dorsolateral prefrontal cortex, anterior cingulate cortex, and inferior parietal lobule (Labudda et al., 2008). DM during the Iowa Gambling Task, on the other hand, has been associated with increased activation in dorsolateral prefrontal cortex, insula, anterior and posterior cingulate cortex, orbitofrontal and ventromedial prefrontal cortex, and ventral striatum (Li et al., 2010). Findings from an fMRI study employing the Cups Task found that all risky choices elicited activation in the dorsomedial prefrontal cortex, whereas the gain and loss domains led to differential activation in the ventromedial prefrontal cortex and nucleus accumbens (Xue et al., 2009). Thus, although research suggests overlap in the frontal areas engaged during these tasks, there is also variability in involvement of other regions.

Studies have shown that a variety of participant characteristics or behaviors can impact DM performance. For instance, sex differences in DM are well-documented (Shulman, Harden, Chein, & Steinberg, 2015; van den Bos, Homberg, & de Visser, 2013). Participant age can also influence DM because the neural circuits underlying executive functions follow a pattern of protracted development during adolescence, causing younger participants to make riskier choices (Casey, Jones, & Hare, 2008). Additionally, cognitive factors such as intelligence have been shown (albeit inconsistently) to moderate DM task performance, (Brand, Laier, Pawlikowski, & Markowitsch, 2009; Toplak, Sorge, Benoit, West, & Stanovich, 2010). Finally, both substance abuse (Bechara & Damasio, 2002; Bechara, Dolan, & Hindes, 2002; Brand, Roth-Bauer, Driessen, & Markowitsch, 2008; Brevers et al., 2014), and use of specific substances such as cannabis (Churchwell, Lopez-Larson, & Yurgelun-Todd, 2010; Gonzalez, Schuster, Mermelstein, & Diviak, 2015), have been linked to poorer DM. It is therefore important to take relevant characteristics into account when assessing DM in a given population.

Because these tasks are increasingly used in both research and clinical settings, further work is needed to determine whether measures of DM assess the same underlying construct, even when measured among participants with different characteristics. The current study aims 1) to determine whether three commonly used DM tasks—the Cups Task, Game of Dice Task, and Iowa Gambling Task load onto the same factor by creating a latent construct of DM, and 2) to examine whether this latent construct and its individual components were commensurate across a set of theoretically-relevant covariates by using moderated non-linear factor analysis (MNLFA). MNLFA allows for simultaneous assessment of measurement invariance and differential item functioning (DIF) across variables reflecting individual differences. Because these three DM tasks involve maximizing monetary gains and minimizing losses by making choices at varying levels of risk and reward, and neuroimaging evidence suggests that similar networks are activated during performance, we hypothesized that the three DM tasks would load onto a single factor, and that the structure of this factor would not be influenced by covariates (e.g. sex, age, IQ, and cannabis use) in our adolescent sample.

Methods

Participants

Participants were 401 (54.1% Male) adolescents ages 14 to 17 (M = 15.40, SD = .72), recruited through Miami-Dade County middle and public schools, flyers posted throughout the community, and word-of-mouth. Participants were primarily of Hispanic/Latino ethnicity (89.8%). Self-reported race was: White (76.8%), Black/African American (7.7%), Mixed Race (12.0%), and Other (3.5%).

The sample consisted of participants from a longitudinal study examining associations between DM, memory, and trajectories of cannabis use (R01 DA031176, PI: Gonzalez). Eligibility for the parent study was ascertained through phone screens. Inclusion criteria were developed to recruit a sample consisting predominantly of adolescents at risk for escalation in cannabis use, i.e. adolescents who reported early experimentation with substances without reaching problematic levels of use at time of screening, and with little to no exposure to substances other than alcohol, cigarettes, or cannabis. The majority of the sample (90%) reported some use, even if minimal, of either alcohol, cigarettes, or other drugs at time of screening. Additional information on participant selection can be found in prior publications with this cohort (Duperrouzel et al., in press; Hawes, Trucco, Duperrouzel, Coxe, & Gonzalez, 2018; Lopez-Quintero et al., 2018; Ross, Graziano, Pacheco-Colón, Coxe, & Gonzalez, 2016). Participants were also between the ages of 14 and 17 at baseline, and able to read and write English. We excluded participants who reported developmental disorders, birth complications, neurological conditions, or a history of diagnosed or significant mood or thought disorders (excluding ADHD), and those who reported frequent or recent use of drugs other than alcohol, nicotine, or cannabis, or whose answers at the time of screening suggested the presence of an alcohol or cannabis use disorder. Participants underwent oral fluid toxicology screening to test for recent drug use. We excluded 7 participants who tested positive for any drug, 5 who met criteria for past dependence on substances other than cannabis, and 15 participants who reported using a drug (besides alcohol, nicotine, cannabis, and hallucinogens) within 14 days of the assessment. Participant and substance use characteristics (displayed in Table 1) and analyses were based on the remaining 372 participants.

Table 1.

Participant demographics at the baseline assessment (M = mean, SD = standard deviation, Md = median, IQR = interquartile range).

Demographics (N = 372)	M ± SD or %
Age	15.39 ± .72
% Male	54.0%
% Hispanic	89.2%
WRAT-4 Reading Score	108.19 ± 14.75
Substance Use Characteristics	Md [IQR]
Lifetime Days of Use
Alcohol	4.50 [1.00, 17.00]
Nicotine	.00 [.00, 1.00]
Cannabis	19.5 [1.00, 120.00]
Past 30-Day Days of Use
Alcohol	.00 [.00, 1.00]
Nicotine	.00 [.00, .00]
Cannabis	1.00 [.00, 5.00]

Procedures

All procedures were approved by the Florida International University IRB. We obtained parental consent and participant assent for all participants. The parent study involves five assessment waves conducted at 6-month intervals over a 2-year period, each of which involves a detailed assessment protocol. Participants received monetary compensation for their time in the study, earning $75 for completing the baseline assessment. The main analyses of the current study focus on data acquired during the baseline assessment. Data from the one-year follow-up assessment were used for a subset of analyses.

Measures

Decision-making.

We assessed DM through three computerized tasks: the Cups Task, Game of Dice Task, and Iowa Gambling Task.

Cups Task.

The Cups Task was designed to assess DM in children and adolescents (Levin, Hart, Weller, & Harshman, 2007). This task measures DM under conditions of specified risk in both gain and loss domains (Levin & Hart, 2003). Performance on this task has been linked to performance in other DM tasks, such as the Iowa Gambling Task (Weller, Levin, & Bechara, 2010). Participants are shown a visual display of 2, 3, or 5 cups on both sides of the screen, and are asked to choose a cup from either side for a total of 54 trials. Choices from one side always yield a reward (e.g. definite gain of one quarter) or loss (e.g. definite loss of one quarter), whereas choices from the other side provide the chance for a greater reward (e.g. chance to gain multiple quarters) or loss (e.g. chance to lose multiple quarters). The latter were considered risky choices. We used the total number of risky choices in the gain domain, and total number of risky choices in the loss domain as our indices of DM for this task.

Game of Dice Task.

The Game of Dice Task (GDT) was designed to assess the influence of executive functioning (e.g. performance monitoring, strategizing) on DM under uncertain or risky conditions, in which participants are explicitly provided with rules and probabilities for gains and losses (Brand et al., 2005). This task has been successfully used with adolescents (Drechsler, Rizzo, & Steinhausen, 2008; Ross et al., 2016). Performance on this task has also been correlated to performance on the IGT (Brand, Recknor, Grabenhorst, & Bechara, 2007). Participants are instructed to win as much money as possible within 18 throws of a die. Before each trial, participants choose a single number, or a combination of two, three, or four numbers. Each choice is associated with specific gains and losses depending on the probability of the occurrence of the participant’s choice. In other words, choices with more numbers have a higher probability of occurring, but are associated with a lesser reward than choices with fewer numbers, which have a lower probability of occurring. We used the total number of risky choices; that is, the number of times the participant chose the options with the lowest probability of occurring (i.e., one or two numbers), as the index of DM for this task.

Iowa Gambling Task.

The Iowa Gambling Task (IGT) assesses DM under conditions of ambiguous risk. This task was developed to capture the DM impairments seen in patients with damage to the ventromedial prefrontal cortex (Bechara, Damasio, Damasio, & Anderson, 1994). Although originally designed for use with adults, the IGT has been successfully used with adolescents (Hooper, Luciana, Conklin, & Yarger, 2004; Ross et al., 2016). In this task, participants are shown a visual display of four decks of cards, and are told that the goal is to win as much money as possible. They are instructed that every time they choose a card they will win money, but sometimes also lose money, and that some decks are worse than others. More choices from good decks lead to a positive total score, whereas more choices from bad decks yield a negative total score at the end of 100 trials. As per the IGT Professional Manual, we used the reverse-scored IGT Net Score, i.e. choices from good decks (Decks C and D) minus choices from bad decks (Decks A and B), as the index of DM for this task (Bechara, 2007).

Substance use.

The Drug Use History Questionnaire is a detailed semi structured interview used to assess frequency and amount of use of 15 different drug classes over a participant’s lifetime (Rippeth et al., 2004). We used frequency (i.e., number of days) of cannabis use in the past 30 days as a covariate in our analyses.

Estimated IQ.

We used the Word Reading subtest of the Wide Range Achievement Test −4^th Edition (Wilkinson & Robertson, 2006) to estimate participants’ IQs, which we used as a covariates in our analyses.

Statistical Analyses

Covariate selection.

We selected a set of covariates based on theoretical relevance. Following the rationale outlined in the Introduction, we included age, sex, estimated IQ, and recent cannabis use as covariates in our analyses.

Correlations.

To better characterize correlations between DM tasks among adolescents, we examined bivariate correlations between all indices of DM used in the current study.

Latent DM construct.

All analyses were conducted using Mplus 8 (Muthén & Muthén, 2012). We conducted a confirmatory factor analysis (CFA), where four DM indices derived from three DM tasks were combined into a single-factor model. Specifically, we used the reverse-scored IGT Net Total, the number of risky choices in the GDT, and the total number of risky choices in the gain and loss domains from the Cups Task. Higher scores in this construct reflect poorer DM. This model used maximum likelihood estimation with standard errors and a chi-square statistic that are robust to non-normality (MLR). Model fit was assessed using the Comparative Fit Index (CFI), Root Mean Square Error of Approximation (RMSEA), and sample-size adjusted Bayesian Information Criterion (BIC). To handle missing data, we used full-information maximum likelihood (FIML) estimates.

Moderated nonlinear factor analysis.

After establishing unidimensionality of the DM construct, we used MNLFA (Bauer, 2017) to examine potential effects of study covariates on parameters of the latent DM model. Specifically, we tested for differences in the DM construct mean and variance, as well as differential item functioning (DIF) of item intercepts and factor loadings as a function of participants’ age, sex, estimated IQ, and past 30-day cannabis use frequency. DIF was examined on an item-by-item basis, accounting for covariate effects on the DM construct mean and variance. We then established a final MNLFA model by retaining all significant covariate effects on the DM factor (mean and variance) and items (thresholds and loadings). We used parameter estimates from this final model to produce maximum a posteriori (MAP) scores on the DM construct, which account for differences in the DM factor mean and variance and item DIF resulting from participants’ sex, age, IQ, and/or cannabis use. Unlike traditional summed score approaches, MAP scores provide information about individual differences by providing participant-specific factor scores for the DM construct (Curran et al., 2014)

Construct stability.

To determine whether the DM construct was stable over time, we recreated our final MNLFA model using data collected at the one-year follow-up assessment of the parent study. We calculated the correlation between participants’ MAP scores on the DM factor at baseline with their MAP scores on the DM factor at the one-year follow-up assessment.

Results

Table 2 shows bivariate Pearson’s correlations between all indices of DM used in the current study among adolescents. The number of risky choices in the GDT was significantly correlated with risky choices in the Cups Task for both the risk and loss domains. Additionally, the reverse-scored IGT Net Total was significantly correlated with the number of risky choices in the loss domain in the Cups Task. Correlations between GDT and IGT indices were not significant.

Table 2.

Bivariate Pearson’s correlations between DM indices from the Cups Task, GDT, and IGT (N = 372).

DM Index	1.	2.	3.	4.
1. Cups Task Risk Gain Domain	-
2. Cups Task Risk Loss Domain	47**	-
3. GDT Risk	.15**	.21**	-
4. IGT Net Total (reverse-scored)	.09	.16**	−.05	-

(DM = decision-making; GDT = Game of Dice Task; IGT = Iowa Gambling Task;

^**indicates significance at p < .005).

The unidimensional CFA model of DM demonstrated good fit (CFI = .989; RMSEA = .038, BIC = 10383.72). All indices loaded significantly onto the DM factor. Table 3 shows detailed MNLFA results. Participant sex significantly influenced mean scores of the DM construct (p = .007), suggesting that female cannabis users demonstrated higher risk-taking than males. There was some evidence of DIF across DM indices. Specifically, participant IQ significantly moderated the IGT index intercept (p = .027), whereas participant age had a marginally significant moderating influence on the GDT index factor loading (p = .048). Although these effects are marginally significant and would attenuate to nonsignificance upon correcting for multiple comparisons, we chose to retain these effects in the final MNLFA model to more conservatively correct for any potential DIF.

Table 3.

Moderated non-linear factor analysis (MNLFA) results and covariate effects. Values represent unstandardized factor loadings and standard errors.

		Covariate Effect
Reference Parameter	Baseline	Age	Sex	IQ	Past 30-day Cannabis Use
DM
Mean	.00a	−.01 (.05)	.20** (.07)	−.00 (.00)	.00 (.01)
Variance	1.00a	−.03 (.09)	−.18 (.14)	−.00 (.00)	−.00 (.01)
Item 1. Cups Task risk gain domain
Intercept/Threshold	16.88** (.29)	.16 (.48)	−.50 (.59)	.02 (.02)	.06 (.03)
Loading	3.03** (.59)	.93 (.64)	.81 (.86)	.02 (.03)	−.02 (.05)
Item 2. Cups Task risk loss domain
Intercept/Threshold	17.10** (.34)	.08 (1.74)	1.31 (1.44)	−.01 (.03)	−.04 (.07)
Loading	5.72** (.99)	.95 (1.00)	.28 (.82)	.03 (.02)	−.18 (.14)
Item 3. GDT risk
Intercept/Threshold	7.72** (.27)	.12 (.53)	.32 (.52)	−.01 (.02)	.02 (.04)
Loading	1.22** (.33)	1.21* (.61)	.03 (.80)	.04 (.03)	−.04 (.05)
Item 4. IGT Net Total (reverse-scored)
Intercept/Threshold	.77 (1.15)	−.19 (1.67)	−1.10 (2.46)	−.20* (.09)	−.13 (.14)
Loading	3.86** (1.36)	2.00 (2.37)	1.11 (4.06)	.10 (.13)	.10 (.21)

^aIndicates that the value of the parameter was fixed (not estimated) to set the scale of the latent variables.

^**Indicates significance at p < .01 level.

^*Indicates significance at p<.05. Bolded effects were retained in the final MNLFA model.

Furthermore, participants’ MAP scores on the DM factor at baseline (N = 372) were moderately and significantly correlated with their MAP scores on the DM factor at the one-year follow-up assessment (N = 356), r = .35, p < .001. This provides evidence of moderate stability of the DM factor across the follow-up window.

Discussion

Research suggests that different commonly-employed DM tasks may measure different aspects of DM (Buelow & Blaine, 2015), which raises questions as to whether these instruments measure the same underlying factor. The current study addressed this issue by examining whether different DM tasks load onto a single latent factor. Despite low correlations between indices from the IGT and those of the GDT and Cups Task, our results indicated four indices derived from the Cups Task, GDT, and IGT loaded significantly onto a single DM factor in our sample composed predominantly of adolescent cannabis users. This suggests that, although these tasks may assess different aspects of DM (e.g., DM under ambiguous vs specified conditions of risk) that may provide nuanced information, they nonetheless sufficiently tap into a single underlying construct of DM common across tasks. Collectively, these measures assess the ability to make optimal choices that maximize rewards in the presence of risk. This finding is consistent with results from neuroimaging studies, which show cross-task overlap in activation during DM in frontal regions, such as the prefrontal cortex, while also showing differences in activation in other regions (e.g. insula, posterior cingulate cortex) (Labudda et al., 2008; Li et al., 2010; Xue et al., 2009).

Furthermore, we used MNLFA to determine whether our DM factor was commensurate across participant age, sex, general cognitive ability, and recent cannabis use (Bauer, 2017). These analyses indicated that, on average, girls made riskier decisions than boys in our sample of adolescents at risk of cannabis escalation. Although previous work suggests that men make riskier choices than women, it is important to consider that the current study assessed DM during adolescence, a period characterized by neuromaturational changes that result in increased risk-taking and sensation-seeking (Casey et al., 2008). Further, females reach peak levels of sensation-seeking at an earlier age (ages 14–17) than males (ages 16–17), which may explain why girls engaged in more risk-taking in our sample (Shulman et al., 2015). We also found that participant characteristics like age and IQ caused DIF in our model. We chose to retain these effects in the final MNLFA model, ensuring that our DM factor was commensurate across participant characteristics. In other words, accounting for these effects in our final model ensures that individual differences in DM factor score represent true differences in DM performance, rather than differences in other characteristics that may not be of interest.

One previous study used exploratory and confirmatory factor analyses to examine whether three DM tasks (IGT, Columbia Card Task, and Balloon Analogue Risk Task) loaded onto a single factor among college students (Buelow & Blaine, 2015). Although this study found overlap between some of the task indices, a 3-factor model provided the best fit, suggesting that all of these tasks assess unique components of DM. Our findings, on the other hand, suggest that a single-factor model provides good fit. Discrepancies in our findings may be explained by several factors. First, it is possible that our tasks are more similar to each other than those employed by Buelow & Blaine (2015). For instance, neuroimaging studies suggest that in addition to activation in frontal areas like the anterior cingulate and dorsolateral prefrontal cortex, DM during the Balloon Analog Risk Task is associated with robust activation in mesolimbic regions, such as the midbrain and ventral and dorsal striatum, which differs from that reported with tasks used in the current study (Labudda et al., 2008; Li et al., 2010; Rao, Korczykowski, Pluta, Hoang, & Detre, 2008; Xue et al., 2009). Second, there are several methodological differences which may have influenced findings, including the specific task indices used as well as the type of analysis. For instance, our latent factor approach employed confirmatory factor analysis to test our hypothesis that the indices loaded onto a single factor. Thus, the number of factors was decided a priori, whereas this is not the case with exploratory factor analysis. Buelow and Blaine (2015) used a combination of exploratory and confirmatory factor analyses, which may have contributed to differences in our findings. Third, the participants in our sample were significantly younger. Although speculative, it is possible that different tasks may load onto a single factor at this developmental stage due to the protracted pattern of development observed in brain areas underlying executive functions such as DM (Casey et al., 2008), and that this single factor may later differentiate into multiple factors as the brain continues to mature.

Finally, scores on our latent construct at baseline were significantly correlated with scores on this construct at the one-year follow-up assessment, suggesting that our construct was relatively stable over time. This effect was moderate in size, and was consistent with previous estimates for individual DM tasks (Buelow & Barnhart, 2018).

These results should be interpreted in light of several limitations. First, our sample consisted of adolescents primarily of Hispanic/Latino descent who were cannabis users, which may limit the generalizability of our findings to other populations. The limited range of age in our sample may have also limited our ability to detect significant age-related DIF. Future studies should aim to replicate these results in a more representative sample of healthy adolescents. In addition, although some of the DM tasks we employed yield several informative and fine-grained indices of DM, the current study only used one or two indices per task, as this was better suited to the purpose of our analyses. Considering ongoing debates about which indices best capture DM in each task (e.g. IGT), future studies should examine whether different indices for these tasks influence loadings onto the DM factor, resulting in improved model fit. Nonetheless, we created a latent factor of DM which accounts for DIF caused by different participant characteristics.

Our results provide further support that various DM measures assessed a single underlying construct of DM when applied to a sample of adolescent cannabis users, which presumably represents the ability to make optimal choices that maximize rewards in the presence of risk. Future studies of DM should apply similar approaches to examine whether their DM tasks tap into a single underlying construct in their sample and whether that construct is commensurate across participants with different characteristics. Applying these methods will help ensure that our measures of DM are comparable for all participants, which will ultimately enhance generalizability of findings.