Item-Based Analysis of Delayed Reward Discounting Decision Making

Abstract

Delayed reward discounting (DRD) is a behavioral economic index of time preference, referring to how much an individual devalues a reward based on its delay in time, and has been linked to a wide array of health behaviors. It is commonly assessed using a task that asks participants to make dichotomous choices between two monetary rewards, one available immediately and the other after a delay. This study sought to shorten an extended iterative DRD assessment to increase its versatility and efficiency. Data were drawn from two young adult samples, an exploratory sample (N = 130) and a confirmatory sample (N = 247). In the exploratory sample, eight items were identified as predicting the majority of the variance in the full task area under the curve (AUC) (R² = .821; p < .001). In the confirmatory sample, the same eight items similarly predicted the majority of variance in the full task AUC (R² = .844, p < .001). These results provide initial support for the validity of a brief 8-item assessment of DRD. Priorities for further validation and potential applications are discussed.

1. Introduction

Impulsivity can be broadly characterized as acting “with relatively little forethought” (Dickman, 1990, p. 99). Rather than being a single construct, however, efforts to operationalize impulsivity have illuminated its multifactorial nature. The diverse aspects of impulsivity include risk taking, an impaired ability to inhibit prepotent responses, lack of judgment of negative consequences, reactivity to emotional states, and delayed reward discounting (DRD) (de Wit, Flory, Acheson, McCloskey, & Manuck, 2007; Logan, Schachar, & Tannock, 1997; Swann, Bjork, Moeller, & Dougherty, 2002; Whiteside & Lynam, 2001).

This latter index, DRD, is a behavioral economic index of how much an individual devalues a reward based on its delay in time. Impulsive DRD is a core feature of several psychiatric disorders that are characterized by excessive choice of immediate rewards at the cost of long-term outcomes. For example, impulsive DRD has been associated with a range of addictive behaviors, such as tobacco, cocaine, opiate, and alcohol dependence (MacKillop et al., 2011). Furthermore, impulsive DRD has been associated with attention-deficit/hyperactivity disorder (ADHD) (Scheres, Tontsch, Thoeny, & Kaczkurkin, 2010; Wilson, Mitchell, Musser, Schmitt, & Nigg, 2011). This over-orientation toward immediate rewards is thought to be a core deficit in ADHD (Tripp & Alsop, 2001). In addition, impulsive DRD has been inversely associated with an array of health behaviors, including mammogram, prostate, and dental screenings; asthma adherence; exercising; responsiveness to hypertension diagnosis, and flu shot utilization (Axon, Bradford, & Egan, 2009; Bradford, 2010; Brandt & Dickinson, 2013).

Delayed reward discounting is characteristically assessed with a behavioral task (MacKillop et al., 2011). Originally, the DRD task was non-randomized and systematically assessed preferences for immediate versus delayed rewards with a titrating function that sequentially decreased (or increased) reward discrepancy and increased delay time (Rachlin, Raineri, & Cross, 1991). Subsequently, randomized tasks have increasingly been used to improve the resolution of participant preferences independent of systematic sequences of choices (Amlung, Sweet, Acker, Brown, & Mackillop, 2012; Boettiger et al., 2007). A fully permuted and randomized task is in many ways an ideal assessment as it systematically characterizes choice preferences at multiple delay lengths and allows for examination of consistency in reward preferences to assess validity. These DRD tasks exhibit good validity, but require as long as twenty minutes to complete (e.g., MacKillop, Anderson, Castelda, Mattson, & Donovick, 2006), reflecting relatively high assessment burden. More recently, iterative tasks have been made more efficient by using adaptive adjusting procedures to hone in on preference reversals (Sheffer et al., 2011). However, even with this step, DRD tasks are often too lengthy for time-limited clinical settings or large-scale epidemiological or economic surveys.

Previous efforts have been made to shorten the task, the most prominent being the Monetary Choice Questionnaire (MCQ; Kirby, Petry, & Bickel, 1999), which consists of 27 randomized choices across three reward magnitudes. The task has even been distilled into single-item and two-item assessments (Anokhin, Golosheykin, Grant, & Heath, 2011; Reimers, Maylor, Stewart, & Chater, 2009), but these measures are necessarily relatively low resolution and have been found to reveal smaller effect size associations with health behaviors (MacKillop et al., 2011). Additionally, matching (i.e., fill-in-the-blank) has been proposed as a method for circumventing ordering effects of choice questions and for shortening length of testing, however, comparatively fewer psychological studies use this strategy and choice formats have been found to reveal higher associations between discounting and health behaviors (Hardisty, Thompson, Krantz, & Weber, 2013). Notably, no studies to date have systematically examined the extended DRD task to determine the most predictive items toward developing a more efficient DRD assessment.

The utility of a brief version extending across delay amounts and times would reduce assessment burden, allowing greater inclusion of DRD assessment in both research and applied contexts. The goal of the current study was to examine item-level performance in the context of a full iterative DRD task. Using exploratory and confirmatory samples, we examined the relationship between individual item performance and over discounting preferences. We hypothesized that a smaller set of items would be able to substantially capture DRD preferences and would generalize across samples.

2. Method

2.1 Samples

Data were drawn from two undergraduate samples at the University of Georgia (exploratory N = 130; confirmatory N = 247). Participants were excluded if they did not respond to more than 25% of the DRD task items (exploratory = 0; confirmatory = 1) and for response consistency less than 75% (i.e., inconsistency in levels of future discounting; exploratory = 4; confirmatory = 3), reflecting poor effort. Participants were primarily female (77.7% exploratory/75.9% confirmatory), European American (85.1%/81.7%), and 20 years-old (20.3/19.6). Assessments took place during one-hour group testing sessions. Participants were compensated research credit or extra credit for their time. All aspects of the studies were approved by the institutional review board and all participants provided informed consent.

2.2 Assessment

Comprehensive demographics were assessed, including sex, age, race, gender, income, education and other descriptive variables. During the DRD task, participants chose between a larger delayed reward (LDR; $100 after 1 day, 1 week, 2 weeks, 1 month, 6 months or 1 year) and a smaller immediate reward (SIR; $1, $10, $20, $30, $40, $50, $60, $70, $80, $90, $99) that was available today. Items were ordered in a semi-randomized sequence that contained no consecutive trials with both adjacent small reward magnitudes and identical delay lengths. The task was administered via a PowerPoint presentation that was projected onto a large screen. Each item was presented for six seconds with a two second interstimulus interval, and there was a 15 second break in the middle of the task. Participants were instructed to check a box on a response sheet corresponding to their choice of the LDR or SIR presented.

2.3 Data Analysis

The data were screened for outliers, defined as Z > 3.29, but none were present (Tabachnick & Fidell, 2007). Points of indifference (i.e., where participants begin to value the SIR over the LDR) were calculated based on the smallest amount of money chosen to be received immediately instead of waiting the specified delay to receive $100 (see Amlung & MacKillop, 2011). Area-under-the-curve (AUC) scores were generated as the index of temporal discounting; smaller AUC values reflect greater future discounting and impulsivity (see Myerson, Green, & Warusawitharana, 2001). In the exploratory sample, stepwise regression was used as a data-driven strategy for identifying the most influential items for predicting the full task AUC. All 66 items from the exploratory sample were entered into a multiple regression that iteratively retained items that accounted for incremental additional variance at p < .10. The trend-level significance threshold was chosen to avoid prematurely eliminating items during successive steps. As considerable overlap was expected among item performance, and was indeed present, items exhibiting a tolerance value < .40 were eliminated due to excessive collinearity (e.g., 0 = 100% collinearity) (Allison, 1998). Then, the regression was rerun with the remaining items that satisfied the tolerance threshold to identify items that offered significant and unique incremental variance in predicting the overall AUC. These remaining items were used to test the hypothesis that the reduced items would predict the majority of the variance accounted for by the full task AUC. The items identified in the exploratory sample were subsequently entered into a linear regression model in the confirmatory sample to test the second hypothesis of replicability across studies. Finally, the predictive relationship of the shortened scale at the individual level was assessed by computing the interquartile range (IQR) of the actual and predicted AUC and examining the difference between these. Of note, the difference in mean AUCs between actual and predicted values was not a meaningful metric because it is inherently zero.

To verify the generalizability of the shortened measure to alternative discounting characterization strategies, a hyperbolic discounting function was also generated utilizing Mazur’s (1987) formula: V = 1 / (A + kD). V is the subjective value of the delayed reward (i.e., the indifference point), A is the amount of the delayed reward (i.e., $100), D is the delay, and k is the index of best-fit, indicating the overall rate of discounting within the model. Because the distribution of the k is typically positively skewed, it was log transformed to improve its distribution.

3. Results

There were no significant differences in the overall levels of impulsivity between the two samples (exploratory M = .50 (SD = .23); confirmatory M = .47 (.24)). Correlations between points of indifference of both samples demonstrated high intercorrelations between each delay and their most proximal delays (Table 1).

Table 1.

Mean points of indifference (PoIs) and correlations among PoIs by sample

Variable	Mean (SD)	1	2	3	4	5
Exploratory Sample
1. PoI-1	96.2 (6.5)	----	----	----	----	----
2. PoI-7	88.6 (11.8)	.68**	----	----	----	----
3. PoI-14	81.5 (14.1)	.58**	.80**	----	----	----
4. PoI-30	70.5 (23.5)	.11	.43**	.52**	----	----
5. PoI-180	41.5 (26.8)	.13	.37**	.48**	.62**	----
6. PoI-365	38.7 (27.1)	.12	.34**	.37**	.66**	.84**
Confirmatory Sample
1. PoI-1	96.4 (7.9)	----	----	----	----	----
2. PoI-7	88.4 (12.8)	.39**	----	----	----	----
3. PoI-14	78.4 (18.3)	.31**	.71**	----	----	----
4. PoI-30	64.9 (26.7)	.23**	.62**	.72**	----	----
5. PoI-180	39.9 (26.3)	.16*	.42**	.58**	.74**	----
6. PoI-365	35.9 (26.9)	.05	.42**	.52**	.69**	.83**

Note: PoI = Point of Indifference;

^*significant correlation at p <.05;

^**significant correlation at p <.01.

In the exploratory sample, the first step of the stepwise regression generated 25 items (out of 66 total items) which substantially predicted the full task AUC, R² = .996, p < .001. In the second stage, 12 items were removed for excessive collinearity (tolerance < .40). The stepwise regression was run again on the remaining 13 items, which resulted in eight items surviving the individual item threshold. The eight items supported the first hypothesis of predicting the majority of the full task AUC from the exploratory sample (R² = .819; p < .001). The eight items were comprised of varying immediate rewards spanning the delay duration continuum (i.e., 14 days, 30 days, 6 months, 1 year; see Appendix).

In the confirmatory sample, the same eight items entered into a linear regression predicting the full task AUC from the confirmatory sample, and accounted for the large majority of the variance (R² = .844, p < .001), supporting the second hypothesis of replicability across studies. Notably, in the confirmatory sample, the item $99 today versus $100 in one month, was not a significant predictor of the variance of the full task AUC. Individual item associations and overall linear regression models are presented in Table 2.

Table 2.

Regression models

		AUC				Log10-k
Delay Duration	Immediate Reward	B	SE	β	T	B	SE	β	T
Exploratory Sample Stepwise Solution
14 Days	$30	.129*	.059	.084	.858	−.290	.173	−.065	.944
30 Days	$99	.113**	.031	.142	.787	−.382**	.091	−.164	.906
	$80	.080**	.024	.174	.599	−.215**	.069	−.159	.536
	$70	.104**	.023	.227	.574	−.299**	.067	−.222	.571
	$40	.144**	.024	.258	.898	−.422**	.070	−.258	.766
6 Months	$80	.118**	.027	.196	.770	−.436**	.079	−.247	.690
	$10	.119**	.024	.201	.690	−.225**	.071	−.130	.844
1 Year	$50	.172**	.027	.281	.700	−.538**	.080	−.300	.703
		R2/Adj R2		.830	.819			.830	.819
Application to Confirmatory Sample
14 Days	$30	.084**	.029	.081	.786	−.351**	.096	−.116	.802
30 Days	$99	.025	.023	.030	.682	.054	.076	.022	.855
	$80	.129**	.016	.267	.571	−.369**	.053	−.260	.579
	$70	.118**	.016	.242	.588	−.346**	.054	−.243	.564
	$40	.097**	.018	.165	.855	−.315**	.061	−.182	.652
6 Months	$80	.144**	.022	.216	.669	−.386**	.073	−.197	.587
	$10	.127**	.019	.205	.619	−.221**	.063	−.122	.680
1 Year	$50	.121**	.022	.183	.611	−.458**	.071	−.236	.599
		R2/Adj R2		.849	.844			.806	.800

Note: B = unstandardized regression coefficient; SE = standard error; β = standardized regression coefficient; T = tolerance;

^*significant at p <.05;

^**significant at p <.01; smaller-sooner rewards coded 1, larger-later rewards coded 2.

Analyses of the IQR of differences between actual and predicted AUC values in both cohorts suggested relatively high overlap in individual values. For the exploratory sample, M AUC = .50, IQR =.32–.63, and the predicted AUC IQR = .39 – .59. More than 70% of the individual participants actual mean AUCs were within .10 of the 8-item predicted values. For the confirmatory sample, M AUC = .47, IQR = .30–61; and the predicted AUC IQR = .29–.62, which is almost identical. In this case, more than 80% of the raw values were within .10 of the predicted values, again suggesting a high degree of overlap. Taken together, the predicted values overlapped considerably with the raw values.

Comparative variance was accounted for when predicting k in both populations from the 8-item measure (exploratory sample: R² = .819; p < .001; confirmatory sample: R² = .800; p < .001). All items provided significant unique variance in the prediction of k in the models with the exception of one item ($30 today versus $100 in 14 days) in the exploratory sample. Detailed results can be found in Table 2.

4. Discussion

This study sought to examine item-level performance of an iterative DRD task to create a briefer version that may be used efficiently in both research and applied contexts. In the exploratory sample, a data-driven approach identified eight items that contributed significant unique variance in predicting the large majority of the variance of the full task AUC. Then, in the confirmatory sample, the 8-item measure replicated these results, accounting for the large majority of the variance of the full task AUC. Furthermore, results demonstrated generalizability across indices of DRD, as the shortened measure was found to predict an equally high amount of variance of k in both samples. The items spanned from 14 days to 1 year, demonstrating the importance of assessing DRD at temporally close and distant delays. The need for assessing reward preference at varying delays was further validated by the findings that the points of indifference from temporally proximal time delays correlated highly significantly while points of indifference at temporally distant time delays correlated less strongly and occasionally non-significantly. Generally speaking, the current findings are supportive of a brief DRD assessment comprising the eight items identified. Going forward, important questions will be whether these items are able to capture the same relationships between DRD and health behaviors as extended tasks and whether they improve other existing brief DRD assessments.

The current findings should be considered in the context of the study’s strengths and weaknesses. One of the strengths of both the full-length and shortened DRD task within this study is the presentation of items in a semi-randomized format, counterbalanced to prevent order effects. Some previous research has suggested that item order can itself affect preferences, such that DRD tasks with ascending orders induce higher discounting than tasks with descending rewards (Hardisty et al., 2013; Reid et al., 2009). Had we conducted the current study using exclusively ascending or descending format, the items identified would have potentially included the order bias. An additional strength was the use of both exploratory and confirmatory samples to systematically identify the most predictive items in an extended DRD task, in both cases capturing a substantial portion of the variance of the full task AUC.

However, the study also had some notable limitations. Both samples were comprised of undergraduate college students with a relatively high median income and had 80% or more Caucasian participants. These demographics may limit the generalizability of this shortened task to lower SES, and a broader range of age, education, and race. As such, it will be important to validate this brief measure on different community populations. Also notable was that one of the items predicting AUC was not individually supported in the confirmatory sample, and one of the items predicting k was not supported in the exploratory sample, raising the question of whether they are necessary. These concerns will also need to be addressed in future studies. Additionally, despite predicting the majority of the full task AUC and k in both samples, between 15–20% of the variance is not accounted for and thus some resolution is lost with using the brief scale. Finally, it should be noted that this task is only a measure of preference for immediate versus future rewards, and does not capture delayed loss discounting, front-end delays (e.g., $100 in one week vs. $200 in one year), or zero/negative discount rates. This is a necessary consequence of the parent task being exclusively oriented toward DRD, but constrains the decision-making that can be assessed nonetheless.

Despite these considerations, this study provides initial support of a shortened item set in two samples. The 8-item measure is 88% shorter than the original 66-item task, with only modest sacrifice in characterization of choice preferences. Notably, although some resolution is lost between the 25-item and 8-item DRD task, the 25-item task would take three times as long and a tolerance < .40 suggests that the 17 items that were removed offered marginally unique variance. This briefer version may provide greater utility in highly time limited research and applied contexts, such as hospitals, healthcare systems, or population surveys. For example, DRD has consistently predicted better treatment outcomes in cigarette smokers (Krishnan-Sarin et al., 2007; MacKillop & Kahler, 2009; Sheffer et al., 2011) and this brief assessment could be readily applied to busy clinical settings. Given that this item set has the potential to connect DRD assessments to a variety of novel contexts, future studies should seek to replicate associations between DRD and addictive disorders and other negative health behaviors (MacKillop et al., 2011). Finally, future research should conduct direct comparisons between full DRD tasks and shortened DRD tasks (e.g., this 8-item measure, MCQ, 1–2 item measures) to examine empirically which measures maximize correspondence with of full task performance and minimize information loss in relation to health behaviors and other domains.

Highlights.

• Item-based analysis was used to develop a more efficient delay discounting measure.

• In an exploratory sample, 8 items predicted >80% of full task performance.

• In a confirmatory sample, the 8 items again predicted >80% of full task performance.

• This item set may be useful for assessing discounting in time-limited settings.

Appendix. Brief DRD Task

Please check the amount you would prefer in the following questions.

1. Would you rather have:	$50	Today ❒	Or,	$100	in 1 Year? ❒
2. Would you rather have:	$70	Today ❒	Or,	$100	in 1 Month? ❒
3. Would you rather have:	$100	in 6 Months ❒	Or,	$10	Today? ❒
4. Would you rather have:	$80	Today ❒	Or,	$100	in 6 Months? ❒
5. Would you rather have:	$100	in 1 Month ❒	Or,	$40	Today? ❒
6. Would you rather have:	$100	in 2 Weeks ❒	Or,	$30	Today? ❒
7. Would you rather have:	$100	In 1 Month ❒	Or,	$99	Today? ❒
8. Would you rather have:	$100	In 1 Month ❒	Or,	$80	Today? ❒