Development and validation of the Delaying Gratification Inventory

Abstract

Deficits in gratification delay are associated with a broad range of public health problems, such as obesity, risky sexual behavior, and substance abuse. However, six decades of research on the construct has progressed less quickly than might be hoped, largely due to measurement issues. Although past research implicates five domains of delay behavior, involving food, physical pleasures, social interactions, money, and achievement, no published measure to date has tapped all five components of the content domain. Existing measures have been criticized for limitations related to efficiency, reliability, and construct validity. Using an innovative Internet-mediated approach to survey construction, we developed the 35-item five-factor Delaying Gratification Inventory (DGI). Evidence from four studies and a large, diverse sample of respondents (N = 10,741) provided support for the psychometric properties of the measure. Specifically, scores on the DGI demonstrated strong internal consistency and test-retest reliability for the 35-item composite, each of the five domains, and a 10-item short-form. The five-factor structure fit the data well and had good measurement invariance across subgroups. Construct validity was supported by correlations with scores on closely-related self-control measures, behavioral ratings, Big Five personality trait measures, and measures of adjustment and psychopathology, including those on the Minnesota Multiphasic Personality Inventory-2-Restructured Form (MMPI-2-RF). DGI scores also showed incremental validity in accounting for well-being and health-related variables. The present investigation holds implications for improving public health, accelerating future research on gratification delay, and facilitating survey construction research more generally by demonstrating the suitability of an Internet-mediated strategy.

Delaying gratification refers to the tendency to forego strong immediate satisfaction for the sake of salient long-term rewards. Although most develop a burgeoning capacity to delay gratification by early adolescence, adulthood is marked by substantial individual differences in delay behavior (Lee, Lou, Wang, & Chiu, 2008). National Institutes of Health (NIH, 2009, p.2) guidelines identify gratification delay as having a non-trivial impact upon public health, with six decades of research linking poor gratification delay to societal problems, including obesity, substance abuse, risky sexual behavior, psychopathology, consumer debt, criminality, and low educational attainment (Baumeister, Vohs, & Tice, 2007; Bembenutty & Karabenick, 2004; DeWall, Baumeister, Stillman, & Galliot, 2007; Gottdiener, Murawski, & Kucharski, 2008; Seeyave et al., 2009; Wulfert, Safren, Brown, & Wan, 1999). Despite its importance, research on gratification delay has not progressed as quickly as might be hoped largely due to measurement limitations and inconsistent findings (Baumeister et al., 2007; Lee et al., 2008; Mauro & Harris, 2000; McLeish & Oxoby, 2007; Richards, Zhang, Mitchell, & de Wit, 1999; Smith & Hantula, 2008). We put forth a framework for conceptualizing gratification delay, examine the relative merits of available measurement strategies, and use a novel, Internet-mediated approach to survey development.

Theoretical Framework

Historically, terms like gratification delay, self-regulation, self-control, impulsivity, and ego resiliency have often been used interchangeably or inconsistently (Funder, Block, & Block, 1983; Gailliot et al., 2007; Lee et al., 2008; Mauro & Harris, 2000; Mischel & Gilligan, 1964; Mischel, Cantor, & Feldman, 1996; Muraven, Baumeister, & Tice, 1999), ignoring subtle distinctions between constructs (see Figure A1 of online supplemental Appendix A). Authoritative reviews emphasize that under the umbrella of self-regulation exists self-control (Baumeister et al., 2007; Gailliot et al., 2007), which can be understood as a continuum of three constructs, with gratification delay occupying conceptual space between impulse control and ego resiliency (Funder & Block, 1989). Delay of gratification is similar to impulse control in that both involve resisting strong rewards, can occasionally be disadvantageous, and have trait-like features (Baumeister & Vohs, 2004; Funder & Block, 1989; Funder et al., 1983; Gottdiener et al., 2008; Ramanathan & Menon, 2006). Alternately, gratification delay is similar to ego resiliency (but different from impulse control) in that both require a future time orientation, involve carefully weighing consequences, and have ability features (Bembenutty & Karabenick, 2004; Funder & Block. 1989; Mischel, Shoda, & Peake, 1988; Twenge, Catanese, & Baumeister, 2003). Thus, operationalizing gratification delay involves posing choices between evocative immediate rewards and salient long-term consequences.

In addition to synthesizing past research to identify a clear definition of gratification delay, there is a need to articulate the content domain of the construct. We examined five domains of gratification delay, involving (1) food, (2) physical pleasures, (3) social interactions, (4) money, and (5) achievement. Baumeister et al. (2007, Box 1, p. 353) described a somewhat similar set of five domains of behavior with evolutionary significance that were vulnerable to ego-depletion. Further, several additional studies have described at least two of the hypothesized five factors of delay behavior (see online supplemental Table B1; e.g., Baumeister et al., 1994; Bembenutty & Karabenick, 1998, 2004; Lee et al., 2008; Mischel et al., 1988; Ramanathan & Williams, 2007). This investigation was designed to develop a scale measuring individual differences in the five domains of gratification delay that have received attention in past research.

Measurement Strategies

In six decades, three types of measures have mainly been used to assess gratification delay: early performance-based strategies, Mischel's paradigm, and delay discounting tasks. Performance measures have included the number of human movement (M) responses on the Rorschach, maze and tracing tasks, time estimation, the Stroop, and stop-signal tasks. In addition to being time consuming, these measures have a disjointed theoretical relationship to gratification delay (Rapaport, 1951) and have demonstrated poor evidence of construct validity (Nederkoorn et al., 2006; Singer, Meltzoff, & Goldman, 1952; Wormith & Hasenpusch, 1979).

Given these pitfalls, Mischel explored the behavioral decision making paradigm as a more suitable method for assessing delay of gratification (Funder et al., 1983; Mischel, 1958; Mischel & Ayduk, 2002; Mischel et al., 1988). As reviewed by Mischel (1996), classic studies involved young children choosing between receiving one marshmallow immediately or two marshmallows after a brief (e.g., 20 min) delay period. Although this constitutes a clear operationalization of the construct, limitations of this strategy include narrow coverage of the content domain, inefficiency, and low suitability to adult participants. Of greater concern, this paradigm often relies upon one or a very limited number of choices, yielding a dichotomous indicator of delay behavior, which attenuates reliability and validity estimates (Funder et al., 1983; Mauro & Harris, 2000; Mischel, 1958; Wormith & Hasenpusch, 1979). Multiple observations of delay behavior yield stronger results (Funder & Block, 1989; Mischel & Gilligan, 1964) but narrow content domain and age limitations remain.

First designed for animal studies, delay discounting tasks have recently gained popularity for assessing gratification delay in human adults (e.g., Reynolds, 2006). Using real or hypothetical monetary reinforcers, the approach examines changes in response curves, or other metrics, as a result of greater delays in reinforcement. The tasks consist of a lengthy series of items that ask participants, for example, to choose between $50 today and $90 tomorrow. The delay period is incrementally increased until the examinee opts for the immediate reward, signaling a maximum delay period. Task limitations include being time consuming, covering only one aspect of the content domain (i.e., money), and being costly if real reinforcers are used. Also, there is stronger evidence for predictive validity (Reynolds, 2006; Shamosh et al., 2008) than construct validity (McLeish & Oxoby, 2007; Richards et al., 1999; Smith & Hantula, 2008; Wormith & Hasenpusch, 1979). This has led Reynolds (2006, p. 665) to call for “a better understanding of what is being assessed with these measures,” as they may tap working memory, logical reasoning, withdrawal sensitivity, or other important constructs beyond gratification delay.

Survey methodology provides practical and psychometric benefits over alternative strategies. The practical advantages of being able to recruit large samples of adults quickly and inexpensively have been shown in studies on the three published scales, including the 12-item Deferment of Gratification Questionnaire (DGQ; Ray & Najman, 1986), 10-item Academic Delay of Gratification Scale (ADOGS; Bembenutty & Karabenick, 1998), and 22-item Multidimensional Delay of Gratification scale (MDG; Ward, Perry, Woltz, & Doolin, 1989). Yet, existing scales have room for improvement in content validity and score reliability. Specifically, no published scale has explicitly addressed the entire content domain relevant to measuring gratification delay. The DGQ was crafted without apparent review of the broad content domain of gratification delay and the ADOGS focuses on achievement. The MDG was intended to measure two sociopolitical aspects of gratification delay in South African Apartheid-era opposition, but the factor structure was unsupported, and additional domains were not included. Internal consistency reliability for scores on the three measures has fallen short of expectations, ranging from .68 to .74, with potential culprits including survey brevity and difficult item wording. Our goal was to extend upon the practical advantages of surveys by developing a five-factor measure of gratification delay that produces scores of high reliability.

Internet Methodology

The growth of publicly available web-based psychology studies suggests potential avenues for scale development research. Compared to traditional laboratory studies conducted in-person with undergraduates, Internet-mediated research permits several advantages, including increased sampling efficiency, greater sample heterogeneity, and decreased research time and costs (Hoerger, 2010; Hoerger & Currell, in press). For example, two web studies of individual differences have surpassed 100,000 participants (Nosek, Banaji, & Greenwald, 2002; Srivastava, John, Gosling, & Potter, 2003). Large, heterogeneous samples are particularly alluring for test construction projects. Power stabilizes item-total correlations, allowing the “best” set of items to be selected from a larger item pool, and sample heterogeneity improves external validity. In contrast, overreliance upon small convenience samples can threaten cross-sample reliability estimates (e.g., the “Subtle” items of the MMPI; Graham, 2006).

The upside of Internet-mediated studies can substantially overshadow risks. Foremost, early methodological concerns about web-based research (e.g., measurement inequivalence) have not been borne out by data (De Beuckelaer & Lievens, 2009; Gosling, Vazire, Srivastava, & John, 2004). Further, technico-ethical issues, such as confidentiality and consent, can be handled effectively for low-risk studies (Hoerger & Currell, in press; Kraut et al., 2004). One realistic concern is that the public is unlikely to complete a lengthy battery of validity measures (Krug, 2005). As such, we initiated a two-step approach to development and validation. During development, the survey was to be administered to a broad public sample, optimizing evidence for reliability and factor structure. Validation evidence from a lengthier set of measures was to come primarily from traditional convenience samples of undergraduates, who are less deterred by study length (Hoerger, 2010).

Present Investigation

The present investigation involved four Internet-mediated studies on the development and validation of the Delaying Gratification Inventory. Studies 1 and 2 focused on scale development, using a large, diverse sample of adults worldwide. These studies were designed to provide evidence regarding internal consistency reliability, factor structure, and measurement invariance, with ancillary analyses also providing provisional evidence of test-retest reliability and construct validity. Study 1 involved administering a large pool of items in order to craft the final 35-item scale, and Study 2 was a cross-sample replication and extension. Studies 3 and 4 focused on validation, using smaller convenience samples of undergraduates. Study 3 provides evidence of test-retest reliability, construct validity, and incremental validity, whereas Study 4 focused exclusively on validity evidence involving adjustment and psychopathology correlates of the resulting survey's scores.

Studies 1 and 2

Method

Procedures

Large, worldwide samples participated in pilot testing and Studies 1 and 2 via the web. We rationally developed a large pool of items, based on our five-factor framework, and these items were piloted extensively online to hundreds of participants. Participant feedback via anonymous comments was used to remove or modify items that were potentially confusing or biased. In combination with psychometric evidence, the item pool was iteratively revised, ultimately resulting in a manageable set of 70 items measuring the five domains of gratification delay. This item pool was administered online during Study 1. The results of Study 1 were used to devise the 35-item Delaying Gratification Inventory (DGI), which was administered online in Study 2. These studies were intended to provide replicable evidence of internal consistency, factor structure, and measurement invariance. Also, a small set of additional items was administered in Study 2 to provide initial evidence of construct validity, and test-retest reliability was examined for a subset of participants completing the measure twice.

These studies were designed to minimize cost, optimize participant recruitment, and meet high ethical standards for Internet-mediated research (Hoerger & Currell, in press). The study web site was accessible through research sites, search engines, relevant Wikipedia pages, Facebook, discussion forums, and blogs. Upon entering the site, participants could access the investigator's contact information, relevant research articles, IRB approval documentation, Frequently Asked Questions (FAQs), and an online consent form to begin participation. Upon completing the survey, participants received additional study information, innocuous tailored personality feedback based on their survey responses (coded using Perl CGI), and optional links for providing anonymous feedback and entering a cash drawing of $100.

Numerous procedures were used to reduce repeat or invalid response contamination. For most browsers, JavaScript ensured that all questions were answered prior to form submission. To guard against multiple submissions, participants were asked directly whether they had participated previously, and the top of the survey include an unusual picture to assist their memory of the site (a picture of an okapi, a zebra-like animal). We also tracked partial IP addresses (see Hoerger & Currell, in press), which are standard IP addresses but with the leading digit removed, thereby ensuring some tracking capability while better ensuring anonymity. Response validity items were also embedded within survey content. Finally, participants had the incentive of responding honestly in order to gain realistic personality feedback.

Measures

All study measures were completed in English. Participants indicated whether they had completed the survey previously, and optionally provided demographic information, including gender, age, ethnicity (based on the 2000 U.S. census categories), location (all U.S. states, Canada, Mexico, Europe, Asia, Australia, South America, or Africa), highest grade completed, and high school GPA.

DGI

The 35-item scale (see Table 1) yields gratification delay scores for five domains (food, physical pleasures, social interactions, money, and achievement), a 35-item composite (DGI-35), and a 10-item short-form composite (DGI-10). Seventeen items are reverse-coded, and participants reported how well each item described themselves using a scale from 1 (Strongly Disagree) to 5 (Strongly Agree).

Table 1. Delaying Gratification Inventory (DGI) Items by Domain.

Item	Text
Food
1	I can resist junk food when I want to.
6	I would have a hard time sticking with a special, healthy diet. *†
11	If my favorite food were in front of me, I would have a difficult time waiting to eat it.*
16	It is easy for me to resist candy and bowls of snack foods.
21	Sometimes I eat until I make myself sick.*
26	I have always tried to eat healthy because it pays off in the long run. †
31	Even if I am hungry, I can wait until it is meal time before eating something.
Physical
2	I am able to control my physical desires.
7	I like to get to know someone before having a physical relationship.
12	My habit of focusing on what “feels good” has cost me in the long run.*
17	I have given up physical pleasure or comfort to reach my goals. †
22	I prefer to explore the physical side of romantic involvements right away.*
27	When faced with a physically demanding chore, I always tried to put off doing it. *†
32	I have lied or made excuses in order to go do something more pleasurable. *
Social
3	I hate having to take turns with other people. *
8	Usually I try to consider how my actions affect others.
13	I think that helping each other benefits society. *
18	I try to consider how my actions will affect other people in the long-term. †
23	I do not consider how my behavior affects other people. *†
28	I value the needs of other people around me.
33	There is no point in considering how my decisions affect other people. *
Money
4	When I am able to, I try to save away a little money in case an emergency should arise.
9	It is hard for me to resist buying things I cannot afford. *
14	I try to spend my money wisely. †
19	I cannot be trusted with money. *†
24	When someone gives me money, I prefer to spend it right away. *
29	I manage my money well.
34	I enjoy spending money the moment I get it. *
Achievement
5	I worked hard in school to improve myself as a person.
10	I have tried to work hard in school so that I could have a better future.
15	In school, I tried to take the easy way out. *
20	I am capable of working hard to get ahead in life.
25	I cannot motivate myself to accomplish long-term goals. *†
30	I have always felt like my hard work would pay off in the end. †
35	I would rather take the easy road in life than get ahead. *

Note.

^*indicates reverse-coded item.

^†indicates inclusion on DGI-10 short form composite.

Personality and behavioral tendencies

In Study 2 only, participants rated themselves on 30 single-item personality trait descriptors (e.g., “Neuroticism: to feel sad, worried, anxious, nervous, depressed, moody, frustrated, irritated, and out of control”) and 17 single-item behavioral tendencies (e.g., “I pay bills on time”); see Results section for a complete listing. Participants rated how well traits described themselves on a scale from 1 (Below Average) to 9 (Above Average), and rated behavioral tendencies on a scale from 1 (Never) to 5 (Always). Very brief measures have shown merit for studies in which hypothesized correlations were sizeable (Donnellan, Oswald, Baird, & Lucas, 2006; Wanous, Reichers, & Hudy, 1997), and brevity was necessary for maintaining participant recruitment (Hoerger, 2010; Krug, 2005).

Participants

Data from a large sample of survey respondents (n = 10,241) in Studies 1 and 2 were cleaned to remove invalid responses prior to core analyses. Reasons for exclusion included missing data, invalid response sets, repeat participation as indicated by self-report, repeat participation as indicated by a recurrent partial IP address with matching demographics, and accidental duplicate form submissions. Ninety five percent of responses were retained as valid (Study 1: 1,900 of 1,982; Study 2: 7,771 of 8,259). Participants were diverse in terms of age (M = 30.6, SD = 10.9 years), ethnicity, location, and education level (see Table 2).

Table 2. Demographic Characteristics for Participants in Studies 1 and 2.

Demographic Characteristic	Study 1		Study 2
	n	%	n	%
Age
18-19	322	17.5%	870	11.7%
20-29	808	44.0%	3,296	44.1%
30-39	369	20.1%	1,805	24.2%
40-49	215	11.7%	917	12.3%
50-59	100	5.4%	452	6.1%
60-69	21	1.1%	114	1.5%
70+	3	0.2%	12	0.2%
Gender
Female	888	47.1%	3,578	46.3%
Male	997	52.9%	4,142	53.7%
Race/Ethnicity
White and non-Latino	1,401	74.4%	5,770	74.6%
Black or African descent	66	3.5%	237	3.1%
Asian descent	210	11.1%	919	11.9%
Hispanic or Latino of any race	71	3.8%	293	3.8%
Biracial or Multiracial	87	4.6%	285	3.7%
Other	49	2.6%	226	2.9%
Location
United States	1,178	62.9%	3,747	49.0%
Canada	146	7.8%	412	5.4%
Mexico	8	0.4%	15	0.2%
Europe	329	17.6%	1,088	14.2%
Asia	103	5.5%	402	5.3%
Australia	84	4.5%	1,879	24.6%
South America	18	1.0%	56	0.7%
Africa	7	0.4%	51	0.7%
Education Level (U.S. Only)
Doctoral Degree	63	5.4%	264	7.1%
Masters Degree	160	13.7%	668	17.9%
Bachelors Degree	296	25.3%	966	25.9%
Some College	423	36.2%	1,214	32.6%
High School Diploma	162	13.9%	425	11.4%
Non- High School Graduate	65	5.6%	192	5.1%

Note. For Study 1, n = 1,900; however, responses missing for age (n = 62), gender (n = 15), ethnicity (n = 16), location (n = 27), education level (n = 9), and GPA (n = 57). For Study 2, n = 7,771, and responses were missing for age (n = 305), gender (n = 51), ethnicity (n = 41), location (n = 121), education level (n = 18), and GPA (n = 134).

Results and Discussion

The results of Study 1 were used to develop a reliable 35-item scale, covering the five domains of gratification delay (for item-total correlations, see Supplemental Tables B2 and B3). Scale means, standard deviations, internal consistency reliabilities, and intercorrelations are show for all U.S. participants from both studies in manuscript Table 3, and separated by study, gender, and location in Supplemental Tables B4 to B8. Across subgroups, internal consistency reliability was strong for scores on the DGI-35 composite scale (α ≥ .90) and good for scores on the DGI-10 short form (α ≥ .77). Thus, both the long and short forms produced reliable measurements of general individual differences in gratification delay. Across participant groups, subscale scores also had good reliability (α = .69 to .89) and were modestly intercorrelated (r = 23 to .60). Item-level statistics for the DGI-35 were stable across samples, and items loaded relatively uniquely on their designated domains. As expected, the five domains had modestly overlapping variance, but also accounted for unique aspects of gratification delay.

Table 3. DGI Scale Score Properties in Studies 1 and 2 for all U.S. Participants.

Scale	M	SD	F	P	S	M	A	10	35
Food	22.3	5.8	(.75)
Physical	22.8	5.2	.43	(.71)
Social	29.3	4.4	.25	.43	(.81)
Money	26.8	6.4	.36	.50	.33	(.89)
Achievement	26.7	6.0	.33	.58	.43	.44	(.85)
DGI-10	36.3	6.9	.59	.73	.60	.66	.75	(.79)
DGI-35	127.9	20.2	.66	.81	.63	.75	.77	.92	(.91)

Note. N = 4,925. F = Food, P = Physical, S = Social, M = Money, A = Achievement, 10 = DGI-10 short form composite, 35 = DGI-35 composite. Table values are means, standard deviations, and correlations, with alphas indicated in parentheses.

Internal consistency and domain intercorrelations were similar across gender and participant location, but some mean differences were present (see Supplemental Tables B5 to B8). Specifically, females scored slightly higher than males in terms of composite gratification delay (DGI-35: d = 0.17 to 0.21), and across most domains, with the greatest female advantage observed in the achievement domain (d = 0.41 to 0.42). In contrast, males reported greater gratification delay in the food domain (d = 0.22 to 0.28). In terms of location, U.S. participants differed from international participants mainly in terms of greater delay of gratification in the achievement domain (d = 0.25 to 0.32). Observed demographic differences were present but generally small to moderate in size and consistent with prior research.

Confirmatory Factor Analysis supported the hypothesized five-factor model, which was robustly upheld across demographic groups. CFA model fit was examined with LISREL 8.80 using the Satorra-Bentler (1988) method, which corrects for interval data. To adequately characterize model fit, most researchers report several fit statistics, such as the Comparative Fit Index (CFI), the Normed Fit Index (NFI), Root Mean Square Error of Approximation (RMSEA), standardized Root Mean Residual (sRMR), the Akaike Information Criterion (AIC), chi-square (χ²), and the ratio of chi-square to degrees of freedom (χ²/df). First, we examined model fit for the entire sample of Study 1 and 2 participants. The hypothesized five-factor model (for a diagram, see Supplemental Figure A2) fit the data well, CFI = .964, NFI = .962, RMSEA = .057, sRMR = .058, AIC = 18,031, Satorra-Bentler scaled χ² = 17,871, df = 550, p < .001, and χ²/df = 32.49. Because the physical and achievement domains were the most related, we compared our theory-driven model to a four-factor model combining these domains. We also used multigroup CFA to compare the five-factor structure by study sample, gender, and participant location. The four-factor model was rejected as it offered no appreciable improvement in fit, and the five-factor model was found to have strong structural, factor, and variance-covariance invariance across samples, genders, and participant locations (see Supplemental Table B9).

Table 4 shows initial validity evidence for DGI scores in Study 2. In general, scores correlated highly with those from closely related constructs, such as self-discipline, self-control, conscientiousness, and moderation. DGI composite scores were also strongly related to health (r = .40 to .43) and well-being (r = .43 to .46), and scores increased slightly with age and educational attainment. Patterns of correlations varied in theoretically-meaningful ways across domains of gratification delay. Among the five domains, the food scale scores correlated most highly with moderation/immoderation, health, anxiety, perceived attractiveness, somatization, fast food consumption, thinking about food, exercise, soda drinking, and watching television. Notable correlates of the physical subscale include sensation-seeking, reported lying, rebelliousness, rule-breaking, sexual thoughts, impulse control, partying excessively, neuroticism, anxiety, excitement-seeking, smoking cigarettes, risk-taking, gambling, and Machiavellianism. The social domain was most distinct, relating to altruism, agreeableness, helping other people, prudence, patience, anger, openness to experience, comprehension, and discussing intellectual topics. Money domain scores were linked to extravagance, buying unneeded purchases, self-control, paying bills on time, smoking cigarettes, checking financial news, and gambling. Finally, the achievement score was predictive of achievement-striving, diligence, self-discipline, conscientiousness, well-being, depression, high school GPA, education level, reading books, extraversion, and perceived intelligence. Correlations between DGI-35 scores and the traits and behavioral tendencies listed in Table 4 varied minimally across gender, location, and participant age decade (average magnitude of deviation in r across subgroups was .03). In summary, the food domain is related to delay behavior involving food, health and activity level; the physical domain to sensation-seeking, drives, and thrills; the social domain to altruistic and prosocial behaviors; the money domain to day-to-day financial management; and the achievement domain to conscientiousness and achievement-striving. The validity of score interpretations was consistent across demographic groups studied. Thus, promising initial findings implicated the need for more detailed follow-up studies.

Table 4. Correlations between DGI Scores, Demographics, Trait Descriptor Ratings, and Behavioral Tendency Ratings in Study 2.

Measure	F	P	S	M	A	10	35
Demographic
Age	.05	.14	.15	.10	.08	.12	.14
High School GPA	.07	.23	.16	.27	.34	.28	.30
Education Level	.03	.16	.19	.21	.25	.23	.23
Trait Descriptors
Moderation	.53	.52	.29	.42	.37	.56	.59
Self-Discipline	.37	.53	.27	.37	.57	.59	.59
Diligence	.32	.50	.29	.33	.63	.59	.58
Conscientiousness	.32	.49	.30	.36	.56	.57	.57
Self-Control	.34	.47	.28	.53	.37	.51	.56
Extravagance	-.28	-.40	-.21	-.74	-.29	-.48	-.55
Immoderation	-.53	-.48	-.22	-.40	-.34	-.50	-.55
Achievement-Striving	.24	.40	.29	.32	.64	.52	.53
Well-Being	.26	.36	.29	.25	.41	.46	.43
Prudence	.17	.34	.38	.32	.29	.36	.41
Patience	.28	.32	.34	.29	.24	.35	.40
Perceived Health	.36	.30	.18	.24	.33	.43	.40
Impulse Control	.30	.33	.27	.26	.25	.36	.39
Rebelliousness	-.13	-.39	-.31	-.27	-.32	-.32	-.39
Agreeableness	.16	.29	.46	.19	.30	.34	.37
Neuroticism	-.26	-.33	-.21	-.23	-.28	-.36	-.36
Depression	-.25	-.28	-.18	-.22	-.30	-.35	-.34
Anger	-.24	-.27	-.30	-.21	-.22	-.30	-.34
Sensation-Seeking	-.08	-.44	-.20	-.26	-.21	-.25	-.33
Altruism	.11	.21	.53	.07	.26	.30	.30
Anxiety	-.25	-.25	-.16	-.19	-.22	-.29	-.30
Perceived Attractiveness	.33	.16	.08	.14	.21	.29	.26
Somatization	-.20	-.16	-.13	-.18	-.15	-.23	-.23
Comprehension	.14	.13	.21	.14	.17	.21	.21
Excitement-Seeking	.00	-.24	-.19	-.21	-.12	-.14	-.21
Perceived Intelligence	.14	.12	.15	.15	.17	.20	.20
Risk-Taking	.01	-.22	-.19	-.20	-.10	-.13	-.19
Openness to Experience	.12	.10	.28	.07	.15	.19	.19
Machiavellianism	-.01	-.16	-.15	-.14	-.08	-.09	-.14
Extraversion	.10	.05	.13	-.05	.18	.15	.11
Behavioral Tendency
Buy unneeded purchases	-.32	-.33	-.20	-.58	-.23	-.41	-.48
Lie or hide the truth	-.23	-.44	-.32	-.32	-.35	-.41	-.46
Pay bills on time	.19	.32	.19	.47	.32	.40	.43
Bend or break the rules	-.11	-.35	-.27	-.29	-.27	-.28	-.36
Eat fast food	-.37	-.23	-.15	-.25	-.17	-.34	-.33
Exercise	.32	.25	.13	.19	.27	.38	.32
Help other people	.12	.26	.44	.08	.31	.30	.32
Party excessively	-.08	-.33	-.14	-.26	-.17	-.20	-.27
Drink pop / soda	-.28	-.17	-.14	-.19	-.12	-.26	-.25
Smoke cigarettes	-.06	-.23	-.11	-.25	-.18	-.22	-.23
Think about sex	-.05	-.35	-.13	-.13	-.15	-.17	-.22
Read books	.11	.19	.19	.10	.20	.21	.21
Check financial news	.16	.13	.06	.23	.16	.23	.21
Think about food	-.39	-.13	-.02	-.13	-.05	-.14	-.21
Gamble	-.08	-.19	-.16	-.16	-.15	-.17	-.20
Discuss intellectual topics	.12	.09	.19	.13	.14	.18	.18
Watch television shows	-.18	-.10	-.06	-.08	-.06	-.12	-.13

Note. n = 7,771. F = Food, P = Physical, S = Social, M = Money, A = Achievement, 10 = DGI-10 short form composite, 35 = DGI-35 composite. To facilitate visual inspection, correlations greater than magnitude r = .30 are in bold, though all greater than magnitude r = .02 are statistically significant (p < .05).

Finally, we conducted an exploratory analysis to examine test-retest reliability. Among all participants, 151 self-reported having completed the measure more than once (repeat responses excluded in previous analyses). For a subsample of them (n = 35), we were able to identify their first and second survey submission based on their partial IP address and matching demographic characteristics. On average, the duration between testing was approximately two months (Mdn = 51.6 days, M = 74.7 days, SD = 94.6 days). Test-retest correlations were strong across scale scores: food (r = .74), physical (r = .84), social (r = .74), money (r = .90), achievement (r = .86), DGI-10 (r = .87), and DGI-35 (r = .90). Although a more detailed follow-up is needed, available evidence supports strong test-retest reliability of DGI scores.

Results support the utility of web-based scale development and provide strong psychometric evidence for DGI scores. Based on a diverse worldwide sample of over 10,000 adult respondents, we found evidence for internal consistency reliability, test-retest reliability, a theoretically-driven factor structure, measurement invariance, and construct validity. Studies 3 and 4 were designed to pick up where Study 2 left off, by examining additional validity evidence in greater depth.

Study 3

Method

Participants and Procedures

Whereas Studies 1 and 2 focused on internal consistency and factor structure, Study 3 was designed to examine test-retest reliability under more controlled circumstances, additional evidence for construct validity, and incremental validity. An undergraduate convenience sample was recruited, given the study length and lack of substantive demographic differences in gratification delay in Study 2. The study was administered in two phases, both through SurveyMonkey.com. In Phase I, participants (n = 207) completed the DGI among measures for other studies; four participants were dropped due to invalid response sets. A subset of participants (n = 64) agreed to complete Phase II two months later, completing the DGI again and other measures of personality and behavior. Participants were mainly young (Age: M = 19.3, SD = 2.4; Standing: 76.4% freshmen), female (65.6%), white (93.8%), and of average academic ability (ACT: M = 22.6, SD = 3.9). DGI scores were unrelated to age, years in school, and gender (p > .10); findings for ACT scores and other cognitive indicators are reviewed later.

Measures

Participants completed the 35-item DGI and the same single-item personality and behavior ratings used in Study 2. They also reported demographic information, including age, race, gender, high school and college GPA, and ACT score.

Self-control constructs

Participants completed three self-control measures. The 36-item Self-Control Scale (SCS; α = .88; Tangney et al., 2004) primarily taps impulse control, but also hits loosely on delay of gratification, competencies, and conscientiousness, with items like, “I am good at resisting temptation,” rated on a scale from 1 (Not at all) to 5 (Very much). The 30-item Barratt Impulsivity Scale (BIS; α = .87; Patton, Stanford, & Barratt, 1995) focuses on impulse control, but also taps broader aspects of self-control and emotional stability. Items like, “I concentrate easily” are rated on a scale from 1 (Never) to 5 (Always). Finally, the 10-item Academic Delay of Gratification Scale (ADOGS; α = .72; Bembenutty & Karabenick, 1998) involves hypothetical decisions between immediate pleasures and long-term academic goals. Participants respond to items like, “(A) Study a little everyday for an exam and spend less time with your friend, OR (B) Spend more time with your friends and cram just before the test?” using a scale from 1 (Definitely Choose A) to 4 (Definitely Choose B).

Big Five

Participant completed a 150-item version of a Big Five personality questionnaire (Goldberg, 2006). The scale measures neuroticism (α = .89), extraversion (α = .87), openness to experience (α = .79), agreeableness (α = .89), and conscientiousness (α = .92), as well as six facets for each domain. Items like, “Prefer variety to routine” are rated on a scale from 1 (Disagree) to 9 (Agree).

Results and Discussion

Basic evidence of internal consistency, test-retest reliability, and construct validity of the DGI support initial findings from Studies 1 and 2. Specifically, internal consistency and test-retest reliabilities were strong for the DGI-35 composite (α = .88, r_retest = .88) and subscale scores (average α = .77, r_retest = .76). Thus, in addition to performing well with a diverse, worldwide sample, scores on the measure also demonstrated adequate reliability in a university convenience sample. Before examining new validity evidence, DGI scores were examined for their associations with trait descriptors and behavioral tendencies previously investigated in Study 2 (for details, see Supplemental Table B10). The pattern of correlations is notably similar to that of Study 2, providing replicable evidence, albeit basic, that DGI scores are validly interpretable, even across distinct samples.

To provide additional evidence of the validity of DGI scores we examined correlates with several indicators of academic achievement, the Big Five personality domains and 30 underlying facets, and several closely-related measures of self-control (see Table 5). Correlations with other self-control measures were generally strong, and the social and achievement domains had sizeable correlations with ACT score and GPA. DGI scores were generally associated with greater conscientiousness and reduced neuroticism. Specific DGI domains were associated with aspects of agreeableness, extraversion, and openness. The DGI social domain was particularly associated with altruism.

Table 5. Correlations between Scores on the DGI, Self-Control Measures, Cognitive Indicators, and Big Five Personality Traits in Study 3.

Measure	F	P	S	M	A	10	35
Self-Control
SCS	.51	.56	.23	.53	.59	.70	.71
BIS	-.35	-.54	-.23	-.57	-.54	-.56	-.66
ADOGS	.38	.28	.23	.30	.55	.51	.51
Cognitive Indicators
ACT Score	-.15	-.04	.25	-.03	.04	.06	.02
College GPA	-.12	.10	.34	-.06	.30	.16	.15
High School GPA	-.07	.04	.15	.08	.25	.04	.13
Big Five Traits
Neuroticism	-.47	-.31	-.22	-.29	-.41	-.49	-.50
Anxiety	-.31	-.06	-.03	-.10	-.22	-.27	-.21
Anger	-.07	-.16	-.39	-.18	-.16	-.28	-.28
Depression	-.39	-.30	-.25	-.11	-.45	-.45	-.43
Self-Consciousness	-.38	-.16	-.02	-.07	-.27	-.27	-.26
Immoderation	-.59	-.38	-.06	-.55	-.34	-.52	-.57
Vulnerability	-.35	-.28	-.13	-.32	-.32	-.35	-.41
Extraversion	.19	.01	.30	.14	.35	.28	.29
Friendliness	.06	.03	.56	.15	.32	.31	.33
Gregariousness	.00	-.15	.18	-.09	.06	.03	.00
Assertiveness	.26	.04	.07	.15	.23	.19	.23
Activity level	.35	.17	-.13	.12	.37	.27	.26
Excitement seeking	.14	-.22	-.14	.05	.10	-.08	-.01
Cheerfulness	.07	.25	.60	.26	.41	.46	.46
Openness	.13	.15	.44	.14	.23	.21	.32
Imagination	-.01	-.03	.31	.20	.16	.07	.19
Artistic interests	.14	.17	.43	.13	.25	.24	.32
Emotionality	-.18	.10	.34	.03	.07	.13	.10
Adventurousness	.24	.15	.26	-.05	.11	.19	.20
Intellect	.22	.26	.20	.32	.32	.29	.39
Liberalism	.06	-.15	.05	-.19	-.13	-.19	-.11
Agreeableness	.03	.35	.74	.11	.32	.44	.44
Trust	.06	.22	.41	-.07	.34	.30	.26
Morality	.03	.39	.50	.12	.34	.40	.39
Altruism	.02	.29	.82	.11	.31	.41	.44
Cooperation	.14	.33	.62	.22	.27	.52	.46
Modesty	-.05	.12	.42	.07	.09	.11	.18
Sympathy	-.09	.23	.62	.03	.09	.26	.24
Conscientiousness	.46	.53	.19	.49	.72	.70	.71
Self-efficacy	.36	.23	.14	.42	.57	.48	.52
Orderliness	.26	.32	-.07	.37	.34	.42	.36
Dutifulness	.27	.46	.44	.36	.52	.58	.60
Achievement-striving	.32	.28	.19	.36	.68	.47	.54
Self-discipline	.52	.52	.04	.26	.57	.58	.55
Cautiousness	.29	.52	.21	.45	.53	.57	.59

Note. n = 64. F = Food, P = Physical, S = Social, M = Money, A = Achievement, 10 = DGI-10 short form composite, 35 = DGI-35 composite. To facilitate visual inspection, statistically significant correlations are bold (p < .05).

Scores on the DGI also showed solid evidence for incremental validity in predicting a broad range of outcomes, including well-being and health-related behaviors. Specifically, we examined how well DGI scores predicted relevant outcomes over and above five closely-related “rival” measures, including the SCS, BIS, ADOGS, Big Five Agreeableness domain, and Big Five Conscientiousness domain. Incremental validity was examined using two analytic approaches. First, we examined whether any of the DGI scores outperformed all five rival measures in predicting any of the 47 traits and behavioral tendencies listed in Table 4 (and Supplemental Table B10). Given that the DGI is composed of 35 items and the rival measures collectively consisted of 136 items, evidence for incremental validity would be notable in these statistically conservative analyses. For 57% of the trait descriptors, at least one of the DGI scales had a higher correlation than all rival scales. Significantly, for 65% of the behavioral tendencies, at least one of the DGI scales outperformed each of the five rival measures. Thus, the DGI correlated more highly with a wide range of theoretically-related constructs than did a number of rival measures that consisted collectively of a much larger item pool.

Hierarchical multiple regression was used to examine incremental validity in predicting well-being and health-related variables. Outcomes variables examined were the single-item ratings of well-being, perceived physical health, frequency of exercise, frequency consuming fast food, and frequency of smoking cigarettes. DGI scores showed incremental validity in predicting well-being, fast food consumption, and smoking, but not perceived health or exercise frequency. In particular, the five rival measures accounted for 34% of the variance in well-being, and the DGI achievement domain accounted for 5% of incremental variance, ΔR² = .05, ΔF(1,57) = 4.00, p < .05. Additionally, the five comparison measures accounted for 10% of the variance in fast food consumption, but the DGI physical domain accounted for an additional 9% of incremental variance, ΔR² = .09, ΔF(1,57) = 6.03, p < .05. Finally, the five comparison measures explained 15% of the variance in smoking, and the DGI money domain accounted for an additional 7% of incremental variance in reported smoking behavior, ΔR² = .07, ΔF(1,57) = 4.89, p < .05. Results involving incremental validity in predicting well-being and health-related behaviors are particularly notable given that variance in the dependent variables was likely attenuated by both the use of single-item measures as well as the relative homogeneity of the undergraduate sample on health-related variables.

In summary, Study 3 extended upon the two previous DGI studies in several ways. This study replicated evidence for core psychometric properties of the scale's scores, including internal consistency, test-retest reliability, and construct validity. The DGI was shown to correlate well with scores on existing survey measures of self-control and the Big Five, providing stronger evidence for construct validity than could be obtained in Study 2. Additionally, although the DGI correlated well with related constructs, it afforded incremental validity in predicting self-reports of well-being and health-related behaviors. Notwithstanding these strengths, evidence linking DGI scores to adjustment and psychopathology would provide greater support for its applied use.

Study 4

Method

Participants and Procedures

Study 4 was designed to gather additional evidence of the validity of DGI scores, with a greater focus on correlates related to adjustment and psychopathology. Due to proprietary restrictions, the Minnesota Multiphasic Personality Inventory-2-Restructured Form (MMPI-2-RF; Ben-Porath & Tellegen, 2008) had to be administered in person; therefore, the study was divided into two phases. First, 293 undergraduate participants completed the majority of study measures online via SurveyMonkey.com. Second, a subset of 58 participants agreed to complete the MMPI-2-RF in a laboratory session two weeks later. We used validity items embedded within phase one measures as well as MMPI-2-RF validity profile analysis (for guidelines, see Ben-Porath & Tellegen, 2008) to eliminate a few invalid responders. The vast majority of phase one (n = 286) and MMPI-2-RF (n = 56) participants responded validly and were retained for analyses. Participants were primarily young (Age: M = 19.7, SD = 2.1; Standing: 51% freshmen), female (65.1%), and white (90.1%). The DGI social subscale scores increased slightly with age, r = .12, p = .04. Additionally, females scored higher than males on the DGI-35 composite (d = 0.11, t(282) = 4.09, p < .001) and physical, social, and achievement subscales (average d = 0.13).

Measures

The DGI was administered again, and internal consistency remained adequate for the DGI-35 composite (α = .87) and subscale scores (average α = .73). Demographic information was also collected, including age, race, and gender.

Self-control constructs

To provide additional evidence for convergent validity, the DGI was compared to three related measures. The 12-item Deferment of Gratification Questionnaire (DGQ; α = .71; Ray & Najman, 1986) assesses gratification delay with items such as, “Do you fairly often find that it is worthwhile to wait and think things over before deciding?” The yes-no response format was altered to a scale from 1 (Completely Untrue) to 9 (Completely True). Ego-resiliency and impulse control were measured with Letzring, Block, and Funder's (2004) 14-item Ego-Resiliency Scale (ERS; α = .74) and 37-item Ego-Undercontrol Scale (EUS; α = .85). Items include, “I usually think carefully about something before acting” and “I tend to buy things on impulse,” rated on a scale from 1 (Does not apply at all) to 4 (Applies very strongly).

Food-related problems

Problematic eating behavior was measured using the 33-item Dutch Eating Behavior Questionnaire (DEBQ; α = .93; van Strien, Frijters, Berger, & Defares, 1986). Items like “Do you have a desire to eat when you are bored or restless?” are rated on a scale from 1 (Never) to 5 (Very Often), and the scale produces three composite scores, including restrained eating, emotional eating, and external eating (eating when snacks are more readily available).

Physically risky behavior

Risky behaviors involving sex, drugs, and alcohol were measured using 30 items adapted from the Add Health Questionnaire (α = .82; Resnick et al., 1997), such as “Have you ever used chewing tobacco?” and “Have you ever paid someone for sex?”

Social problems

The 32-item Inventory of Interpersonal Problems – Short Circumplex form (α = .92; Soldz, Budman, Demby, & Merry, 1995) was used to measure social adjustment problems along eight dimensions: domineering, vindictive, cold, socially avoidant, nonassertive, exploitable, overly nurturant, and intrusive. Items like, “I keep other people at a distance too much,” were rated on a scale from 1 (Completely Disagree) to 5 (Completely Agree).

Money problems

Financial problems were assessed using 10 items adapted from the Add Health Questionnaire (α = .57; Resnick et al., 1997), including, “In the past 12 months, was there a time when you had to borrow money from a friend?” and “Do you have any credit card debt?”

Achievement problems

The 25-item Academic Maladjustment scale from the Student Adaptation to College Questionnaire (α = .86; Baker & Siryk, 1989) measured achievement problems, using items like, “I am attending classes regularly,” rated from 1 (Applies very closely to me) to 9 (Doesn't apply to me at all).

Psychopathology

The 338-item Minnesota Multiphasic Personality Inventory-2-Restructured Form (MMPI-2-RF; Ben-Porath & Tellegen, 2008) measures psychopathology along 50 overlapping dimensions, using a true-false format. The MMPI-2-RF was administered in a group setting two weeks after the initial measures were completed online.

Results and Discussion

Scores on the DGI were related to those on a variety of other measures of self-control, adjustment, and psychopathology. As hypothesized, the DGI demonstrated continued evidence of construct validity in correlating well with self-control measures, and domains were related to adjustment problems in theoretically-meaningful ways (see Table 6). The food domain was associated with binge eating and idle snacking (external eating). The physical domain was related to substance use and number of sexual partners. The social domain predicted a broad range of interpersonal problems, though several domains had relevant correlates. The money domain was mildly predictive of problematic personal financial behaviors as well as substance use. Further, the achievement domain was strongly related to academic adjustment.

Table 6. Correlations between Scores on the DGI, Self-Control Measures, and Adjustment Measures in Study 4.

Measure	F	P	S	M	A	10	35
Self-Control
DGQ	.36	.47	.31	.73	.39	.58	.67
Ego Undercontrol Scale	-.29	-.44	-.28	-.38	-.27	-.40	-.48
Ego Resiliency Scale	.12	.10	.27	.01	.28	.27	.21
Eating Pathology (DEBQ)
Restrained Eating	.10	.05	.13	.04	.07	.12	.11
Emotional Eating	-.39	-.09	-.05	-.17	-.08	-.19	-.23
External Eating	-.39	-.30	-.09	-.19	-.08	-.20	-.30
Physically Risky Behavior (AHQ)
Substance Problems	-.18	-.48	-.13	-.22	-.15	-.25	-.33
Cigarette Smoking	-.10	-.26	-.03	-.14	-.14	-.23	-.20
Alcohol Use	-.15	-.38	-.14	-.19	-.19	-.23	-.30
Marijuana Use	-.03	-.27	-.06	-.16	-.09	-.15	-.18
Risky Sexual Behavior	-.08	-.16	-.09	-.15	-.10	-.12	-.17
Frequency of Sex	-.02	-.14	.03	-.16	-.01	-.04	-.09
Number of Sexual Partners	-.09	-.32	-.17	-.14	-.09	-.14	-.22
Social Problems (IIPSC)
Domineering	-.31	-.23	-.26	-.07	-.13	-.26	-.28
Vindictive	-.27	-.25	-.29	-.08	-.19	-.27	-.30
Cold	-.17	-.16	-.24	.04	-.18	-.24	-.19
Socially Avoidant	-.10	-.10	-.12	.01	-.20	-.22	-.14
Nonassertive	-.06	-.06	.11	.05	-.04	-.02	.00
Exploitable	-.07	-.10	.10	-.06	-.08	-.10	-.07
Overly Nurturant	-.07	.02	.20	-.03	.00	-.03	.02
Intrusive	-.17	-.26	-.13	-.16	-.12	-.17	-.24
Monetary Distress (AHQ)
Financial Problems	-.07	-.15	-.04	-.17	-.03	-.06	-.14
Borrowed Money	-.14	-.16	.00	-.17	-.10	-.09	-.17
Paid Research Subject	-.11	-.17	-.03	-.25	-.09	-.14	-.20
Achievement Problems (SACQ)
Academic Maladjustment	-.17	-.34	-.29	-.25	-.62	-.48	-.48

Note. n = 284. F = Food, P = Physical, S = Social, M = Money, A = Achievement, 10 = DGI-10 short form composite, 35 = DGI-35 composite, DGQ = Deferment of Gratification Questionnaire, DEBQ = Dutch Eating Behavior Questionnaire, AHQ = Add Health Questionnaire, IIPSC = Inventory of Interpersonal Problems - Short Circumplex, SACQ = Student Adaptation to College Questionnaire. To facilitate visual inspection, statistically significant correlations are bold (p < .05).

DGI scores were also associated with scores on the MMPI-2-RF (see Table 7). In general, DGI scores were associated with scores indicating positive psychological adjustment, as noted by associations with validity indices and negative correlations with substantive scales. In particular, DGI scores were powerfully predictive of decreased externalizing symptoms, as measured by the BXD (Behavioral/Externalizing scale), RC4 (Antisocial Behavior scale), JCP (Juvenile Conduct Problems scale), SUB (Substance Abuse scale), and related scales. Correlates varied across domains. Those scoring high on the food domain were less likely to have substance problems, atypical or unhelpful thinking patterns, hypomanic symptoms, and behavior problems. They were also more likely to be introverted. The physical domain was robustly related to fewer externalizing problems (e.g., r = -.57 with JCP and r = -.51 with BXD scores), better self-control, fewer hypomanic symptoms, and less anger, aggression, and resentment. The social domain was solely related to externalizing behaviors, and the money domain to substance abuse. Finally, the achievement domain was associated with reduced symptoms of depression, increased activation, fewer juvenile conduct problems, and more cerebral interests. Results demonstrate the utility of the DGI in predicting adjustment issues and follow-up studies involving clinical samples are warranted.

Table 7. Correlations between Scores on the DGI and MMPI-2-RF Psychopathology Scales in Study 4.

MMPI-2-RF Scale	F	P	S	M	A	10	35
Response Validity Scales
Variable Responding (VRIN-r)	-.05	-.09	-.05	.17	.12	.05	.03
True Response Inconsistency (TRIN-r)	-.14	-.02	-.26	.08	.12	-.03	-.05
Infrequent Responses (F-r)	-.03	-.31	-.07	.02	-.07	-.14	-.13
Infrequent Psychopathology Resp (Fp-r)	-.17	-.25	-.21	.15	-.07	-.06	-.15
Infrequent Somatic Responses (Fs)	.01	-.02	.07	.09	.09	-.01	.06
Symptom Validity (FBS-r)	.04	.00	.17	.20	.07	.13	.14
Uncommon Virtues (L-r)	.31	.35	.05	.19	.06	.24	.29
Defensiveness (K-r)	.30	.19	.06	.03	.14	.17	.21
Higher Order Scales
Emotional/Internalizing (EID)	-.01	-.05	.11	.05	-.12	-.04	-.01
Thought Dysfunction (THD)	-.33	-.18	.13	-.06	.19	-.08	-.10
Behavioral/Externalizing (BXD)	-.19	-.51	-.31	-.23	-.20	-.29	-.41
Restructured Clinical Scales
Demoralization (RCd)	-.13	-.06	.00	.07	-.14	-.08	-.07
Somatic Complaints (RC1)	-.02	-.11	-.06	.14	.01	-.06	-.01
Low Positive Emotions (RC2)	.11	-.04	.10	.08	-.31	-.02	-.01
Cynicism (RC3)	-.29	-.26	-.13	-.14	-.11	-.17	-.27
Antisocial Behavior (RC4)	-.31	-.49	-.36	-.23	-.09	-.36	-.42
Ideas of Persecutions (RC6)	-.35	-.36	-.05	-.01	.02	-.20	-.23
Dysfunctional Negative Emotions (RC7)	-.28	-.08	.00	.07	.14	-.08	-.05
Aberrant Experiences (RC8)	-.30	-.22	.09	-.10	.06	-.12	-.15
Hypomanic Activation (RC9)	-.35	-.36	-.19	-.12	-.01	-.20	-.30
Personality Psychopathology Five
Aggressiveness (AGGR-r)	-.14	-.21	-.14	-.13	.07	-.11	-.16
Psychoticism (PSYC-r)	-.22	-.13	.09	-.03	.07	-.04	-.08
Disconstraint (DISC-r)	-.12	-.46	-.23	-.21	-.17	-.26	-.34
Negative Emotionality (NEGE-r)	-.16	-.09	.05	-.02	.18	-.05	-.03
Introversion (INTR-r)	.28	.10	-.03	.09	-.24	.05	.08
Somatic
Malaise (MLS)	-.01	-.03	.17	.20	-.04	.13	.08
Gastro-Intestinal Complaints (GIC)	-.05	-.12	.00	.11	-.07	-.04	-.03
Head Pain Complaints (HPC)	.05	-.13	.13	.01	.02	.02	.02
Neurological Complaints (NUC)	-.20	-.26	-.23	.02	-.07	-.16	-.20
Cognitive Complaints (COG)	-.11	-.17	.05	.05	-.05	-.08	-.07
Internalizing
Suicidal/Death Ideation (SUI)	-.14	-.10	-.02	.14	.01	.03	-.03
Helplessness/Hopelessness (HLP)	-.11	-.23	.02	.05	-.27	-.12	-.15
Self-Doubt (SFD)	-.08	.06	.09	.06	-.08	-.06	.02
Inefficacy (NFC)	-.23	-.09	-.05	-.10	-.13	-.17	-.17
Stress/Worry (STW)	-.09	-.04	.08	-.07	.06	.00	-.03
Anxiety (AXY)	.02	-.07	.24	.23	.10	.08	.14
Anger Proneness (ANP)	-.22	-.30	-.19	-.07	-.07	-.23	-.24
Behavior-Restricting Fears (BRF)	-.02	.07	.08	.15	.12	.04	.11
Multiple Specific Fears (MSF)	-.23	-.13	.13	-.08	-.02	.03	-.11
Externalizing
Juvenile Conduct Problems (JCP)	-.27	-.57	-.40	-.18	-.33	-.36	-.49
Substance Abuse (SUB)	-.37	-.49	-.26	-.30	.06	-.33	-.40
Aggression (AGG)	-.17	-.31	-.14	-.14	-.07	-.18	-.24
Activation (ACT)	-.35	.03	.19	.01	.35	.05	.04
Interpersonal Scales
Family Problems (FML)	-.09	.10	.10	.14	.18	.03	.12
Interpersonal Passivity (IPP)	.11	.10	.04	.12	-.20	-.04	.06
Social Avoidance (SAV)	.14	.13	-.09	.12	-.13	.04	.07
Shyness (SHY)	-.09	-.07	-.16	-.10	-.17	-.17	-.16
Disaffiliativeness (DSF)	.21	.10	-.15	.08	-.14	-.05	.05
Interest Scales
Aesthetic-Literary Interests (AES)	-.16	.01	.10	.16	.28	.15	.10
Mechanical-Physical Interests (MEC)	.22	-.18	-.12	-.05	-.23	-.10	-.09

Note. n = 56. MMPI-2-RF = Minnesota Multiphasic Personality Inventory-2-Restructured Form, F = Food, P = Physical, S = Social, M = Money, A = Achievement, 10 = DGI-10 short form composite, 35 = DGI-35 composite. Parenthetical acronyms refer to official MMPI-2-RF scale names. To facilitate visual inspection, statistically significant correlations are bold (p < .05).

General Discussion

The present investigation drew upon six decades of research to shape the development of the first known theoretically-driven five-factor measure of individual differences in the tendency to delay gratification, the Delaying Gratification Inventory (DGI). The DGI was designed to provide practical and psychometric advantages over past methods of measurement (Mauro & Harris, 2000; Nederkoorn et al., 2006; Smith & Hantula, 2008), thereby accelerating social and behavioral public health research (DeWall et al., 2007; Gottdiener et al., 2008; NIH, 2009; Seeyave et al., 2009). Our innovative web-based strategy to survey development and validation bolstered the recruitment of over 10,000 adult participants worldwide, diverse in age, race, and educational attainment. Scores on the resulting scale showed strong internal consistency, test-retest reliability, factor structure, and construct validity. Further, findings have implications for clinical and public health strategies, follow-up studies, and test construction projects.

Consistent with the hypothesized multidimensionality of gratification delay, the study procedures produced a psychometrically strong survey measuring five domains of delay behavior, involving food, physical pleasures, social interactions, money, and achievement. Across studies and samples, scores on the DGI-35 composite, DGI-10 short form, and five domains met conventional standards for internal consistency and test-retest reliability. The full 35-item form clearly provides the richest set of information, but the reliability of the short form composite supports its stand-alone use for studies in which a lengthier measure may be overly burdensome. Additionally, the theoretically-derived five-factor model fit the data well and was relatively robust when examined across various subgroups, showing little measurement invariance when constrained by factor structure, factor loadings, or the factor variance-covariance matrix.

The four studies in the present investigation clearly demonstrate that DGI composite and domain scores correlate with theoretically-relevant traits, behavioral tendencies, adjustment problems, and psychopathology symptoms. People who generally delay gratification well, as indicated by the DGI-35 composite score, also scored highly on other measures of self-control, conscientiousness, self-discipline, and achievement-striving, supporting the basic construct validity of the measure. Additionally, delaying gratification was modestly associated with improved health and well-being, increased exercise, better diet, altruistic and agreeable traits, and openness to experience. Those scoring high on delay of gratification also have somewhat reduced levels of binge eating, neuroticism, depression, anxiety, anger, rebelliousness, sensation seeking, substance use, risky sexual behavior, interpersonal problems, externalizing problems, and hypomanic symptoms. These results corroborate and extend upon prior findings relating gratification delay to improved psychosocial adjustment (Bembenutty & Karabenick, 2004; Funder & Block, 1989; Mischel & Mischel, 1983; Funder & Block, 1989; Funder et al., 1983; Lee et al., 2008; Ramanathan & William, 2007). The general pattern of correlations was similar, though slightly attenuated, for scores on the DGI-10 short form composite scale, indicating its utility when administration of a lengthier scale might be untenable. Additionally, correlates of DGI domain scores were distinct and varied in theoretically-meaningful ways. The food domain was related to dietary habits, preoccupation with food, and activity level. The physical domain was the greatest predictor of externalizing behaviors, including aggression, sensation-seeking, substance use, risky sexual behavior, and rule breaking. The social domain was related to altruism, interpersonal warmth, open-mindedness, and prosocial behavior. The money domain was related to splurging, paying bills on time, and financial distress. Finally, the academic domain was related to conscientiousness, achievement-striving, academic adjustment, and well-being.

The present investigation must be qualified by several important limitations. One, findings were based primarily on a broad, diverse sample of general adults, and no claims are made about the generalizability of DGI correlates to specialized or clinical populations. Studies linking gratification delay to specific physical and mental health problems are encouraged, particularly given that findings involving the MMPI-2-RF were restricted to an undergraduate sample. Two, outcome measures relied upon self-report. The incorporation of structured behavioral observations, medical record data, or other non-self report methods would provide additional tests of validity. Three, although test-retest correlations provide evidence for the stability of DGI scores, this investigation does not directly address the degree to which gratification delay is modifiable, indicating that prospective intervention studies on emotional skill development are warranted. Four, additional evidence documenting the association between DGI money scores and significant real-world financial behavior is needed. Potential avenues include predicting business success, bankruptcy, credit card debt, and retirement planning.

Study limitations are balanced by several strengths worth noting. First, the use of large, diverse worldwide samples in Studies 1 and 2 afforded substantial statistical power and facilitated the generalizability of the resulting scale. Second, the technico-ethical rigor of the study web site reinforced APA ethical guidelines for Internet-mediated research (Hoerger & Currell, in press), facilitated participant recruitment and enjoyment, and suggests strategies for future web-based researchers. Third, the DGI was designed for flexible and widespread use. The measure is non-proprietary, and clinicians and researchers have the option to administer the 35-item inventory or 10-item short form, depending on their constraints and interests.

In conclusion, we hope the development of the DGI will provide a significant step forward in research on gratification delay. Scores on the DGI have strong psychometric properties and can be administered efficiently to diverse samples of adults. Regarding the investigation's clinical and public health implications, the DGI could be used by trained individual therapists, Department of Social Services workers, school counselors, community social workers, prison group therapists, Alcoholics Anonymous, occupational rehabilitation programs, and pastoral counselors to identify individuals at risk for particular adjustment problems and better route clients to appropriate intervention services. The scale can also be incorporated into public health studies attempting to predict academic achievement, externalizing behavior problems, psychopathology, consumer financial planning, and healthcare decision making. Finally, the methodological approach suggests avenues for efficient, low-cost survey development.