Assessing the Psychometric Properties of Two Food Addiction Scales

Abstract

Background

While food addiction is well accepted in popular culture and mainstream media, its scientific validity as an addictive behavior is still under investigation. This study evaluated the reliability and validity of the Yale Food Addiction Scale and Modified Yale Food Addiction Scale using data from two community-based convenience samples.

Methods

We assessed the internal and test-retest reliability of the Yale Food Addiction Scale and Modified Yale Food Addiction Scale, and estimated the sensitivity and negative predictive value of the Modified Yale Food Addiction Scale using the Yale Food Addiction Scale as the benchmark. We calculated Cronbach’s alphas and 95% confidence intervals (CIs) for internal reliability and Cohen’s Kappa coefficients and 95% CIs for test-retest reliability.

Results

Internal consistency (n=232) was marginal to good, ranging from α=0.63 to 0.84. The test-retest reliability (n=45) for food addiction diagnosis was substantial, with Kappa=0.73 (95% CI, 0.48–0.88) (Yale Food Addiction Scale) and 0.79 (95% CI, 0.66–1.00) (Modified Yale Food Addiction Scale). Sensitivity and negative predictive value for classifying food addiction status were excellent: compared to the Yale Food Addiction Scale, the Modified Yale Food Addiction Scale’s sensitivity was 92.3% (95% CI, 64%–99.8%), and the negative predictive value was 99.5% (95% CI, 97.5%–100%).

Conclusions

Our analyses suggest that the Modified Yale Food Addiction Scale may be an appropriate substitute for the Yale Food Addiction Scale when a brief measure is needed, and support the continued use of both scales to investigate food addiction.

1. Introduction¹

While food addiction has a prominent presence in popular culture, it is not currently included in the Diagnostic and Statistical Manual of Mental Disorders (DSM). Its scientific validity as a mental disorder and addictive behavior is still under investigation. To empirically examine the prevalence and validity of food addiction and whether certain eating behaviors are indicative of addiction, consistent and accurate operationalized measures (e.g., scales) of the construct are needed.

If food addiction is truly an addictive behavior, it should manifest as the compulsive relationship between eating (a behavior) and positively reinforcing foods (the substance) and associated neurological manifestations.^{1, 2} Consistent with this operationalization, studies^{3–15,16, 17} suggest that animals consume certain types of foods in an addictive manner, and more than forty studies in humans suggest that the prevalence of food addiction ranges from 5%^{18, 19} in the general population to over 40%^20–23 among obese individuals. Findings indicate that food addiction is positively associated with binge eating behaviors,^{20, 24–27} depression,^{24, 26–28} food cravings,^{25, 26, 29} and impulsivity.²⁶ Many of these studies suggest that food addiction is associated with other theoretically-related constructs, which contributes evidence for its validity as a psychiatric disorder.

Researchers have generally used two scales to measure food addiction in adults—the 25-item Yale Food Addiction Scale (YFAS) and the 9-item Modified Yale Food Addiction Scale (mYFAS). Several studies^{26, 30–33} suggest that the YFAS has moderate to good psychometric properties—including internal consistency, convergent validity and discriminant validity. However, there is limited investigation of how the scale performs over time; one recent study³³ examined its test-retest reliability and found moderate agreement over 18 months (κ=.50, 95% confidence interval (CI) (0.23–0.77)). To date, the mYFAS—an abridged version of the YFAS—has been included in far fewer studies than the YFAS. The Nurses’ Health Study (NHS) cohorts piloted this shorter scale, and three additional studies^{18, 34, 35} have been published using this measure; however, only one¹⁸ reported on its psychometric properties.

Using data from two community-based convenience samples, we assessed the internal consistency and test-retest reliability of the YFAS and mYFAS and the sensitivity and negative predictive value of the mYFAS using the YFAS as the benchmark.

2. Materials and methods

2.1 Samples: Yale Health Behaviors Surveys 2008 and 2010

Researchers at Yale University created the 2008 and 2010 Yale Health Behaviors Surveys to examine alcohol, smoking, and obesity-related behaviors.^{36, 37} The research team recruited participants using flyers posted around campus and other locations throughout New Haven, Connecticut. The team also recruited through online Craigslist postings for the 2010 survey. The 2008 survey recruited 235 participants 18 years and older, while the 2010 survey recruited 51 participants aged 18–30 years.

In the 2008 survey, participants came to a lab in the Department of Psychology at Yale where they provided consent, completed an hour-long online questionnaire and height and weight were measured. Participants answered questions about food addiction using the YFAS, and other health behaviors (e.g., nicotine and illicit drug use, alcohol consumption, eating, and gambling), family history of problems with alcohol and drug use, and demographic information. Participants were compensated $10.³⁷

The 2010 survey was designed as a test-retest reliability study. Participants were informed that the purpose of the study was to better understand the relative stability of several health-related behaviors over time.³⁶ At the first session, subjects provided consent and completed a series of computerized self-report questions about food addiction using the YFAS, alcohol use, cigarette smoking and eating, and had their height and weight measured. They returned two weeks later to have height and weight measured and complete the surveys administered at the first session. Participants earned $5 at the first visit and $15 at the second.

The research team stored the data for the 2008 and 2010 surveys at Yale on a password-protected computer file, and the Human Subjects Committee of Yale University approved both studies.³⁶

2.2 Measures

2.2.1 The Yale Food Addiction Scale

Gearhardt and colleagues³⁰ developed the YFAS to identify individuals who reported symptoms of substance dependence based on food as the substance of abuse.³⁰ They adapted the DSM-IV-R criteria for substance dependence with items assessing addictive-like eating in the past twelve months. Each of the 25 items taps into one of the seven criteria for substance dependence. For example, items 1–3 assess “substance taken in larger amount and for longer period than intended,” and items 4, 22, 24, and 25 assess “Persistent desire or repeated unsuccessful attempt to quit.” The scale evaluates clinical significance through two items about whether eating behavior causes significant impairment or distress. Experts in addiction and binge eating as well as patients in treatment for binge eating disorder reviewed and approved the items proposed for the scale. Based on the YFAS, food addiction status is met if a person endorses at least 3 of the 7 dependence symptoms and meets criterion for clinical significance (i.e., impairment and/or distress).

In the first reliability and validity study³⁰ of the YFAS, researchers randomly selected 1,440 students from the roster of all students enrolled at Yale University in 2007. Twenty-five percent of students (n=353) initiated the survey. Students answered questions about food addiction, eating behaviors, alcohol consumption, gambling, and smoking. In this validation study, the scale showed good internal consistency reliability (Cronbach’s α=0.86), moderate to good convergent validity (r=0.46 to 0.61, p=0.01) with measures of similar constructs (emotional eating and eating troubles scores, respectively), and good discriminant validity (low correlations between diagnostic and food addiction symptom scores and alcohol problems, r=0.16 and 0.17).³⁰ While this study provided some indication of the scale’s reliability and validity, it did not evaluate the test-retest reliability of the scale.

2.2.2 The Modified Yale Food Addiction Scale

The mYFAS includes nine of the 25 items in the YFAS. Harvard Medical Center piloted the mYFAS in the 2008 and 2009 follow-up studies of the NHS and NHSII cohorts. Researchers chose one item for each of the seven diagnostic criteria for substance dependence and included two items to assess clinical significance. Unlike the version of the mYFAS used in this study, they slightly modified and shortened the wording of each item. If a person endorses at least 3 of the 7 dependence symptoms and meets criterion for clinical significance, the person meets food addiction status (same as the YFAS).

Harvard researchers tested the reliability and validity of the mYFAS in the same sample of 353 Yale college students described above.¹⁸ The students did not fill out a shortened version of the scale; rather, the researchers created the mYFAS by including the same nine items from the YFAS corresponding to the items in the NHS version of the mYFAS. They found that the mYFAS estimated a food addiction prevalence of 9.0% compared with 11.4% found by the YFAS.¹⁸ The internal consistency was α=0.75, convergent validity with similar constructs (emotional eating and difficulty eating) ranged from r=0.40 to 0.50, and correlations with discriminant measures were similar to those of the YFAS (−0.04 to 0.27).¹⁸ These psychometric properties are similar to those of the YFAS, although the mYFAS estimated a slightly lower prevalence of food addiction and slightly lower internal consistency reliability.

Participants in the 2008 and 2010 Yale Health Behaviors Surveys (used for the current analyses) filled out the 25-item YFAS. The mYFAS was created by including the nine items from the YFAS corresponding to the nine items in the NHS version of the mYFAS.

2.3 Data Analysis

Using the 2008 Yale Health Behavior Survey, the prevalence of food addiction and the internal consistency of the YFAS and mYFAS were calculated separately for men and women. Internal consistency reliability was calculated for the seven substance dependence symptoms by calculating Cronbach’s alphas and 95% CIs. We also estimated the sensitivity and negative predictive value and 95% CIs³⁸ of the mYFAS by comparing its estimate of food addiction status to that made by the YFAS (the benchmark).

The 2010 survey was used to evaluate the test-retest reliability of both scales by calculating Cohen’s Kappa coefficients and 95% CIs for food addiction status from time 1 to 2. Test-retest Kappas were also calculated for the seven substance dependence symptom clusters in the YFAS.

Standard thresholds of Cronbach’s alpha≥.70,³⁹ Kappa≥.61,⁴⁰ and sensitivity and negative predictive values>0.70⁴¹ were used to indicate acceptable internal consistency, test-retest reliability, and sensitivity and negative predictive value, respectively.

All analyses were conducted in Stata/MP 11.0 for Mac.

3. Results

3.1 Descriptive statistics

A total of 235 and 51 individuals participated in the 2008 and 2010 surveys, respectively. Three participants in 2008 and six in 2010 did not provide food addiction data and were excluded from analyses. According to the YFAS, the prevalence of food addiction was 5.6% in the 2008 survey and 11.0% in the 2010 survey. The majority of both samples was female, 18–25 years of age, and college-educated. Fifty percent of the participants in both surveys were Caucasian, and about 30% were Asian or African American. The remainder were Hispanic or of mixed race/ethnicity (Table 1).

Table 1.

Characteristics of 2008 and 2010 Yale Health Behavior Surveys

	2008		2010
	(n=232)		(n=45)
	n	%	n	%
Food Addiction Statusa
No	219	94.4	40	89.0
Yes	13	5.6	5	11.0
Age (years)
18–25	174	76.0	33	73.3
26–35	42	18.3	12	26.7
36+	13	5.7	0	0.0
Gender
Male	96	41.6	22	48.9
Female	135	58.4	23	51.1
Education
≤ High
School/Vocational	32	13.8	8	17.8
Some or all of College	157	67.7	24	53.3
≥ Graduate School	43	18.5	13	28.9
Race/ethnicity
Other/Unknown	5	2.2	0	0.0
Mixed	18	7.8	5	11.1
African American	23	10.0	6	13.3
Hispanic	14	6.0	2	4.4
Asian	56	24.2	8	17.8
Caucasian	115	49.8	24	53.3
Body Mass Index (kg/m2)
19.7– ≤ 24.9	178	78.5	20	44.4
25–29.9 (Overweight)	36	15.8	16	35.6
30+ (Obese)	14	6.1	9	20.0

^aFood Addiction Status According to Yale Food Addiction Scale

3.2 Food Addiction Prevalence and Scale Reliability and Validity

3.2.1. Prevalence

Prevalence estimates for the YFAS and mYFAS were similar in the 2008 survey; 5.6% (95% CI, 2.6%–8.6%) for the YFAS and 5.2% (95% CI, 2.3%–8.0%) for the mYFAS. The prevalence was roughly twice as high for women as men (Table 2).

Table 2.

Food Addiction Scale Prevalence and Reliability in the 2008 (n=232) and 2010 (n=45) Yale Health Behavior Surveys

		Yale Food Addiction Scale	Modified Yale Food Addiction Scalea
2008
Prevalenceb		5.6 (2.6–8.6)	5.2 (2.3–8.0)
Men		3.1 (0.0–6.6)	3.1 (0.0–6.6)
Women		7.4 (2.9–11.9)	6.6 (2.4–10.9)
Internal Consistency
Reliabilityc		0.84 (0.76–0.91)	0.67 (0.54–0.79)
Men		0.81 (0.45–1.17)	0.63 (0.35–0.91)
Women		0.84 (0.77–0.92)	0.68 (0.54–0.82)
2010 Test Re–Test Reliabilityd		0.73 (0.48–0.88)	0.79 (0.66–1.00)

^aThis scale has 9 of the 25 items in the Yale Food Addiction Scale

^bPercent and 95% confidence intervals

^cCronbach's alpha and 95% confidence intervals

^dCohen's Kappa coefficients and 95% confidence intervals

3.2.2. Reliability

In the 2008 survey, the YFAS and mYFAS had marginal to good internal consistency reliability for the YFAS seven dependence symptoms (Table 2).

The internal consistency of the YFAS seven dependence symptoms was good (α=0.84; 95% CI, 0.76–0.91). The internal consistency of the mYFAS marginal (α=0.67; 95% CI, 0.54–0.79). When stratified by gender, the reliability was somewhat higher for women than men; however, the confidence intervals overlapped appreciably.

In the 2010 survey, both scales showed good test-retest reliability for food addiction status (Table 2). The test-retest reliability of the YFAS between times 1 and 2 was Kappa=0.73; 95% CI, 0.48–0.88, indicating substantial agreement over time.⁴⁰ The test-retest reliability of the mYFAS was Kappa=0.79; 95% CI, 0.66–1.00, also indicating substantial agreement over time. However, small samples and wide confidence intervals indicate that these estimates are not precise and should be interpreted with caution.

The test-retest reliability Kappa values for the seven food dependence symptoms ranged from 0.40 to 0.76 (Table 3), demonstrating moderate to substantial agreement.

Table 3.

Test Re-Test Reliability Estimates for Food Addiction Dependence Symptoms using the Yale Food Addiction Scale in the 2010 Yale Health Behavior Survey (n=45)

	Cohen's Kappa (95% CI)	Items included in YFAS symptom cluster
Food Addiction Dependence Symptoms
1 Substance taken in larger amount and for     longer period than intended	0.67 (0.54–0.91)	1, 2, and 3
2 Persistent desire or repeated unsuccessful     attempt to quit	0.59 (0.44–0.68)	4, 22, 24, and 25
3 Much time/activity to obtain, use, recover	0.76 (0.50–0.86)	5, 6, and 7
4 Important social, occupational, or recreational     activities given up or reduced	0.40 (0.39–0.50)	8, 9, 10, and 11
5 Characteristic withdrawal symptoms; substance     taken to relieve withdrawal	0.48 (0.14–0.85)	12, 13, and 14
6 Use continues despite knowledge of adverse     consequences	0.63 (0.48–0.76)	19
7 Tolerance (marked increase in amount; marked     decrease in effect)	0.44 (0.35–0.57)	20, 21

CI, Confidence Interval; YFAS, Yale Food Addiction Scale

Questions 17, 18 and 23 in the YFAS are primer questions and are not scored

Landis and Koch interpretation of Kappa: <0.00 = poor agreement; 0.00–0.20 = slight agreement; 0.21–0.40 = fair agreement; 0.41–0.60 = moderate agreement; 0.61–0.80 = substantial agreement; and 0.81–1.00 = almost perfect agreement

According to benchmarks proposed by Landis and Koch,⁴⁰ Symptom 4 had fair agreement, Symptoms 2, 5, and 7 had moderate agreement, and Symptoms 1, 3, and 6 had substantial agreement over time.

3.2.3. Validity: Comparison of the Modified Yale Food Addiction Scale to the Yale Food Addiction Scale

Using the YFAS as the standard, the mYFAS had excellent sensitivity and negative predictive value in the 2008 survey. Of the 13 people identified with food addiction by the YFAS, the modified scale correctly identified 12. The modified scale’s sensitivity was 92.3%; 95% CI, 64%–99.8%, and the negative predictive value was 99.5%; 95% CI, 97.5%–100%. Among men, the scale’s sensitivity and negative predictive values were 100%. Among women, the sensitivity was 90%; 95% CI, 55.5%–99.7%, and the negative predictive value was 99.2%; 95% CI, 95.7%–100%.

4. Discussion

Overall, the YFAS and mYFAS had good psychometric properties in the 2008 and 2010 Yale Health Behavior Surveys. The modified scale performed well as a substitute for the YFAS in the 2008 survey, and estimations of food addiction prevalence were consistent for both versions of the scales. Both scales had marginal to good internal consistency reliability for the seven symptoms of substance dependence (0.63<α<0.84), although Cronbach’s alphas were consistently higher for the longer version of the scale. This is not surprising as the number of items in a scale influences internal consistency; scales with more items are typically more reliable.⁴² In the 2010 survey, test-retest reliability estimates were good for both scales, with Cohen’s Kappas>0.73, and in the 2008 survey, the mYFAS had excellent sensitivity and negative predictive value using the YFAS as the benchmark.

The validity of the mYFAS as a substitute for the YFAS necessarily depends on the assumption that the YFAS is a tool that can validly assess food addiction. Use of the YFAS as a benchmark seemed appropriate, as the YFAS is the most widely used measure of food addiction; the measurement development study has been cited more than 150 times since its publication in 2009. In addition, preliminary evidence from other study samples suggests that it has good psychometric properties; the majority of these studies found that the internal consistency is good to excellent (α>0.80). These findings have been replicated among university students,³⁰ the general community,²⁵ the overweight and obese,^{24, 43} and in German,³² Italian,⁴⁴ French,⁴⁵ and Spanish³¹ populations. The current analyses corroborate this early evidence.

Although the mYFAS showed excellent sensitivity and negative predictive value in the current study, it was not possible to examine the specificity and positive predictive value because the items in the mYFAS (the test) were derived from a subset of the YFAS (the benchmark). Thus, participants could not meet the criteria for food addiction using the mYFAS unless they also met criteria using the YFAS. Therefore, the modified scale could not identify false positives, forcing its specificity and positive predictive value to be 100%. Nonetheless, this is the first study to date to evaluate the sensitivity and negative predictive value of the mYFAS.

Our analyses demonstrated that the YFAS and mYFAS have good test-retest reliability (Kappa>0.73). This test-retest score is comparable to those found in other datasets for other substance use and eating disorders (alcohol use disorder (κ=.69),⁴⁶ binge eating disorder (κ=0.75),⁴⁷ and food addiction (κ=.50))³³. However, test-retest studies have limitations. For example, the time interval between tests influences reliability estimates. Typically, the shorter the time gap, the higher the correlation between tests.⁴⁸ Test-retest investigators try to choose a time period that provides a reasonable balance between potential memory bias and actual (unwanted) clinical change.⁴⁹ In our study, despite the rather short time interval of two weeks, the prevalence of food addiction fell. This drop in prevalence is unlikely to reflect real behavior change. Rather, participants may have believed the second questionnaire was intended to amplify the first and did not feel the need to repeat their answers,⁵⁰ or participants answered “no” to move quickly through the questionnaires. It is also possible that factors unrelated to food addiction (e.g., the participants’ moods, fatigue levels, health) affected test-taking, and thus test scores.⁴² However, these factors would not necessarily lead to a systematic change in food addiction endorsement. Finally, test-retest methods are only suitable for characteristics that are stable over time. This potential limitation likely would not affect our results, as addiction diagnoses do not fluctuate day to day (unlike moods such as anger or anxiety).⁵¹ These potential limitations are unlikely to negate our high Kappa coefficients. However, wide confidence intervals, in part due to small sample size, indicate that our estimates were not precise.

Another limitation of the current study is that the nine-item mYFAS was not identical to the nine-item version used in the NHS cohorts. While the items extracted from the YFAS were identical, the NHS version included several wording changes (Table S3). Previous evidence^{52, 53} suggests that under some circumstances, even minor changes to diagnostic criteria can have major effects on prevalence estimates, which could ultimately complicate scientific theory as well as public health efforts.⁵³ This potential limitation is particularly relevant if one intends to extrapolate the reliability and validity findings of the mYFAS used in the current analyses to the version used in the NHS cohorts. While we believe the items’ meanings in both versions of the scales were similar, we were unable to evaluate what effect, if any, these word modifications had on estimates of reliability and validity. While no study to date has evaluated the psychometric properties of the NHS version of the mYFAS per se, three papers^{18, 34, 35} examined correlates of food addiction using this scale, and all found strong associations between food addiction and variables expected to be associated with food addiction such as body mass index,¹⁸ child abuse,³⁴ and post-traumatic stress disorder.³⁵ Therefore, in practice, the NHS version of the mYFAS has begun to help us better understand the food addiction construct.

In summary, the current study evaluated several psychometric properties of two measures of food addiction using two community-based convenience samples. The seven substance dependence symptoms in both versions of the scale had marginal to good internal consistency, the scales had good test re-test reliability, and the mYFAS had excellent sensitivity and negative predictive value using the YFAS as the benchmark. Our analyses suggest that the mYFAS may be an appropriate substitute for the YFAS, although we were unable to test whether the context of being asked only nine versus the full array of items influenced people’s answers. Our findings support the continued use of the YFAS and mYFAS to investigate whether the construct of food addiction is a valid psychiatric disorder.

Highlights.

• Psychometric study of two frequently used food addiction scales

• Scales had marginal to good internal consistency and good test re-test reliability

• Modified scale had excellent sensitivity and negative predictive value

• Shorter, modified scale may be appropriate substitute for longer, original scale