Are Dietary Restraint Scales Valid Measures of Dietary Restriction? Additional Objective Behavioral and Biological Data Suggest Not

Abstract

Prospective studies find that individuals with elevated dietary restraint scores are at increased risk for bulimic symptom onset, yet experiments find that assignment to energy-deficit diet interventions reduce bulimic symptoms. One explanation for the conflicting findings is that the dietary restraint scales used in the former studies do not actually identify individuals who are restraining their caloric intake. Thus, we tested whether dietary restraint scales showed inverse relations to objectively measured caloric intake in three studies. Four dietary restraint scales did not correlate with doubly labeled water estimates of caloric intake over a 2-week period (M r = .01). One scale showed a significant inverse correlation with objectively measured caloric intake during a regular meal ordered from an ecologically valid menu (M r = −.30), but a significant positive relation that was qualified by a significant quadratic effect, to objectively measured caloric intake during multiple eating episodes in the lab (M r = .32). In balance, results suggest that dietary restraint scales are not valid measures of dietary restriction, replicating findings from prior studies that examined objective measures of caloric intake.

Theorists have asserted that dieting increases risk for onset and maintenance of binge eating and bulimia nervosa (Fairburn, 1997; Huon, 1996; Levine & Smolak, 2006; Neumark-Sztainer, 2005; Polivy & Herman, 1985). Dieting, or dietary restraint¹, refers to intentional and sustained restriction of caloric intake for the purposes of weight loss or maintenance (Herman & Mack, 1975; Wadden, Brownell, & Foster, 2002; Wilson, 2002). Dietary restriction must result in a negative energy balance for weight loss or a balance between intake and output for weight maintenance. Polivy and Herman (1985) argue that dieters’ chronic hunger increases the risk of binge eating and that a reliance on cognitive controls over eating leaves dieters vulnerable to uncontrolled eating when these cognitive processes are disrupted. Binge eating theoretically precipitates redoubled dietary restraint and the use of compensatory weight control techniques (e.g., vomiting), which may escalate into a binge-purge cycle (Fairburn, 1997).

In support of this theory, prospective studies indicate that females with high versus low scores on dietary restraint scales are at greater risk for future onset of binge eating, bulimic symptoms, and bulimic pathology (Neumark-Sztainer et al., 2006; Killen et al., 1996; Stice, Davis, Miller, & Marti, 2008; Stice, Killen, Hayward, & Taylor, 1998) and increases in bulimic symptoms (Johnson & Wardle, 2005; Stice, 2001; Wertheim, Koerner, & Paxton, 2001). These studies primarily used the Restraint Scale (RS; Polivy, Herman, & Warsh, 1978) and the Dutch Restrained Eating Scale (DRES; van Strien, Frijters, van Staveren, Defares, & Deurenberg, 1986). Given the consistency of these prospective findings, it is widely accepted that dieting plays a causal role in the onset of bulimic pathology (Fairburn, 1997; Levine & Smolak, 2006; Neumark-Sztainer, 2005). Thus, eating disorder prevention programs often advise against dieting (e.g., Smolak, Levine, & Schermer, 1998; Stewart, Carter, Drinkwater, Hainsworth, & Fairburn, 2001), and some researchers have evaluated interventions that reduce dietary restriction and propose a moratorium on dieting (Bacon, et al., 2002; Polivy & Herman, 1992).

In contrast to the results from prospective studies, randomized trials have found that assignment to weight loss diet interventions reduce binge eating and bulimic symptoms. Trials indicate that assignment to 5- to 6-month energy-deficit weight loss interventions, versus waitlist control conditions, resulted in significantly greater decreases in binge eating for overweight and obese women (Klem, Wing, Simkin-Silverman, & Kuller, 1997; Goodrick, Poston, Kimball, Reeves & Foreyt, 1998; Reeves et al., 2001). Trials also indicate that assignment to 6-week energy-deficit weight loss interventions, versus waitlist control conditions, produced significantly greater decreases in bulimic symptoms among normal weight adolescent girls and young women (Groesz & Stice, 2007; Presnell & Stice, 2003) and women with bulimia nervosa (Burton & Stice, 2006). Participants in these interventions are instructed to reduce caloric intake and increase physical activity to achieve the negative energy balance necessary for weight loss. Further, assignment to a weight maintenance intervention that significantly reduced risk for weight gain and obesity onset over a 3-year period resulted in decreased bulimic symptoms and reduced risk for future onset of eating disorders in adolescent girls relative to assessment-only controls (Stice, Marti, Spoor, Presnell, & Shaw, 2008). Participants in this intervention were encouraged to bring their caloric intake into balance with their energy expenditure to avoid unhealthy weight gain.

It is important to determine why these contradictory findings have emerged because they have opposing public health implications. If dieting causes bulimic pathology, interventions should attempt to decrease dieting. Yet, if dieting reduces bulimic symptoms and facilitates weight control, interventions should help individuals diet more effectively. The evidence that 45% of adolescent girls report dieting underscores the import of determining whether dieting has adverse effects (Neumark-Sztainer, 2005).

One potential explanation for the inconsistent findings is that the dietary restraint scales used in the prospective studies are not valid measures of dietary restriction. The original dietary restraint scale was developed to identify individuals currently suppressing their weight through dietary restriction (Herman & Polivy, 2008; Polivy, Herman, & Warsh, 1978). Other dietary restraint scales were developed to provide more valid measures for identifying people engaging in dietary restriction for weight control purposes (van Strien et al., 1986). If the scales used in the prospective studies do not identify individuals who are actually achieving the energy-deficit diet necessary for weight loss, it could explain why these studies produce findings that are discrepant from those emerging from experimental trials involving energy-deficit diets. That is, if the experiments are placing people on energy deficit diets that result in documented weight loss, whereas the prospective studies are studying people who desire, but are not achieving an energy-deficit diet, it could explain why results from these two lines of research do not accord; the experiments are studying caloric deficit diets and the prospective studies are not. The evidence that people often under-report caloric intake, particularly those with elevated dietary restraint scores (Bandini, Schoeller, Dyr, & Dietz, 1990; Lichtman et al. 1992; Prentice et al., 1986), suggests this is a reasonable supposition.

We conducted four studies that investigated whether five dietary restraint scales showed inverse correlations with directly observed caloric intake during single eating episodes (Stice, Fisher, & Lowe 2004). We used caloric intake as the criterion because the original validity studies used self-reported intake as the criterion (French, Jeffery, & Wing, 1994; Kirkley, Burge, & Ammerman, 1988; Neumark-Sztainer, Jeffery, & French, 1997; van Strien, et al., 1986; Wardle & Beales, 1987). All five dietary restraint scales were developed to assess intentional dietary restriction for the purposes of weight control: the RS (Polivy et al., 1978), Three Factor Eating Questionnaire-Restraint scale (TFEQ-R; Stunkard & Messick, 1985), DRES (van Strien et al., 1986), Eating Disorder Examination Questionnaire-Restraint Subscale (EDEQ-R, Fairburn & Beglin, 1994), and Dietary Intent Scale (DIS; Stice et al., 2004). These scales showed weak and generally non-significant correlations with objectively measured caloric intake during unobtrusively observed eating episodes across the four studies (M r = −.07, range: −.34 to .20; Stice et al., 2004). For instance, the average correlation between three dietary restraint scales and observed caloric intake of students eating meals in dorm cafeterias was −.09. Our findings replicate results from other studies that examined objectively measured caloric intake during single eating episodes (Epstein et al., 2004; Hetherington et al., 2000; Jansen, 1996; Ouwens, van Strien, & van der Staak, 2003; Sysko, Walsh, & Wilson, 2007; Wardle & Beales, 1987).

Studies have also tested whether dietary restraint scales correlated with objective measures of caloric intake during multiple eating episodes, which should provide a more representative index of habitual caloric intake. Lean and overweight adults with high versus low scores on the TFEQ-restraint scale did not show significant differences in caloric intake during three meals and a snack consumed during a 20-hour monitoring period in the lab (Rolls et al., 1997). The EDE-restraint scale did not correlate significantly with observed caloric intake during three separate taste tests of snack foods for normal weight pre-adolescents (Jansen et al., 2003). The TFEQ-restraint scale did not correlate significantly with observed caloric intake during four separate healthy meals consumed by normal weight young women (Martin et al., 2005). The TFEQ-restraint scale, DIS, EDEQ-restraint scale, and EDE-restraint scale (Fairburn & Cooper, 1993) did not correlate significantly with observed caloric intake of a yogurt shake eaten during two sessions by women with anorexia nervosa (Sysko, Walsh, Schebendach, & Wilson, 2005).

Other studies have tested whether dietary restraint scales correlated with objectively measured caloric intake over longer time intervals. One study found that the TFEQ-restraint scale did not correlate with the caloric content of lunches purchased at workplace cafeterias over a 3-month period (Stice, Cooper, Schoeller, Tappe, & Lowe, 2007). Other studies used doubly labeled water (DLW) to estimate habitual caloric intake over a 2-week period. DLW uses isotopic tracers to assess total carbon dioxide production, which can be used to generate accurate estimates of habitual caloric intake (Schoeller et al., 1986). It is considered to be the gold standard measure of habitual caloric intake because participants can be kept blinded to the objective of the study, it provides a precise estimate of total caloric intake over the monitored period, and requires minimal effort on the part of participants (Schoeller et al., 1986). Only one dietary restraint scale has been evaluated using DLW; the TFEQ-restraint scale did not shown significant inverse correlations with DLW estimates of caloric intake over a 2-week period among normal weight women (Bathalon et al., 2000; Tuschl et al. 1990) or overweight women (Stice et al., 2007).

Although this literature suggests that self-report dietary restraint scales are not valid measures of dietary restriction, there are gaps in this literature. First, the validity studies that assessed caloric intake over extended time periods involved only the TFEQ-R; no studies have tested whether other widely used dietary restraint scales correlate with DLW estimates of caloric intake. Second, many of the prior validity studies involved small samples, limiting confidence in the findings. Third, the studies that assessed caloric intake over extended time periods focused solely on adults; it would be useful to study adolescents because most risk factor studies implicating dieting in the etiology of bulimic pathology have focused on teens. Fourth, we thought it useful to investigate the validity of dietary restraint scales among individuals with eating disorders, as only two prior studies have addressed this question. Fifth, we also tested whether dietary restraint scales showed quadratic relations to caloric intake because it is possible that particularly high scores (e.g., the upper quartile) accurately identify individuals who exhibit dietary restriction, whereas participants with lower scores consume similar amounts of calories. To our knowledge, no previous study has tested for quadratic effects.

We analyzed data from three studies that used objective measures of caloric intake to extend the evidence-base regarding the validity of dietary restraint scales. Study 1 tested whether the TFEQ dietary restraint scale correlated with objectively measured caloric intake during a meal consumed in the lab with men and women ranging from lean to obese. This is the first study to use an ecologically valid paradigm in which participants order from a menu containing a range of healthy and unhealthy foods that are typical of restaurants. Study 2 tested whether the TFEQ-restraint scale correlated with caloric intake during consumption of multiple meals in the lab in lean women, obese women, and women with binge eating disorder (BED). Participants were instructed to consume a regular meal on one occasion and to binge eat on the other occasion. This is the first study to examine the relation between a dietary restraint scale and objectively measured caloric intake during a normal meal and a binge meal among patients with BED and similarly overweight participants. Study 3 tested whether a broader array of dietary restraint scales, including the RS, TFEQ-restraint scale, DRES, and DIS, correlated with DLW-assessed total caloric intake over a 2-week period with a sample of late-adolescent women who ranged from lean to obese. This is the first study to examine the relation between multiple dietary restraint scales and DLW estimated caloric intake, which is important because most risk factor studies that have suggested that dietary restraint increases risk for bulimic pathology have used the RS and the DRES. It is also the first validity study to use DLW with late-adolescent females.

Study 1

Participants and Procedures

Data were drawn from a study assessing the impact of restaurant menu labels on food choices and objectively measured intake during a dinner meal (Roberto, Larsen, Agnew, Baik, & Brownell, in press). Participants were 303 adults recruited from a small US city via flyers, word-of-month, newspaper advertisements, and internet postings. The sample comprised 147 males (50%) and 148 females (M age = 30.5, SD = 12.4). In total, 26 individuals were excluded from these analyses, including 10 individuals who did not complete the TFEQ. Participants were randomly assigned to one of three menu conditions: 1) a menu without any calorie labels (No Calorie Labels); 2) a menu with calorie labels (Calorie Labels); 3) a menu with calorie labels and a statement at the top of the menu which read “the recommended daily caloric intake for an average adult is 2000 calories” (Calorie Labels plus Normative Information). Under the guise of consumer market research, participants were asked to order dinner from a restaurant menu and food choices were recorded. Participants were then served the meal they ordered and overall caloric intake was unobtrusively measured. After the study meal, participants completed the TFEQ. All participants provided written informed consent and the local Human Subjects Committee approved this study and the other studies reported herein.

Measures

Three Factor Eating Questionnaire – Restraint Scale

The TFEQ-restraint scale (Stunkard & Messick, 1985) assesses dietary behaviors designed to produce weight loss or maintenance, monitoring of body shape, and importance of thinness (sample item: I count calories as a conscious means of controlling my weight). This scale has shown internal consistency (α’s ranged from .85 to .93) and temporal reliability (1-month test-retest r = .98; French et al., 1994; Stunkard & Messick, 1985).

Objective measure of caloric intake

The weight of the serving dishes was measured before and after the meal using a digital scale accurate up to ± 0.1 grams. The weight of each serving dish after the meal was subtracted from the weight of each serving dish before the meal. This information, in conjunction with data on the caloric density of each food, allowed us to generate an unobtrusive and objective measure of total caloric intake for each participant (kcal), following the approach commonly used in lab feeding studies (e.g., Telch & Agras, 1996).

Results and Discussion

Preliminary analyses indicated that there were no significant differences across conditions on age (M = 30.51, SD = 12.41), Body Mass Index (BMI M = 25.16, SD = 6.11 kg/m²), degree of liking of the study meal, hunger prior to the meal, fullness following the meal, frequency of visiting fast food restaurants, or TFEQ subscale scores, suggesting that random assignment produced initially equivalent groups. During the study meal, individuals in the No Calorie Labels condition consumed an average of 1459 (SD = 725) kcal, individuals in the Calorie Labels condition consumed an average of 1335 (SD = 621) kcal, and individuals in the Calorie Labels plus Normative Information condition consumed an average of 1256 (SD = 688) kcal.

TFEQ-restraint scores showed a significant negative correlation with total calories consumed for the study meal for the overall sample (r = −0.30, p < 0.001). Although TFEQ-restraint scores tented to show higher inverse correlations with caloric intake in the conditions with versus without caloric density information (see Table 1), multiple regression analysis indicated that these correlations did not differ significantly across experimental condition (p > .20 for all contrasts). Multiple regression analyses also confirmed that participant BMI did not significantly moderate the relation between TFEQ-restraint scores and caloric intake (p > .20).

Table 1.

Correlations between Three-factor Eating Questionnaire-Restraint scores and objectively measured caloric intake for three menu label conditions among a community sample in Study 1

	Full Sample	No Calorie Labels	Calorie Labels	Calorie Labels plus Information
TFEQ - Restraint	−.30***	−0.14	−0.33**	−0.41***

^*Note: p < .05

^**p < .01

^***p < .001 TFEQ = Three Factor Eating Questionnaire

Analyses next tested whether there was evidence of a quadratic relation between TFEQ-restraint scores and total caloric intake across the overall sample. A multiple regression model indicated that the quadratic term did not show a significant relation to overall caloric intake (r = −.096, p = 0.094).

Thus, Study 1 provided evidence of an inverse relation between TFEQ-restraint scores and unobtrusively observed caloric intake among adults. This correlation corresponds to a medium effect size, with TFEQ-restraint scores accounting for 9% of the variance in caloric intake. Although this is relatively modest for a validity coefficient, this appears to be the first study to find that elevated TFEQ-restraint scores are significantly associated with objectively measured caloric intake; prior studies have not found significant relations between dietary restraint scores and objectively measured caloric intake in single eating episodes (e.g., Hetherington et al., 2000; Jansen, 1996; Martin et al., 2005; Ouwens et al., 2003; Stice et al., 2004). There was no evidence of a significant quadratic relation between TFEQ-restraint and caloric intake.

Study 2

This study tested whether the TFEQ-restraint scale correlated with multiple episodes of objectively measured caloric intake (normal meal and a binge eating-instruction meal) and whether there was a quadratic component to this relation.

Participants and Procedures

Participants were 61 women between the ages of 18 and 45 (M age = 33.2, SD = 5.8), including 44 overweight individuals and 17 normal-weight controls. In the overweight group, 25 individuals met DSM-IV criteria for BED and 19 individuals denied eating disorder symptoms and were considered obese controls. Participants were recruited in a large US city by advertisements offering either monetary compensation (obese controls, normal-weight controls) or outpatient treatment in exchange for research participation (patients with BED). The average BMI score was 38.9 (SD = 5.4) for the BED group, 41.7 (SD = 8.9) for the obese group, and 22.8 (SD = 2.5) for the normal-weight controls. No differences were observed between the mean BMI for the BED and obese groups; however, the overweight groups had significantly higher BMI scores than normal weight controls (F [2, 59] = 48.70, p < 0.001).

Data were drawn from two studies of eating behavior that used similar recruitment strategies, inclusion and exclusion criteria, and experimental procedures (Guss et al., 2002; Sysko et al., 2007). Both studies included lab test meals on three non-consecutive days; an adaptation meal, a normal meal, in which participants were asked to eat normally, and a binge meal, in which participants were instructed to binge eat. Data are not presented on intake from the adaptation meal, as this meal was solely intended to allow participants the opportunity to become accustomed to eating in the lab. Participants in both studies consumed a 300 kcal standardized breakfast on the morning of the test meals, and were asked not to consume any food or liquid, other than water, before returning to the lab 6 hours later for the normal or binge meals. Guss and colleagues (2002) provided participants with a multi-item meal, including foods typical of normal meals (e.g., chicken, fish, rice, apples, etc.) and binge meals (e.g., ice cream, cookies, potato chips). Participants in the Sysko and colleagues (2007) study received macaroni and cheese for both normal and binge eating meals. Prior to the meals, all participants completed the TFEQ-restraint scale.

Measures

Three Factor Eating Questionnaire – Restraint Scale

The TFEQ-R (Stunkard & Messick, 1985) was used in Study 2 (see Study 1 for a description and psychometric details).

Objective measure of caloric intake

Each serving dish was unobtrusively weighted before and after the test meal using a digital scale accurate up to ± 0.1 grams. Like the procedure used in Study 1, the reduction in weight of the each participants’ serving dish following the meal, in conjunction with data on the caloric density of each food, allowed us to generate an unobtrusive and objective measure of total caloric intake for each participant (kcal).

Results and Discussion

We first estimated preliminary regression models that tested whether there were mean differences in TFEQ-restraint scores and caloric intake, and whether the correlation between TFEQ-restraint scores and caloric intake differed across the two studies (Guss et al., 2002; Sysko et al., 2007). Because none of these preliminary analyses revealed significant effects (all p-values > .10), data from the two studies were combined. Next, we conducted regression models that tested whether the magnitude of the correlation between the TFEQ-restraint scores and caloric intake differed according to binge versus non-binge instruction conditions. Because this difference did not reach significance (p > .10), data from the binge and non-binge meals were averaged to provide a measure of caloric intake across multiple eating episodes in the BED, obese, and healthy control groups. The average caloric intake across meal conditions was 1516 (SD = 570) kcal for BED participants, 892 (SD = 382) kcal for obese participants, and 867 (SD = 449) kcal for lean participants. We then estimated regression models that tested whether the magnitude of the correlation between the TFEQ-restraint scores and caloric intake during binge and during non-binge meals differed significantly according to BMI or BED status. Because none of these effects reached significance (all p-values > .10), data were combined across participants in the three groups (BED, obese, and controls).

Although participants with elevated TFEQ-restraint scores consumed significantly more calories than those with lower scores (r = 0.32, p=0.011), results indicated that there was a quadratic relation between TFEQ-restraint scores and average test meal intake (r = −.37, p = 0.015). As shown in Figure 1, average caloric intake increased linearly with TFEQ scores from the first quartile through the third quartile, but then decreased again for the fourth quartile. That is, participants with the highest and lowest TFEQ-restraint scores consumed fewer calories than participants with moderate scores. The significant positive main effect between TFEQ-restraint scores and objectively measured caloric intake in Study 2 is inconsistent with previous studies that have found that dietary restraint scales have not shown significant relations with objectively measured caloric intake during multiple eating episodes among participants with (Sysko et al., 2005) and without eating disorders (Jansen et al., 2003; Martin et al., 2005; Rolls et al., 2007), however, this main effect was qualified by a significant negative quadratic relation. To our knowledge, this is the first study to provide evidence of a significant quadratic relation between a dietary restraint scale and objectively measured caloric intake, suggesting it will be important to replicate in an independent sample.

Figure 1.

Graph of the quadratic relation between TFEQ-restraint scores and average intake across the two eating episodes.

Study 3

Participants and Procedures

Participants in Study 3 were a randomly selected subsample from a large obesity prevention trial targeting young women with body image concerns recruited from a state university in the US. Participants were 63 late adolescent females between the ages of 18 and 20 (M age = 18.4, SD = 0.53) (M BMI = 23.7, SD = 4.2). Exclusion criteria included those who had diabetes, conditions requiring supplemental oxygen, or pregnancy. Participants provided data during three visits to the lab: baseline, 2–4 weeks after baseline, and 4–6 weeks after baseline. Participants were also required to avoid traveling more than 200 miles from the study site in the 2-weeks between the second and third visit to the lab.

Measures

Restraint Scale

The RS (Polivy et al., 1978) assesses dieting behaviors, preoccupation with eating, binge eating behaviors, and past weight fluctuations (sample item: How often do you diet?). This scale has shown internal consistency (α’s range from .79 to .86) and temporal reliability (2-year test-retest r = .74; French et al., 1994; Klesges, Isbell, & Klesges, 1992).

Three Factor Eating Questionnaire – Restraint Scale

The TFEQ-R (Stunkard & Messick, 1985) was also used in Study 3 (see Study 1 for a description and psychometric details).

Dutch Restrained Eating Scale

The DRES (van Strien et al., 1986) assesses dietary behaviors designed to produce weight loss and weight maintenance (sample item: Do you deliberately eat less in order not to become too heavy?). This scale has shown internal consistency (α’s range from .93 to .95) and temporal reliability (2-week test-retest r = .82; Stice et al., 2004; van Strien et al., 1986).

Dietary Intent Scale

The DIS (Stice et al., 2004) assesses behaviors used for weight loss or weight maintenance purposes (sample item: I eat diet foods in an effort to control my weight). This scale has shown internal consistency (α’s range from .93 to .94) and temporal reliability (1-month test-retest r = .92; Stice et al., 2004).

Objective measure of caloric intake

Energy intake was calculated from the sum of energy expenditure from DLW and the estimated change in body energy stores from serial body weight measurements performed at baseline, 2–4 weeks later immediately before dosing, and 2-weeks after dosing. Change in body energy stores was calculated assuming 7800 kcal/kg for any change in weight. This was divided by 180 d to calculate the daily source of energy substrates from weight loss or storage of excess energy intake as weight gain (Forbes, 2000).

For the measurement of energy expenditure women arrived at the lab having fasted for 5–15 hours. DLW for measurement of total body water (TBW) and total energy expenditure (TEE) was administered immediately after testing negatively for pregnancy. The doses were 2.0–2.5g H₂18O (10 atom percent)/kg estimated total body weight. Spot urine samples were collected immediately before deuterium was administered, and 1, 3, and 4 hours post dosing during the second visit to the lab. Two-weeks later during the third visit to the lab, 2 additional spot urine samples were collected at the same time of day as the 3 and 4 hour post dosing samples. None of the samples were the first void of the day.

Energy expenditure was calculated using equation A6 as published in Schoeller et al (1986), the dilution space ratios of Racette et al (1994), and the modified Weir (1949) equation as described by Black, Prentice, and Coward (1986). DLW estimated kilocalorie intake per day (kcal/d) was calculated using TEE data and weight change over the 4 weeks adjacent to the measured TEE. The equation used for each participant was: DLW kcal/d = TEE + ((Wt. change for 90d*7,800)/90d). The 7,800 kcal/kg is an estimate of the energy density of adipose tissue (Forbes, 2000).

Estimated energy requirement (EER) in kcal/d, was calculated based on the 2002 Dietary Reference Intake (DRI). DRI values were chosen because they are based on recent scientific knowledge with respect to energy requirements for healthy populations. The DRI prediction requires an estimate of physical activity level (PAL), which were calculated from TEE divided by basal energy expenditure (PAL = TEE/BEE). The measured PAL values range from 1.0 to 2.5 and fit into one of four categories: sedentary (PAL ≥ 1.0 < 1.4), low active (PAL ≥ 1.4 < 1.6), active (PAL ≥ 1.6 < 1.9), and very active (PAL ≥ 1.9 < 2.5). Corresponding values from those categories were applied to the equation to calculate individual EER values.

Results and Discussion

Table 2 reports the correlations between the RS, TFEQ-restraint scale, DRES, DIS, and DLW estimates of habitual caloric intake per day. The four dietary restraint scales showed significant inter-correlations. However, none of the dietary restraint scales correlated with the DLW estimate of kcal per day (r = −.09 – .07), with a mean r = .01. Post hoc analyses confirmed that participant BMI did not moderate these relations (all p-values > .10), as was observed in Study 1 and 2, as well as in prior studies (Stice et al., 2004; Stice et al., 2007). There was no evidence that any of these dietary restraint scales showed a significant quadratic relation to objectively measured caloric intake (all p-values > .10), converging with results from Study1, but not Study 2.

Table 2.

Correlations between dietary restraint scales and DLW estimates of habitual caloric intake over a 2-week period in Study 3

	1.	2.	3.	4.	5.
1. Restraint Scale		.60***	.73***	.80***	.07
2. Three-factor Eating Questionnaire-Restraint			.69***	.60***	.03
3. Dutch Restrained Eating Scale				.85***	−.09
4. Dietary Intent Scale					.01
5. Double labeled water kcal intake per day

^*Note: p < .05

^**p < .01

^***p < .001

The fact that the results converge with the findings from four other DLW studies (Bathalon et al., 2000; Stice et al., 2007; Tuschl et al., 1990) provides increased confidence in the findings. Collectively, these five DLW studies suggest that the TFEQ-restraint scale does not show an inverse correlation with objective biological estimates of caloric intake in lean, overweight, and obese populations. A novel contribution of Study 3 is that it is the first to provide evidence that other dietary restraint scales, including the RS, DRES, and DIS, likewise did not show significant inverse correlations with DLW estimates of habitual caloric intake. Another noteworthy feature of Study 3 is that it involved late-adolescent females, as opposed to the four previous doubly labeled water studies that involved only adults. This is important because dozens of studies have used dietary restraint scales with late adolescent samples. Late adolescent participants are also widely used in studies examining the implication that dietary restriction is a risk factor for bulimic pathology onset.

General Discussion

The primary aim of this report was to test whether the dietary restraint scales showed the expected inverse correlations with objectively measured caloric intake that was suggested by the earlier validity studies that relied on self-reported caloric intake. Study 1 found a significant inverse relation between TFEQ-restraint scores and unobtrusively observed caloric intake among adults that was medium in magnitude, with TFEQ-restraint scores accounting for 9% of the variance in caloric intake. This result is consistent with findings from validity studies that examined self-reported caloric intake (e.g., French et al., 1994; Kirkley et al., 1988; Neumark-Sztainer et al., 1997; Wardle & Beales, 1987), but is inconsistent with studies that have found that dietary restraint scales typically show non-significant relations to objectively measured caloric intake (e.g., Epstein et al., 2004; Hetherington et al., 2000; Jansen, 1996; Ouwens et al., 2003; Stice et al., 2004; Sysko et al., 2007; Wardle & Beales, 1987). Study 2 found a significant positive relation between the TFEQ-restraint scale and objectively measured intake during two eating episodes, which was moderate in magnitude, accounting for 10% of the variance in caloric intake. Because no previous study has found dietary restraint scores to correlate positively with caloric intake, it is important to consider the possibility that this effect resulted from some unique characteristic of this sample or procedure or that this effect is a chance finding. Study 3 found that the RS, TFEQ-restraint scale, DRES, and DIS snowed non-significant relations with DLW estimates of habitual caloric intake, with these relations accounting for less than 1% of the variance in caloric intake. These null findings accord with results from previous validity studies that have examined objectively measured caloric intake reviewed in the introduction (e.g., Bathalon et al., 2000; Hetherington et al., 2000; Jansen et al., 2003; Martin et al., 2005; Rolls et al., 2007; Stice et al., 2007; Tuschl et al., 1990). Study 3 makes a novel contribution to the literature because it is the first DLW study to examine the validity of multiple dietary restraint scales; past DLW validity studies have focused solely on the TFEQ-restraint scale. This is also the first DLW study to investigate the validity of dietary restraint scales with a sample of late-adolescents, which is important because numerous studies that have used dietary restraint scales have involved female college students.

Given that the findings from Studies 1 and 2 do not converge with most previous validity findings, it is important to consider explanations for the two significant linear relations. With regard to the significant inverse correlation from Study 1, the effect sizes provided in Table 2 suggest that providing information about the caloric content of foods on menus may allow those who desire to exercise restraint over the caloric intake to succeed. Although the magnitude of the correlations did not differ significantly across the menu labeling conditions, dietary restraint scores showed significant inverse relations to objectively measured caloric intake when menus provided information on the caloric content of the foods, but that this relation was non-significant when the menus did not provide caloric content information. This pattern of findings suggests that providing caloric content information may promote healthier eating behavior.

With regard to the significant positive correlation from Study 2, it is possible that this unexpected relation emerged because some participants had BED, whereas others did not, and virtually all previous studies have not included participants with this eating disorder. However, post hoc analyses indicated that the dietary restraint scores showed a moderate positive correlation with objectively measured caloric intake for BED participants, obese participants, and controls in this study. It is also possible that the unexpected positive relation emerged because participants were instructed to binge eat on one occasion and eat normally on the other occasion. However, post hoc analyses indicated that dietary restraint scores showed a moderate positive correlation with caloric intake for the binge instruction day and the normal eating day when caloric intake data from these two conditions were analyzed separately.

Another possibility worth considering is that the two significant correlations reported in Studies 1 and 2 are simply chance findings given that virtually all previous validity studies reported non-significant relations between dietary restraint scores and objectively measured caloric intake. Including all of the studies reviewed in the introduction and the studies in the present report, a total of 44 correlations between dietary restraint and objectively measured caloric intake have been reported. In total, only 3 of these 44 relations were statistically significant (6.8%), which is remarkably close to the 5% that would be expected based on chance (studies typically used a .05 p-value). Previous studies have found that correlations ranged from negative to positive (e.g., Stice et al., 2004; range: −.34 to .20, as was observed in the present studies (range −.30 to .32). Also noteworthy is the fact that the few significant relations for the two dietary restraint scales have not replicated across studies. Although a significant inverse relation between the TFEQ-restraint scale and objectively measured intake emerged in Study 1, this same scale showed a significant positive relation in Study 2 and non-significant relations in all the other studies that have examined this scale (e.g., Bathalon et al., 2000; Martin et al., 2005; Stice et al., 2007; Sysko et al., 2005). Similarly, although the DIS showed a significant inverse relation to objectively measured intake in one prior study (Stice et al., 2004), this relation did not replicate in other studies that have examined this scale (e.g., Sysko et al., 2005; Sysko et al., 2007), including the present report. Collectively, these results imply that the few significant effects are chance findings.

A secondary aim of this report was to test whether dietary restraint scores showed quadratic relations to objectively measured caloric intake, because it is possible that individuals with particularly high scores may exhibit dietary restriction, whereas those in the lower range do not. Although none of the four dietary restraint scales examined in Study 3 showed a significant quadratic relation to DLW estimates of caloric intake, the TFEQ-restraint scale showed a marginally significant relation to caloric intake in Study 1 and a significant quadratic relation in Study 2. Results from this latter study indicated that caloric intake increased linearly as TFEQ scores increased from the first quartile through the third quartile, but then decreased for the fourth quartile. Although the finding that participants with particularly elevated dietary restraint scores consumed fewer calories than those with moderate scores, it was puzzling that individuals with the lowest dietary restraint scores consumed less calories as well. Moreover, a quadratic relation emerged for only one of the four dietary restraint scales examined herein and the significant quadratic effect emerged in only one of the three studies that examined the TFEQ restraint scale, suggesting that this was not a robust relation. The fact that the quadratic relation only emerged in the study involving individuals with BED may suggest that this population contributed to the quadratic relation. In support of this possibility, post hoc analyses confirmed that the quadratic effect became non-significant when the participants with BED were excluded from the analyses (the r decreased from −.37 to −.33). We also tested whether the fact that participants were expressly asked to binge eat contributed to the quadratic effect. In line with this possibility, post hoc analyses found that the magnitude of the quadratic effect became smaller when we examined only caloric intake in the non-binge condition (r = −.28) relative to when we examined caloric intake averaged across the binge and non-binge conditions (r = −.37). Interestingly, the quadratic effect became trivial when we both excluded participants with BED and solely focused on caloric intake in the non-binge condition (r = − .16), suggesting that both factors contributed to the isolated quadratic effect.

In balance, given that the average correlation between dietary restraint scales and objectively measured caloric intake was .02, the present results seem to suggest that the four dietary restraint scales examined in the present studies do not show inverse correlations with objectively measured caloric intake. These results converge with the results from previous validity studies that have found that six dietary restraint scales were not correlated with objective measures of caloric intake (Bathalon et al., 2000; Epstein et al., 2004; Hetherington et al., 2000; Jansen et al., 2003; Martin et al., 2005; Ouwens et al., 2003; Rolls et al., 1997; Stice et al., 2004; Stice et al., 2007; Sysko et al., 2005; Sysko et al., 2007; Tuschl et al., 1990). Similar effects have emerged for lean, overweight, and obese individuals, as well as for adolescents and adults. These effects have been observed in a wide variety of settings, including controlled labs, restaurants, and cafeterias. Further, studies from numerous independent groups, including studies conducted by researchers who developed dietary restraint scales, identify similar effects regardless of whether the study investigated single eating episodes, multiple eating episodes, or 2–12 week observation periods, or used unobtrusive or biological estimates of caloric intake.

These results stand in contrast to the findings from earlier validity studies that only examined the relation between dietary restraint and self-reported caloric intake (e.g., French et al., 1994; Kirkley et al., 1988; Neumark-Sztainer et al., 1997; van Strien, et al., 1986; Wardle & Beales, 1987). A probable explanation for the discrepant findings is that self-reported caloric intake is inaccurate. Such reporting inaccuracies are probably partially rooted in social desirability biases, as this under-reporting is greatest for overweight individuals (Prentice et al., 1986), and underreporting of dietary intake correlates positively with social desirability scales (Maurer et al., 2006; Tooze et al., 2004).

It is important to acknowledge the limitations of the present studies. First, the samples for Studies 2 and 3 were relatively small. Second, the participants in these three studies were not random samples of individuals from well-defined populations, suggesting that the results should be generalized with care. Nonetheless, the fact that similar findings have emerged from 16 validity studies from several countries suggests that the relations are robust. Third, each objective measure of caloric intake probably has certain biases. Nonetheless, the fact that similar effects have emerged from a variety of paradigms for objectively measuring caloric intake is reassuring.

One important implication resulting from this collection of validity studies is that findings from studies using these dietary restraint scales should be reinterpreted, including those suggesting that dietary restraint increases risk for future onset of bulimic pathology. If dietary restraint scales do not identify people who are actually restricting their caloric intake, this implies that it may not be dietary restriction that increases the risk for bulimic pathology. That is, studies drawing conclusions from restraint scales about the effect of diet-induced negative energy balance on the etiology of bulimic pathology should be reconsidered, because restraint scales do not appear to identify individuals who are restricting their caloric intake. One possibility is that it is cognitive attention to dietary intake or attempted dietary restriction, rather than successful dietary restraint, that increases risk for bulimic pathology. Yet, this would not explain why negative energy balance weight loss interventions consistently reduce bulimic symptoms, as participants in these interventions would have to devote more cognitive attention to reducing caloric intake and efforts to lose weight because they typically show future weight loss, whereas those with elevated dietary restraint scores typically show future gain weight.

Another possibility is that it is fasting, rather than more modest dietary restriction, which increases risk for bulimic pathology. Perhaps it is these individuals who drive the relation between elevated scores on dietary restraint scales and future risk for onset of bulimic pathology. This interpretation accords with several established findings. First, it aligns with evidence from animal experiments that have investigated the effects of severe dietary restriction. Rats randomized to extreme caloric deprivation conditions (in which they lost between 7% to 20% of their body mass) consume significantly more calories during ad lib feeding and show a preference for high-fat foods immediately after the deprivation period than non-deprived control rats (Hagan, Chandler, Wauford, Rybak, & Oswald, 2003; Lusas & Sclafani, 1992; Ogawa et al., 2005; Sclafani & Ackroff, 1993). Second, this explanation accords with experiments that indicate that enforced periods of caloric deprivation result in greater reinforcement value of food, as assessed by operant tasks that measure how hard participants will work to earn food and by actual caloric intake (Epstein et al., 2003; Raynor & Epstein, 2003). Third, it dovetails with evidence that individuals who retrospectively report severe diet-induced weight suppression show a poorer response to treatment for bulimia nervosa (Butryn, Lowe, Safter, & Agras, 2006). Fourth, it is consistent with recent evidence that fasting is a more potent risk factor for future risk for bulimic pathology onset than elevated dietary restraint scores (Stice, Davis et al., 2008).

Collectively, results from the validity studies suggest that dietary restraint scales may assess relative dietary restriction rather than absolute dietary restriction. Individuals with elevated dieting scores may be eating less than they desire and thus may perceive this relative restriction as dietary restraint (Lowe & Levine, 2005; Timmerman & Gregg, 2003; van Strien et al., 1986), even if they are not achieving the negative energy balance necessary for weight loss. This interpretation is consistent with evidence that (a) intermittent dieters temporarily arrest a weight gain trajectory while they are attempting to engage in a weight loss diet, but do not lose weight (Presnell, Stice, & Tristan, 2008), (b) dietary restraint scores often increase when people are placed on energy-restriction diets relative to non-dieting controls (e.g., Williamson et al., 2007), and (c) individuals with elevated dietary restraint scores consume significantly more calories than those with low dietary restraint scores, but did not feel they have overeaten (Jansen, 1996).

It seems theoretically possible that individuals who experience greater reward from food intake or anticipated food intake (Stice, Spoor, Bohon, Veldhuizen, & Small, 2008) may be particularly likely to evidence an overeating tendency, which they subsequently attempt to curb through attempted dietary restriction. Alternatively, abnormalities in gut peptides and hormones that influence appetite (e.g., leptin) may also give rise to this overeating tendency. Unfortunately, it would appear that attempts to curb this overeating tendency are typically unsuccessful. This interpretation implies that people who overeat and are unable to modulate this tendency are at risk for binge eating and bulimia nervosa. In short, it may be the inability to successfully limit their dietary intake that increases risk for binge eating, not the attempts to limit their caloric intake.

Future Directions

Because dietary restraint scales do not appear to be valid measures of dietary restriction, a priority for future research will be to determine what latent construct these scales assess. The fact that these scales consistently predict bulimic pathology onset suggests that a resolution of this question would have important implications for etiologic theories of this condition and for the design of prevention and treatment interventions. Future research should also explore the possibility that there are qualitatively different types of dieting and that some forms increase and others decrease the risk for onset of bulimic symptoms (e.g., strict fasting versus replacing high-fat foods with fruits and vegetables). It is also possible that dieting that takes the form of acute fasting between episodes of overeating increases risk for bulimic pathology. Finally, it will be important to develop a dieting scale that is a valid measure of dietary restriction. Without a valid measure of dietary restraint, it will be virtually impossible to determine whether dietary restraint plays a role in the development of eating pathology.