Conventional Energy and Macronutrient Variables Distort the Accuracy of Children’s Dietary Reports: Illustrative Data from a Validation Study of Effect of Order Prompts

Abstract

Objective

Validation-study data are used to illustrate that conventional energy and macronutrient (protein, carbohydrate, fat) variables, which disregard accuracy of reported items and amounts, misrepresent reporting accuracy. Reporting-error-sensitive variables are proposed which classify reported items as matches or intrusions, and reported amounts as corresponding or overreported.

Methods

58 girls and 63 boys were each observed eating school meals on 2 days separated by ≥4 weeks, and interviewed the morning after each observation day. One interview per child had forward-order (morning-to-evening) prompts; one had reverse-order prompts. Original food-item-level analyses found a sex-x-order prompt interaction for omission rates. Current analyses compared reference (observed) and reported information transformed to energy and macronutrients.

Results

Using conventional variables, reported amounts were less than reference amounts (ps<0.001; paired t-tests); report rates were higher for the first than second interview for energy, protein, and carbohydrate (ps≤0.049; mixed models). Using reporting-error-sensitive variables, correspondence rates were higher for girls with forward- but boys with reverse-order prompts (ps≤0.041; mixed models); inflation ratios were lower with reverse- than forward-order prompts for energy, carbohydrate, and fat (ps≤0.045; mixed models).

Conclusions

Conventional variables overestimated reporting accuracy and masked order prompt and sex effects. Reporting-error-sensitive variables are recommended when assessing accuracy for energy and macronutrients in validation studies.

Introduction

Dietary validation studies examine the effectiveness of methods to elicit accurate reports by comparing two sets of information – reported information, such as 24-hour dietary recalls, and reference information, such as direct observations. Each set of information consists of (food) items and their respective amounts. As defined in Table 1, reference-set items are matches and omissions, and reported-set items are matches and intrusions (Smith, 1991; Smith et al., 1991). Matches are reference items that are reported. Omissions are unreported reference items. Intrusions are reported items not in the reference set. For each match, the reported amount can be entirely corresponding, part corresponding and part unreported, or part corresponding and part overreported. For each omission, the entire reference amount is unreported. For each intrusion, the entire reported amount is overreported.

Table 1.

Definitions of terms

Conventional variables: Variables which are indifferent to reporting errors. Report rate is an example of a conventional variable.

Correspondence rate: For an individual, correspondence rate = (corresponding amount from matches/reference amount) × 100. It is a genuine measure of reporting accuracy that is sensitive to reporting errors. It has a lower bound of 0%, which indicates that nothing in the reference set was reported eaten. It has an upper bound of 100%, which indicates that all items and amounts in the reference set were reported correctly. Higher correspondence rates reflect better reporting accuracy.

Corresponding amount from a match: The smaller of the reported and reference amounts (or the reported amount if it is equal to the reference amount).

Inflation ratio: For an individual, inflation ratio = {[(overreported amount from matches) + (overreported amount from intrusions)]/(reference amount)} × 100. It is a measure of reporting error. It has a lower bound of 0%, which indicates that there were no intrusions and that no amounts of matches were overreported. It has no upper bound. Lower inflation ratios reflect better reporting accuracy.

Intrusion: A food item that is not eaten, but is reported eaten.

Match: A food item that is actually eaten and is reported eaten.

Omission: A food item that is actually eaten but is not reported eaten.

Order prompts: Prompts used with children during the first phase of each interview to report the previous day’s intake in forward order (beginning with yesterday morning and going forward through the day to yesterday evening) or in reverse order (beginning with yesterday evening and going backward through the day to yesterday morning).

Overreported amount from a match: The amount by which the reported amount exceeds the reference amount (or zero if the reported amount is less than or equal to the reference amount).

Overreported amount from an intrusion: All of the reported amount.

Reference amount: Corresponding amount from matches + unreported amount from matches + unreported amount from omissions.

Reference set of information: Food items actually eaten (i.e., matches and omissions) and their respective amounts.

Report rate: For an individual, report rate = (reported amount/reference amount) × 100. It is a conventional measure of reporting accuracy that is indifferent to reporting errors. It has a lower bound of 0%, which indicates nothing was reported. Although 100% is typically interpreted to indicate perfect reporting accuracy, a report rate has no upper bound because there is no limit on what an individual can report. Report rates of ~100%, >100%, and <100% have been interpreted as high reporting accuracy, overreporting, and underreporting, respectively. For an individual, report rate = correspondence rate + inflation ratio.

Reported amount: Corresponding amount from matches + overreported amount from matches + overreported amount from intrusions.

Reported set of information: Food items reported eaten (i.e., matches and intrusions) and their respective amounts.

Reporting errors: Intrusions and their overreported amounts, and overreported amounts from matches.

Reporting-error-sensitive variables: Variables which are sensitive to reporting errors, and therefore better represent reporting accuracy. Correspondence rate and inflation ratio are examples of reporting-error-sensitive variables.

Sequence: Sequence refers to a child’s first or second interview. Each child was interviewed twice, once with forward-order prompts and once with reverse-order prompts. Half of the children in each race/sex group were randomly assigned to forward-order prompts during the first interview and reverse-order prompts during the second interview; the other half of each group received the complementary assignment.

Unreported amount from a match: The amount by which the reference amount exceeds the reported amount (or zero if the reference amount is less than or equal to the reported amount).

Unreported amount from an omission: All of the reference amount.

Conventional approaches to evaluating dietary reporting accuracy typically transform sets of reference information and reported information to energy and nutrients, cumulate values within each set of information for each subject, and then use statistical tests to compare total reported energy and nutrients to total reference energy and nutrients. We contend that conventional approaches provide a distorted picture of dietary reporting accuracy because they use variables that ignore distinctions between matches and intrusions, and between corresponding and overreported amounts. (Hereafter, we use reporting errors to refer to intrusions, their overreported amounts, and overreported amounts from matches.) Although people report dietary intake as items, reporting accuracy is typically assessed indirectly for nutrients (but see Feskanich et al., 1993, as an example of an exception); this is problematic because accuracy may appear high for some nutrients but not others as intrusions may be similar to items actually eaten in some nutrients but not others (Smith, 1991; Smith et al., 1991).

In this article, we use validation-study data to illustrate that conventional energy and macronutrient (protein, carbohydrate, fat) variables that disregard reporting errors misrepresent reporting accuracy; we propose alternative reporting-error-sensitive variables. We use data from a validation study conducted to examine the effect of order prompts (morning-to-evening versus evening-to-morning) on the accuracy of children’s reports of school meals obtained during 24-hour dietary recalls (Baxter et al., 2003c). The goal of this article is not to identify “the way” to assess dietary intake, but to encourage a more sensitive approach to analyzing validation-study data when investigating reporting accuracy.

Methods

Descriptions of the sample and methodology are abbreviated because details have been described elsewhere (Baxter et al., 2003c). The appropriate institutional review board provided approval for human research. Parents and children were informed that each child could be interviewed up to two times, but were not informed that order prompts would differ between any child’s two interviews.

Summary of Sample and Data Collection

In Fall, 2000, all 915 fourth-grade children from 11 elementary schools in one district were invited to participate. From the 669 children providing written child assent and parental consent, a random sample of 121 children (31 black boys, 29 black girls, 32 white boys, 29 white girls) was selected who ate school meals (breakfast, lunch) on days when one or more observers were present. Only children who ate meals obtained at school were selected because it is difficult to unobtrusively identify food items brought from home (Simons-Morton et al., 1992). Although the sample was stratified by race and sex, the study was not powered to detect race or sex differences.

Reference information consisted of two days of observations for each of the 121 children, with two school meals (breakfast, lunch) observed on each day and at least four weeks separating the two days. For each observation, one of four research dietitians observed one to three children simultaneously and recorded food items and amounts eaten in servings on observation forms (Baxter et al., 2002; Baxter et al., 2003c). Entire meal periods were observed so food trading could be noted (Baxter et al., 2001). Inter-observer reliability, assessed weekly between pairs of observers, was satisfactory (Baglio et al., 2004; Baxter et al., 2003c).

Reported information consisted of two interviews for each of the 121 children; each interview concerned the previous day’s intake, and was conducted in the morning after each day of observation. Of the 121 children, half in each race/sex group were randomly assigned to forward-order prompts during the first interview and reverse-order prompts during the second interview; the other half of each group received the complementary assignment. Each interview was conducted by one of four research dietitians who hand-recorded children’s reports. Interviews were audio-recorded and transcribed. Interview protocols were modeled on the computerized interview of the Nutrition Data System for Research (NDS-R 4.03, Nutrition Coordinating Center, University of Minnesota, Minneapolis, MN, 2000). An interviewer had not observed her interviewee’s school meals on the previous day. No child was interviewed more than once by any interviewer. Quality control for interviews, assessed weekly, indicated that interviewers followed interview protocols (Baxter et al., 2003c; Shaffer et al., 2004).

Analytic Variables

Variables were prepared for two analytic approaches; each involved arithmetic transformations of reference information and reported information to energy and macronutrients. One approach used conventional variables that disregarded reporting errors. The other approach used reporting-error-sensitive variables.

Reference information was available for only school meals; thus, analyses were restricted to reported information about these parts of the children’s 24-hour dietary recalls. Children’s reports were considered reports about school meals if children named the meal appropriately, indicated “school” as the meal’s location, and reported mealtimes to within one hour of observed mealtimes (Baxter et al., 2002; Baxter et al., 2003c). Meals reported by the 121 children that did not satisfy these criteria were not treated as reports about school meals.

Reference amounts and reported amounts were recorded using qualitative terms and assigned quantities as none=0.00, taste=0.10, little bit=0.25, half=0.50, most=0.75, all=1.00, or the actual number of servings if more than one was observed or reported (Baxter et al., 2002; Baxter et al., 2003c). For reference items and reported items, per serving information about energy and macronutrients was obtained from the NDS-R database; for items not in that database, product information and recipes provided by the school district’s nutrition program were used.

Conventional energy and macronutrient variables

The quantified servings of reference items and reported items were multiplied by the per-serving energy and macronutrient values to calculate amounts of energy and macronutrients. To calculate total reference information for a child for a school day, reference amounts for energy and each macronutrient were summed across items observed eaten by the child at that day’s school meals. To calculate total reported information for a child for a school day, the same process was applied to items reported eaten by a child for that day’s school meals. Table 2 provides the calculations used to compute values of reference energy and reported energy, the conventional variables used to analyze reporting accuracy for energy, for one child on one school day.

Table 2.

Classifications and calculations used for assessing accuracy of reported energy in kilocalories compared to reference (observed) energy for one child for school breakfast and school lunch on one school day^a

Food itemb	Energy per servingc	Reference(observed) amount (in servings)	Reported amount (in servings)	Reference (observed) energyd	Reported energyd	M, O, or Ie	Over-reported energy from intrusionsf	Over-reported energy from matchesg	Corresponding energy from matchesh	Un-reported energy from matchesi	Un-reported energy from omissionsj
Breakfast
Vanilla milk	146	0.75	0.10	110	15	M	0	0	15	95	0
Cheese toast	169	0.50	0.00	85	0	O	0	0	0	0	85
Applesaucek	96	0.00	0.00	--	--	--	--	--	--	--	--
Syrup	90	0.00	0.10	0	9	I	9	0	0	0	0
French toast	240	0.00	0.25	0	60	I	60	0	0	0	0
Lunch
Hot dog	233	1.50	0.00	350	0	O	0	0	0	0	350
Vanilla milk	146	0.75	1.50	110	219	M	0	109	110	0	0
Brownie	229	1.00	1.00	229	229	M	0	0	229	0	0
Pineapplek	33	0.00	--	--	--	--	--	--	--	--	--
Hamburger	260	0.00	0.10	0	26	I	26	0	0	0	0
Total				884	558		95	109	354	95	435

^aAt school breakfast, this child was observed eating vanilla milk and cheese toast; for school breakfast, this child reported eating vanilla milk, syrup, and French toast. At school lunch, this child was observed eating hot dog, vanilla milk, and brownie; for school lunch, this child reported eating hamburger, vanilla milk, and brownie.

^bEvery food item that was observed, reported, or both observed and reported is listed.

^cFrom the Nutrition Data System for Research database (NDS-R, version 4.03, Nutrition Coordinating Center, University of Minnesota, Minneapolis, MN, 2000), product information, or recipes obtained from the school district’s nutrition program.

^dAll energy values are products of serving amounts and energy per serving. Reference energy is either corresponding or unreported; thus, in each row, corresponding energy from matches + unreported energy from matches + unreported energy from omissions = reference energy. Reported energy is either corresponding or overreported; thus, in each row, overreported energy from intrusions + overreported energy from matches + corresponding energy from matches = reported energy.

^eM = match; O = omission; I = intrusion. (See Table 1 for definitions.)

^fOverreported energy from intrusions = energy from intrusions.

^gOverreported energy from matches = the part of reported energy from matches that exceeded reference energy.

^hCorresponding energy from matches = the part of reported energy from matches that was less than or equal to reference energy.

ⁱUnreported energy from matches = the part of energy from matches for which reference energy exceeded reported energy.

^jUnreported energy from omissions = energy from omissions.

^kApplesauce was observed on this child’s tray, but none was eaten; this child mentioned applesauce during the interview but reported that none was eaten. Pineapple was observed on this child’s tray but none was eaten and pineapple was not reported by this child during the interview. Neither applesauce nor pineapple was included when calculating reporting accuracy for this child.

A report rate was calculated for energy and each macronutrient for each interview for each child. A report rate—defined as (reported amount/reference amount) × 100 (see Table 1)—has a lower bound of 0%, but no upper bound because what an individual can report is limitless. Report rates of ~100%, >100%, and <100% have been interpreted as high reporting accuracy, overreporting, and underreporting, respectively (Reynolds et al., 1990; Samuelson, 1970; Todd and Kretsch, 1986).

Reporting-error-sensitive energy and macronutrient variables

For each child, for each school meal on each school day, reference items were classified as matches or omissions, and reported items were classified as matches or intrusions. Reported items were classified as matches unless it was clear children did not describe items observed eaten (Baxter et al., 2002; Baxter et al., 2003c). The constituent energy and macronutrient values of matches, omissions, and intrusions were classified as corresponding, unreported, or overreported. Corresponding, unreported, and overreported amounts of energy and macronutrients were obtained by multiplying each corresponding, unreported, and overreported number of servings by the appropriate per-serving values of energy and macronutrients. For each school day for each child, total energy and total grams of each macronutrient for each of five categories of amounts—overreported from intrusions, overreported from matches, corresponding from matches, unreported from matches, and unreported from omissions—were calculated by summing across items. Table 2 illustrates these five categories for energy for one school day for one child.

A correspondence rate—a genuine measure of reporting accuracy—was calculated for energy and each macronutrient for each school day for each child (see Table 1). The correspondence rate—the percentage of the reference amount that is reported correctly—may range from 0% to 100% (indicating all items and amounts were reported correctly).

An inflation ratio—a measure of reporting error—was calculated for energy and each macronutrient for each school day for each child. An inflation ratio reflects reports of intrusions and/or overreported amounts of matches (see Table 1); it is the difference between a report rate and a correspondence rate. An inflation ratio has a lower bound of 0% (indicating no overreported amounts), but no upper bound because what an individual can report is limitless.

Analyses

Previously published food-item-level analyses, for which items were classified as matches, omissions, or intrusions, showed a sex-x-order prompt interaction for omission rates (the percentage of observed items not reported) (mixed-model analysis, p<0.008): Boys had lower omission rates (better accuracy) with reverse-order prompts, and girls had lower omission rates with forward-order prompts (Baxter et al., 2003c). Because order prompts were the focus of the original study, we investigated the sex-x-order prompt interaction in both the conventional and reporting-error-sensitive energy and macronutrient variables for this article.

Conventional energy and macronutrient variables

For each interview, t-tests for paired values compared mean differences of reported and reference energy and macronutrients to zero. For each interview, Pearson correlations were calculated between reference and reported energy and macronutrients. For report rates for energy and each macronutrient, separate mixed-model analyses of variance (ANOVA) were conducted to determine whether reporting accuracy depended on order prompts (forward, reverse), interview sequence (first, second), sex, race, and/or interactions. Interviewer and interview day were random effects in each model. For each analysis, a model with all main effects and all two-factor interactions was tested; nonsignificant interactions (p>0.10) were removed, and the analysis was run again. A significance criterion of 0.05 was established; all p values are two-tailed. Least squares means (LSMs) for significant effects and interactions were calculated from the final models.

Reporting-error-sensitive energy and macronutrient variables

For correspondence rates and inflation ratios for energy and each macronutrient, separate mixed-model ANOVAs were conducted to determine whether reporting accuracy depended on order prompts, sequence, sex, race, and/or interactions. Interviewer and interview day were random effects in each model. Models were tested as was described for report rates, and LSMs were calculated.

Caveat

In each set of analyses, because multiple energy and macronutrient variables were calculated from single sets of food items, the energy and macronutrient variables are not independent. However, treating such variables separately appears to be customary in dietetics and nutritional epidemiology.

Results

Conventional energy and macronutrient variables

Table 3 shows, for each order prompt for each sex, descriptive statistics for reference and reported energy and macronutrients, and statistics traditionally used to assess reporting accuracy. For each sex in each order prompt, reported amounts for energy and macronutrients were significantly less than reference amounts (ts<−4.00, ps<0.001).

Table 3.

Descriptive statistics for reference (observed) and reported energy (in kilocalories) and macronutrients (in grams), and traditional accuracy statistics (paired t-tests, Pearson correlations, report rates) using conventional variables,^a for each sex and each order prompt (forward, reverse)

		Reference	Reported	tb	Pearson Correlation	Report Rate Mean (SD)
		- - - - Mean (SD) - - - -
Girls (n=58)
Energy	Forward	825 (251)	577 (282)	−7.16****	0.51****	69% (5%)
	Reverse	803 (262)	443 (278)	−8.42****	0.28*	58% (5%)
Protein	Forward	31 (13)	25 (13)	−4.00***	0.57****	80% (6%)
	Reverse	31 (13)	20 (12)	−6.43***	0.41**	71% (6%)
Carbohydrate	Forward	123 (38)	86 (43)	−7.11****	0.53****	80% (6%)
	Reverse	120 (40)	66 (44)	−8.63****	0.37**	65% (6%)
Fat	Forward	26 (9)	16 (10)	−7.23****	0.50****	65% (6%)
	Reverse	24 (10)	12 (9)	−7.38****	0.17	55% (5%)
Boys (n=63)
Energy	Forward	915 (282)	582 (313)	−7.22****	0.25	67% (5%)
	Reverse	900 (265)	569 (341)	−7.12****	0.28*	62% (5%)
Protein	Forward	36 (13)	26 (14)	−4.82****	0.32*	82% (6%)
	Reverse	36 (13)	25 (14)	−5.33****	0.19	73% (6%)
Carbohydrate	Forward	138 (45)	86 (49)	−7.06****	0.21	73% (6%)
	Reverse	137 (44)	85 (53)	−7.19****	0.28*	72% (6%)
Fat	Forward	26 (11)	16 (10)	−7.07****	0.42***	67% (5%)
	Reverse	25 (8)	16 (11)	−6.93****	0.40**	62% (5%)

^aConventional variables are indifferent to reporting errors which include intrusions and their overreported amounts, and over-reported amounts from matches.

^bPaired t-tests compared the mean reported minus reference values to 0.

^*p<0.05

^**p<0.01

^***p<0.001

^****p<0.0001

All p values are two-tailed.

For girls, except for fat with reverse-order prompts, Pearson correlations between reference and reported amounts with each order prompt for energy and each macronutrient were significant (ps<0.05), ranging from 0.28 to 0.57. For boys, Pearson correlations were significant with forward-order prompts for protein (0.32) and fat (0.42) (ps<0.05), and with reverse-order prompts for energy, carbohydrate, and fat (ps<0.05), ranging from 0.28 to 0.48.

For girls, mean report rates for energy and macronutrients ranged from 65% to 80% with forward-order prompts and 55% to 71% with reverse-order prompts; for boys, they ranged from 67% to 82% with forward-order reports and 62% to 73% with reverse-order reports. Report rates were unrelated to order prompts, sex, or race, but significant main effects of sequence were found: Report rates were significantly higher for the first than second interview for energy (first=71% [LSM]; second=62%; F=3.90; p=0.049), protein (first=85%; second=72%; F=4.72; p=0.031) and carbohydrate (first=71%; second=61%; F=4.37; p=0.038) (but not fat, although the pattern was similar [first=67%; second=60%; F=1.76; p=0.19]).

Report rates were unrelated to any interaction including the sex-x-order prompt interaction.

Reporting-error-sensitive energy and macronutrient variables

Table 4 shows LSMs, standard errors, and inferential statistics for correspondence rates and inflation ratios for energy and macronutrients for each order prompt for each sex.

Table 4.

Least squares means (and standard errors) for reporting-error-sensitive variables^a (correspondence rates and inflation ratios) for energy and macronutrients for each sex and each order prompt (forward, reverse) (n = 58 girls and 63 boys)

		Correspondence Rateb		Inflation Ratioc
		Girls	Boys	Girls	Boys	Girls and Boys
Energy	Forward	45% (3%)	39% (3%)	26% (4%)	31% (3%)	29% (3%)
	Reverse	36% (3%)	45% (3%)	22% (4%)	20% (3%)	21% (3%)
		F=4.89, p=0.028				F=4.90, p=0.028
Protein	Forward	54% (4%)	46% (4%)	29% (5%)	40% (5%)	35% (4%)
	Reverse	42% (4%)	53% (4%)	30% (5%)	21% (5%)	25% (4%)
		F=6.02, p=0.015				F=3.13, p=0.078
Carbohydrate	Forward	43% (3%)	38% (3%)	29% (4%)	31% (4%)	30% (3%)
	Reverse	35% (3%)	43% (3%)	23% (4%)	21% (4%)	22% (3%)
		F=4.21, p=0.041				F=4.27, p=0.040
Fat	Forward	46% (4%)	38% (4%)	22% (4%)	32% (4%)	27% (3%)
	Reverse	36% (4%)	45% (4%)	20% (4%)	17% (4%)	18% (3%)
		F=4.27, p=0.040				F=4.08, p=0.045

^aReporting-error-sensitive variables are sensitive to reporting errors and therefore better represent reporting accuracy.

^bCorrespondence rates (defined in Table 1) were significantly better for girls with forward-order prompts but for boys with reverse-order prompts; F values are for tests of interaction of sex and order prompt from mixed models.

^cInflation ratios (defined in Table 1) were significantly worse with forward- than reverse-order prompts for energy, carbohydrate, and fat (but not protein, although the pattern was similar); F values are for tests of main effect of order prompt from mixed models. Least squares means and standard errors are provided for girls and boys separately by order prompt for descriptive purposes only.

All p values are two-tailed.

For girls, mean correspondence rates for energy and macronutrients ranged from 43% to 54% with forward-order prompts and 35% to 42% with reverse-order prompts; for boys, they ranged from 38% to 46% with forward-order reports and 43% to 53% with reverse-order reports. Only the sex-x-order prompt interactions were significant: Correspondence rates were higher for girls with forward-order prompts but boys with reverse-order prompts (ps≤0.041).

Inflation ratios for energy and macronutrients were unrelated to sequence, sex, race, or any interaction, including the sex-x-order prompt interaction. However, a significant main effect for order prompt was found: Inflation ratios were lower with reverse- than forward-order prompts for energy, carbohydrate, and fat (see Table 4; ps≤0.045) (but not for protein, although the pattern was similar). Mean inflation ratios for energy and macronutrients ranged from 27% to 35% with forward-order prompts and 18% to 25% with reverse-order prompts.

Table 5 shows descriptive statistics for the five categories of amounts for energy and macronutrients for each sex and each order prompt. The values in this table clarify that correctly-reported parts of reference amounts (i.e., corresponding amounts from matches) were higher for girls with forward-order prompts and boys with reverse-order prompts. Unreported amounts from omissions were considerable, and were not offset by overreported amounts from intrusions.

Table 5.

Means and standard deviations of reference and reported energy (in kilocalories) and macronutrients (in grams) according to five categories of amounts^* for reporting-error-sensitive variables for each sex and each order prompt (forward, reverse) (n = 58 girls and 63 boys)

	Overreported amount from intrusions		Overreported amount from matches		Corresponding amount from matches		Unreported amount from matches		Unreported amount from omissions
	Girls	Boys	Girls	Boys	Girls	Boys	Girls	Boys	Girls	Boys
Energy
Forward	151 (165)	161 (165)	51 (67)	61 (94)	375 (229)	358 (263)	64 (82)	71 (111)	386 (248)	486 (299)
Reverse	109 (137)	117 (143)	48 (64)	58 (85)	286 (213)	393 (270)	41 (65)	69 (115)	476 (299)	437 (342)
Protein
Forward	6 (8)	7 (8)	2 (3)	3 (5)	17 (11)	16 (12)	3 (4)	3 (6)	12 (11)	16 (13)
Reverse	4 (6)	4 (7)	2 (3)	3 (4)	13 (11)	18 (12)	2 (3)	3 (6)	16 (13)	15 (16)
Carbohydrate
Forward	24 (24)	23 (23)	8 (12)	10 (16)	54 (33)	52 (38)	10 (13)	11 (17)	60 (34)	75 (46)
Reverse	16 (20)	18 (20)	8 (12)	9 (13)	42(33)	57 (41)	7 (10)	12 (18)	71 (41)	68 (51)
Fat
Forward	4 (6)	5 (6)	1 (1)	1 (2)	11 (9)	10 (10)	2 (3)	2 (4)	12 (10)	14 (10)
Reverse	3 (5)	3 (5)	1 (2)	1 (3)	8 (7)	11 (8)	1 (2)	1 (3)	15 (12)	13 (10)

^*The five categories of amounts are defined in Table 1. Reporting-error-sensitive variables are sensitive to reporting errors and therefore better represent reporting accuracy.

Discussion

Conventional energy and macronutrient variables provided the following picture of reporting accuracy: First, underreporting was significant. Underreporting is the concern on which most attention has been focused (Livingstone and Robson, 2000; Maurer et al., 2006; Tooze et al., 2004). Nevertheless, second, report rates were moderately high. Third, report rates were significantly related to sequence: For energy, protein, and carbohydrate, they were significantly lower for the second than first interview. Effects of sequence, with less intake reported during a second than first administration, have been found when children completed food frequency questionnaires (FFQs) twice, at intervals from two weeks to one year (Cullen et al., 1999; Domel et al., 1994a; Jensen et al., 2004; Rockett et al., 1995). Children have provided multiple 24-hour dietary recalls for FFQ relative-validity studies, but sequence effects on recalls usually have not been investigated; instead, intake has been averaged across recalls and then compared to FFQ(s) (Cullen et al., 1999; Field et al., 1999; Jensen et al., 2004; Rockett et al., 1997).

Analyses of reporting-error-sensitive energy and macronutrient variables provided a different picture of reporting accuracy: First, correspondence rates were lower than report rates, and inflation ratios were considerable. These results indicate that much of what was reported was incorrect, so reporting accuracy for energy and macronutrients was worse than what conventional report rates suggested. Second, correspondence rates indicated that correct reporting of energy and macronutrients was better for girls with forward-order prompts but boys with reverse-order prompts. Third, inflation ratios indicated that children had fewer reporting errors for energy, carbohydrate, and fat with reverse-than forward-order prompts. Fourth, means for five categories of amounts showed that energy and macronutrients from omissions were not balanced by energy and macronutrients from intrusions. Omissions and intrusions may result from distinct psychological mechanisms and processes (see Koriat and Goldsmith, 1994; Koriat and Goldsmith, 1996).

Previously-published food-item-level analyses showed that reporting accuracy was better for boys with reverse-order prompts and girls with forward-order prompts (Baxter et al., 2003c). Current analyses of conventional energy and macronutrient variables indicated that children, although moderately accurate, underreported, and that reporting accuracy was unrelated to sex and order prompts, but was related to sequence; these results conflict with food-item-level results. In contrast, current analyses of reporting-error-sensitive energy and macronutrient variables indicated that children’s reporting accuracy was much worse than implied by conventional measures, and was better for boys with reverse-order prompts and girls with forward-order prompts.

Research on autobiographical memory (Jobe et al., 1990; Jobe et al., 1993; Loftus, 1984; Loftus and Fathi, 1985; Loftus et al., 1992; Whitten and Leonard, 1981) and eyewitness testimony (Geiselman and Callot, 1990; Geiselman et al., 1985; Geiselman et al., 1986) has suggested that accuracy of recollection may depend on order prompts. Numerous validation studies have assessed children’s reporting accuracy by comparing dietary recalls to observation (Baranowski et al., 2002; Baxter et al., 1997; Baxter et al., 2000; Baxter et al., 2002; Baxter et al., 2003a; Baxter et al., 2003b; Baxter et al., 2003c; Baxter et al., 2004; Carter et al., 1981; Crawford et al., 1994; Domel et al., 1994b; Lytle et al., 1993; Lytle et al., 1998; Reynolds et al., 1990; Samuelson, 1970; Stunkard and Waxman, 1981; Todd and Kretsch, 1986; Warren et al., 2003). To our knowledge, the study in which these data were collected (Baxter et al., 2003c) is the only validation study to have examined systematically the effect of order prompts on children’s dietary reporting accuracy. The results provided here extend previously-published food-item-level results and suggest that children’s reporting accuracy for energy and macronutrients for the previous-day’s intake is better for boys with reverse-order prompts and for girls with forward-order prompts.

Macronutrients were not energy-adjusted because that could erroneously deflate or inflate macronutrient values. For example, if an individual underreported soft drink consumption, then reported energy and reported carbohydrate would be too low, but reported protein and reported fat would not; thus, this individual’s energy-adjusted reported protein and reported fat would be erroneously inflated.

Limitations of this illustration include specific aspects of the original study’s design and methods. Only two school meals were observed instead of an entire 24-hour period. Quantification involved conversion of qualitative serving-sizes and amounts of standard servings; however, these processes were applied consistently to both reference information and reported information to obtain estimates of energy and macronutrients. Because the subjects were children, a lenient criterion was used to classify reported items as matches; this may have overestimated reporting accuracy.

Several strengths offset these limitations. The validation method, observation of school meals, may be the best method for validating dietary reports (Mertz, 1992). Quality control for observations and interviews, both assessed throughout data collection, was acceptable. Children’s reports were obtained without parental assistance so that errors in children’s reporting accuracy could be identified; this was appropriate because the observed school meals were eaten when parents were absent.

Conclusions

These results have several implications for dietary validation studies which, in turn, inform the direction of dietary intake analyses for studies that rely on self-reports to assess diet and disease relationships or to examine the effectiveness of dietary change interventions. Conventional report rates contain a component based on error and thus have little logical connection to reporting accuracy. Conventional energy and macronutrient variables such as report rates, which disregard reporting errors, 1) almost surely overestimate reporting accuracy, and 2) may mask the effects of predictor variables, such as order prompts and sex, on reporting accuracy. Dietary validation studies that investigate reporting accuracy, or that test specific methods (such as order prompts) to improve the accuracy of self-reports, should use energy and macronutrient variables that are sensitive to reporting errors instead of simply comparing total reported information to total reference information.