Shortening the retention interval of 24-hour dietary recalls increases fourth-grade children’s accuracy for reporting energy and macronutrient intake at school meals

Abstract

Background

Accurate information about children’s intake is crucial for national nutrition policy and for research and clinical activities. To analyze accuracy for reporting energy and nutrients, most validation studies utilize the conventional approach which was not designed to capture errors of reported foods and amounts. The reporting-error-sensitive approach captures errors of reported foods and amounts.

Objective

To extend results to energy and macronutrients for a validation study concerning retention interval (elapsed time between to-be-reported meals and the interview) and accuracy for reporting school-meal intake, the conventional and reporting-error-sensitive approaches were compared.

Design and participants/setting

Fourth-grade children (n=374) were observed eating two school meals, and interviewed to obtain a 24-hour recall using one of six interview conditions from crossing two target periods (prior-24-hours; previous-day) with three interview times (morning; afternoon; evening). Data were collected in one district during three school years (2004–2005; 2005–2006; 2006–2007).

Main outcome measures

Report rates (reported/observed), correspondence rates (correctly reported/observed), and inflation ratios (intruded/observed) were calculated for energy and macronutrients.

Statistical analyses performed

For each outcome measure, mixed-model analysis of variance was conducted with target period, interview time, their interaction, and sex in the model; results were adjusted for school year and interviewer.

Results

Conventional approach — Report rates for energy and macronutrients did not differ by target period, interview time, their interaction, or sex. Reporting-error-sensitive approach — Correspondence rates for energy and macronutrients differed by target period (four P-values<0.0001) and the target-period by interview-time interaction (four P-values<0.0001); inflation ratios for energy and macronutrients differed by target period (four P-values<0.0001), and inflation ratios for energy and carbohydrate differed by the target-period by interview-time interaction (both P-values<0.005).

Conclusions

Shorten the retention interval of dietary recalls to increase accuracy for reporting energy and macronutrients. For validation studies, obtain reference information from a method that provides details about foods and amounts consumed, and use an analytic approach that captures errors of reported foods and amounts.

INTRODUCTION

Despite decades of refining dietary-assessment methods, the accuracy of dietary intake data continues to be problematic (1–5). Studies that examine relationships between diet and disease often have null findings and inconsistent results (6). As Schatzkin and colleagues stated in 2009 (6), “The inconsistency and uncertainty in the nutritional epidemiology…can be interpreted in two ways: (a) important, public health-relevant, causal links…are few, and many of the long-standing hypotheses are simply wrong; (b) many of these long-standing hypotheses are right, but methodologic difficulties have prevented us from generating the requisite evidence. The first interpretation is really one of exclusion: as long as methodologic problems prevent us from seeing the truth we cannot rule out the truth.” Clearly, improved dietary assessment tools could be immensely valuable.

Accurate information about children’s intake is crucial for national nutrition policy and for research and clinical activities. For millions of children, school meals are major sources of food (7,8). Accurate information about children’s school-meal intake is increasingly needed to address concerns about whether school meals promote children’s health and well-being (9,10). For many studies, children must self-report school-meal intake because parents lack first-hand knowledge of children’s intake at school. Self-report methods such as dietary recalls are generally used with children over age nine years, or third grade (11). Children in upper elementary-school grades have, without parental assistance, provided recalls of intake at school, and 24-hour recalls, for national surveys (12,13) and research studies (14–21). Dietary recalls are appropriate for children considering concerns that children lack the cognitive skills needed to complete food frequency questionnaires (15,22–24) and that completing food records may alter eating behavior (24–26).

Methodological research demonstrates that study design influences the accuracy of dietary reports obtained from children. For example, the primary aim of a 2009 validation study was to investigate the effects of retention interval (elapsed time between to-be-reported meals and the interview) on children’s accuracy for reporting school-meal intake during 24-hour dietary recalls (27). Fourth-grade children were observed eating two school meals (breakfast and lunch) and interviewed to obtain a 24-hour recall in one of six interview conditions created by crossing two target periods (prior-24-hours [the 24 hours immediately preceding the interview]; previous-day [midnight to midnight of the day before the interview]) with three interview times (morning; afternoon; evening). Food-item-level analyses for omission rates (percentage of observed but unreported items), intrusion rates (percentage of reported but unobserved items), and total inaccuracy (a measure that combined reporting errors for items and amounts) found that children’s accuracy for reporting school-meal intake was better for prior-24-hour recalls than previous-day recalls, best for prior-24-hour recalls obtained in the afternoon and evening, and worst for previous-day recalls obtained in the afternoon and evening (27).

This article examines reporting accuracy for energy and macronutrients (protein, carbohydrate, fat) in the retention-interval validation study (27). Although people report intake as food, it is common to investigate accuracy of reported energy and nutrients. However, concern has been raised about how this is accomplished (28,29). In three 2007 articles (30–32), two approaches were compared. The “conventional approach” was not designed to capture reporting errors because all reported items and their reported amounts are converted to energy and nutrients. The conventional approach uses paired t-tests and correlations to compare mean differences between reported and reference energy and macronutrients, and calculates a report rate (reported/reference) for energy and each nutrient. In contrast, the “reporting-error-sensitive approach” is sensitive to reporting errors for food items and amounts because it classifies reported items as matches (items in both the reference and reported information) or intrusions (items in the reported information but not in the reference information), and then classifies reported amounts as corresponding, unreported, or overreported. The reporting-error-sensitive approach calculates a correspondence rate (correctly reported/reference) and an inflation ratio (intruded/reference) for energy and each nutrient. In the three articles (30–32), each of which used data from a unique validation study (33–35) conducted with different samples of children, results showed that the conventional approach overestimated accuracy for reporting energy and macronutrients and failed to reveal effects of manipulated aspects of dietary recall interviews. Specifically, in the first article (30), the conventional approach failed to detect improvements in accuracy for reporting energy and macronutrients over multiple interviews that were evident with the reporting-error-sensitive approach, and with food-item-level analyses conducted earlier (33). In the second article (31), the conventional approach found a sequence effect (first versus second interview) on accuracy for reporting energy and macronutrients that was not found with food-item-level analyses conducted earlier (34), but failed to detect effects of reporting-order prompts (forward [morning-to-evening]; reverse [evening-to-morning]) and sex that were evident with the reporting-error-sensitive approach, and with food-item-level analyses conducted earlier (34). In the third article (32), although no significant effect of interview modality (in person; by telephone) on accuracy was found with the conventional or reporting-error-sensitive approaches, or with food-item-level analyses conducted earlier (35), the conventional approach’s report rates for energy and macronutrients were higher than the reporting-error-sensitive approach’s correspondence rates.

In extending results for the retention-interval validation study (27) to energy and macronutrients, the current article compared the conventional and reporting-error-sensitive approaches. This article’s goal is to encourage a better approach to analyzing data from validation studies so that dietary reporting errors can be better understood.

METHODS

Summary of sample, study design, and data collection

This section summarizes the sample, study design, and data collection for the retention-interval validation study which have been described in detail elsewhere (27).

The University of South Carolina’s institutional review board for human subjects approved the project. Written parental consent and child assent were obtained.

During the 2004–2005, 2005–2006, and 2006–2007 school years, children from fourth-grade classes at 17, 17, and 8 elementary schools, respectively, in one district were invited to participate. Across the three school years, of the 2,391 children invited, 1,780 children (74%) agreed. Offer-versus-serve foodservice had been implemented by the district, so children could refuse some meal items (36). Each of 374 children (50% girls; 96% African American; mean±SD age=10.00±0.88 years) was observed eating school breakfast and school lunch, and interviewed to obtain a 24-hour recall using one of six interview conditions created by crossing two target periods (prior-24-hours; previous-day) with three interview times (morning; afternoon; evening). Assignment of children to interview condition was random with the constraint that each condition in the final sample had 62 or 64 children (50% girls) (27).

Reference information was obtained by research staff who observed children eating school meals. Observers followed a written protocol based on procedures used earlier (33–35,37–39). For children randomized to prior-24-hour recalls in the morning, lunch was observed on one day and breakfast on the next day; for all other children, breakfast and lunch were observed on the same day. An observer watched one to three children simultaneously and noted food items and amounts eaten in servings of standardized school-meal portions. Observations occurred with children seated according to their school’s typical arrangement, and during entire, regular meal periods to note food trades (40–42). Interobserver reliability was assessed using established procedures (34,35,37–39,43) at least weekly for each observer throughout data collection. For the three school years, mean agreement between observers to within one-fourth serving on amounts eaten ranged from 98% to 100% for breakfast, and from 94% to 97% for lunch (27); these levels of agreement are satisfactory (42,44).

Reported information was obtained by non-observing research staff who interviewed individual children to obtain 24-hour dietary recalls without parental assistance. Morning and afternoon interviews were conducted in person in private locations at children’s schools after breakfast and lunch, respectively; evening interviews were conducted by telephone between 6:30 p.m. and 9:00 p.m. In an earlier validation study (35), no significant differences were found between in-person and telephone dietary recalls in fourth-grade children’s accuracy. Written interview protocols, described in detail elsewhere (27), were similar to ones used earlier (37,38) and modeled on the Nutrition Data System for Research (NDSR) protocol (Nutrition Coordinating Center, University of Minnesota, Minneapolis). Instead of using NDSR software during interviews, interviewers used paper forms to note information reported by children. As in earlier studies (33–35,37–39), children reported amounts eaten in qualitative terms (e.g., taste, little bit, most). Each interview was audio recorded and transcribed. Quality control for interviews was assessed using established procedures (34,35,37–39,45); only interviews that passed quality control were analyzed. As described in detail elsewhere (27), of 442 interviews conducted, 46 failed quality control due to interviewer errors during interviews, and an additional 22 interviews were excluded from analyses for other reasons (e.g., observation errors; telephone problems).

Analytic variables

Analyses were restricted to reports of school meals because reference information was available only for school meals. Following criteria used earlier (33–35,37–39) and applied consistently to all recalls, meals in children’s 24-hour recalls were treated as referring to school meals if children identified “school” as the location, referred to breakfast as “school breakfast” or “breakfast”, referred to lunch as “school lunch” or “lunch”, and reported mealtimes to within one hour of observed mealtimes.

As in earlier studies (33–35,37,39), qualitative labels used to record reference information from observations and reported information from recalls were assigned numeric values as none=0.00, taste=0.10, little bit=0.25, half=0.50, most=0.75, all=1.00, or as the actual number of servings if >1.00 serving was observed and/or reported. For reference items and for reported items, information about energy and macronutrients for standardized school-meal portions was obtained primarily from the NDSR database, but sometimes from the school district’s nutrition program.

Variables were prepared for two analytic approaches — conventional and reporting-error-sensitive. Each approach involved arithmetic conversion of reference information and reported information to energy (in kilocalories) and macronutrients (in grams).

Conventional variables

For reference items, quantified servings were multiplied by per-serving energy and macronutrient values; for reported items, the same process was applied. Reference amounts for energy and each macronutrient were summed across items a child was observed to have eaten during the target period’s two school meals. Reported amounts for energy and each macronutrient were summed across all items a child reported as having eaten for the target period’s two school meals irrespective of whether items or amounts were reported correctly.

As in earlier studies (30–32), a report rate for energy and each macronutrient was calculated for each child (see Table 1, footnote h). Report rates have a lower bound of 0% and no upper bound. Customarily, report rates close to 100%, >100%, and <100% have been interpreted as indicating high reporting accuracy, overreporting, and underreporting, respectively (46–48).

Table 1.

Results from the conventional approach ^a for analyzing energy and macronutrients, by target period ^b and interview condition ^c

	n	Reference amount d	Reported amount e	tf	rg	Report Rate (in %) h
		Mean ± SD				Mean ± SD	Minimum; Median; Maximum i
Energy (kcal)
Target period
Prior 24 hour	172	797 ± 262	626 ± 297	–7.09 ****	0.36 ****	85 ± 55	7; 80; 496
Previous day	163	789 ± 263	657 ± 296	–5.27 ****	0.36 ****	90 ± 46	10; 84; 298
Mixed-model results j						F = 2.03, P = 0.1550
Interview condition
Prior 24 hour / morning	58	750 ± 220	551 ± 306	–4.48 ****	0.20	85 ± 76	7; 70; 496
Prior 24 hour / afternoon	58	858 ± 287	655 ± 325	–5.53 ****	0.59 ****	77 ± 34	23; 75; 186
Prior 24 hour / evening	56	782 ± 268	672 ± 241	–2.52 *	0.18	95 ± 46	22; 90; 319
Previous day / morning	48	848 ± 280	699 ± 308	–3.16 **	0.39 **	87 ± 41	24; 82; 225
Previous day / afternoon	54	766 ± 232	608 ± 275	–3.82 ***	0.29 *	85 ± 42	10; 83; 190
Previous day / evening	61	761 ± 271	668 ± 302	–2.25 *	0.36 **	98 ± 51	16; 90; 298
Mixed-model results k						F = 0.52, P = 0.5969
Protein (g)
Target period
Prior 24 hour	172	30 ± 12	24 ± 13	–5.37 ****	0.33 ****	105 ± 180	2; 86; 2124
Previous day	163	29 ± 10	25 ± 13	–3.51 ***	0.34 ****	99 ± 68	0; 88; 547
Mixed-model results j						F = 1.39, P = 0.2400
Interview condition
Prior 24 hour / morning	58	29 ± 10	21 ± 14	–3.83 ***	0.29 *	112 ± 275	2; 72; 2124
Prior 24 hour / afternoon	58	31 ± 13	24 ± 14	–3.57 ***	0.43 **	85 ± 52	3; 75; 276
Prior 24 hour / evening	56	29 ± 12	26 ± 10	–1.81	0.26	117 ± 137	17; 100; 1033
Previous day / morning	48	31 ± 10	29 ± 14	–1.54	0.60 ****	93 ± 39	13; 93; 203
Previous day / afternoon	54	28 ± 9	22 ± 11	–3.85 ***	0.18	83 ± 50	10; 77; 266
Previous day / evening	61	27 ± 12	26 ± 13	–0.98	0.24	117 ± 91	0; 102; 547
Mixed-model results k						F = 1.38, P = 0.2540
Carbohydrate (g)
Target period
Prior 24 hour	172	104 ± 33	84 ± 40	–6.42 ****	0.39 ****	86 ± 46	8; 85; 264
Previous day	163	107 ± 40	88 ± 41	–5.19 ****	0.38 ****	90 ± 45	10; 87; 327
Mixed-model results j						F = 0.79, P = 0.3743
Interview condition
Prior 24 hour / morning	58	98 ± 32	73 ± 37	–3.85 ***	0.04	87 ± 60	8; 74; 264
Prior 24 hour / afternoon	58	111 ± 36	87 ± 46	–5.55 ****	0.70 ****	78 ± 31	10; 80; 153
Prior 24 hour / evening	56	104 ± 31	92 ± 35	–2.17 *	0.27 *	94 ± 41	25; 89; 255
Previous day / morning	48	114 ± 44	91 ± 41	–3.69 ***	0.44 **	85 ± 38.	22; 83; 207
Previous day / afternoon	54	105 ± 36	84 ± 39	–3.44 **	0.29 *	88 ± 45	10; 88; 204
Previous day / evening	61	102 ± 39	91 ± 43	–1.97	0.41 ***	97 ± 51	27; 88; 327
Mixed-model results k						F = 0.40, P = 0.6740
Fat (g)
Target period
Prior 24 hour	172	29 ± 15	21 ± 13	–6.58 ****	0.46 ****	97 ± 150	2; 75; 1765
Previous day	163	27 ± 13	23 ± 13	–4.05 ***	0.35 ****	102 ± 90	0; 82; 791
Mixed-model results j						F = 3.43, P = 0.0650
Interview condition
Prior 24 hour / morning	58	27 ± 12	19 ± 14	–4.51 ****	0.48 ***	107 ± 241	4; 61; 1765
Prior 24 hour / afternoon	58	32 ± 17	23 ± 14	–4.61 ****	0.55 ****	82 ± 61	2; 70; 319
Prior 24 hour / evening	56	27 ± 17	22 ± 11	–2.46 *	0.36 **	101 ± 73	8; 98; 506
Previous day / morning	48	30 ± 13	25 ± 15	–2.14 *	0.35 *	96 ± 71	13; 83; 371
Previous day / afternoon	54	26 ± 12	21 ± 11	–2.72 **	0.31 *	92 ± 55	5; 80; 214
Previous day / evening	61	27 ± 13	23 ± 14	–2.19 *	0.37 **	116 ± 122	0; 88; 791
Mixed-model results k						F = 1.19, P = 0.3058

^aThe conventional approach is an analytic approach to evaluate accuracy for reporting energy and nutrients; it was not designed to capture reporting errors because all reported food items along with their reported amounts are converted to energy and nutrients.

^bTarget period is the period of time covered by a 24-hour dietary recall. The prior-24-hour target period concerns the 24 hours immediately preceding the interview. The previous-day target period concerns midnight to midnight of the day before the interview.

^cThe six interview conditions were created by crossing two target periods with three interview times (morning; afternoon; evening); they are labeled and defined as 1) prior 24 hour / morning — recall about the prior-24-hour target period obtained in a morning interview; 2) prior 24 hour / afternoon — recall about the prior-24-hour target period obtained in an afternoon interview; 3) prior 24 hour / evening — recall about the prior-24-hour target period obtained in an evening interview; 4) previous day / morning — recall about the previous-day target period obtained in a morning interview; 5) previous day / afternoon — recall about the previous-day target period obtained in an afternoon interview; and 6) previous day / evening — recall about the previous-day target period obtained in an evening interview.

^dThe reference amount was the amount observed eaten at school breakfast and school lunch. It was calculated for energy and each macronutrient for each child.

^eThe reported amount was from the school breakfast and school lunch parts of children’s 24-hour dietary recalls. It was calculated for energy and each macronutrient for each child.

^fFor each target period and interview condition, paired t-tests were conducted to compare mean differences between reported and reference amounts of energy and of each macronutrient with zero. Differences were calculated as reported minus reference, so negative t values indicate underreporting. P values are indicated as * for P<0.05, ** for P<0.01, *** for P<0.001, and **** for P<0.0001.

^gFor each target period and interview condition, Pearson correlations were calculated, over children, between reference and reported energy and each macronutrient. Pearson correlations were tested for differences with zero. P values are indicated as * for P<0.05, ** for P<0.01, *** for P<0.001, and **** for P<0.0001.

^hReport rate is the reported percentage of the reference (i.e., observed) amount, calculated for energy and each macronutrient for each child as: ([sum of reported amounts] / [sum of reference amounts]) × 100. It is a measure of reporting accuracy which is indifferent to reporting errors. It has a lower bound of 0%, which indicates nothing was reported. It has no upper bound because there is no limit on what a person can report. Customary interpretation of report rates is that values close to 100%, >100%, and <100% indicate high reporting accuracy, overreporting, and underreporting, respectively.

ⁱChildren’s reports were compared to direct observation of school meals, so there was nothing suspect about maximum values, and it would be inappropriate to classify them as outliers and to exclude them from analyses.

^jMixed-model ANOVA results concerning the effect of target period on report rates.

^kMixed-model ANOVA results concerning the interaction of target period with interview time on report rates.

Reporting-error-sensitive variables

For each child, reference items were classified as matches or omissions, and reported items were classified as matches or intrusions. Following procedures used earlier (33–35,37–39), reported items were classified as matches unless it was clear that children had not described items observed eaten. As detailed in the Table 3 legend, the constituent amounts of energy and macronutrients of matches were classified as corresponding, unreported, or overreported; the constituent amounts of energy and macronutrients of omissions were classified as unreported; and the constituent amounts of energy and macronutrients of intrusions were classified as overreported (30–32). Each corresponding, unreported, and overreported number of servings was multiplied by the appropriate per-serving values of energy and macronutrients, and summed across a child’s items for the target period’s two school meals to create per-child energy and macronutrient values for each amount category.

Table 3.

Descriptive statistics for five amount categories used to create variables for the reporting-error-sensitive approach ^a for analyzing energy and macronutrients, by target period ^b and interview condition ^c

	n	Overreported amount from intrusions d	Overreported amount from matches e	Corresponding amount from matches f	Unreported amount from matches g	Unreported amount from omissions h
		Mean ± SD
Energy (kcal)
Target period
Prior 24 hour	172	154 ± 169	47 ± 70	425 ± 246	86 ± 126	286 ± 231
Previous day	163	309 ± 228	42 ± 80	306 ± 255	57 ± 93	425 ± 264
Interview condition
Prior 24 hour / morning	58	173 ± 193	43 ± 61	335 ± 221	57 ± 87	358 ± 224
Prior 24 hour / afternoon	58	132 ± 146	46 ± 71	477 ± 277	100 ± 108	281 ± 235
Prior 24 hour / evening	56	156 ± 165	51 ± 78	464 ± 212	101 ± 167	216 ± 214
Previous day / morning	48	238 ± 244	45 ± 72	416 ± 275	88 ± 117	344 ± 285
Previous day / afternoon	54	309 ± 195	40 ± 74	258 ± 221	48 ± 90	460 ± 229
Previous day / evening	61	364 ± 229	42 ± 91	262 ± 242	41 ± 67	458 ± 266
Protein (g)
Target period
Prior 24 hour	172	4 ± 7	2 ± 3	17 ± 11	4 ± 6	9 ± 10
Previous day	163	10 ± 10	2 ± 4	13 ± 11	2 ± 4	13 ± 11
Interview condition
Prior 24 hour / morning	58	5 ± 8	2 ± 3	14 ± 11	2 ± 4	12 ± 10
Prior 24 hour / afternoon	58	4 ± 7	2 ± 4	18 ± 11	5 ± 6	8 ± 10
Prior 24 hour / evening	56	4 ± 6	2 ± 3	19 ± 9	4 ± 8	5 ± 8
Previous day / morning	48	8 ± 10	3 ± 5	18 ± 13	3 ± 4	10 ± 10
Previous day / afternoon	54	9 ± 8	1 ± 2	11 ± 9	3 ± 5	15 ± 10
Previous day / evening	61	13 ± 11	2 ± 3	11 ± 11	2 ± 4	15 ± 12
Carbohydrate (g)
Target period
Prior 24 hour	172	24 ± 24	6 ± 10	54 ± 32	10 ± 13	41 ± 30
Previous day	163	44 ± 30	5 ± 10	38 ± 34	8 ± 14	60 ± 37
Interview condition
Prior 24 hour / morning	58	27 ± 26	6 ± 9	41 ± 25	8 ± 13	50 ± 31
Prior 24 hour / afternoon	58	21 ± 21	6 ± 8	61 ± 36	11 ± 13	40 ± 29
Prior 24 hour / evening	56	25 ± 25	7 ± 11	60 ± 30	10 ± 13	34 ± 29
Previous day / morning	48	33 ± 29	5 ± 7	52 ± 38	11 ± 17	51 ± 44
Previous day / afternoon	54	45 ± 27	5 ± 10	33 ± 29	7 ± 14	65 ± 32
Previous day / evening	61	52 ± 31	5 ± 12	33 ± 33	6 ± 9	63 ± 36
Fat (g)
Target period
Prior 24 hour	172	4 ± 8	2 ± 3	16 ± 12	4 ± 7	9 ± 11
Previous day	163	10 ± 11	2 ± 4	11 ± 11	2 ± 4	14 ± 12
Interview condition
Prior 24 hour / morning	58	5 ± 9	1 ± 2	13 ± 11	2 ± 3	12 ± 9
Prior 24 hour / afternoon	58	4 ± 7	2 ± 3	18 ± 13	4 ± 5	10 ± 12
Prior 24 hour / evening	56	4 ± 7	2 ± 4	16 ± 10	5 ± 11	6 ± 10
Previous day / morning	48	8 ± 12	2 ± 3	15 ± 11	4 ± 5	11 ± 11
Previous day / afternoon	54	10 ± 10	1 ± 3	9 ± 9	1 ± 3	15 ± 11
Previous day / evening	61	11 ± 11	2 ± 4	10 ± 11	1 ± 3	16 ± 12

^aThe reporting-error-sensitive approach is an analytic approach to evaluate accuracy for reporting energy and nutrients; it is sensitive to reporting errors for food items and amounts.

^bTarget period is the period of time covered by a 24-hour dietary recall. The prior-24-hour target period concerns the 24 hours immediately preceding the interview. The previous-day target period concerns midnight to midnight of the day before the interview.

^cThe six interview conditions were created by crossing two target periods with three interview times (morning; afternoon; evening); they are labeled and defined as 1) prior 24 hour / morning — recall about the prior-24-hour target period obtained in a morning interview; 2) prior 24 hour / afternoon — recall about the prior-24-hour target period obtained in an afternoon interview; 3) prior 24 hour / evening — recall about the prior-24-hour target period obtained in an evening interview; 4) previous day / morning — recall about the previous-day target period obtained in a morning interview; 5) previous day / afternoon — recall about the previous-day target period obtained in an afternoon interview; and 6) previous day / evening — recall about the previous-day target period obtained in an evening interview.

^dThe overreported amount from an intrusion is the entire reported amount for each intrusion. An intrusion is food item in the reported information but not in the reference (i.e., observed) information; in other words, an intrusion is a food item that was not eaten but was reported eaten in some non-zero amount for that meal.

^eThe overreported amount from a match is the amount by which the reported amount exceeds the reference amount (or zero if the reported amount is less than or equal to the reference amount). A match is a food item in both the reference information and reported information; in other words, a match is a food item that was eaten in some non-zero amount and was reported eaten in some non-zero amount for that meal.

^fThe corresponding amount from a match is the smaller value of the reported amount and the reference amount (or the reported amount if it is equal to the reference amount).

^gThe unreported amount from a match is the amount by which the reference amount exceeds the reported amount (or zero if the reference amount is less than or equal to the reported amount).

^hThe unreported amount from an omission is the entire reference amount for each omission. An omission is a food item in the reference information but not in the reported information; in other words, an omission is a food item that was eaten in some non-zero amount but was not reported eaten for that meal.

As in earlier studies (30–32), a correspondence rate for energy and each macronutrient was calculated for each child (see Table 2, footnote d). Correspondence rates have a lower bound of 0% and an upper bound of 100%. Larger correspondence rates indicate better reporting accuracy.

Table 2.

Results from the reporting-error-sensitive approach ^a for analyzing energy and macronutrients, by target period ^b and interview condition ^c

	n	Correspondence Rate (in %) d		Inflation Ratio (in %) e
		Mean ± SD	Minimum; Median; Maximum f	Mean ± SD	Minimum; Median; Maximum f
Energy (kcal)
Target period
Prior 24 hour	172	53 ± 26	0; 56; 100	32 ± 52	0; 19; 496
Previous day	163	37 ± 27	0; 33; 100	53 ± 47	0; 44; 266
Mixed-model results g		F = 32.08, P < 0.0001		F = 37.42, P <0.0001
Interview condition
Prior 24 hour / morning	58	45 ± 26	0; 42; 100	40 ± 75	0; 24; 496
Prior 24 hour / afternoon	58	54 ± 23	0; 59; 94	23 ± 24	0; 15; 102
Prior 24 hour / evening	56	61 ± 25	0; 66; 100	33 ± 43	0; 20; 285
Previous day / morning	48	48 ± 29	0; 55; 100	39 ± 43	0; 24; 225
Previous day / afternoon	54	33 ± 25	0; 31; 85	52 ± 36	0; 43; 147
Previous day / evening	61	32 ± 25	0; 31; 91	66 ± 55	0; 55; 266
Mixed-model results h		F = 12.60, P < 0.0001 i		F = 5.48, P = 0.0046 j
Protein (g)
Target period
Prior 24 hour	172	59 ± 31	0; 64; 100	45 ± 182	0; 13; 2124
Previous day	163	43 ± 32	0; 36; 100	55 ± 69	0; 34; 522
Mixed-model results g		F = 21.26, P < 0.0001		F = 30.14, P < 0.0001
Interview condition
Prior 24 hour / morning	58	49 ± 33	0; 45; 100	63 ± 279	0; 17; 2124
Prior 24 hour / afternoon	58	60 ± 28	0; 62; 100	25 ± 38	0; 8; 204
Prior 24 hour / evening	56	69 ± 28	0; 77; 100	48 ± 144	0; 17; 1033
Previous day / morning	48	56 ± 33	0; 63; 100	37 ± 42	0; 22; 178
Previous day / afternoon	54	39 ± 31	0; 33; 100	44 ± 40	0; 34; 232
Previous day / evening	61	37 ± 31	0; 33; 100	80 ± 95	0; 46; 522
Mixed-model results h		F = 10.84, P < 0.0001 i		F = 2.08, P = 0.1264
Carbohydrate (g)
Target period
Prior 24 hour	172	51 ± 25	0; 53; 100	35 ± 40	0; 25; 258
Previous day	163	35 ± 26	0; 31; 100	56 ± 45	0; 44; 327
Mixed-model results g		F = 36.21, P < 0.0001		F = 35.07, P < 0.0001
Interview condition
Prior 24 hour / morning	58	43 ± 25	0; 41; 100	44 ± 54	0; 26; 258
Prior 24 hour / afternoon	58	53 ± 22	0; 57; 94	25 ± 21	0; 22; 95
Prior 24 hour / evening	56	58 ± 25	0; 64; 100	36 ± 37	0; 27; 214
Previous day / morning	48	45 ± 28.	0; 49; 100	40 ± 39	0; 29; 207
Previous day / afternoon	54	31 ± 24	0; 28; 89	57 ± 39	0; 45; 174
Previous day / evening	61	30 ± 24	0; 24; 85	67 ± 52	0; 57; 327
Mixed-model results h		F = 11.17, P < 0.0001 i		F = 6.90, P = 0.0012 j
Fat (g)
Target period
Prior 24 hour	172	56 ± 31	0; 57; 100	41 ± 151	0; 6; 1765
Previous day	163	39 ± 32	0; 35; 100	63 ± 92	0; 32; 724
Mixed-model results g		F = 23.07, P < 0.0001		F = 30.11, P < 0.0001
Interview condition
Prior 24 hour / morning	58	46 ± 30	0; 45; 100	61 ± 244	0; 6; 1765
Prior 24 hour / afternoon	58	55 ± 28	0; 59; 100	27 ± 52	0; 6; 237
Prior 24 hour / evening	56	66 ± 31	0; 71; 100	34 ± 74	0; 8; 506
Previous day / morning	48	52 ± 32	0; 56; 100	43 ± 71.	0; 14; 371
Previous day / afternoon	54	35 ± 32	0; 35; 100	57 ± 51	0; 47; 177
Previous day / evening	61	32 ± 30	0; 24; 100	84 ± 125	0; 46; 724
Mixed-model results h		F = 11.84, P < 0.0001 i		F = 2.46, P = 0.0870

^aThe reporting-error-sensitive approach is an analytic approach to evaluate accuracy for reporting energy and nutrients; it is sensitive to reporting errors for food items and amounts.

^bTarget period is the period of time covered by a 24-hour dietary recall. The prior-24-hour target period concerns the 24 hours immediately preceding the interview. The previous-day target period concerns midnight to midnight of the day before the interview.

^cThe six interview conditions were created by crossing two target periods with three interview times (morning; afternoon; evening); they are labeled and defined as 1) prior 24 hour / morning — recall about the prior-24-hour target period obtained in a morning interview; 2) prior 24 hour / afternoon — recall about the prior-24-hour target period obtained in an afternoon interview; 3) prior 24 hour / evening — recall about the prior-24-hour target period obtained in an evening interview; 4) previous day / morning — recall about the previous-day target period obtained in a morning interview; 5) previous day / afternoon — recall about the previous-day target period obtained in an afternoon interview; and 6) previous day / evening — recall about the previous-day target period obtained in an evening interview.

^dCorrespondence rate is the percentage of the reference amount (i.e., the amount observed eaten at school breakfast and school lunch) that was reported correctly in the school breakfast and school lunch parts of children’s 24-hour dietary recalls. It was calculated for energy and each macronutrient for each child as: (corresponding amount from matches / reference amount) x 100. A match is a food item in both the reference information and reported information. The corresponding amount from a match is the smaller value of the reported amount and the reference amount, or the reported amount if it is equal to the reference amount. Correspondence rate is a measure of reporting accuracy that is sensitive to reporting errors. It has a lower bound of 0%, which indicates that no reference items were reported eaten. It has an upper bound of 100%, which indicates that all reference items and amounts were reported correctly. Larger values indicate better reporting accuracy.

^eInflation ratio is a non-negative augmentation to correctly reported information which is based on inaccurate reporting. It was calculated for energy and each macronutrient for each child as: {[(overreported amount from matches) + (overreported amount from intrusions)] / (reference amount)} × 100. The overreported amount from a match is the amount by which the reported amount exceeds the reference amount (or zero if the reported amount is less than or equal to the reference amount). An intrusion is a food item in the reported information but not in the reference information. The overreported amount from an intrusion is the entire reported amount for each intrusion. Inflation ratio is a measure of reporting error. It has a lower bound of 0%, which indicates that there were no intrusions and that no amounts of matches were overreported. Inflation ratio has no upper bound because there is no limit on what a person can report. Smaller values indicate better reporting accuracy.

^fChildren’s reports were compared to direct observation of school meals, so there was nothing suspect about maximum values, and it would be inappropriate to classify them as outliers and to exclude them from analyses.

^gMixed-model ANOVA results concerning the effect of target period.

^hMixed-model ANOVA results concerning the interaction of target period with interview time.

ⁱResults from pairwise comparisons for the six interview conditions found that correspondence rates for energy and each macronutrient were better for prior-24-hour recalls in the afternoon and evening than previous-day recalls in the afternoon and evening (16 P-values≤0.0005), for prior-24-hour recalls in the evening than prior-24-hour recalls in the morning (four P-values≤0.0007), and for previous-day recalls in the morning than previous-day recalls in the evening (four P-values<0.002). Also, for energy, correspondence rate was better for previous-day recalls in the morning than previous-day recalls in the afternoon (P<0.002).

^jResults from pairwise comparisons for the six interview conditions found that inflation ratios for energy and carbohydrate were better for prior-24-hour recalls in the afternoon and evening than previous-day recalls in the afternoon and evening (eight P-values≤0.0001), for prior-24-hour recalls in the morning than previous-day recalls in the afternoon and evening (four P-values<0.002), and for previous-day recalls in the morning than previous-day recalls in the evening (both P-values≤0.0009).

Also, as in earlier studies (30–32), an inflation ratio for energy and each macronutrient was calculated for each child (see Table 2, footnote e). Inflation ratios have a lower bound of 0% and no upper bound. Smaller inflation ratios indicate better reporting accuracy.

Note that the sum of the reporting-error-sensitive approach’s correspondence rate and inflation ratio is the conventional approach’s report rate; thus, report rate is actually the sum of a measure of accuracy and a measure of error (30–32).

Analyses

Analyses were conducted using Stata 10.0 (Stata, Inc., College Station, TX) and SAS 9.0 (SAS Institute, Inc., Cary, NC). The main effects of interest for each analytic approach were target period, interview time, and their interaction (because target period and interview time together determine retention interval). Because neither report rates nor inflation ratios have upper bounds, these variables were rank-transformed for analyses.

Conventional approach

For each target period and interview condition, paired t-tests were conducted to compare mean differences between reported and reference amounts of energy and of each macronutrient with zero. Also, for each target period and interview condition, Pearson correlations were calculated, over children, between reference and reported values of energy and of each macronutrient.

Separate mixed-model analyses of variance (ANOVAs) were conducted to determine whether rank-transformed report rates for energy and each macronutrient depended on target period, interview time, their interaction, and/or sex. Results were adjusted for school year and interviewer. For each analysis, a full model was fit, non-significant terms (P>0.05) were removed, and the model was re-estimated.

Reporting-error-sensitive approach

Separate mixed-model ANOVAs were conducted to determine whether correspondence rates for energy and each macronutrient depended on target period, interview time, their interaction, and/or sex; this approach was also used for rank-transformed inflation ratios. Results were adjusted for school year and interviewer. For each analysis, a full model was fit, non-significant terms (P>0.05) were removed, and the model was re-estimated. When the target-period by interview-time interaction was significant, means for each of the 15 pairs of six conditions were compared using a Bonferroni-adjusted significance criterion of 0.0033.

RESULTS

Analyses did not include data from 39 children (18 girls) who failed to meet criteria for reporting both school meals. Thus, results presented are from 335 children (169 girls).

Conventional approach

Table 1 shows results from the conventional approach for analyzing energy and macronutrients, by target period and interview condition. For each target period, for energy and each macronutrient, reported amounts were less than reference amounts (eight P-values≤0.0006; paired t-tests). For all six interview conditions for energy, three conditions for protein, five conditions for carbohydrate, and all six conditions for fat, reported amounts were less than reference amounts (20 P-values<0.04; paired t-tests). For each target period, for energy and each macronutrient, Pearson correlations between reference and reported amounts ranged from 0.33 to 0.46 and were different from zero (eight P-values<0.0001). For four interview conditions for energy, three conditions for protein, five conditions for carbohydrate, and all six conditions for fat, Pearson correlations between reference and reported amounts ranged from 0.27 to 0.70 and were different from zero (18 P-values<0.05).

None of the effects of target period, interview time, their interaction, and sex was significant for report rates for energy or any macronutrient. Mean report rates for energy and macronutrients ranged from 85% to 105% for the two target periods, and from 77% to 117% for the six interview conditions.

Reporting-error-sensitive approach

Table 2 shows results from the reporting-error-sensitive approach for analyzing energy and macronutrients, by target period and interview condition. Mixed-model ANOVAs of correspondence rates showed that for energy and each macronutrient, there was a significant effect of target period (four P-values<0.0001) and a significant target-period by interview-time interaction (four P-values<0.0001). Concerning target period, correspondence rates for energy and each macronutrient were better for prior-24-hour recalls (means of 51% to 59%) than previous-day recalls (means of 35% to 43%). Concerning the six interview conditions, pairwise comparisons showed that correspondence rates for energy and each macronutrient were better for prior-24-hour recalls in the afternoon and evening than previous-day recalls in the afternoon and evening (16 P-values≤0.0005), for prior-24-hour recalls in the evening than prior-24-hour recalls in the morning (four P-values≤0.0007), and for previous-day recalls in the morning than previous-day recalls in the evening (four P-values<0.002). Also, for energy, correspondence rate was better for previous-day recalls in the morning than previous-day recalls in the afternoon (P<0.002).

Mixed-model ANOVAs of inflation ratios showed that for energy and each macronutrient, there was a significant effect of target period (four P-values<0.0001), and for energy and carbohydrate, a significant target-period by interview-time interaction (energy P<0.005, carbohydrate P<0.002). Concerning target period, inflation ratios for energy and each macronutrient were better for prior-24-hour recalls (means of 32% to 45%) than previous-day recalls (means of 53% to 63%). Concerning the six interview conditions, pairwise comparisons showed that inflation ratios for energy and carbohydrate were better for prior-24-hour recalls in the afternoon and evening than previous-day recalls in the afternoon and evening (eight P-values≤0.0001), for prior-24-hour recalls in the morning than previous-day recalls in the afternoon and evening (four P-values<0.002), and for previous-day recalls in the morning than previous-day recalls in the evening (both P-values≤0.0009).

Table 3 shows descriptive statistics for the five amount categories for energy and macronutrients, by target period and interview condition. Statistical tests were not run on these amount categories because they were used to calculate the variables analyzed for the reporting-error-sensitive approach. Descriptive statistics in Table 3 show that unreported amounts from omissions were considerable, and were not offset by overreported amounts from intrusions. Means for corresponding amounts from matches for energy and macronutrients were larger for prior-24-hour recalls than previous-day recalls, clarifying why correspondence rates for energy and macronutrients were better for prior-24-hour recalls than previous-day recalls. Means for overreported amounts from intrusions for energy and macronutrients were smaller for prior-24-hour recalls than previous-day recalls, clarifying why inflation ratios were better for prior-24-hour recalls than previous-day recalls for energy and macronutrients.

DISCUSSION

The conventional approach depicted accuracy for reporting energy and macronutrients as follows: Underreporting was evident in paired t-tests between reported and reference values although Pearson correlations showed significant associations. Customary interpretation of report rates suggested high reporting accuracy. Analyses of report rates for energy and macronutrients did not indicate variation in reporting accuracy over retention intervals.

The reporting-error-sensitive approach provided a substantially different picture of accuracy for reporting energy and macronutrients: Correspondence rates were decidedly smaller than report rates, and inflation ratios were considerable. Analyses of correspondence rates and inflation ratios for energy and macronutrients showed differences in reporting accuracy by retention interval. Specifically, reporting accuracy was better with a shorter than with a longer retention interval—when the target period was the prior-24-hours instead of the previous-day, and when the interview was on the same day as both school meals in the target period instead of on the subsequent day.

Food-item-level analyses conducted earlier (27) found that children’s accuracy for reporting school-meal intake was best for the shortest retention interval; specifically, accuracy was better for prior-24-hour recalls than previous-day recalls, and accuracy was best for prior-24-hour recalls in the afternoon and evening, and worst for previous-day recalls in the afternoon and evening. This article’s conventional approach for analyzing accuracy for reporting energy and macronutrients indicated that accuracy was high and did not depend on retention interval. Conclusions from the conventional approach conflict with conclusions from food-item-level analyses (27); this conflict is logical because the conventional approach was not designed to capture errors of reported food items or amounts. In contrast, this article’s reporting-error-sensitive approach for analyzing accuracy for reporting energy and macronutrients indicated that accuracy was moderate to low, but better for prior-24-hour recalls than previous-day recalls, best for prior-24-hour recalls in the afternoon and evening, and worst for previous-day recalls in the afternoon and evening. Conclusions from the reporting-error-sensitive approach agree with those from food-item-level analyses (27); this agreement is logical because the reporting-error-sensitive approach captures errors of reported food items and amounts.

For validation studies, separating the evaluation of reporting errors for food items and amounts provides insight into what contributes to errors, which in turn provides insight into whether improvements are needed for reporting of food items, amounts, or both. Research indicates that children have considerable difficulty accurately estimating quantity eaten (21,41,47,49,50), even after training. However, this article’s results for the amount categories showed that when the correct items were reported, children were fairly accurate in reporting amounts in qualitative terms (e.g., taste, little bit, most). Also, unreported amounts from omissions accounted for more energy and macronutrients than overreported amounts from intrusions; thus, omissions and intrusions, with their respective unreported and overreported amounts, do not balance each other. These results concerning amounts are similar to those from three 2007 articles (30–32); collectively, they suggest that using dietary assessment tools that help children report the correct food items will yield a bonus of improving children’s ability to report amounts.

Results from this article agree with results from three 2007 articles (30–32) which showed that the conventional approach both overestimated children’s accuracy for reporting energy and macronutrients and provided a distorted picture of it. Thus, the current results further confirm the importance of using a reporting-error-sensitive approach when analyzing validation-study data to investigate accuracy for reporting energy and macronutrients, and demonstrate the important influence of retention interval on children’s accuracy for reporting energy and macronutrients.

In some investigations of accuracy for reporting energy in dietary recalls, reference information has been total energy expenditure (TEE) estimated using the doubly labeled water (DLW) technique (5,51–56). Because TEE from DLW lacks details about food items and amounts consumed, it cannot differentiate whether reporting errors are due to reports of the wrong items, unreported items, or incorrectly reported amounts. Equality of a person’s reported energy intake and his or her TEE from DLW does not imply that the person reported the correct items and amounts; it would be possible to have such equality without a single reported item or amount being correct! Because reference information obtained using TEE from DLW does not permit reporting-error-sensitive analyses, DLW data alone do not permit full investigation or understanding of the complexities of dietary-reporting error. In future validation studies, methodological differentiation between food items and amounts actually consumed and those reported (beyond what DLW data alone can provide) may help resolve remaining issues with the accuracy of dietary intake data. Increasing the accuracy of dietary intake data in future studies could better pinpoint true relationships between diet and disease.

In some investigations of dietary recall accuracy in which direct observation of intake has been used to obtain reference information (21,41,46–49,57–60), results for accuracy for reporting energy and macronutrients have been provided using the conventional approach only (21,46–49,57–60). For investigations that used the conventional approach only, based on three 2007 articles (30–32) and this article, it is possible that conclusions concerning accuracy for reporting energy and macronutrients would be different if the reporting-error-sensitive approach were used.

The current analyses were limited by aspects of the original study’s design. Children’s ages and race/ethnicities were homogeneous. Analyses were restricted to the school-meal parts of 24-hour recalls because only school meals were observed. Qualitative terms were converted to quantitative terms for amounts of standardized school-meal portions, although these processes were applied consistently to reference information and to reported information.

There are several strengths. Reference information was obtained by direct observation, which is considered the gold standard for validation (4). Also, quality control was assessed regularly throughout data collection for observations and interviews. In addition, observations were conducted in a setting (i.e., school) and manner that minimized reactivity (i.e., reciprocation or acting in response) and enhanced generalizability. Results from the secondary aim of the retention-interval validation study (61) showed that school-meal observations did not influence fourth-grade children’s 24-hour recalls; thus, conclusions about 24-hour recalls by fourth-grade children observed eating school meals in validation studies are generalizable to 24-hour recalls by comparable but unobserved children in non-validation studies (e.g., epidemiologic studies; interventions).

For many national surveys (62–67), adult household members help children ages six to 11 years report their intake. To our knowledge, this common practice has not been validated. A 1989 study by Eck and colleagues (68) is often incorrectly cited as a rationale to use joint parent-child recalls of children’s intake. That study (68) found that joint recalls by mother, father, and child better reflected observed intake of a cafeteria meal by four- to ten-year-old children than did recalls by the mother or father alone. However, children by themselves did not provide recalls, so no comparison could be made of the accuracy of child-only, parent-only, and joint parent-child recalls of children’s intake. Also, studies have found relationships between self-reported intake and various characteristics of adults (especially among women) such as body mass index (54,69–75) and social desirability (76–80); it is plausible that adult characteristics could impact accuracy of joint parent-child recalls of children’s intake. Validation studies are needed to compare the accuracy of child-only, parent-only, and joint parent-child recalls of children’s intake.

To our knowledge, six validation studies have examined age differences in dietary recall accuracy by elementary-school children; five of these studies found improvement with age (46,81–84), and the other (48) found no effect of age. As four of those studies concerned accuracy for a single meal per child, additional studies concerning intake for multiple meals (e.g., breakfast and lunch) by children from each grade level may be beneficial. However, validation studies to identify “the age” at which elementary-school children “achieve” dietary recall accuracy seem somewhat misplaced considering longstanding concerns about errors in adults’ dietary recalls. More benefit may be achieved by future methodological validation studies focused on improving dietary recall accuracy. For example, is dietary recall accuracy better with NDSR’s interviewer-administered electronic protocol (85–87), the US Department of Agriculture’s interviewer-administered automated multiple-pass method (88–91), or the National Cancer Institute’s new self-administered web-based protocol (92) which is being adapted for use by children? Does each pass in a 24-hour recall multiple-pass protocol improve accuracy enough to warrant its use? Do practice dietary recalls improve accuracy enough to justify their time and effort? Is the consistency of dietary recall accuracy better for prior-24-hour recalls than previous-day recalls? Does the combined influence of retention interval and prompts improve dietary recall accuracy, and do so differently by sex?

This article’s results have several important applications: First, to increase children’s accuracy for reporting school-meal intake, shorten the retention interval between intake and report. For example, obtain dietary recalls in the afternoon about that day’s school meals. For this study, for prior-24-hour recalls in the afternoon and evening compared to previous-day recalls in the afternoon and evening, correspondence rates and inflation ratios for energy and macronutrients improved by one-third to one-half. These improvements demonstrate that the level of confidence in elementary-school children’s self-reported school-meal intake depends on methodological variables (e.g., retention interval) that are clearly under investigators’ and practitioners’ control. Second, include details about retention interval (target period and interview time) in publications of studies utilizing dietary recalls; simply stating “recalls” is inadequate. Third, for validation studies, obtain reference information from a method (such as direct observation) that provides details about food items and amounts consumed; DLW data alone cannot provide these details. Finally, when analyzing validation-study data to investigate accuracy for reporting energy and nutrients, use an analytic approach that is sensitive to reporting errors of food items and amounts; the conventional analytic approach overestimates reporting accuracy and fails to detect effects such as retention interval on reporting accuracy.

With the incorporation of web-based, self-administered dietary recalls into prospective cohort studies, all indications are that dietary recalls will not only continue to be used, but will have a more prominent role in future research and clinical practice in the US and worldwide (6). Decisions made by investigators and practitioners about how and when to obtain dietary recalls can improve or impede accuracy, and decisions about data collection methods and analytic approaches have important implications for the quality of results and conclusions concerning reporting accuracy for energy and macronutrients. Applying the reporting-error-sensitive approach to past, current, and future validation studies may refine methods for improving the accuracy of dietary recalls.