A pilot study of the effects of interview content, retention interval, and grade on accuracy of dietary information from children

Abstract

Objective

Investigate differences in dietary recall accuracy by interview content (diet-only; diet-and-physical-activity), retention interval (same-day; previous-day), and grade (3^rd; 5^th).

Methods

Thirty-two children observed eating school-provided meals and interviewed once each; interview content and retention interval randomly assigned. Multivariate analysis of variance on rates for omissions (foods observed but unreported) and intrusions (foods reported but unobserved); independent variables—interview content, retention interval, grade.

Results

Accuracy differed by retention interval (P = .05; better for same-day [omission rate, intrusion rate: 28%, 20%] than previous-day [54%, 45%]) but not interview content (P > .48; diet-only: 41%, 33%; diet-and-physical-activity: 41%, 33%) or grade (P > .27; 3^rd: 48%, 42%; 5^th: 34%, 24%).

Conclusions and Implications

Although the small sample limits firm conclusions, results provide evidence-based direction to enhance accuracy; specifically, to shorten the retention interval. Larger validation studies need to investigate the combined effect of interview content, retention interval, and grade on accuracy.

INTRODUCTION

Schools are common targets for interventions to improve children’s dietary intake.^1–3 As most parents lack first-hand knowledge of their children’s intake at school, investigators rely on elementary-school children in upper-grade levels to provide dietary recalls.^4–6 Validation studies have identified errors in children’s dietary recalls.^7–14 The few studies that have examined elementary-school children’s age and dietary recall accuracy found it was better for children in upper than lower grades.^9,10,13 Validation studies provide insight for improving 4^th- and 5^th-grade children’s dietary recall accuracy; for example, minimizing the retention interval (elapsed time between to-be-reported meals and the report) improves accuracy.^8,11,14

One approach to possibly improving accuracy is to combine recall of dietary intake with recall of physical activity.⁵ Episodic memory and semantic memory have been differentiated;¹⁵ episodic memories are context-bound (i.e., particular events occurring in particular contexts) while semantic memory is situation-independent knowledge (e.g., general information). Simultaneously recalling both dietary intake and physical activity might yield more accurate information than recalling dietary intake only by facilitating memory for specific/correct episodes of eating. Frank and colleagues¹⁶ used activity prompts (e.g., sports, party) with children’s dietary recall to link the day’s events with intake. Moore and colleagues⁵ evaluated a computer program to assess children’s simultaneous reports of dietary and physical activity behaviors. Children completed an interviewer-administered dietary recall immediately after a computer-administered diet-and-physical-activity recall. Relative validity of dietary information (comparing each child’s 2 recalls) showed good agreement for 18 of 21 food groups, although accuracy for food items and interviewer-administered dietary recalls were not assessed.⁵ If combining recall of dietary intake with physical activity lengthens interviews but fails to improve dietary recall accuracy, or has deleterious effects on it, the practical value of having children provide integrated recalls of dietary intake and physical activity would be unclear.

This study investigated differences in dietary recall accuracy by interview content (diet-only; diet-and-physical activity), retention interval (same-day recalls in the afternoon; previous-day recalls in the morning), and grade (3^rd; 5^th). It was hypothesized that accuracy would be better for diet-and-physical-activity than diet-only content, for same-day recalls in the afternoon than previous-day recalls in the morning, and for 5^th- than 3^rd-grade children.

METHODS

The University of South Carolina Institutional Review Board approved the study. Written parental consent and child assent were obtained.

Participants and Design

Recruitment occurred in 7 3^rd-grade and 6 5^th-grade classes during the 2009–2010 school year at 2 schools in one district in Columbia, SC. Of 230 children (51% boys; 85% Black) invited to participate, 152 children (43% boys; 85% Black) agreed. In Spring, 2010, 32 randomly selected children were each observed eating school-provided breakfast and lunch in the cafeteria on a school day. These 32 children were then interviewed about time at school (from arrival at school until school dismissal) that occurred on their observation day; assignment to interview content and retention interval was random with the following constraints: The final sample of 32 children had 16 3^rd-grade children (half girls) and 16 5^th-grade children (half girls); interview content was diet-only for 8 children (half girls) per grade, and diet-and-physical-activity for 8 children (half girls) per grade; and retention interval was same-day recalls in the afternoon for 4 children (half girls) per content per grade, and previous-day recalls in the morning for 4 children (half girls) per content per grade. School staff and children did not know in advance when observations and/or interviews would occur, nor the interview content and/or retention interval to which children were assigned. More children were recruited than needed to ensure random selection, and so children could not determine who specifically was being observed and/or would be interviewed.

Meal Observations

Two researchers observed school-provided meals according to an established, written protocol.^8,11 Observers were trained with practice and assessment of pre-data-collection interobserver reliability. During regular meal periods, each observer simultaneously watched one to 3 children seated according to their school’s typical arrangement. Researchers used paper forms to record items and amounts eaten in servings of standardized school-meal portions. During data collection, interobserver reliability was assessed weekly (on 4 days total) using established procedures^8,11 on 7 children (4 girls) for breakfast and 8 children (4 girls) for lunch. Mean agreement between observers to within 1/4^th serving on amounts eaten was acceptable (breakfast = 95%; lunch = 93%), and Kappa between observers for absolute agreement was substantial (breakfast = .80; lunch = .68).¹⁷

Interviews

Three researchers conducted individual, in-person interviews in private locations at children’s schools after lunch on Mondays through Fridays (for same-day recalls in the afternoon), and after breakfast on Tuesdays through Fridays (for previous-day recalls in the morning). Although 2 researchers also conducted observations, a child’s interviewer had not observed that child’s meals. Interviewers were trained using modeling, practice, and assessment of pre-data-collection quality control for interviews. Four written multiple-pass interview protocols were created by crossing 2 interview contents (diet-only; diet-and-physical activity) with 2 retention intervals (same-day recalls in the afternoon; previous-day recalls in the morning). Table 1 describes the interview protocols which were modeled on the Nutrition Data System for Research protocol (NDSR, Nutrition Coordinating Center, Minneapolis, MN, 2007) and adapted for retention interval as in other studies.^8,11 Interviewers used paper forms to note information reported by children, and document each interview’s beginning and ending times. Interviews were audio recorded and transcribed. Each interviewed child was mailed a $10 check. Throughout data collection, quality control for interviews was assessed using established procedures;^8,11 each interview’s audio recording and typed transcript were reviewed by 2 researchers — an interviewer other than the one who conducted the interview and a non-interviewing researcher. Each interview had satisfactory adherence to the specific protocol.

Table 1.

Overview of multiple-pass interview protocols used to obtain recalls from 32 3^rd- and 5^th-grade children

Pass	Description a,b
1st	Collect list: “After you got to school this morning at (time), what was the first time you had something to eat or drink, or were physically active, moving around, or exercising? What time was that? What did you eat or drink at that time? Did you eat or drink anything else at that time? What moving around did you do at that time? Did you do any other moving around at that time? What was the next time at school today that you ...” Repeat this process to cover today at school in chronological order. “Can you remember any other times at school today that you ate or drank something - either in your classroom, in the cafeteria, or somewhere else at school today? Can you remember any other times at school today that you were physically active, moving around, or exercising - either in your classroom or somewhere else at school today?”
2nd	Obtain details: In chronological order, for each intake time, ask the name of the eating occasion (breakfast, lunch, snack), location of meal (cafeteria, classroom, somewhere else at school), additions to items, details about items, and amounts consumed. For each physical activity time, ask how long the activity was done and the pace (fast, medium, slow).
3rd	Review: In chronological order, review and confirm details for each intake time and each physical activity time. For each intake time, ask whether anything else was consumed. For each physical activity time, ask whether any other moving around was done. “Can you remember any other times at school today that you ate or drank something? Can you remember any other times at school today that you were physically active, moving around, or exercising?”

^aWhen the retention interval was “previous-day recalls in the morning,” the child was asked about yesterday at school.

^bUnbolded text was for “diet-only” interview content. Bolded text was included for “diet-and-physical-activity” interview content.

Analyses

Accuracy was assessed for only the school-meal parts of recalls because only school meals were observed. As in earlier studies,^8,11 for reported items to be treated as reports about school meals, children had to identify school as the location where items were eaten, refer to breakfast as school breakfast or breakfast, refer to lunch as school lunch or lunch, and report mealtimes to within one hour of observed mealtimes.

As people report intake as foods, accuracy was assessed for foods rather than nutrients. For each meal per child, there were 2 sets of food items; one set contained items observed eaten, and the other set contained items reported eaten. As in other studies,^8,11 a meal-component weight was assigned to each item observed eaten and/or reported eaten at a school meal (see Table 2, footnote d). According to an established classification system,^{7–9,11,18,19} items in both sets were matches, items only in the reported set were intrusions, and items only in the observed set were omissions. For each interviewed child’s 2 school meals, weighted matches, omissions, and intrusions were summed, and omission rate and intrusion rate were calculated (see Table 2, footnotes e and f). Smaller rates indicate better accuracy.

Table 2.

Results for 3 dietary response variables for a total of 32 children by interview content, retention interval, and grade

Variable	n	Omission Rate (%)d,e	Intrusion Rate (%)d,f	Total Inaccuracy (servings)d,g
		Means (standard deviations) and 95% confidence intervals for differences in means †
Interview Contenta
Diet-only	16	40.7 (21.8)	32.9 (24.2)	6.3 (2.9)
Diet-and-physical activity	16	41.1 (23.0)	32.5 (25.6)	5.4 (3.1)
95% Bonferroni Confidence Intervals		−0.4 ± 20.4	0.5 ± 22.8	0.9 ± 2.2
Retention Intervalb
Same-day recalls in the afternoon	16	27.6 (25.1)	20.4 (27.9)	4.6 (3.4)
Previous-day recalls in the morning	16	54.2 (22.3)	45.0 (24.8)	7.2 (3.0)
95% Bonferroni Confidence Intervals		26.6 ± 15.9 *	24.6 ± 17.7 *	2.6 ± 2.5 *
Gradec
3rd (8 or 9 years)	16	47.7 (22.3)	41.9 (24.8)	6.5 (3.0)
5th (10 or 11 years)	16	34.0 (21.9)	23.6 (24.4)	5.3 (2.9)
95% Bonferroni Confidence Intervals		13.7 ± 20.4	18.3 ± 22.8	1.3 ± 2.1

^*P < .05;

^† Confidence intervals (CIs) are (center) ± (margin of error). CIs for mean differences in total inaccuracy are unadjusted 95% CIs; other CIs are Bonferroni-adjusted family wise 95% CIs;

^aNo differences in omission rate-intrusion rate combination (P > .48; multivariate analysis of variance [MANOVA]) or in total inaccuracy (P > .50; analysis of variance [ANOVA]);

^bOmission rate-intrusion rate combination better for same-day recalls in the afternoon than previous-day recalls in the morning (P = .05; MANOVA); no difference in total inaccuracy (P = .07; ANOVA);

^cNo differences in omission rate-intrusion rate combination (P = .27; MANOVA) or in total inaccuracy (P > .29); ANOVA);

^dAccuracy assessed for school-meal parts of recalls; meal-component weight assigned to each item (combination entrée = 2, condiment = .33, remaining meal components = 1); amounts eaten observed, reported, and scored in servings of standardized school-meal portions as none = 0, taste = .1, little bit = .25, half = .5, most = .75, all = 1, or actual number of servings if > 1;

^eOmission rate (i.e., percentage of items observed eaten but unreported) calculated as (sum of weighted omissions/[sum of weighted omissions + sum of weighted matches]) × 100%; values range from 0% to 100% with smaller rates better;

^fIntrusion rate (i.e., percentage of items reported eaten but unobserved) calculated as (sum of weighted intrusions/[sum of weighted intrusions + sum of weighted matches]) × 100%; defined values range from 0% to 100% with smaller rates better;

^gTotal inaccuracy calculated as sum of three components: (a) sum, over matches, of absolute difference between amounts observed and reported for each match × weight, (b) sum, over intrusions, of each intruded amount × weight, and (c) sum, over omissions, of each omitted amount × weight; this measure cumulates errors (in servings) for items and amounts without indicating whether errors are due to omissions, intrusions, or incorrect amounts for matches; smaller values are better.

Amounts eaten were observed, reported, and scored in servings of standardized school-meal portions (see Table 2, footnote d). These procedures, which have been used in other school-based dietary-reporting validation studies,^8,11 were applied consistently to observed and reported information irrespective of interview content, retention interval, and grade.

A measure of total inaccuracy, developed and used previously,^8,9,11 was calculated (see Table 2, footnote g) to combine reporting errors for items and amounts for each interviewed child’s 2 school meals. This measure does not indicate whether errors are due to amounts unreported for omissions, amounts reported for intrusions, or amounts incorrectly reported for matches. Smaller values indicate better accuracy.

For the response variables omission rate and intrusion rate, a multivariate analysis of variance (MANOVA) was conducted with interview content, retention interval, grade, their 2-way interactions, and sex as independent variables. Race was not included because 94% of the final sample was Black. Residuals were tested for normality and showed no violations.

Total inaccuracy was treated as a single response variable, and analysis of variance (ANOVA) model was conducted with interview content, retention interval, grade, their 2-way interactions, and sex as independent variables. Residuals were tested for normality and showed no violations.

Each interview’s length was calculated by subtracting beginning time from ending time. Summary statistics were calculated; significance testing was not conducted because interview length does not assess accuracy.

Statistical analyses were performed using SAS/STAT® software (Version 9.2, SAS Institute Inc., Cary, NC, 2002–2008). A Bonferroni adjustment was used to maintain the family significance level at .05 for tests on 4 independent variables (interview content, retention interval, grade, sex).

RESULTS

Table 2 shows means and standard deviations for omission rate, intrusion rate, and total inaccuracy by interview content, retention interval, and grade. No 2-way interaction (between each pair of interview content, retention interval, and grade) was significant, so main-effect tests for each measure were interpreted directly, and Table 2 shows group means for the 3 factors separately (averaged across levels of other factors). As sex was not significant (P = .68; MANOVA), Table 2 does not show information by sex.

Concerning interview content (diet-only; diet-and-physical-activity), there was no effect on the omission rate-intrusion rate combination (P > .48; MANOVA). Bonferroni-adjusted family wise 95% confidence intervals (CIs) were centered nearly at 0 (see Table 2), suggesting that this nonsignificant result potentially is not attributable to the small sample.

Concerning retention interval (same-day recalls in the afternoon; previous-day recalls in the morning), there was a significant effect on the omission rate-intrusion rate combination (P = .05; MANOVA). Accuracy was better for same-day recalls in the afternoon than previous-day recalls in the morning (see Table 2).

Concerning grade (3^rd; 5^th), there was no effect on the omission rate-intrusion rate combination (P = .27; MANOVA). Bonferroni-adjusted family wise 95% CIs (see Table 2) were not centered at 0, suggesting that this nonsignificant result potentially is attributable to the small sample.

For total inaccuracy, no 2-way interactions were significant so main-effects tests were interpreted directly. There were no effects on total inaccuracy of interview content, retention interval, grade, and sex (P values ≥ .07; ANOVA). Table 2 shows individual 95% CIs for differences in total inaccuracy by interview content, retention interval, and grade; the CI for sex was 0 ± 2.2.

Average interview length in minutes (mean [standard deviation]) of recalls varied by interview content; diet-and-physical-activity content recalls (20.4 [4.1] minutes) were longer than diet-only content recalls (12.2 [3.9] minutes). However, average interview length of recalls was similar by retention interval (same-day 16.4 [4.5] minutes; previous-day 16.1 [4.0] minutes) and by grade (3^rd 17.7 [4.0] minutes; 5^th 14.9 [3.9] minutes).

DISCUSSION

This study is unique as no earlier validation study has investigated differences in children’s dietary recall accuracy by interview content (diet-only; diet-and-physical-activity). Although the sample was small and several variables were tested, care was taken to evaluate significance in the presence of multiple tests. Results include CIs as they are a more useful complement to nonsignificant tests than power calculations.²⁰

The lack of significant differences in accuracy by interview content was surprising based on past assumptions that accuracy would be better for interviews which combined recall of dietary intake with recall of physical activity.^5,16 The cognitive burden of recalling both dietary intake and physical activity is greater than that of recalling only dietary intake, and may have negatively impacted children’s accuracy for recalling dietary intake.²¹ The CIs for interview content were centered nearly at 0 indicating that a larger study (having similar summary statistics) would yield CIs presumably still containing 0, thus also finding no significant differences in accuracy across the 2 interview contents.

Retention interval results for omission rate and intrusion rate showed that accuracy was better for same-day recalls in the afternoon than previous-day recalls in the morning. These results, which show that 3^rd- and 5^th-grade children’s accuracy was better with a shorter retention interval, are consistent with earlier research with 4^th-grade^8,11 and 5^th-grade¹⁴ children.

The CIs for grade were not centered at 0; this potentially indicates that a larger study would yield narrower CIs and find significant differences in accuracy across the 2 grades. Three past studies that examined elementary-school children’s dietary recall accuracy and grade (i.e., age) found that accuracy improved as age increased.^9,10,13

Although analyses included 2-way interactions between each pair of interview content, retention interval, and grade, no 2-way interaction was significant, potentially because of the small sample. Larger studies are needed to investigate the combined effect of interview content, retention interval, and grade on children’s dietary recall accuracy. (For example, a larger study might find an interaction between interview content and grade, perhaps with better accuracy for diet-only content for one grade, but diet-and-physical-activity content for another grade.)

Concerning interview length, one would expect interviews about both dietary intake and physical activity to take more time than interviews about only dietary intake. This small study found that the time (8 minutes on average) to include physical activity in dietary recalls did not yield the hypothesized improvement in children’s dietary recall accuracy.

This study had limitations. The sample was small due to budgetary constraints and the expense of conducting direct meal observations. Interviews concerned only time when children were at school because the only meals observed were breakfast and lunch. Children who brought lunch from home and ate it at school were not observed because it is difficult to unobtrusively identify contents of those lunches. However, the vast majority (average of 79%) of children at the 2 schools ate school-provided lunch. Weight and height were not obtained, which prevented investigation of a relationship between body mass index and dietary recall accuracy. It was not feasible to investigate the relationship between socioeconomic status (i.e., eligibility for free or reduced-price school meals) and dietary recall accuracy while considering this pilot study’s key independent variables (interview content, retention interval, grade) because 29 of the 32 children in the final sample received free school meals.

This study had several strengths. The validation method of direct meal observations avoided the problem of comparing children’s recalls to information reported by children via back-to-back recalls or another method (such as food records) or by parents (who are not present to witness children’s intake at school). Rigorous and consistent quality control procedures were implemented for observations and interviews.

IMPLICATIONS FOR RESEARCH AND PRACTICE

Although the small sample limits firm conclusions, these results provide evidence-based direction for future studies to enhance children’s dietary recall accuracy. Specifically, these results provide preliminary evidence showing that children’s dietary recall accuracy was similar for 2 interview contents (diet-only; diet-and-physical-activity); this emphasizes the importance and necessity of conducting validation studies which investigate methodological aspects of obtaining dietary recalls from children to avoid reliance on speculation and assumptions concerning how to improve dietary recall accuracy. These results expand the body of compelling evidence which indicates that children’s dietary recall accuracy is better with shorter retention intervals (e.g., same-day recalls in the afternoon) than longer ones (e.g., previous-day recalls in the morning) to younger (i.e., 3^rd-grade) children. This pilot study contributes to the few studies with elementary-school children that examined dietary recall accuracy and age. Larger validation studies are needed to provide empirical evidence concerning the combined effect on children’s dietary recall accuracy of interview content, retention interval, and grade.