Accuracy of Fourth-Graders' Dietary Recalls of School Breakfast and School Lunch Validated with Observations: In-Person versus Telephone Interviews

Abstract

Objective: To investigate the accuracy of children's dietary recalls of school breakfast and school lunch validated with observations and obtained during in-person versus telephone interviews.

Design: Each child was observed eating school breakfast and school lunch and was interviewed that evening about that day's intake.

Setting: Ten elementary schools.

Participants: A sample of fourth-graders was randomly selected within race (black, white) and gender strata, observed, and interviewed in person (n = 33) or by telephone (n = 36).

Main Outcomes Measured: Rates for omissions (items observed but not reported) and intrusions (items reported but not observed) were calculated to determine accuracy for reporting items. A measure of total inaccuracy was calculated to determine inaccuracy for reporting items and amounts combined.

Analysis: Analysis of variance; chi-square.

Results: Interview type (in person, telephone) did not significantly affect recall accuracy. For omission rate, intrusion rate, and total inaccuracy, means were 34%, 19%, and 4.6 servings for in person recalls and 32%, 16%, and 4.3 servings for telephone recalls of school breakfast and school lunch.

Conclusions and Implications: The accuracy of children's recalls of school breakfast and school lunch is not significantly different whether obtained in person or by telephone. Whether interviewed in person or by telephone, children reported only 67% of items observed; furthermore, 17% of items reported were not observed.

INTRODUCTION

In a recent review regarding the evaluation measures used in nutrition education intervention studies conducted between 1980 and 1999, Contento et al reported that almost all studies assessed dietary intake.¹ Despite decades of developing various methods for obtaining dietary intake, the accuracy of dietary intake data continues to be a major problem.² The most commonly used method for dietary surveys in the United States is the 24-hour recall.³ In Contento et al's review, of the 62 interventions with schoolchildren that evaluated dietary intake, 26 (42%) used 24-hour recalls.¹ Recent validation studies have indicated that the accuracy of children's dietary recalls is poor.^4-6 Numerous publications (we refer to 5 as examples) have documented the need for validated methods for assessing children's dietary intake.^7-11

Telephone dietary surveys represent an economical way to increase the scope of nutrition research; advantages include cost savings and safety of the interviewer.¹² With telephone interviews, respondents and interviewers do not need to travel to a common location, and it is possible to conduct interviews without respondents knowing in advance exactly when interviews will occur.³ Telephone surveys eliminate potential interviewer bias that may exist in face-to-face interviews¹²; furthermore, the anonymity of the interviewer may improve reporting on sensitive topics.¹³ National surveys have used telephone 24-hour recalls.¹² Among adults, numerous studies have indicated that dietary recall information obtained by telephone was, in general, similar to or indicated slightly greater mean intakes than information from in-person dietary recalls in previous national surveys,^14,15 food records,¹⁶ and in-person dietary recalls.^17-21 However, these studies lacked true validation because they compared one self-report method with another self-report method, without investigators knowing the respondents' actual consumption. Validation studies among adults have indicated that dietary recall information obtained by telephone was similar to information from energy intake for weight maintenance,²² observation of congregate noon meal,²³ and tray cards with visual estimation in a college dining hall.²⁴ However, a validation study by Tran et al with adults indicated that although no significant differences were found in mean daily energy intake between inperson and telephone recalls, mean energy intake from both in-person and telephone recalls was significantly lower than total energy expenditure assessed using doubly labeled water.²¹ In summary, among adults, information from dietary recalls is similar whether obtained in person or by telephone, although under-reporting is evident when compared to doubly labeled water.

Children have provided telephone dietary recalls in many studies^11,25-35; however, parents often assisted children,^11,25-35 so conclusions cannot be made regarding the accuracy of the children's telephone recalls. Two studies compared children's telephone recalls to parental records of children's intake, but neither study validated the parental reports.^27,28 One study compared telephone recalls to doubly labeled water, but children were younger than 7 years, and recalls were performed by the mother and child.³⁰ Other studies compared children's telephone recalls to food records³¹ and food frequencies.³³ Several studies used children's telephone recalls but lacked comparison methods.^{11,25,26,29,32,34} To our knowledge, only a study by Wyon et al has obtained telephone recalls from children without parental assistance and validated the recalls against a method that did not rely on self-reports from children or parents.³⁵ The Wyon et al study was conducted to examine the effect of energy intake at breakfast on school performance the same morning, although children were given the impression that they were participating in a marketing survey. Four breakfast packs, of either 2 identical high-energy meals and 2 identical low-energy meals or 3 identical high-energy meals and 1 low-energy meal, were delivered to each child's home on a Monday evening. Parents were asked to put uneaten items in a bag labeled with the weekday on which the meal was eaten; these bags were collected on Friday evening that same week. Recalls of breakfast from 24-hour telephone recalls by 166 children (10 years old) were compared to analysis of uneaten breakfast items. Results indicated a correlation of 0.89 for energy intake, with good agreement for boys and girls. The short (ie, 4 days) and unusual nature of the study (ie, provision of breakfast packs similar in content) may have contributed to the children's high recall accuracy of energy intake at breakfast.

Because children have limited attention spans, eye contact made during in-person recalls may help hold their attention, thus leading to increased recall accuracy. However, eye contact is not possible during telephone recalls; therefore, children's recall accuracy may be less for telephone versus in-person recalls. Furthermore, telephone surveys are not advantageous for developing rapport and increasing the motivation and interest of the respondent¹³; this could be critical when respondents are children, especially considering their limited attention spans. To our knowledge, although children have provided dietary recalls by telephone in many studies, information obtained from children's telephone recalls has not been validated against observation or compared with in-person recalls. The purpose of this study was to test the hypothesis that the accuracy of children's dietary recalls of school breakfast and school lunch validated with observations would be significantly greater when obtained during in-person rather than telephone interviews. Accuracy was defined as the extent to which the school breakfast and school lunch portion of the recalls, which were obtained in the evening and regarded that day's intake, provided correct information compared to observations. Accuracy measures were assessed for foods rather than nutrients because people report what they have eaten as foods. This study was the third in a series of 3 studies investigating the accuracy of children's dietary recalls; 1 study has been conducted each school year during 3 consecutive school years with fourth-grade children from a total of 12 elementary schools in 1 district.^4,5 In each study, observations of school breakfast and school lunch were used to validate these portions of children's dietary recalls. Each study was approved by the Human Assurance Committee at the Medical College of Georgia.

METHODS

Participants

All children from each fourth-grade class (n = 42) at 10 public elementary schools in 1 district during the 2001-2002 school year were invited to participate. The 10 schools were selected to obtain a final sample with equal numbers of black (B), white (W), male (M), and female (F) children with high participation in school breakfast and school lunch. A mean of 59% (median = 60%, minimum = 27%, maximum = 79%) of the children across all grades at the respective 10 schools was eligible to receive free or reduced-price school meals during the school year of data collection. Of the 799 fourth-graders (21% BM, 21% BF, 27% WM, 26% WF, 2% other [O] M, 3% OF) invited to participate, 56% (n = 451; 22% BM, 23% BF, 26% WM, 25% WF, 1% OM, 3% OF) agreed by providing child assent and parental consent. Of these, a sample of 69 children was randomly selected within race (black, white) and gender strata. Due to their small number, children of other races were not sampled. The sampling scheme was stratified by race and gender to ensure race/gender representativeness within the sample, but the study was not powered to detect race or gender differences. A 2-arm parallel design was used; each child was observed eating school breakfast and school lunch once and was interviewed once. Half of the children randomly selected within each race/gender group were randomly assigned to be interviewed in person and the other half by telephone. Power calculations were based on a factorial analysis of variance (ANOVA) using an α of 0.025 to establish an overall confidence level of at least 95% for the 2 primary dependent measures (omission rates and intrusion rates). A difference in omission and/or intrusion rates between interview types (in-person, telephone) of at least 10% was considered large enough to be meaningful. The power analysis indicated that 50 children per interview type would provide 80% power to detect a difference of the specified size.

Procedures for Observations and Interobserver Reliability

Observations were conducted by a total of 3 trained research dietitians (RDs) who followed written procedures used previously.^4,5 Because it can be difficult to unobtrusively observe contents of meals brought from home and eaten at school,³⁶ only children who participated in school breakfast and school lunch were observed. Observations covered the entire school breakfast and school lunch periods to account for trading of foods.^37,38 An RD stood by tables where classes or groups of children sat and used a recording form while observing 1 to 3 children simultaneously. Although children in general knew when they were being observed, individual children did not know who would be interviewed. To lessen reactivity during data collection by acquainting children with the presence of observers, practice observations were conducted with each class prior to data collection.³⁹

According to procedures used previously,⁵ interobserver reliability (IOR) was assessed between pairs of RDs weekly throughout data collection to ensure that all observations were conducted according to protocol. After each IOR, each RD (whether in the pair or not) completed an IOR checklist to identify discrepancies; IOR checklists were reviewed by the principal investigator (PI) and each RD. During the 6 weeks of data collection, IOR was conducted on 6 days at 6 schools on 10 children (3 M; 5 B, 4 W, 1 O), with each RD participating on 3 to 5 days. Children observed for IOR were not interviewed that evening. Results from IOR for school breakfast and school lunch for the 10 children indicated 93% mean agreement (median = 92%, minimum = 80%, maximum = 100%) across observers for food items in which the amounts observed eaten were within one-fourth serving. This percent agreement is considered satisfactorily high.^36,40

Procedures for Interviews and Quality Control for Interviews

Interviews were conducted by 1 of 2 trained RDs on Monday through Thursday evenings; although 1 of these 2 RDs also conducted observations, she never conducted observations and interviews on the same day. Interviews followed a written multiple-pass protocol with 4 passes (Table 1) that was patterned after that used by the Nutrition Data System for Research (NDS-R, version 4.03, Nutrition Coordinating Center, University of Minnesota, Minneapolis, Minn, 2000), which has often been used with children.^{6,11,25-27,29,33,34} However, information was written on an interview form instead of using the computerized NDS-R version. A similar version of the forward interview protocol with the same 4 passes was used recently in our previous 2 studies in which fourth-graders were interviewed in the morning regarding the previous day's intake.^4,5 Minor changes were made in the wording of the interview protocol for the current study because children were interviewed in the evening of the day they were observed eating school breakfast and school lunch and interviews regarded that day's intake. Interviews were audiorecorded and transcribed. Beginning and ending times were documented to determine length.

Table 1.

Description of Multiple-Pass Protocol Used for In-Person and Telephone Interviews

First Pass

Children were told that the interviewer wanted to know about everything they ate or drank that day. A quick list was obtained to provide an outline of what was eaten by asking, “After you got up today, when was the first time you had something to eat or drink? What did you eat or drink at that time? Did you eat or drink anything else at that time? When was the next time after [time] that you had something to eat or drink? What did you eat or drink at that time? Did you eat or drink anything else at that time? … Can you remember any other times today that you had something to eat or drink?”

Second Pass

The quick list was reviewed to identify forgotten foods by asking, “After you got up today, the first time you ate was at [time]. You ate [read all foods] and drank [read all drinks]. Can you think of anything else you ate at that time? Can you think of anything else you drank at that time?”

Third Pass

The name of each meal was obtained by asking, “What was the name of the meal you ate at [time]? Please listen to all of these choices and choose one: school breakfast, breakfast, school lunch, lunch, dinner/supper, or snack.”

The location where each meal was eaten was obtained by asking, “Where did you eat this? Please listen to all of these choices and choose one: home, school, or somewhere else.”

Additions to each item were obtained by asking, “Did you add anything to [name of item]?”

Details regarding the kinds of items eaten were obtained by asking, “What kind of [name of item] was it?”

Amounts eaten were obtained by asking, “How much of your serving of [name of item] did you eat (or drink)? A serving or helping is how much you were given or how much you got yourself. Please listen to all of these choices and choose one: none, taste, little bit, half, most, all, or more than one serving.”

Fourth Pass

Information reported was reviewed and confirmed by asking, “For [meal] at [time] at [location], you had [amount] of your [food/drink]. Is this correct? Did you eat or drink anything else at that time? … Can you remember any other times today that you had something to eat or drink?”

In-person interviews were conducted in a research van parked outside children's homes. One of 2 police officers hired from the Medical College of Georgia Police Bureau drove the van and accompanied the RD for her safety and that of the child; this was explained in the assent and consent forms signed by children and parents. The officer stood outside the van or sat in the cab while the interview was conducted in the back of the van. Because of scheduling demands, the officers were available only 2 evenings per week. Thus, 2 evenings (Monday and Wednesday or Tuesday and Thursday) each week were randomly assigned to each condition (in-person, telephone).

Parents were asked to provide telephone numbers on consent forms and were telephoned by RDs to obtain information regarding what time the child ate supper and went to bed, which evenings the child would be available for an interview the following week, and directions to the home. Parents were asked not to tell children they had been contacted because not all children at each school would be observed and interviewed the following week. Parents and children did not know in advance exactly when children would be interviewed or whether specific children would be interviewed in person or by telephone.

According to procedures used previously,⁵ quality control for interviews (QCI) was conducted daily throughout data collection to ensure that interviews were conducted according to protocol. The audiorecording and typed transcript from one interview conducted each evening was randomly selected and reviewed by another RD, who completed a QCI checklist. The QCI checklist had 56 items with response options of “adequate,” “needs improvement,” and “not applicable.” Each completed QCI checklist was reviewed by the principal investigator and RD who conducted the interview. A total of 22 QCI checklists was completed on 69 interviews (32%) conducted on 22 evenings. Across all 22 QCI checklists, 97% of items (937 of 970) that were not rated “not applicable” were rated “adequate.” Of the 33 items rated “needs improvement,” there were 6 items on 13 QCI checklists for interviews conducted by one RD (RD1) and 17 items on 9 QCI checklists for interviews conducted by another RD (RD2). Two or fewer items were rated “needs improvement” on 11 QCI checklists for interviews conducted by RD1 and on 6 QCI checklists for interviews by RD2. These results indicate that RD1 and RD2 adequately followed the interview protocol.

Recall Accuracy

Although each interview was conducted in the evening and regarded intake since the child got up that morning, only items for the school breakfast and school lunch portions of the recalls were analyzed for accuracy because only these meals were validated with observations. As in our 2 previous studies,^4,5 to determine which meals in the recalls referred to school breakfast and school lunch, children had to (1) indicate “school” as the location where the meal was eaten, (2) refer to breakfast as “school breakfast” or “breakfast,” (3) refer to lunch as “school lunch” or “lunch,” and (4) report the mealtime within 1 hour of the observed mealtime. Each item observed and/or reported eaten at school breakfast or school lunch was grouped into 1 of 10 meal components: lunch entree, combination entree, bread/cereal, vegetable, fruit, dessert, condiment, beverage, breakfast meat, and miscellaneous (foods such as candy and chips that did not fit into any of the other meal components).^4,5 As in our previous studies,^4,5,41 errors regarding reporting combination entrees counted more than errors for condiments and remaining meal components, so statistical weights were assigned by meal component with combination entree (eg, hamburger, sausage biscuit) = 2, condiment (eg, ketchup) = 0.33, and remaining meal components (eg, chicken, milk, apple, corn, roll, brownie) = 1. Combination entrees were considered a single meal component and were counted only once during analyses.

Accuracy for reporting items. Each item reported and/or observed eaten was classified as a match (observed and reported eaten), omission (observed but not reported eaten), or intrusion (reported but not observed eaten). Intrusions have also been called “phantom foods.”^6,8,42 Because foods can be reported in many ways, items were scored as matches unless it was clear that a child's recall did not describe an observed food. This broad interpretation maximized the scored correctness of the child's recall, so true correctness could be overestimated. Examples of items observed and reported that matched were all types of pizza (eg, cheese, sausage, pepperoni) and white milk (eg, skim, 1%, whole). Milk flavors (eg, chocolate, strawberry, white), vegetables (eg, green beans, green peas), and fruit juices (eg, orange, apple) that differed were not matched. These requirements were used in our 2 previous studies.^4,5 To analyze accuracy for reporting food items, irrespective of amounts, matches, omissions, and intrusions were tallied for each child's recall, and omission and intrusion rates were calculated as in our previous studies.^4,5,42 Omission and intrusion rates are defined in the legend of Table 2. Intrusion rates were undefined if a child was observed to have eaten something yet failed to report eating anything at school breakfast and school lunch. Omission rates and defined intrusion rates could have values from 0% to 100%, with 0% considered perfect. A child's recall could have low accuracy due to either high omission and low intrusion rates, low omission and high intrusion rates, or high omission and high intrusion rates.

Table 2.

Descriptive Results for Omission Rates, Intrusion Rates, Total Inaccuracy, Absolute and Arithmetic Differences per Serving for Matches, and Amounts per Serving for Omissions and Intrusions for In-Person and Telephone Recalls of School Breakfast and School Lunch

Outcome Variable	n	Mean	SD	Minimum	Maximum	Median
In-person recalls of school breakfast and school lunch
Omission rates* † (in %)	33	34	28	0	100	33
Intrusion rates* ‡ (in %)	32	19	23	0	100	13
Total inaccuracy* § (in servings)	33	4.6	2.6	0.0	10.8	4.0
Absolute differences for matches* ∥	31	0.28	0.30	0.00	1.50	0.22
Arithmetic differences for matches* ¶	31	−0.09	0.36	−1.50	0.45	−1.50
Amounts for omissions* #	30	0.92	0.42	0.10	2.00	0.87
Amounts for intrusions*,**	20	0.82	0.27	0.25	1.25	0.88
Telephone recalls of school breakfast and school lunch
Omission rates* † (in %)	36	32	20	0	79	30
Intrusion rates* ‡ (in %)	36	16	19	0	60	11
Total inaccuracy* § (in servings)	36	4.3	2.6	0.3	8.7	4.2
Absolute differences for matches* ∥	36	0.19	0.15	0.00	0.67	0.17
Arithmetic differences for matches* ¶	36	0.00	0.17	−0.59	0.31	0.00
Amounts for omissions* #	33	0.69	0.26	0.10	1.15	0.75
Amounts for intrusions*,**	20	0.72	0.25	0.25	1.00	0.75

^*A statistical weight was assigned to each item by meal component with combination entree = 2, condiment = 0.33, and remaining meal components = 1.

^†Omission rate = [(sum of weighted omissions for each child's recall of school breakfast and school lunch) ÷ (sum of weighted omissions + sum of weighted matches for each child's recall of school breakfast and school lunch)] * 100. Omission rates could have values from 0% to 100%, with 0% considered perfect.

^‡Intrusion rate = [(sum of weighted intrusions for each child's recall of school breakfast and school lunch) ÷ (sum of weighted intrusions + sum of weighted matches for each child's recall of school breakfast and school lunch)] * 100. The sample size was 32 instead of 33 for in-person recalls for this variable because intrusion rates were undefined if a child was observed to have eaten something yet failed to report eating anything for school breakfast and school lunch (1 child's recall obtained in-person). Defined intrusion rates could have values from 0% to 100%, with 0% considered perfect.

^§Total inaccuracy = (absolute difference between amounts reported and observed for each match * statistical weight) + (each omitted amount * statistical weight) + (each intruded amount * statistical weight) summed over all items at school breakfast and school lunch for each child's recall. A score of 0 servings indicated a perfect recall compared to observation.

^∥Absolute differences per serving for matches = {sum ([absolute difference between amounts reported and observed for each match] * statistical weight)} ÷ {weighted number of matches for each child's recall of school breakfast and school lunch}. The sample size was 31 instead of 33 for in-person recalls for this variable because absolute differences per serving for matches were undefined for recalls of school breakfast and school lunch with no matches (2 children's recalls obtained in person).

^¶Arithmetic differences per serving for matches = {sum ([amount reported − amount observed for each match] * statistical weight)} ÷ {weighted number of matches for each child's recall of school breakfast and school lunch}. The sample size was 31 instead of 33 for this variable for in-person recalls because arithmetic differences per serving for matches were undefined for recalls of school breakfast and school lunch with no matches (2 children's recalls obtained in person).

^#Amounts per serving for omissions = {sum ([amount observed but not reported for each item] * statistical weight)} ÷ {weighted number of omissions for each child's recall of school breakfast and school lunch}. The sample sizes were 30 instead of 33 for in-person recalls and 33 instead of 36 for telephone recalls for this variable because amounts per serving for omissions were undefined for recalls of school breakfast and school lunch with no omissions (3 children's recalls obtained in person and 3 children's recalls obtained by telephone).

^**Amounts per serving for intrusions = {sum ([amount not observed but reported for each item] * statistical weight)} ÷ {weighted number of intrusions for each child's recall of school breakfast and school lunch}. The sample sizes were 20 instead of 33 for in-person recalls and 20 instead of 36 for telephone recalls for this variable because amounts per serving for intrusions were undefined for recalls of school breakfast and school lunch with undefined intrusion rates as well as for recalls of school breakfast and school lunch with no intrusions (13 children's recalls obtained in person and 16 children's recalls obtained by telephone).

Accuracy for reporting amounts. Amounts were observed and/or reported in servings as none, taste, little bit, half, most, all, or the actual number if > 1 serving (eg, 2) and scored respectively as 0.0, 0.10, 0.25, 0.50, 0.75, 1.0 or the actual number if > 1. To analyze accuracy for reporting amounts for matches, absolute and arithmetic differences were calculated between amounts observed and reported eaten.^4,5,42 Absolute differences measured the closeness of observed and reported amounts regardless of under- or overreporting. Thus, a score of 0 indicated perfect reporting of amounts for matches, and a score of 1.5 indicated a 1.5-serving difference between observed and reported amounts for matches but did not differentiate between under- and overreporting. Arithmetic differences measured the closeness of observed and reported amounts, but under- and overreporting could offset each other. Thus, a score of –1.5 indicated that a child underreported amounts for matches by 1.5 servings, and a score of +1.0 indicated that a child overreported amounts for matches by 1.0 serving. Amounts were calculated for omissions and intrusions separately to assess whether these errors in reporting involved small or large amounts of servings.^4,5 The footnotes to Table 2 define absolute and arithmetic differences per serving for matches and amounts per serving for omissions and intrusions; accuracy was high for each of these 4 variables when close to 0.

Accuracy for reporting items and amounts combined. A single measure of total inaccuracy was used to capture inaccuracy for reporting both items and amounts for school breakfast and school lunch combined.^4,5,41 The footnotes to Table 2 define total inaccuracy. By summing (a) absolute differences between amounts reported and observed for each match multiplied by its statistical weight, (b) amount for each omission multiplied by its statistical weight, and (c) amount for each intrusion multiplied by its statistical weight, a single measure of inaccuracy was created that cumulated errors in servings for all items whether they were matches, omissions, or intrusions for school breakfast and school lunch for a child. A total inaccuracy score of 0 servings indicated a perfect recall compared to observation; as a score for total inaccuracy increased, recall accuracy decreased. A total inaccuracy score of 4.5 servings indicated that across all matches, omissions, and intrusions for school breakfast and school lunch for a child, the cumulative difference between amounts reported and observed for each item multiplied by its statistical weight was 4.5 servings. Although total inaccuracy was based on all items and amounts in a recall compared to an observation, it failed to indicate whether errors were due to omissions, intrusions, or incorrectly reported amounts.

Analyses

To determine whether accuracy differed by interview type (in-person, telephone), interviewer, or weekday, ANOVA was conducted on omission rates and total inaccuracy separately. Race and gender terms were included in the model to control for possible confounding due to differences in means among strata. Total inaccuracy was square root transformed because the variances were found to be related to the mean in one of our previous studies.⁴ Because omission and intrusion rates may both range from 0% to 100%, with 0% considered perfect, an arbitrary pass/fail criterion was used to define acceptable accuracy,^4,5 which established omission rates, intrusion rates, and worse-case omission and intrusion rates combined of ≤ 30% as passing and > 30% as failing. An ANOVA was not conducted on intrusion rates because the underlying structure of the data (with 0% intrusion rates for 12 [36%] of the 32 in-person recalls of school breakfast and school lunch with defined intrusion rates and 16 [44%] of the 36 telephone recalls of school breakfast and school lunch) did not fit the required normality assumption. Thus, a chi-square test was conducted on the proportion of in-person and telephone recalls of school breakfast and school lunch with passing versus failing intrusion rates. The hypothesis would be supported when omission rates, intrusion rates, and/or total inaccuracy were less for recalls of school breakfast and school lunch obtained in person compared to by telephone. Statistical calculations were conducted using SPSS for Windows (version 7.5, SPSS, Chicago, Ill, 1996) and SAS (Release 8.2 [TS2m0], SAS Institute, Inc, Cary, NC, 2001).

RESULTS

In-person interviews were conducted with 33 children (10 BM, 8 BF, 8 WM, 7 WF) and telephone interviews with 36 children (8 BM, 11 BF, 8 WM, 9 WF). Evening interviews regarding that day's intake lasted an average of 13 minutes (median = 13, minimum = 7, maximum = 25) when conducted in person and an average of 15 minutes (median = 15, minimum = 9, maximum = 27) by telephone. Results from ANOVAs on omission rates and total inaccuracy failed to indicate significant effects of interview type, interviewer, or weekday (all P's > .096). Table 2 provides descriptive results for omission rates, intrusion rates, total inaccuracy, absolute and arithmetic differences per serving for matches, and amounts per serving for omissions and intrusions for in-person and telephone recalls of school breakfast and school lunch. Results from a chi-square test failed to indicate significant differences in the proportion of passing versus failing intrusion rates between in-person and telephone recalls of school breakfast and school lunch (P = .62). Table 3 provides the cumulative distribution of omission rates, intrusion rates, and worse-case omission and intrusion rates combined by accuracy cutpoint for in-person and telephone recalls of school breakfast and school lunch.

Table 3.

Cumulative Distribution of Omission Rates, Intrusion Rates, and Worse-Case Omission and Intrusion Rates Combined by Accuracy Cutpoint^* for In-Person and Telephone Recalls of School Breakfast and School Lunch

	In-Person Recalls of School Breakfast and School Lunch (n = 33)			Telephone Recalls of School Breakfast and School Lunch (n = 36)
Accuracy Cutpoint (Cumulative % of Children)	Omission Rates†‡	Intrusion Rates†§	Omission and Intrusion Rates Combined†∥	Omission Rates†‡	Intrusion Rates†§	Omission and Intrustion Rates Combined†∥
0	9	36	6	8	44	6
10	15	42	12	14	47	11
20	48	61	42	28	67	28
30	48	70	45	53	75	50
40	61	88	58	81	81	72
50	79	88	76	83	92	78
60	85	91	85	92	100	92
70	85	94	85	92	100	92
80	91	94	91	100	100	100
90	94	94	94	100	100	100
100	100	97	97	100	100	100
Undefined¶	0	3	3	0	0	0

^*To define acceptable accuracy, we applied an arbitrary pass/fail criterion, which establishes omission rates, intrusion rates, and worse-case omission and intrusion rates combined of ≤ 30% as passing and > 30% as failing.

^†A statistical weight was assigned to each item by meal component with combination entree = 2, condiment = 0.33, and remaining meal components = 1.

^‡Omission rate = [(sum of weighted omissions for each child's recall of school breakfast and school lunch) ÷ (sum of weighted omissions + sum of weighted matches for each child's recall of school breakfast and school lunch)] * 100. Omission rates could have values from 0% to 100%, with 0% considered perfect.

^§Intrusion rate = [(sum of weighted intrusions for each child's recall of school breakfast and school lunch) ÷ (sum of weighted intrusions + sum of weighted matches for each child's recall of school breakfast and school lunch)] * 100. The sample size was 32 instead of 33 for in-person recalls for this variable because intrusion rates were undefined if a child was observed to have eaten something yet failed to report eating anything for school breakfast and school lunch (1 child's recall obtained in person). Defined intrusion rates could have values from 0% to 100%, with 0% considered perfect.

^∥To examine accuracy for telephone and in-person recalls of school breakfast and school lunch, children were grouped according to cutpoint for omission rate, intrusion rate, and worse-case omission and intrusion rates combined. For example, a child with an omission rate of 25% and an intrusion rate of 15% would be in the 30% cutpoint for omission rate, the 20% cutpoint for intrusion rate, and the 30% cutpoint for omission and intrusion rates combined (because his/her maximum omission and intrusion rates were 25%).

^¶Percentages of children with undefined rates are actual, not cumulative.

DISCUSSION

To our knowledge, this is the first validation study to use observation to evaluate the accuracy of children's dietary recalls obtained in person versus by telephone. Results indicated that the accuracy of children's dietary recalls of school breakfast and school lunch did not differ significantly whether obtained in person or by telephone. Mean omission rates, intrusion rates, and total inaccuracy were 34%, 19%, and 4.6 servings for in-person recalls of school breakfast and school lunch and 32%, 16%, and 4.3 servings for telephone recalls of school breakfast and school lunch. Thus, whether interviewed in person or by telephone, children failed to report 33% of the items they were observed eating at the 2 school meals; furthermore, 17% of the items children reported eating during these 2 school meals were not observed eaten.

Results from general memory research indicate that as the length of time increases between an event and its recall, so does the amount of error in the information recalled.⁴³ Thus, the sooner something is recalled, the more accurate it is. Surprisingly, the application of this finding to dietary assessment has been investigated, to our knowledge, in only 2 studies, 1 with adults and 1 with children. In a study by Smith et al, adults completed food diaries for 2 or 4 weeks and then recalled what they had eaten either 0, 2, 4, or 6 weeks later.⁴⁴ Although recalls were not validated against a non–self-report method, results indicated that overall memory performance decreased as the time interval between eating and recall increased. In a validation study by our group, fourth-grade children were observed eating school lunch and asked to recall intake for only school lunch within 90 minutes of eating, the next morning, or on Monday morning (regarding Friday's lunch).⁴² Results indicated that accuracy decreased significantly as the time interval increased between eating and recalling school lunch intake.

Accuracy for reporting items in the current study was better than accuracy obtained in our 2 previous studies conducted with fourth-graders during the 2 prior school years in many of the same schools as the current study.^4,5 These 2 previous studies had means of approximately 51% for omission rates, 39% for intrusion rates, and 7.1 servings for total inaccuracy. The major difference between the current and previous studies is that for the current study, interviews were conducted in the evening regarding that day's intake as opposed to the next morning regarding the previous day's intake. The current study was not designed to assess accuracy of morning recalls regarding the previous day's intake versus evening recalls regarding that day's intake. However, comparing results from the current study and these 2 previous studies,^4,5 along with results from general memory research⁴³ and the studies by Smith et al⁴⁴ and our group,⁴² it is logical to assume that one way to improve the accuracy of children's dietary recalls of school breakfast and school lunch is to obtain recalls on the same day as intake instead of waiting until the next morning.

Although accuracy from the current study was better than in our 2 previous studies,^4,5 approximately half of the children provided recalls of school breakfast and school lunch with failing omission rates, one fourth provided recalls of school breakfast and school lunch with failing intrusion rates, and half provided recalls of school breakfast and school lunch with failing rates for worse-case omission and intrusion rates combined (see Table 3). Thus, even when interviewed in the evening regarding that day's intake, many of the children in the current study provided recalls of school breakfast and school lunch with failing accuracy. Results regarding accuracy for reporting amounts during both in-person and telephone interviews from the current study were similar to those found in our 2 previous studies.^4,5 That is, accuracy for reporting amounts for matches (items observed and reported eaten) in qualitative terms (ie, taste, little bit, half, etc) was quite high (see Table 2). However, as indicated in Table 2, amounts of items omitted and intruded were often almost full servings. These results imply that to improve children's accuracy for reporting amounts, efforts must focus first on helping children report only matches (ie, items observed and reported eaten). As discussed previously, one apparent method for enhancing children's ability to report more matches and fewer omissions and intrusions appears to be to decrease the time interval between eating and recall. Validation studies with children are needed to determine other methods for obtaining dietary recalls that actually improve children's recall accuracy, whether in-person or by telephone. Researchers and practitioners are cautioned to use validated methods because sometimes what is assumed will help does not improve recall accuracy and may actually decrease it. For example, in another of our previous studies that used observations to validate recalls of school lunch,⁴¹ prompting improved accuracy of recalls for school lunch for 19% of 48 first graders and 25% of 48 fourth-graders, did not change it for 37% and 60%, respectively, and actually decreased recall accuracy for 44% and 15%, respectively.

Post hoc ANOVAs were conducted regarding (1) the weighted number of items observed eaten at the 2 school meals, (2) the weighted number of items reported eaten at the 2 school meals, and (3) the total number of meals and snacks reported eaten during interviews (conducted in the evening regarding that day's intake) by type, interviewer, and weekday. Results from all 3 post hoc ANOVAs failed to indicate significant differences. The weighted number of items observed eaten at the 2 school meals was 8.4 ± 2.5 during in-person interviews and 8.9 ± 2.1 during telephone interviews (P = .21). The weighted number of items reported eaten at the 2 school meals was 6.7 ± 2.8 during in-person interviews and 7.2 ± 2.0 during telephone interviews (P = .46). The mean number of meals and snacks reported eaten was 3.6 ± 1.7 (median = 3, minimum = 2, maximum = 10) during in-person interviews and 3.7 ± 0.9 (median = 4, minimum = 2, maximum = 6) during telephone interviews (P = .6630). The average beginning time was 7:15 PM for in-person interviews and 7:28 PM for telephone recalls. In one of our previous studies with fourth-graders interviewed in the morning regarding the previous day's intake, the mean number of meals and snacks reported eaten was higher, at 4.4 ± 1.4.⁵ Thus, studies that obtain recalls in the evening regarding that day's intake, as was done in the current study, will miss some meals and snacks compared with studies that obtain recalls in the morning regarding the previous day's intake. In that previous study, across a total of 1099 meals and snacks reported during a total of 242 recalls, meals and snacks were reported after 6:59 PM in 76% of recalls and accounted for 21% of the total meals and snacks reported; meals and snacks were reported after 7:59 PM in 55% of recalls and accounted for 14% of the total meals and snacks reported. However, the accuracy of these evening meals and snacks is unknown because the only meals validated with observations were school breakfast and school lunch. To obtain evening recalls that cover 24 hours of intake, recalls could cover that day's intake, as well as intake for the previous evening. Another option for obtaining 24 hours of intake for a single day (ie, midnight to midnight) would be to obtain evening recalls regarding that day's intake and then obtain another recall from each participant the following morning regarding the previous evening's intake (ie, after the beginning time of the interview).

In-person interviews were attempted with 38 children and obtained from 33 children (87%); telephone interviews were attempted with 47 children and obtained from 36 children (77%). Thus, the “miss rate” was 13% for in-person interviews and 23% for telephone interviews, even though parents were called during the week prior to when interviews were attempted to obtain the child's evening schedule for the next week. In-person interviews were missed primarily because no one answered the door when the interviewer knocked. Telephone interviews were missed primarily because no one answered the telephone. Some of the missed interviews may have been avoided had exact interview dates been scheduled with parents; however, this was not done because of concerns that it would create excessive reactivity from the children and possibly impact recall accuracy. Several of the telephone numbers provided by parents on the consent forms in September were disconnected when telephone calls to parents were attempted in January and February. This problem may have been reduced slightly if consent forms had been distributed and collected closer to the data collection period. However, disconnected and/or incorrect telephone numbers could be a hindrance for research studies using the telephone to interview children. Other problems with telephone interviews include call waiting and subjects' use of cell phones with poor reception.

An important strength of this study is the use of observation (which did not rely on the child's memory) to validate the school breakfast and school lunch portion of the children's dietary recalls. Observations of school meals provide an excellent opportunity to validate portions of children's dietary recalls.^37,39 Observations conducted in homes may be intrusive⁹ and, thus, cause excessive reactivity⁴⁵; however, children are accustomed to eating at school and to being watched while eating at school. Although reactivity from observations is an issue,⁶ procedures such as those in this study may be taken to minimize it. Another important strength of this study is that accuracy was analyzed several ways regarding items, amounts, and items and amounts combined using methods from previous studies.^4,5,41,42

There are several limitations of this study. First, the study included a relatively small number of children. Although the planned sample size of 50 children per interview type provided adequate power to detect differences in omission and/or intrusion rates of 10%, analyses conducted at the two-thirds point of data collection (with data from 69 children) indicated only 2% differences in omission rates and 3% differences in intrusion rates between in-person and telephone recalls of school breakfast and school lunch. If data collection had been continued on an additional 31 children, the differences in omission and/or intrusion rates between in-person and telephone recalls from these 31 children would have to have been very large (ie, greater than 26%) to indicate differences of at least 10% with a total sample of 100 children. Thus, data collection was stopped. Second, the percentage of children who agreed to participate by providing child assent and parental consent was moderate, at 56%, and the 10 public elementary schools from which the children were recruited were all in the same school district and were not selected randomly. However, the race and gender profiles of the fourth-graders who agreed to participate were similar to those of the total population of fourth-graders at the 10 schools who were invited to participate. Participation in our 2 previous studies was much higher, around 73%, when interviews were conducted in the morning at school and children were paid $10 per interview.^4,5 Perhaps participation in the current study was lower even though children were paid $25 per interview because of the intrusiveness of evening interviews obtained in-person or by telephone. Third, analyses for items were limited to the school breakfast and school lunch portion of the recalls because these were the only meals validated with observations. Finally, recalls did not cover an entire 24-hour period. Although RDs used information from parents regarding when supper would be eaten so that children could be interviewed after supper, some children (13 interviewed in-person, 9 interviewed by telephone) claimed that they had not eaten supper when an RD conducted the interview. However, the majority of interviews did cover all meals and snacks consumed, except for bedtime snacks, on the day of the interviews.

IMPLICATIONS FOR RESEARCH AND PRACTICE

Results from the current study indicate that whether interviewed in-person or by telephone in the evening regarding that day's intake, children failed to report one third of the items they were observed eating at school breakfast and school lunch; in addition, almost one fifth of the items they reported eating at these 2 school meals were not observed eaten. This level of accuracy is insufficient for most purposes for which dietary information would be used. Validation studies with children are needed to determine how to obtain more accurate information regarding actual intake. Results from the current study provide direct evidence that the accuracy of children's dietary recalls of school breakfast and school lunch is not significantly different whether obtained during in-person or telephone interviews. This is important because of the potential savings in time and money with telephone interviews. Although not directly tested in the current study, decreasing the time interval between eating and recall appears to be one method for enhancing the accuracy of children's dietary recalls, whether recalls are obtained in-person or by telephone. Thus, researchers, practitioners, and policy makers are encouraged to consider the time interval between eating and recall when obtaining dietary recalls from children. Authors of publications of studies using children's dietary recalls are encouraged to specify details regarding the recalls. These details could include the time of day the recalls were obtained (ie, morning, afternoon, evening), target period covered by the recalls (previous day's intake versus intake for the 24 hours prior to the beginning time of the interview), interview format (ie, manner in which children are initially instructed to report intake), and prompting (ie, questions asked or cues given by interviewers). More accurate information regarding what children have actually eaten is needed to guide the development and evaluation of nutrition education programs to help children eat more healthfully.