A Systematic Review of Methods to Assess Children’s Diets in the School Context^,,

Abstract

To evaluate the impact of school-based nutrition interventions, accurate and reliable methods are needed to assess what children eat at school. The primary objective of this study was to systematically review methodological evidence on the relative accuracy and reliability of dietary assessment methods used in the school context. The secondary objective was to assess the frequency of methods and analytical approaches used in studies reporting in-school dietary outcomes. Three health databases were searched for full-text English-language studies. Twenty-two methodological studies were reviewed. For school meal recalls, the majority of studies (n = 8 of 12) reported poor accuracy when accuracy was measured by using frequencies of misreported foods. However, when energy report rates were used as a measure of accuracy, studies suggested that children were able to accurately report energy intake as a group. Results regarding the accuracy of food-frequency questionnaires (FFQs) and food records (FRs) were promising but limited to a single study each. Meal observations offered consistently good interrater reliability across all studies reviewed (n = 11). Studies reporting in-school dietary outcomes (n = 47) used a broad range of methods, but the most frequently used methods included weighed FRs (n = 12), school meal recalls (n = 10), meal observations by trained raters (n = 8), and estimated FRs (n = 7). The range of dietary components was greater among studies relying on school meal recalls and FRs than among studies using FFQs. Overall, few studies have measured the accuracy of dietary assessment methods in the school context. Understanding the methodological characteristics associated with dietary instruments is vital for improving the quality of the evidence used to inform and evaluate the impact of school-based nutrition policies and programs.

Introduction

Given growing concerns about nutrition-related chronic diseases among children (1, 2), there is now strong international interest in evaluating and improving children’s diets within school settings (3, 4). Schools are a proposed site for public health action to improve children’s diet quality because schools reach a large number of children, can facilitate nutrition education and health promotion programs, and can influence dietary behaviors (5–7). For example, school meal and snack programs have been promoted as a means to reduce food insecurity for children living in low-income households (8–10) and to improve diet quality at the population level (11–13). Many Canadian (14, 15) and US school districts (16–18) have implemented food policies, such as school food and beverage sales guidelines, aiming to improve school food environments (14, 15). Given that school meal programs and policies require substantial investments to implement and sustain (19, 20), evidence is needed to assess whether and how they impact children’s diet quality, both in the school context and for the whole day.

Evaluating the impact of school-based nutrition interventions on children’s diet quality requires both accurate and reliable instruments. Accuracy refers to the proximity of measurement results to their true values (21). Accuracy exists on a continuum and is relative to another (ideally true, error-free) measure. However, in nutrition research, there are few gold standards, and seldom is the true value for intake ever known. Instead, researchers assess relative accuracy wherein results from a test instrument (e.g., a school meal recall) are compared with those from another measure believed to hold less measurement error (e.g., a weighed food record). Moreover, the degree of accuracy that is required depends on context—whether it is good enough for a specific purpose. As a result, researchers aim to select tools that fit a study’s stated purpose.

Reliability broadly refers to the ability of an instrument to produce consistent or repeatable results (21). Test-retest reliability [sometimes referred to as reproducibility (21)] refers to an instrument’s ability to produce consistent results with repeated measurements, whereas interrater reliability examines the extent to which different raters (or observers) provide similar estimates. Intrarater reliability refers to how consistent results are for repeated ratings by a given rater.

Approaches to measure dietary intakes within school settings can be broadly categorized as self-report (the subject is doing the reporting of foods and beverages consumed) or observational methods (wherein a researcher or observer evaluates dietary consumption in real time). Self-reported methods include meal-specific recalls, estimated food records (FRs)⁴, and FFQs. Observational methods include meal observations (MOs) by trained raters, digital photography (DP), and the school food checklist (SFC). Weighed FRs can be classified as either self-report tools if the weighing is done by participants or observational if weighing is done by researchers. Self-report methods are often used in large, population-based dietary surveys, because they are typically less labor intensive (thus less expensive) to conduct than observational methods (22). However, validation studies have confirmed important measurement errors inherent to self-report methods (23–25). Although observational methods are thought to be free of reporting bias and hence more objective, they are labor intensive and thus not easily scalable to large, population-based studies. Understanding the types and degrees of measurement errors associated with different dietary assessment methods is crucial when designing a study and appropriately interpreting results (26).

Although several reviews have examined the accuracy of dietary assessment instruments for use among children (25, 27–29), no comprehensive review has focused specifically on eating contexts, such as schools, and whether current dietary assessment methods capture in-school dietary intakes accurately. Schools present both unique methodological challenges and opportunities for collecting dietary data. For example, parents are not present to assist in reporting food intakes, and foods may come from different sources (home, school food service operations, and off-campus locations). [In the context of this study, school food service operations include any food or beverage provided or sold on school grounds (e.g., cafeterias, à la carte services, vending machines).] On the other hand, dietary intakes can be observed more easily because consumption typically occurs in large groups at set times.

To appropriately measure the impact of school-based nutrition interventions, such as school meal programs and policies regarding access to specific foods on school grounds, researchers and practitioners need to know the strengths, limitations, and sources of measurement error associated with different dietary instruments. Therefore, the primary aim of this study was to review methodological evidence on the relative accuracy and reliability of dietary assessment methods used in the school context. A secondary aim was to assess the frequency of instrument use and the range of analytical approaches utilized in studies reporting in-school dietary outcomes.

Methods

The review process was informed by the Preferred Reporting Items for Systematic reviews and Meta-Analyses 2015 framework (30).

Eligibility and exclusion criteria.

The search strategy aimed to identify articles that developed, validated, and/or applied dietary assessment methods to characterize in-school dietary intakes. Participants were school-aged children (6–17 y) who participated in experimental or observational studies that measured food and beverage consumption in the school context, such as lunches or snacks consumed at school. Studies were excluded if they focused on childcare or daycare settings, included only preschool-aged children, took place in low-income countries, did not specify whether dietary intakes were consumed at school or elsewhere, or only included measures of food availability or sales but not actual intake. Commentaries and editorials were also excluded.

Information sources and search strategy.

Three health databases (MEDLINE Complete, CINAHL Complete, and PubMed) were searched for full-text English-language publications published in 2015 or before by using keywords (“school meal*" OR “school diet*" OR “school intake*" OR “school dietary intake*" OR “school nutrient* intake*" OR “school food*" OR “school lunch*" OR “packed lunch*" OR “bagged lunch*" OR “school snack*” OR “school breakfast*”) AND (children OR youth OR adolescent* OR student*) AND (tool* OR measure* OR assessment* OR instrument* OR indicator* OR index). [The asterisk (*) was used at the end of key search terms to include any additional characters for a key word search (e.g., “school meal” and “school meals”).] Reference lists were further screened to identify additional studies.

Study selection and classification.

Articles were screened and assessed for eligibility by the first author. Studies were classified into 2 groups based on their main purpose: 1) methodological studies evaluating the accuracy and/or reliability of dietary instruments (e.g., school meal recalls) used in the school context compared with a reference instrument (e.g., weighed FRs), and 2) studies whose primary outcome or outcomes were measures of in-school dietary intakes.

Approach for evaluating relative accuracy and reliability in methodological studies.

Studies were first categorized based on whether they used a self-reported or observational tool. Key characteristics of each study, including the study reference, context, sample characteristics, test and reference instrument used, statistical approach, and outcome measures of relative accuracy and/or reliability, were extracted into summary tables by the first author.

Table 1 describes the key measures for evaluating the relative accuracy and reliability and the scoring criteria to appraise each methodological study. Common approaches used in the literature included reporting the frequency of discrepancies between values obtained for the test and the reference measures. For example, Baxter and colleagues (31, 32, 36–38) used omission, intrusion, and match rates to assess the degree of misreporting for individual foods (regardless of whether amounts were accurate). Arithmetic and absolute differences were used to assess children’s ability to accurately report amounts consumed. [When assessing the accuracy of a self-report method (meal recalls) compared with visual estimation from trained raters, precision was assessed up to 0.1 serving for the majority of studies reviewed.] Statistical assessments of relative accuracy also included paired t tests, correlation coefficients⁵, and energy report rates.

TABLE 1.

Approaches for evaluating the relative accuracy and reliability of dietary instruments¹

		Cutoffs for assessing relative accuracy and reliability
	Definitions	Poor or failing	Acceptable
Measures of relative accuracy
Omission rate	A measure of reporting error that reflects the rate of foods reported relative to all foods observed. Calculated as the sum of weighted values for omitted foods / (sum of weighted values for matched foods + sum of weighted values for omitted foods). Ranges from 0% to 100%.	>15%	≤15%
Intrusion rate	A measure of reporting error that reflects the rate of foods reported but not observed. Calculated as the sum of weighted values for intrusion2 foods / (sum of weighted values for matched foods + sum of weighted values for intrusion foods). Ranges from 0% to 100%.	>15%	≤15%
Match rate	The ratio of foods reported to be consumed over foods that were consumed. Ranges from 0% to 100%.	<85%	≥85%
Arithmetic difference3	The difference (expressed in svgs) between observed and reported amounts for matched foods, but under- and overreports can offset one another. These differences for each meal item are multiplied by a statistical weight4 and then summed up for each meal for each child. The sign of the arithmetic difference provides an indication as to whether, on average, children tend to over- or underreport foods and beverages consumed.	<−0.5 or >0.5 svg/meal	−0.5 to 0.5 svg/meal
Absolute difference5	The difference (expressed in svgs) between observed and reported amounts for matched foods, but under- and overreports do not cancel each other. These differences for each meal item are multiplied by a statistical weight and then summed up for each child. This represents the average number of svgs misreported in a given meal for a group of children.	>1 svg/meal	≤1 svg/meal
2-Sample t test	A statistical test used to assess the difference between 2 means (obtained from the reference and test instrument).	Significant difference	No significant difference
Energy report rate	The percentage of reported energy consumed divided by the total observed energy consumed and multiplied by 100. The closer to 100%, the more valid the instrument.	<85% or >115%	85–115%
Correlation coefficient	Measures the agreement between individual values between a test and reference method.3	r < 0.6	r ≥ 0.6
Measures of reliability6
Cohen’s κ coefficient	Statistical measure of the amount of agreement between 2 measures of the same concept (can be used to assess test-retest or interrater reliability).	<0.6	0.6–0.8: acceptable; >0.8: good
Intraclass correlation coefficient	A statistical test that measures intrarater reliability (how similar the ratings are for a given observer over time).	<0.6	0.6–0.8: acceptable; >0.8: good
Percentage agreement between raters	A statistical test to measure interrater reliability (how different observers rate the same observation).	<60%	60–80%: acceptable; >80%: good

¹svg, serving.

²Intrusion foods are foods that were not observed by raters and observers but reported by children (also sometimes referred to as phantom foods).

³Correlation coefficients provide only an indirect measure of accuracy. A correlation coefficient measures the agreement between individual values for 2 methods. For example, the individual values for protein intakes obtained by a 24-h dietary recall and an FFQ designed to measure usual daily protein intake could be highly correlated, but the absolute difference in individual values estimated via each method could substantially differ.

⁴Subjective weights are used when adding each meal component to generate an omission or intrusion rate for the whole meal: meal entrée = 2, condiments = 0.33, and other meal items = 1 (31–33).

⁵A limitation of arithmetic and absolute difference measures is that that they are svg based, but svg sizes vary greatly across different types of foods. A 0.1-svg difference in reporting of a food with low energy density, such as lettuce, or a food typically consumed in small portions, such as ketchup, has a very different impact on the accuracy of total reported intake compared with a 0.1-svg of a food that is energy dense, such as pizza or a chocolate bar.

⁶Cutoffs for defining poor, acceptable, and good test-retest reliability and inter- and intrarater reliability were derived from methodological literature on reliability (34, 35).

There are currently no established cutoffs classifying dietary instruments as having acceptable or poor accuracy. Therefore, to facilitate the interpretation and comparison of findings across studies, we selected cutoffs described in Table 1 that align with the commonly used cutoffs for establishing acceptable rates for reporting energy intake defined as between 85% and 115% of the true value (25, 27). Although Baxter and colleagues (31, 32) used ±30% as an acceptable cutoff for omission and intrusion rates, this would mean that a substantial number of foods could be misreported. Thus, we applied a more conservative cutoff of ±15%, which is consistent with the reporting of energy intake. Cutoffs for defining poor, acceptable, and good test-retest reliability and inter- and intrarater reliability were derived from methodological literature on reliability (34, 35).

Frequency of instrument use and dietary outcomes assessed in school contexts.

Empirical studies were reviewed, and study reference, context, sample characteristics, type of dietary instrument used, dietary outcome(s), and key findings were extracted into summary tables. We examined whether studies captured dietary outcomes using multiple single dietary components (e.g., absolute amounts of micronutrients and/or foods, proportion of energy from macronutrients) or whether a study relied on a more complex analytical approach (e.g., a composite diet quality index). Finally, we examined whether certain tools and instruments tended to more frequently report certain types of dietary measures than others.

Results

Figure 1 outlines the article identification and selection process. A total of 436 unique references were identified, of which 59 were selected for qualitative synthesis. An additional 10 articles were identified after screening reference lists from reviewed studies in the primary search (n = 69 studies in the final review). Twenty-two studies were classified as methodological studies, and 47 included ≥1 outcome measure describing in-school dietary intakes. No study fell into both categories; hence, all 69 articles were considered unique records.

FIGURE 1.

Preferred Reporting Items for Systematic reviews and Meta-Analyses (30) flow diagram of the study selection process.

Methodological evidence on the relative accuracy and reliability of dietary assessment tools

Table 2 provides descriptive information on each of the dietary assessment instruments used in the school context, outlining key strengths and limitations and an overall summary grade for relative accuracy and reliability. Table 3 synthesizes key details of methodological studies by dietary instrument.

TABLE 2.

Key characteristics of dietary instruments to measure in-school diet quality among children (n = 22 studies)¹

Dietary instrument	Strengths	Limitations	Evidence of relative accuracy2	Evidence of reliability3
Self-report methods
School meal recalls	Opportunity to probe for detailed dietary information and portion sizes, low burden on subjects	Costly if interviewer-administered, relies on memory, subject to misreporting, prone to social desirability bias	Poor accuracy for individual foods reported (n = 8 of 12 studies); acceptable accuracy when reporting amounts consumed (n = 4 of 5 studies), acceptable energy report rates (n = 2 of 3 studies)	N/A
Estimated food records (by study subjects)	Detailed dietary information (quantities and types of food), self-administered, does not rely on memory	Higher burden on subjects, subject to misreporting, prone to reactivity and social desirability bias	Acceptable accuracy with daily monitoring but poor when children are monitored on a weekly basis (n = 1 study)	N/A
FFQs	Can provide estimates of usual intakes, self-administered (so lower costs)	Finite food list, lower precision for amounts consumed, difficult to capture contextual information (e.g., time and place of consumption), cognitively challenging for younger age groups, subject to misreporting, prone to social desirability bias	Acceptable accuracy for measuring select beverages and snack foods (n = 1 study)	N/A
Observational methods
In-person meal observations	Precise information about amounts and types of foods; lowest burden on subjects, more objective method	Costly, labor-intensive, potentially intrusive	Acceptable accuracy (n = 1 study)	Good interrater reliability (n = 11 studies)
Weighed food records (by research staff)	Precise information about amounts and types of foods; lowest burden on subjects, more objective method	Most costly method, labor-intensive, not well suited for environments where students bring home-packed lunches	No evidence4	No evidence
Digital photography methods	Low burden on participants; quick to administer, can be self-administered (so lower cost), more objective method	Costly (labor-intensive for dietary data entry), difficult to capture hidden or wasted or spilled or traded foods, can be challenging to interpret data from photos	Acceptable accuracy (n = 2 studies)	Acceptable interrater reliability (n = 2 studies)
School food checklist	Only validated to measure energy intake for a group; quick to administer (∼3 min/child), more objective method	Finite food list, labor-intensive, potentially intrusive, not validated to assess the quality of intake (only energy)	Acceptable accuracy (n = 1 study)	Acceptable interrater reliability (n = 1 of 2 studies); good intrarater reliability (n = 1 study)

¹N/A, not applicable.

²A measure of accuracy was expressed by using categorical ratings (poor or failing compared with acceptable) defined with cutoffs shown in Table 1.

³Interrater and intrarater reliability is applicable to observational tools only.

⁴There is no gold standard in dietary assessment methodology, but weighed food records by trained raters and observers are regarded as the closest to a gold standard. We found no methodological study testing the relative accuracy of the weighed food record. The plate-waste method (a form of weighed food record in which weights of foods remaining on the plate at the end of the meal are subtracted from the amounts served) is an adaptation of the weighed food record.

TABLE 3.

Summary of methodological studies evaluating dietary instruments for measuring in-school consumption (n = 22 studies)¹

Study, year (ref), country	Dietary instrument	Reference method	Sample characteristics	Results2	Key findings3
School meal specific recalls4 (n = 15 studies)
Baxter et al., 2002 (31), United States	Breakfast and lunch recall (school meals), same day (evening), no parental assistance	MOs	104 children (10 y old)	Omission rate: 41% Intrusion rate: 24% Absolute difference: 0.24 svg Arithmetic difference: −0.08 svg 89% agreement across raters for within 0.25 svg for MOs	Poor accuracy for types of foods reported. However, children were able to report amounts consumed for matched foods with acceptable accuracy. Good interrater reliability for in-person meal observations.
Baxter et al., 2003 (37), United States	Breakfast and lunch recall (school meals), same day (evening) via person and telephone, no parental assistance, multiple-pass protocol	MOs	69 children (10 y old)	For in-person and via telephone recalls, respectively:  Omission rates: 34% and 32%  Intrusion rates: 19% and 16%  Absolute difference: 0.28 and 0.19 svg  Arithmetic difference: −0.09 and 0 svg 93% agreement across raters for within 0.25 svg for MOs	Poor accuracy for types of foods reported, regardless of telephone vs. in-person interview. However, children were able to report amounts consumed for matched foods with acceptable accuracy. Good interrater reliability for the reference method.
Baxter et al., 2003 (32), United States	Breakfast and lunch recall (school meals), next morning, no parental assistance, multiple-pass protocol (4 passes)	MOs	121 children (10 y old)	For reverse and forward recalls, respectively:  Omission rates: 57% and 56%  Intrusion rates: 32% and 39%  Absolute difference: 0.23 and 0.24 svg  Arithmetic difference: −0.08 and −0.08 svg 90% agreement across raters for within 0.25 svg for MOs	Poor accuracy for foods reported, regardless of the recall order. However, children were accurate at reporting amounts for matched foods. Good interrater reliability for the reference method.
Baxter et al., 2009 (38), United States	Breakfast and lunch recall (school meals), same and next day (different retention intervals), no parental assistance	MOs	374 children (10 y old)	For lunch recalls (rates for different time retention periods, ranges):  Omission rates: 28–55%  Intrusion rates: 16–49%  Energy report rates: 78–88%  Reported good interrater reliability for MOs but percentage agreement not reported by authors	Poor accuracy for types of foods reported (both on same day and next day). Poor accuracy for energy reported when recall occurred on the next day but acceptable accuracy when recall occurred on the same day.
Baxter et al., 2010 (39), United States	Breakfast and lunch recall (school meals), different retention intervals, no parental assistance	MOs	374 children (10 y old)	Mean report rates for all interview conditions combined:  Energy report rates: 85%  Protein report rates: 105%  Carbohydrates report rates: 86%  Fat report rates: 97%94–97% agreement across raters for MOs	Acceptable accuracy for energy, protein, carbohydrates, and fat report rates; report rates varied by nutrient. Retention period did not affect energy, protein, carbohydrate, or fat report rates. Good interrater reliability for reference method.
Baxter et al., 2000 (33), United States	Lunch recall (school meals), morning of the next day, no parental assistance, with and without prompting	MOs	96 children (mean 7.2 and 10.1 y old)	Next day, no prompting:  Total inaccuracy5: 2.7 svg (grade 1)  Total inaccuracy: 1.7 svg (grade 4) Next day, with prompting:  Total inaccuracy: 2.6 svg (grade 1)  Total inaccuracy: 1.8 svg (grade 4)	Accuracy described as poor for both grades, but accuracy measures were higher for grade 4 than for grade 1 students.
Baxter et al., 1997 (36), United States	Lunch recall (school meals), same day and next day, no parental assistance	MOs	260 children (10 y old)	Rates for same day and next day, respectively:  Omission rates: 16%, 32%  Intrusion rates: 5%, 13%  Absolute difference: 0.10 and 0.14 svg  Arithmetic difference: −0.02 and −0.01 svg No interaction effects found by sex or ethnicity	Poor accuracy for types of foods reported, but omission and intrusion rates improved when recall was conducted on the same day as the lunch meal. Children were able to accurately report amounts consumed for matched foods (small absolute and arithmetic differences).
Biltoft-Jensen et al., 2013 (40), Denmark	Lunch recall (school meals), same day (evening), web-based software (WebDASC), parental assistance	Weighed FRs and DP method	81 children (8–11 y old)	Omission rate: 3% Intrusion rate: 14%	Acceptable accuracy for types of foods reported.
Biltoft-Jensen et al., 2015 (41), Denmark	Lunch recall (school meals), same day (evening), web-based software (WebDASC), parental assistance	Weighed FRs and DP method	193 children (8–11 y old)	Omission rate: 9% Intrusion rate: 6%	Acceptable accuracy for types of foods reported.
Guinn et al., 2010 (42), United States	School meal recall, various retention periods, no parental assistance	MOs	327 children (10 y old)	Mean energy report rate: 88% Report rates for energy decreased with increasing points on the social desirability scale and BMI percentile categories >90% interrater agreement for the reference method	Acceptable accuracy for energy report rates. Accuracy was inversely associated with social desirability bias and BMI. Good interrater reliability for the reference method
Hunsberger et al., 2013 (43), Sweden	School meal recall, previous day, RD administered using a web-based dietary recall software	Weighed FRs	25 children (6–8 y old)	Overall match rate: 90% (range: 67–100%). Difference in child-reported energy and observed energy: 7 ± 50 kcal (P = 0.49); strong correlation (r = 0.92; P < 0.001) between children’s recall and MOs	Acceptable accuracy for reporting individual foods and for estimating energy intake of the group.
Lyng et al., 2013 (44), Denmark	Lunch recall (home-packed lunches), same day (early afternoon), no parental assistance	DP method	114 children (11 y old)	For girls and boys, respectively:  Match rates: 78%, 74%  Omission rates: 22%, 26%  Intrusion rates: 18%, 24%	Poor accuracy for reported food items.
Medin, 2015 (45), Norway	Lunch recall (school meals), web-based self-administered recall, same day, with parental assistance	MOs	117 children (8–9 y old)	Match rate: 73% Omission rate: 27% Mean intrusion rate: 19% Higher parental education associated with better accuracy (77% vs. 52% match rate) (P = 0.008)92% agreement across raters for MOs	Poor accuracy for reported foods. Good interrater reliability for the reference method.
Paxton et al., 2011 (46), United States	Lunch recall (school meals), paper-and-pencil questionnaire, same day, no parental assistance	MOs	18 children (8–10 y old)	6% mean omission rate 10% mean intrusion rate Absolute difference: 0.06 svg Arithmetic difference: 0.01 svg 85% agreement between raters for MOs	Acceptable accuracy for both reported food items and for amounts of foods reported. Good interrater reliability for reference method.
Warren et al., 2003 (47), United Kingdom	Lunch recall (school meals and home-packed lunches), same day (2 h after meal), no parental assistance	MOs	303 children (5–7 y old)	Home-packed and school lunches, respectively:  Match rates: 70%, 58%  Intrusion rates: 22%, 8% Prompting significantly increased match rates, which increased from 66% to 80% for the whole sample Interrater reliability not reported for MOs	Poor accuracy for reported food items, regardless of lunch type. Nondirective prompts increased recall accuracy.
Estimated FRs (n = 1 study)
Domel et al., 1994 (48), United States	Estimated lunchtime FRs, no parental assistance	MOs	24 students (9–10 y old)	Using a daily monitoring approach (students were prompted by data collectors to complete their FRs), Pearson r ranged from 0.16 to 0.85 for different meal components (mean r = 0.66). However, using the weekly monitoring approach, Pearson r values dropped: range −0.21 to 0.69 (mean r = 0.25) 90% agreement between raters for MOs	Overall school meal accuracy was acceptable only for children who were monitored on a daily basis to complete their FRs (but not among children on the weekly monitoring group). Accuracy also varied considerably depending on the meal component. Good interrater reliability for MOs.
FFQs (n = 1 study)
Neuhouser et al., 2009 (49), United States	19-item beverage and snack FFQ (recall period: past week)	Estimated 4-d FRs (administered 1 wk before FFQ)	46 children (mean age 12.7 y old)	Pearson r = 0.71, 0.70, and 0.69 for beverages, snacks and total fruit and vegetables, respectively Test-retest reliability r > 0.7 for 19 items	Acceptable accuracy.Acceptable test-retest reliability.
In-person meal observations (n = 1 study)
Richter et al., 2012 (50), Canada	In-person meal observation (by nutrition students)	Weighed FRs	32 children (elementary school, no age provided)	Raters accurately identified 86% of premeasured lunches (within 0.25 svg) and over- or underreported amounts for the other meal items in 14% of the lunches Majority of ICCs >0.8	Acceptable accuracy.Good interrater reliability.
DP methods (n = 2 studies)
Sabinsky et al., 2013 (51), Denmark	DP for home-packed lunches	Weighed FRs	191 children (7–13 y old)	Spearman r range: 0.89–0.97 for amounts of meal items. No statistical difference between amounts of fish, fat, starch, whole grains, and overall lunch meal quality index scores between the test and reference instrument Bland-Altman analyses suggest negligible bias (mean bias for fruit: −4.27 g; LOA −29.4 to 20.8 g) (mean bias for vegetables −6.19 g; LOA −34.5 to 22.2 g). κ coefficients: 0.59–0.82	Acceptable accuracy. Bland-Altman plots suggest a tendency for the DP method to underestimate fruit and vegetable consumption. Acceptable interrater reliability.
Taylor et al., 2014 (52), United States	DP for school meals	Weighed FRs	958 children (8–10 y old)	Pearson r used to assess correlations between amounts estimated from DP vs. from weighed FRs for various meal lunch items. Correlation coefficient range: 0.59–0.98, all r values >0.8 except for leafy greens (r = 0.59) and lasagna (r = 0.62) Mean fruit and vegetable consumption with the use of photography (97 g) was within 1 g of reference method and not significantly different from reference method (P = 0.56). LOA for individual-tray fruit and vegetable consumption were –32.9 to 31.3 g. 96% agreement across raters; mean ICC was 0.92	Acceptable accuracy. DP was accurate at estimating amounts eaten at the group level. There was no evidence of bias from Bland-Altman analyses. Good interrater reliability.
SFC (n = 2 studies)
Kremer et al., 2006 (53), Australia	SFC	Weighed FRs	106 children (5–12 y old)	Relative accuracy for energy measured by using mean difference between test and reference instrument: Mean difference 15 kJ (95% CI: −107, 138 kJ) (P > 0.05). κ coefficient: 0.51	The SFC provides acceptable accuracy to measure energy intake for the group. Interrater reliability was poor.
Mitchell et al., 2010 (54), Australia	SFC, home-packed lunches, SFC + meal observations, and SFC + DP method	No reference measure6	176 children (5–8 y old)	Results shown are ICCs range for different lunch items By using the SCF with meal observations:  ICCs = 0.78–1 (intrarater reliability)  ICCs = 0.50–0.95 (interrater reliability) (majority ICCs >0.7 for all meal items except for noodles and leftovers) By using the SCF with DP:  ICCs = 0.57–0.98 (intrarater reliability)  ICCs = 0.34–0.92 (interrater reliability) (majority ICCs >0.7 for all meal items except for leftovers)	The SFC has good intrarater reliability; acceptable interrater reliability for majority of meal items.

¹DP, digital photography; FR, food record; ICC, intraclass correlation coefficient; LOA, limits of agreement; MO, meal observation; ref, reference; SFC, school food checklist; svg, serving; WebDASC, Web-based Dietary Assessment Software for Children.

²Results presented are group means unless specified otherwise.

³To provide an overall rating for relative accuracy and reliability (when relevant), we used cutoffs for measures of relative accuracy. Those measures of accuracy and reliability are presented in Table 1.

⁴Unless specified, the school meal recall method was interviewer-administered.

⁵Total inaccuracy combines both the type of error from misreporting meal items with measures of accuracy (33). Total inaccuracy is calculated as (absolute difference between amounts reported and observed eaten for each match × statistical weight) + (each omitted amount × statistical weight) + (each intruded amount × statistical weight) summed over all items for a given meal for each child. There is no upper limit for total inaccuracy, and this measure of reporting can be sensitive to the number and types of meal components. For example, a meal with multiple small entrées may result in greater total inaccuracy values than a meal with only 1 entrée. Thus, total inaccuracy is not appropriate to compare the accuracy of different instruments because these measures may vary depending on the meal context. Hence, this indicator was not selected as a criterion for comparing accuracy between types of instruments.

⁶The study goal was to test intra- and interrater reliability for estimating energy intake from lunches by using the SFC.

School meal recalls (n = 15 studies).

Adapted from the 24-h dietary recall method, school meal recalls rely on children’s memories to report all foods and beverages consumed at school (breakfast and/or lunches), often using specific prompting methods to elicit detailed information on types and amounts of foods consumed. This method poses minimal burden on children and may be designed to collect detailed, contextual information about consumption patterns (e.g., where was the food consumed and with whom). The main source of measurement error of this method is driven by children’s ability to recall foods and beverages consumed. The school recall method can also be affected by social desirability bias, a type of systematic error (a type of measurement error in which measurements consistently depart from the true value in the same direction) when subjects selectively misreport certain foods because of their norms and beliefs about what they should eat. However, the school meal recall is thought to hold less systematic error than an FFQ because it can capture foods and amounts consumed fairly precisely. Moreover, the cognitive task of recalling food consumed is less challenging than estimating usual intakes over a longer time period (55). For example, the Observing Protein and Energy Nutrition study, which used recovery biomarkers to estimate true intakes of energy and protein, indicated that the 24-h dietary data collected were less affected by systematic error than data in FFQs (55). Finally, school meal recalls are not subject to reactivity bias, a type of systematic error occurring when the respondent changes his or her eating behavior as a result of the act of recording food intake.

All 15 studies testing the accuracy of the school meal recall method were conducted with elementary-school children, the majority of whom were aged 9–10 y (Table 3). The relative accuracy of school meal recalls was always compared with an observational method, most frequently by using in-person MOs (n = 11 studies), DP methods (n = 3 studies), or weighed FRs (n = 1 study). The majority of studies reported poor accuracy when measured by using frequencies of misreported foods for the school meal recall (omission and intrusion rates >15% for 8 of 12 studies, see Table 3). Match, omission, and intrusion rates also varied considerably across studies, likely reflecting the influence of children’s age and the wide variety of interview conditions across methodological studies. For example, omission rates ranged from 3% (40) to 57% (32) and intrusion rates ranged from 5% (36) to 49% (56).

Four of 12 studies reported acceptable match, omission, and intrusion rates. In 2 of these studies (40, 41), recall of the school lunch meal was conducted with parental assistance in the evening by using Web-based Dietary Assessment Software for Children, which includes a built-in prompting system to help children recall frequently forgotten foods. In the third study (46), children were asked to recall foods they had consumed for lunch immediately after the lunch period by using a paper-and-pencil questionnaire. The prompting was minimal, but the retention period (time between consumption and recall) was extremely short. In the fourth study (43) the school meal recall was administered by a dietitian using web-based dietary assessment software for children. Unlike the previous 2 studies, the recall retention period was larger (children were asked to recall their school lunch meal from the previous day), yet all accuracy measures reported were acceptable.

The 5 studies reporting absolute and arithmetic differences (between amounts reported and amounts observed to be consumed by observers) suggest that children were able to report amounts consumed with acceptable accuracy for matched foods, with arithmetic differences being ≤0.1 serving/meal in all 5 studies (31, 32, 36, 37, 46). In all but 1 study (46) arithmetic differences in amounts were negative but by very small amounts (range: −0.09 to −0.02 servings/meal), suggesting that children tended to slightly underreport amounts consumed at school. Two of the 3 studies assessing the ability of the school meal recall to estimate energy intake of children (as a group) reported energy report rates falling within the acceptable margin for in-school consumption (85–115%) (39, 42). The study that reported variable energy report rates (38) had acceptable energy report rates when the lunch meal occurred the same day as the recall.

School meal recall accuracy can be potentially impacted by factors such as the age of the respondents, retention period, and use of prompting. Only one study compared the relative accuracy of the school meal recall between children of different ages and reported that grade 1 children (∼7 y old) had lower accuracy measures than their peers in fourth grade (∼10 y old) (33). Three studies compared the accuracy of the school meal recalls by using various retention periods (36, 38, 39). There were no differences in accuracy measures (energy and macronutrient report rates) across different target retention periods for one of the studies reviewed (39). There were improved (but still not acceptable) accuracy measures in the 2 other studies when shorter retention periods were used. Five (31, 34, 44, 45, 47) of the 8 studies (31, 37, 40, 44, 44–47), which conducted school meal recall the same day as when consumption took place, still reported poor accuracy for reported food items. One study reported significant increases in match rates with any prompting (47) whereas another reported no changes in accuracy measures when prompting was used (33).

A key limitation of the school meal recall method is the high labor cost associated with the interviewer administration of the recall. However, 3 studies demonstrated acceptable accuracy by using cost-saving methods, such as self-reported, paper-and-pencil school meal recall questionnaires (46) and web-based dietary recall software (40, 41). Technological advances making the school meal recall both web based and self-administered (e.g., the Automated Self-Administered 24-h dietary recall) could make this approach more cost-effective (57). The Automated Self-Administered 24-h dietary recall is a fully automated, web-based, self-administered 24-h dietary recall that includes optional modules asking respondents to report the meal occasion, location of consumption, with whom meals are eaten, and the source of the foods consumed (57). Hence, this method could be potentially adapted to explore in-school dietary intakes by isolating foods consumed at school from all foods consumed within a 24-h period.

In summary, evidence suggests that the relative accuracy of the school meal recall is poor when using measures evaluating the frequency of discrepancies for individual foods reported at lunchtime by children aged 9–10 y. However, children appear to accurately report the amounts of foods for matched foods (within 0.25 servings), and the school meal recall method was able to provide an acceptable estimate of energy intake for the group. Relative accuracy also increased when reporters were older children. However, only one study compared the accuracy of this method between children aged 7 and 10 y, and no studies have included older children or adolescents.

Estimated lunchtime FRs (n = 1 study).

The lunchtime FR is a self-administered instrument in which children are asked to record all foods consumed (types of foods and estimated amounts) for their lunch meal. Because they are self-administered, estimated FRs are less expensive to conduct than interview-administered dietary recall methods or observational instruments. Estimated FRs can provide detailed information about foods consumed without relying on children’s memory (Table 2). However, this tool is prone to reactivity bias because the acts of recording and eating occur at the same time. Like the school meal recall, the estimated FR is also subject to reporting error.

One study drawing from a small sample (n = 24 children, aged 9–10 y) tested the accuracy of estimated FRs at lunchtime (test instrument) compared with in-person MOs for a 2-wk period (reference instrument) (48). Students were randomly assigned to 1 of 2 groups: a daily monitoring group (reminded to fill out their FRs and prompted for any missing items on a daily basis) or a weekly monitoring group. Accuracy was measured by using Pearson’s correlation coefficients between number of servings reported in students’ FRs and those measured using in-person MOs. Among students who were monitored daily by trained data collectors, overall accuracy for the meal was acceptable (mean r = 0.66) but varied considerably across individual meal items (r range: 0.16–0.85). However, overall accuracy for the meal was poor among students who were monitored on a weekly basis (mean r = 0.25; range: −0.21– to 0.57). This suggests that the estimated FR method may be prone to reporting decay and perhaps reliance on memory (it is possible that children forgot to complete these food records at lunchtime and instead relied on their memory to recall foods consumed). These findings suggest that estimated FRs could result in acceptable accuracy if children are reminded daily to complete these at mealtime. However, the relative accuracy of this method appears to be highly variable depending on the meal component. There is a clear need for more methodological studies with larger and more diverse samples before recommending the use of estimated FRs to assess in-school dietary intakes. Unlike other contexts, FRs completed at school cannot be modified by parents who are not present to observe dietary intake during school hours.

FFQs (n = 1 study).

Designed to evaluate usual dietary consumption, FFQs typically include a finite list of food groupings and/or categories and ask respondents to report usual frequency of consumption of each food category over a specific time period (e.g., ≤1 mo among children because of the difficulty associated with estimating usual intakes over time). FFQs hold the benefit of being typically self-administered (hence being low cost), and they are designed to assess habitual dietary intakes (Table 2). However, FFQs can be prone to systematic error because of the difficulty in estimating intakes over longer periods and social desirability bias. Because they include a predetermined list of foods and beverages, they are not designed to capture all the foods and beverages consumed at a particular meal.

Only one methodological study assessed the relative accuracy of a 19-item FFQ [a beverage and snacks questionnaire (BSQ)] developed to measure (separately) in-school and out-of-school snack- and beverage-consumption patterns in US middle-school children (49). The BSQ, which queries intake over a 1-wk period, was administered on 2 occasions to assess test-retest reliability, separated by 4–6 wk. During the week before the second BSQ administration, participants completed a 4-d FR (which also queried location of food consumption). Accuracy was assessed by using correlation coefficients for in-school and out-of-school dietary intakes between the test instrument (BSQ) and average intakes from the 4-d estimated FR. The majority of 19 questionnaire items assessing in-school dietary intakes were significantly associated with amounts obtained from the estimated FR (mean r = 0.69 for all food and beverage items; P < 0.05). However, there were several limitations to this study. First, measurement error from the test and reference instruments were not independent (both the BSQ and the FRs were prone to reporting error). [To appropriately measure the accuracy of a self-report instrument, it is preferable to use an observational instrument (or one not prone to reporting error) (21).] Second, interpreting the test-retest reliability of the BSQ is challenging given that it captured different time periods for which dietary patterns could reasonably be different because of intra-individual variability in dietary intakes in a 4- to 6-wk period. Finally, it is possible that the act of recording foods with the use of a food diary may have helped children respond more accurately subsequent to the BSQ when administered a second time. Therefore, the relative accuracy of using the BSQ to measure in-school dietary patterns remains unclear, especially in younger children who may find it difficult to remember usual consumption patterns (27, 58) for specific time reference periods (school hours on school days). Lastly, FFQs have not been validated to measure amounts consumed during school hours, only types of foods and beverages consumed.

MOs by trained raters (n = 11 studies).

MO is a method in which trained observers and raters visually estimate amounts of foods consumed at lunchtime from children in group settings. Unlike self-report tools, MOs avoid the measurement errors associated with underreporting and social desirability bias associated with self-report tools. Like the school meal recall and FR, MOs provide detailed information on the types and amounts of food and beverages consumed, but this method can also be labor intensive, requiring extensive training on visual estimation of food consumption, and is potentially intrusive to children (Table 2). Finally, this method is also subject to reporting error from the observer or rater.

MOs were the most commonly used reference method among methodological studies reviewed. Only one study assessed the relative accuracy of the MOs by comparing visually estimated amounts from trained raters (dietetics students) with premeasured home-packed lunches (50). The percentage of items for which an accurate amount had been reported by observers in packed lunches (n = 32 lunches) was 86%, which was deemed acceptable (no variability estimates were provided) (Table 3). Because this method is often used as the reference method in methodological studies, there is a need to establish the relative accuracy of this method compared with weighed FRs in school settings.

Eleven studies measured the interrater reliability of the MO method (Table 3). Interrater reliability was consistently good, with percentage of agreements for amounts consumed across raters ranging from 85% to 97% (31, 32, 37–39, 42, 45, 46, 48) and intraclass correlation coefficients (ICCs) reported >0.8 (50). Collectively, these findings support the MO as a reliable method for estimating types and amounts of food consumed in school settings.

DP methods (n = 2 studies).

DP methods capture food intake through cameras before and after consumption. Trained staff take digital images of the lunch meal following a standardized protocol (e.g., taking pictures before and after consumption from different camera angles to capture depth), and research staff (typically dietitians) use these images to estimate consumption (59, 60). DP offers similar strengths as in-person MOs but is potentially lower in labor costs as more lunch meals can be captured by a single photographer (Table 2). However, data entry can still be labor intensive and requires trained staff to translate photos to estimated intakes. In general, available literature suggest that DP methods are fairly accurate to estimate intakes among adults with only small over- or underestimates (61). However, DP methods can omit foods that are spilled, traded, wasted, or not easily visible (for example, margarine spread on bread, milk in opaque containers). Finally, the relative accuracy of DP methods may vary greatly depending on whether children consume school meals (of known portions and nutrient composition) compared with home-packed lunches or whether children take photos themselves (e.g., using cameras from mobile phones) as opposed to trained research staff.

A limited number of studies (n = 2) suggest that DP can be an accurate method to assess the content of school meals and lunches (Table 3). In Denmark, the relative accuracy of DP to measure the content of students’ home-packed lunches was assessed relative to weighed FRs by using correlation coefficients, 2-sample t tests for differences in means, and Bland-Altman plots (51). Correlation coefficients were strong and positive (r range: 0.89–0.97), and no statistically significant differences were found in mean amounts for different lunch meal components estimated by using the DP and the reference method. Bland-Altman analyses suggested a tendency to slightly underestimate fruit [mean bias: −4.27 g; limits of agreement (LOAs): −29.4 to +20.8 g] and vegetables (mean bias: −6.19 g; LOA −34.5 to + 22.2 g). When measuring meal quality by using the Meal Index of dietary Quality (a composite diet quality index for the lunch meal with scores ranging from 0 to 28, and 28 being a perfect score), the Bland-Altman plots suggested no evidence of bias (mean bias: 0.07 units; LOA −2.26 to 2.40 units). In another study, the accuracy of the DP method was tested by using weighed FRs (52). All 11 school meal items had a correlation coefficient >0.7 (r range: 0.76–0.98) except for leafy greens (r = 0.59) and lasagna (r = 0.62). The group’s mean for meal items was within 1 g of the reference method, and there were no statistical differences in group means for other meal components and no evidence of bias in Bland-Altman analyses. Both of these studies suggest the potential of the DP method to provide a valid method for estimating dietary intakes (in terms of the types of food consumed and the amounts consumed) in the context of home-packed lunches (51) as well as school meals (52). However, because of the paucity of studies, additional research is needed to establish this method as an accurate approach for assessing dietary intakes in the school context.

The SFC (n = 2 studies).

The SFC was designed to facilitate data collection for MOs conducted in Australian elementary schools (Table 2) (53). The SFC is a 1-page form listing 29 foods and beverages commonly found in Australian students’ lunches. Trained observers monitor children at mealtime, typically 1 observer/4–5 students. Portion sizes are estimated by using predefined standard portion sizes (e.g., 15 mL of peanut butter, 1 slice of bread). There are many limitations to this instrument, the most important one being that it does not stand alone and still requires either DP or in-person meal observers to measure intake. Moreover, the SFC has a finite food list and may miss other foods not listed in the SFC. Thus, measurement error can also arise from misreporting from the observers and raters observing children.

We found only one methodological study examining the ability of the SFC to estimate energy intake among children (53). The mean difference in estimated energy intake between the weighed FR and the SFC was 15 kJ (95% CI: −107 to 138 kJ). However, the LOA (±2 SD) were large (±1270 kJ), suggesting wide variations between raters. The SFC overestimated the energy from breads and fruit drinks and underestimated energy from fat spreads, crackers, granola bars, and fruit. Findings suggested the SFC with MOs by trained raters demonstrated poor interrater reliability, with a mean κ coefficient of 0.51 (53). However, in a more recent study comparing the intra- and interrater reliability of the SFC when used with MOs and the DP method (54), ICCs for intrarater reliability ranged from 0.57 to 1 for different meal components. In contrast with the earlier study, the ICCs for interrater reliability tended to be higher, and interrater reliability was deemed acceptable (ICC >0.7) for the majority of meal components except noodles and leftovers.

Frequency of instrument use and dietary outcomes assessed in school contexts

Table 4 lists the frequency of dietary assessment methods used in studies examining dietary outcomes in the school context. Detailed information from each study is provided in Supplemental Tables 1–7. Overall, observational methods (n = 27 of 47 studies) were slightly more common than self-report instruments (n = 22 of 47 studies), with 2 studies using >1 method. The most common methods were weighed FRs (n = 12 studies) (62–73), followed by school meal recalls (n = 10 studies) (74–83), MOs (n = 8 studies) (84–91), estimated FRs (n = 7 studies) (92–98), FFQs (n = 5 studies) (99–103), DP methods (n = 5 studies) (104–108), and the SFC (n = 2 studies) (90, 105).

TABLE 4.

Frequency of dietary instruments used and types of dietary outcomes used to assess dietary intakes in the school context (n = 47 studies)¹

	Meal recalls	Estimated FRs	FFQs	Weighed FRs	MOs	DP	SFC
All studies,2 n = 47	10	7	5	12	8	5	2
Studies using specific analytical methods to measure in-school dietary intakes
Multiple dietary components,2 n = 44	8	8	4	12	8	4	2
Dietary components,3 n	17 (1–58)	28 (19–64)	11 (7–19)	17 (1–34)	13 (1–27)	8 (1–26)	20 (13–26)
Macronutrients, amounts and/or densities, n = 22	2	7	0	9	3	1	0
Energy density, n = 3	1	1	0	1	0	0	0
Micronutrients, amounts and/or densities, n = 19	1	6	0	8	3	1	0
Fruit and/or vegetable intake exclusively, n = 5	1	0	0	2	1	1	0
Food groups,4 n = 32	5	6	3	8	5	3	2
Composite diet quality index, n = 3	1	0	1	0	0	1	0

¹Values are number of studies unless otherwise indicated. DP, digital photography; FR, food record; MO, meal observation; SFC, school food checklist.

²Some studies used >1 instrument so the row total does not match the number of studies reviewed. Two studies used 2 instruments to assess overall diet quality in the school context.

³Values are mean (range) number of dietary components among studies.

⁴These studies included a wide range of food group outcomes. Outcomes were expressed as the proportion of children consuming specific types of food groups and/or amount in servings of specific food groups. Some studies also reported on either frequency of intake of absolute intake of food groups considered minimally nutritious, such as sugar-sweetened beverages; low-nutrient energy dense foods; fast foods; desserts, pastries, and confections; and minimally nutritious packaged snacks.

Ninety-four percent of studies (n = 44 of 47) used multiple single dietary components to describe in-school diet quality. The number of components varied greatly, ranging from a single food group or nutrient to 64 dietary components (mean ± SD: 18 ± 14). The mean number of dietary components was higher for instruments eliciting more detailed information about types of foods and amounts consumed. For example, the mean number of dietary components was 28 for estimated FRs. In contrast, the mean number of dietary components was 11 for FFQs and included only food-based outcomes. There was no clear pattern connecting type of dietary instrument and type of dietary outcomes measures used. However, studies using FFQs and the SFC reported only food-based outcomes.

Only 3 of 47 studies reviewed developed or applied a composite measure to evaluate the overall quality of the lunch meal in the school context (77, 101, 108). A composite meal-based diet quality index (the Meal Index of dietary Quality) was developed, validated, and applied to measure the quality of foods in home-packed lunches for Danish children aged 7–13 y by using the DP method (108). Seven dietary components (fat, saturated fat, snacks, whole grains, fish, fruit, and vegetables) provided a composite score (ranging from 0 to 28 points) to reflect adequacy, balance, and moderation. In Finland, researchers used a simpler measure of meal-based diet quality based on the balance dimension of diet quality (101). Usual dietary patterns were assessed by using a web-based 37-item FFQ with the school week as reference period. Children were classified as balanced or unbalanced eaters based on whether they reported consuming 3 meal components (main dish, salad, and bread) from school lunches on most school days in 1 wk. In the United States, the Healthy Eating Index (HEI) 2005 [a composite diet quality index reflecting dimensions of adequacy, balance, and moderation (109)] was used to compare in-school breakfast and lunch diet quality on school days compared with breakfast and lunch diet quality on weekend days. Although the US HEI was originally developed to assess diet quality based on a whole day using 24-h dietary recall data, it can potentially be applied to a portion of the school day because its scoring standards are expressed on an energy density basis.

Discussion

To our knowledge, this is the first methodological review to rate the relative accuracy of dietary assessment methods for assessing dietary intakes in the unique consumption context of schools. Assessing dietary intakes in school is methodologically difficult from the perspectives of both data collection and data analysis. We provided a comprehensive review of the advantages, limitations, and types of measurement error associated with diverse dietary assessment approaches. A study’s specific objectives and outcomes of interest should be taken into consideration when choosing a dietary instrument, as each tool offers trade-offs between costs and ease of administration (often affected by whether the tool is self-administered or not). Instruments also vary in their sources of measurement error and the types of dietary outcomes that can be derived.

Overall, the relative accuracy of the school meal recall method was poor when accuracy was assessed by reporting the frequencies of discrepancies between foods reported and foods observed to be consumed, as most studies exceeded the ±15% cutoff applied here. We chose this cutoff because it corresponded to cutoffs established for accuracy of energy reporting. If accuracy had been evaluated by using less-stringent cutoffs, such as those proposed by Baxter and colleagues (31, 32, 36–38) allowing ±30% differences, the school meal recall would have offered acceptable accuracy in 6 of the 12 studies reporting match, omission, and intrusion rates. This last point illustrates the challenges and implications of establishing cutoffs for acceptable levels of accuracy. Although studies reviewed suggested that the school meal recall method was rarely accurate for measuring the types of foods consumed by children at lunchtime, other studies suggested that the school meal recall could still provide relatively accurate estimates of mean energy intake for a group of children. This underscores the importance of considering accuracy measures used by researchers conducting validation studies.

The variations in accuracy measures reported across studies testing the school meal recall method are consistent with a recent review that found that the accuracy of dietary recall methods among children can vary widely depending on children’s age, interview conditions [e.g., the retention period, type of prompting, and use (or not) of parental assistance] (110). This review similarly suggests that the accuracy of this method varies widely depending on the sample characteristics (e.g., age of children) and interview conditions. The school meal recall could potentially offer acceptable accuracy for estimating energy intake at lunchtime for a group of children when using well-designed, 24-h dietary recall software and/or skilled interviewers. Only one study, which included 7- and 10-y-olds, compared the accuracy of this method between children of different ages. Clearly, more validation studies among older children or adolescents are needed.

In contrast, methodological evidence on other self-report tools for use with children in the school context is limited. The relative accuracy of FFQs in the school context was tested in only one study (49). More research on the accuracy of FFQs for assessing in-school dietary intakes (preferably with the use of observational instruments as reference instruments) is needed for future application of FFQ-based approaches to examine in-school dietary intakes. although FRs present some benefits similar to school meal recalls providing potentially rich information about types and amounts of foods consumed, evidence regarding the accuracy of their use in school contexts was also limited to a single study, suggesting reporting decay when children were not reminded or prompted by researchers to record foods consumed at lunchtime (48). There may also be considerable variation in the relative accuracy of using FRs as a means to assess dietary intakes in the school context depending on the study conditions and characteristics of the population. More studies in larger, diverse samples are recommended before recommending FFQs and estimated FRs tools for capturing in-school dietary intakes.

Across all studies reviewed, observational instruments such as weighed FRs and MOs were seen as having less measurement error and were commonly used as reference instruments. The ease of applying observational instruments may vary depending on specific conditions within the school context. For example, lunch source (meaning where the majority of children obtain their lunch food from, e.g., a school meal compared with a home-packed lunch) appears to play a decisive role when deciding on a dietary assessment instrument. It is not surprising that weighed FRs were commonly used in US studies, where a substantial proportion of children participate in the National School Lunch Program (NSLP) (8). The plate-waste method, which is a form of weighed food record in which amounts of unconsumed food are subtracted from the amounts served, can be an effective, nonobtrusive way of estimating consumption (111, 112). This works well in school contexts where the majority of children obtain their meals from school food-service operations where standardized menus are in place. However, in school contexts such as Canada, which does not have a national school lunch program and where students commonly bring a home-packed lunch (81), the plate-waste method is not well suited because nutritional information on meals offered and served is not easy to obtain.

Studies including in-school dietary outcomes used a broad range of instruments. The most frequently used instruments were weighed FRs by trained observers, followed closely by school meal recalls, direct MOs by trained raters, and estimated FRs. The types of analytical approaches used to evaluate dietary consumption were diverse, and the range of dietary components was greater among studies relying on school meals recalls and FRs (weighed or estimated). The vast majority of studies used multiple single dietary components to capture in-school dietary intakes. Using multiple single components to assess diet quality is appropriate when there are food-based standards available to provide a nutritional benchmark for evaluation. For example, dietary data from meal recalls can be compared to specific meal-based standards such as those issued by the US National School Meal Programs stipulating that school lunches must include a minimum of 1/2 cup whole fruit, 3/4 cup vegetables, 1 ounce grain without added sugars, 1 ounce meat or meat alternate, and 1 cup fluid milk (113). Because school meal recalls are designed to assess not only types of foods consumed but also amounts, the dietary data generated from meal recalls is more flexible in terms of being operationalized into various outcome measures for assessing overall diet quality. For example, dietary data from school meal recalls has been used to examine overall diet quality by using an a priori diet quality index (the US HEI 2005) between NSLP and non-NSLP participants (77).

Summary and recommendations for future research

The limited availability of appropriately tested dietary assessment methods poses a challenge for the evaluation of diets among children. Clearly more methodological work is needed to improve the accuracy of self-report instruments for assessing in-school consumption. Among the potential self-report tools available, school meal recalls can be used if the study’s goal is to assess overall energy intake for a group of children. If school meal recalls are used, the quality of measures can be improved by using web-based tools and/or skilled interviewers to help elicit detailed information about foods consumed. Among observational tools reviewed, MOs and DP methods appear to offer acceptable accuracy, but the methodological evidence remains limited. Understanding the advantages, limitations, sources of measurement error, and dietary outcomes associated with dietary assessment methods is vital for improving the quality of the evidence used to inform and evaluate the impact of school-based nutrition policies and programs.