School-Age Children Can Recall Some Foods and Beverages Consumed the Prior Day Using the Automated Self-Administered 24-Hour Dietary Assessment Tool (ASA24) without Assistance

ABSTRACT

Background

Technological innovations allow for collection of 24-h recalls (24HRs) in a broader range of studies than previously possible. The web-based Automated Self-Administered 24-Hour Dietary Assessment Tool (ASA24) has been shown to be feasible and to perform well in capturing true intake among adults. However, data to inform use with children are limited.

Objective

This observational feeding study was conducted to evaluate children's ability to accurately report a lunchtime meal using ASA24 without assistance.

Methods

The study was conducted among children (n = 100) aged 10–13 y within a school setting. Students were served an individual cheese pizza, baby carrots, ranch dip, yogurt, a cookie, and 1 choice of water, juice, or milk. Plate waste was collected and weighed. The next day, participants completed ASA24 and a sociodemographic questionnaire. Descriptive statistics were generated to determine match rates by food item and age, and linear regression analyses were conducted to examine associations between sociodemographic characteristics and accuracy of reported energy and nutrient intake. Associations between true and reported energy and nutrient intakes and portion sizes were assessed with use of t tests.

Results

Just under half (49%) of children fully completed ASA24 (median time, 41 min). Children reported an exact, close, or far match for 58% of all foods and beverages consumed, ranging from 29% for dip to 76% for pizza, but also reported some items not consumed as part of the study meal. Older children completed the recall in a shorter time than younger children (mean 31 among 13 y compared with 52 min among 10 y). Intakes of energy (39%), protein (33%), and sodium (78%) were significantly overestimated, whereas portion sizes for cookies (53%) and juice (69%) were underestimated.

Conclusions

Children can report some foods and drinks consumed using ASA24, but our findings suggest challenges with independent completion, necessitating research to examine strategies, such as training and resources, to support data quality.

Introduction

The capacity to collect accurate dietary data in surveillance and epidemiological research is a widely discussed topic in the field of nutrition (1–3). Validation studies have shown that 24-h dietary recalls (24HRs) capture diet with less bias than other self-report methods (4, 5). However, the use of 24HRs has traditionally not been realistic for many studies given the need for trained interviewers and coders (6, 7). Recent advances in dietary assessment have leveraged technology to ease the burden of collecting recall data (6, 8). Resulting tools, such as the Automated Self-Administered 24-Hour Dietary Assessment Tool (ASA24) developed by the US National Cancer Institute (9) and tailored for use with populations in Canada and Australia, have enabled collection of 24HRs in a wide range of studies.

Among school-age children, there are unique considerations related to the collection of dietary intake data. Depending on age, children's cognitive abilities may not be fully developed, impacting their ability to recall foods eaten over a given period of time, to accurately estimate portion sizes, and to remain engaged during an assessment period (10–12). Social desirability biases may also impact reporting, leading to misestimation of intake (13). In addition, children may lack knowledge of cooking or preparation methods and ingredients (12). Although online data collection tools cannot overcome all of these issues, they may facilitate engagement and thus accuracy of reporting through a dynamic interface and digital images that offer a range of portion sizes tailored to the food or beverage type (14, 15).

Prior research conducted with children aged 9–11 y in the United States found that recalls completed using ASA24 were significantly less accurate than interviewer-administered recalls, but both performed poorly compared to true intake (16). Further, the accuracy and ease-of-use of ASA24 has been shown to vary by age, with younger children (ages 8 and 9 y) excluding a greater number of items truly consumed than older children (ages 10 and 11 y) (17). These findings are consistent with prior research suggesting that 10 y may be the lower cutoff at which children are able to accurately retrieve memories of what foods and beverages were consumed and when (18). For this reason, surveys such as NHANES use proxy and proxy-assisted reporting for children up to the age of 12 y (19).

In this study, the extent to which children aged 10–13 y could accurately report their intake for a single meal using ASA24 without assistance was assessed. Secondary objectives included examining completion times by age, as well as characteristics associated with accuracy of reporting.

Methods

This study received ethics clearance from Public Health Ontario and the University of Waterloo Office of Research Ethics (#21155).

Participant recruitment

Children aged 10–13 y (grades 5 through 8) who were attending schools within school boards located in southwestern Ontario, Canada were recruited to participate. Although the initial aim was to recruit children aged 11 and 12 y (grades 6 and 7), this limited inclusion of mixed-grade classes and the eligibility criteria were thus broadened. Recruitment flowed through 4 hierarchical levels: 1) school boards; 2) schools; 3) classrooms; and 4) children (Figure 1). First, representatives of 8 publicly funded school boards that had the capacity to provide research clearance for studies of this nature were contacted. Three boards agreed to participate and 1 was included in the study, allowing use of a single caterer to prepare and deliver lunchtime meals to multiple schools within the catchment area. A recruitment package was emailed to principals of schools not on a designated Aboriginal reserve or settlement and that had high-speed wireless Internet capabilities and classroom computers. The package contained an information letter and samples of study materials to be disseminated to teachers and parents. Three schools and 7 teachers within those schools agreed to participate.

FIGURE 1.

Study flow diagram detailing recruitment of participants through school boards, schools, classrooms, and children.

Teachers were provided with information packages to be sent home with children. The packages included an information letter detailing the study and eligibility criteria (parental consent; comfortable reading and writing in English; did not have a physical, mental, or visual limitation that would inhibit their ability to recall diet or use a computer; and did not have food allergies or food avoidances that would limit consumption of provided foods and beverages), and a consent form. Overall, 173 children were provided with information packages and parental consent was provided for 129 (75%). In participating classes, 2 consecutive study days were scheduled and teachers were asked to notify parents that children would be provided with lunch on the first of the 2 days.

Data collection

The data collection processes were adapted from a validation study conducted among adults (20). Data were collected in March and April 2016 on 2 consecutive days in each school.

Day 1 procedures

Assent process

On Day 1, trained research assistants visited the classrooms at the beginning of the scheduled lunch period to describe the study, noting that the aim was to learn about children's eating habits. Children who had parental consent were given an opportunity to ask questions and to then provide written assent if they wished to participate. Of the 129 eligible children, 112 were present on Day 1 and 111 provided assent. Of these, 1 was excluded because of a food allergy (Figure 1), leaving 110. Once the assent process was complete, research assistants delivered a food tray to each child who had parental consent and who had assented.

Preparation and serving of lunch meals

Foods served included an individual cheese pizza, a single-serve container of yogurt, a bag of mini carrots, a container of ranch dip, and a chocolate brownie cookie. Children were offered a choice of bottled water, a juice box (orange or apple), or a carton of milk. These foods and beverages were chosen to provide variation in terms of degree of healthfulness (actual and/or perceived) as well as format (e.g., an unpackaged cookie compared with a juice box). As well, these foods could be standardized in terms of serving sizes.

Foods were delivered to the schools by the caterer approximately 30 min before the start of the lunch period to ensure that hot foods were kept hot and cold foods were kept cold. Individual trays (Supplemental Figure 1) were then prepared by trained research assistants in a space not visible to children. While trays were being prepared, 5 samples of each food item were weighed twice with a WR-12 K scale (My Weigh), which has an accuracy of 0.05 oz. or 1 g, and the weights recorded in an Excel spreadsheet. If the 2 weights were not identical, a third weight was taken by the same research assistant and the average of the 3 weights was recorded. For foods served in containers (e.g., pizza), weights were tared. The average weight across the 5 samples of each food was used as the starting weight for calculating amounts consumed for each participant (based on plate waste, described below). This process was repeated at each school because the format of particular foods differed slightly (e.g., the local branches of the catering service provided ranch dip and yogurt in different formats). One exception to the weighing process was carrots, which the caterer standardized by number despite requests to standardize by weight. It was not feasible to weigh each bag of carrots while also preparing the trays in the available preparation time; thus, it was not possible to assess the accuracy of portion size reporting for this item.

An identification sticker was placed on the top and the bottom of each tray; as the trays were distributed, the sticker on the top was removed and placed on the child's assent form. Once all trays were distributed, children were offered a choice of beverage, with their selection recorded unobtrusively on the assent forms. Meals were consumed in the classroom, with research assistants present to discourage sharing of foods and beverages. When children were finished eating, they were asked to leave packaging and any leftover food on their trays.

Measurement of plate waste

After all children had exited at the completion of the meal, each tray was wrapped in plastic and taken to the preparation area, where plate waste was weighed and recorded. Research assistants took 2 weights for each item on the tray, with a third taken in cases in which the first 2 were not identical. Empty containers were used to tare scales so that recorded weights represented the remaining foods and beverages (i.e., plate waste). From the starting weights and plate waste weights, the amount of each food and beverage consumed (i.e., true intake) by each child was calculated. In some cases, children left plastic spoons in yogurt containers. Whether the weights were taken with or without spoons was recorded, but as their contribution was negligible (<1 g), this was not accounted for in the calculation of amounts consumed.

For 2 children, plate waste was not weighed because it was inadvertently discarded before weighing or because the identification label was missing. Data for these 2 children were excluded, leaving 108 children with true intake data. Further, although children were asked to leave all packaging and remaining food on trays at the end of the meal, in a small number of cases, packaging (e.g., a water bottle or bag of carrots) was taken out of the classroom; as a result, it was not possible to assess accuracy of reporting of 7 food items served to 5 participants. The participants were retained for analyses, but the data for these 7 food items were excluded.

Day 2 procedures

On Day 2, research assistants returned to the classrooms for a period ranging from 50 to 60 min and asked participating children to independently complete ASA24 on school-provided computers (nonparticipating children were provided with a link to a nutrition-oriented game). Eight children were not present on the second day of data collection and did not complete ASA24. Thus, the final sample size of children with true intake data and who completed ASA24 was 100.

Teachers were aware of the scheduled data collection on Day 2, but children were not informed of the timing of this part of the study nor the task. Children were provided with a short sociodemographic and health survey that included a cover sheet with their identification as well as corresponding ASA24 username and password information, and were asked to begin by completing ASA24. The ASA24 system prompts participants to report all foods and beverages consumed in the prior 24-h period, including optional queries regarding the meal time, location, and sources of foods and beverages; although true intake data were recorded for lunch only, to avoid focusing the children on that meal specifically, these instructions were not altered. Research assistants provided guidance to children when needed in terms of opening an Internet browser, navigating to the ASA24 website, and entering usernames and passwords, but did not provide other assistance with ASA24 completion, aside from prompting to “do your best” when children asked questions. When children asked each other questions or made comments aloud about what they had eaten the day before, research assistants asked them to focus on their own work.

Most children (n = 86) were asked to complete ASA24-2016, the most recent version of the tool (the 2016 version of ASA24-Canada was not available at the time of data collection so the 2016 US version was used as the interface is identical to that of the Canadian version subsequently released in fall 2016). The use of ASA24-2016 avoided known issues with the Silverlight plug-in that were encountered with the 2014 versions of ASA24, including the Canadian version. For comparative purposes, in 1 classroom in which it was known in advance that the Silverlight plug-in was supported, children (n = 14) completed ASA24-2014-Kids (21); ASA24-Kids has since been discontinued but was tailored for use with children through child-friendly terms (e.g., bubbly instead of carbonated water) and fewer prompts about food preparation.

When they had completed the ASA24 recall or there were approximately 10 min left in the period (whichever occurred first), children were prompted to complete the paper-based sociodemographic/health survey. If they finished early, they were provided with a link to an online nutrition-oriented game and/or provided with an activity sheet.

Data preparation

Completeness of recall data

Of the 100 eligible recalls, 49 were deemed “complete” by the ASA24 system (and 43 of these had at least 1 food item reported for lunch), whereas 51 were incomplete (“quit”), with the latter indicating that the recall was not fully finished. Among these 51 recalls, 33 included a lunch meal with items similar to those that were served; 12 did not include any items for the lunch meal; and 6 included only food and beverage items that were different from those served (i.e., no reported foods matched the provided lunch). These respondents were included in the analyses to avoid overestimating the accuracy of reported intakes.

Coding of true intake data

Each food and beverage served was matched to a food code from the USDA Food and Nutrient Database for Dietary Studies (FNDDS) (22) (which is also used in ASA24-2016). This was conducted by 1 research assistant and verified by a second. The assignment of FNDDS codes allowed calculation of true energy and nutrient intakes.

Identification of matches between true and reported intake

Although children were prompted to complete ASA24 for the day prior (midnight to midnight), the focus was on the lunch meal for which true intake data were collected. Foods and drinks reported at other eating occasions were examined for those similar to the lunch served to identify children who potentially misreported the eating occasion, but as this was not found to be the case, data for all other mealtimes were excluded.

The “My Selections” file was downloaded from the ASA24 system to generate a list of all food and beverage items reported for the lunch meal by all children, and each was determined to be an exact, close, or far match for the foods and beverages offered. For example, cheese pizza with thin crust was considered an exact match, cheese pizza with thick crust a close match, and pepperoni pizza a far match. This activity was conducted without consulting the true intake data and resulted in a match key that was subsequently applied to examine whether children reported a match for each food and beverage they actually consumed. Through this process, exclusions (i.e., foods that were consumed but not reported) and intrusions (i.e., foods that were reported but not consumed) were also identified.

In some cases, children (n = 37) did not provide sufficient details on foods or beverages for food codes to be assigned within the ASA24 system (e.g., pizza was reported but details such as type and portion size were not indicated). In a real-world study, it is possible that values for such foods and beverages may be imputed. Therefore, to allow inclusion in the analyses considering the accuracy of energy and nutrient intakes based on reported consumption, FNDDS food codes were assigned to these foods and beverages. A trained research assistant simulated children's responses by entering the food item (i.e., pizza) into ASA24 and selecting “don't know” for each subsequent prompt (e.g., “how much was eaten?”). The resulting food codes assigned within ASA24 were applied for the purpose of comparing true and reported energy and nutrient intakes.

Covariates

Several participant characteristics were assessed with a health/sociodemographic survey. Gender was assessed by asking whether children identified as boy, girl, other, or “prefer not to answer”; the latter 2 categories were collapsed as only 2 children selected these options. Children self-reported their eating habits as poor/fair, good, or very good/excellent (a missing category was also included). Weight perception (if children considered themselves “very underweight,” “underweight,” “about the right weight,” “overweight,” or “very overweight”) was also considered; the first 2 and latter 2 categories were collapsed, and an additional category was created to identify children who did not respond. Family affluence (based on indicators such as whether the child has their own bedroom, and number of family vehicles) was determined through the Family Affluence Scale II (FAS II) (23), and children were classified as having low/medium (FAS score 0–5) or high (FAS score 6–9) family-level affluence.

Statistical analyses

Analyses were conducted with SAS, version 9.4 (SAS Institute). Descriptive statistics were generated to describe participant characteristics, as well as ASA24 completion rates and session durations across the sample and by age and gender. Further, chi-square analyses were conducted to detect differences in match rates between participants who completed ASA24-2016 and ASA24-Kids; as there were no significant differences in the rates between these 2 subsets (data not shown), the data were combined. Descriptive statistics were then generated to determine match rates (exact, close, far, exclusions) by food/beverage and age. The overall match rate was calculated for all children in the study, and repeated excluding the 12 respondents with incomplete ASA24 status who did not report a lunchtime meal.

Two-tailed paired t tests were conducted to examine differences between true and estimated intake of energy and nutrients among all children in the study. In particular, energy, protein, potassium, and sodium were examined because previous studies assessing errors associated with estimation of these components by comparing self-report to recovery biomarkers have found that misreporting among adults and children is associated with characteristics such as education and body weight status (5, 24, 25).

Linear regression analyses were conducted to examine associations between characteristics that may impact reporting accuracy (age, gender, eating habits, weight perception, and family affluence) and i) mean match rates (overall, exact, close, and far) and ii) the mean ratio of true to estimated intakes for energy and the selected nutrients. To account for multiple comparisons and decrease the false discovery rate, the Benjamini-Hochberg procedure (26) was conducted for both sets of regression analyses.

True and reported portion sizes were plotted and 2-tailed paired t tests were used to examine whether they were significantly different by item (excluding the carrots because of issues with standardization of weight of portions offered, described earlier), excluding the 12 children who did not report a lunchtime meal and the 6 who reported items different from those served (n = 82).

Results

Most of the children were aged 11 or 12 y (79%) and had high family affluence (58%) (Table 1). The median number of minutes spent logged in to ASA24 was 41, ranging from 3 to 304 (Table 2). Boys and girls had similar session durations, with girls generally taking slightly longer to complete ASA24. On average, older children completed the recall in a shorter time than their younger peers.

TABLE 1.

Characteristics of school-age children who participated in observational feeding study to examine the validity of ASA24 (N = 100)¹

Characteristic	%
Gender, age	—
Girl	49
10 y	3
11 y	26
12 y	16
13 y	4
Boy	49
10 y	5
11 y	20
12 y	15
13 y	9
Other/prefer not to answer	2
12 y	2
Eating habits2	—
Poor	4
Fair	19
Good	37
Very good	21
Excellent	4
Don't know/unsure	13
Prefer not to answer	2
Family affluence3	—
Low/medium	23
High	58
Not stated	19
Weight perception4	—
Underweight	18
About the right weight	54
Overweight	11
Don't know/prefer not to answer	17

¹ASA24, Automated Self-Administered 24-Hour Dietary Assessment Tool; FAS, Family Affluence Scale.

²Eating habits were assessed with use of a self-report item inquiring whether participants considered their eating habits as poor, fair, good, very good, or excellent. The first 2 and the last 2 categories were combined for subsequent analyses.

³Family affluence was assessed with the FAS II (23), based on indicators such as whether the child has their own bedroom, and number of family vehicles. Children were classified as having low/medium (FAS score 0–5) or higher (FAS score 6–9) family-level affluence.

⁴Weight perception was assessed with use of a self-report measure, whereby children identified perceiving themselves as “very underweight,” “underweight,” “about the right weight,” “overweight,” or “very overweight.” The “underweight” and “overweight” subcategories were collapsed for subsequent analyses.

TABLE 2.

Time spent logged in to ASA24 among school-age children, by age and gender (N = 98)¹

		Gender
		Boys (n = 49)	Girls (n = 49)
Age, y	n	Min	Min
10	8	51 (39, 81)2	52 (51, 57)
11	46	43 (34.5, 48)	47 (28, 54)
12	31	31 (13, 45)	37 (30, 49)
13	13	34 (31, 37)	28 (25, 31)
Overall	98	40 (29, 47)	43 (29, 51)

¹Values are medians (25th, 75th percentiles). Excludes 2 children who self-reported gender as “other” or “prefer not to answer.” ASA24, Automated Self-Administered 24-Hour Dietary Assessment Tool.

²Two children had session durations that were substantially higher than the remainder (81 and 304 min), indicating that they did not log out of their sessions until much later.

Overall, children reported an exact, close, or far match for 58.3% of all foods and beverages actually consumed at lunch [when children who did not report a lunchtime meal (n = 12) were excluded, the overall match rate for all foods and beverages increased to 66.1%]. Just under 90% of children reported at least 1 food item that was consumed. The overall match rates were 60.4% for 10-y-olds, 65.0% for 11-y-olds, 42.8% for 12-y-olds, and 71.6% for 13-y-olds. Pizza and water had the highest overall match rates (76.0% and 70.8%, respectively), but carrots and water were most often classified as exact matches (34.0% and 66.7%) (Table 3). After adjusting for multiple comparisons, linear regression analyses indicated that older children reported significantly fewer far matches (Supplemental Table 1). There were no other significant associations with match rates.

TABLE 3.

Proportion of school-age children reporting exact, close, and far matches reported and excluding items truly consumed at a lunchtime meal (n = 100)¹

	Foods and beverages offered (number of children who truly consumed the item)
	Pizza (n = 100)	Carrots (n = 97)	Dip (n = 90)	Cookie (n = 98)	Yogurt (n = 84)	Water (n = 20)	Milk (n = 35)	Juice (n = 40)
Exact matches, %	24.0	34.0	18.9	21.4	26.2	66.7	5.7	50.0
Close matches, %	40.0	21.6	10.0	38.8	31.0	4.2	54.3	7.5
Far matches, %	12.0	7.2	0.0	2.0	0.0	0.0	2.9	0.0
All matches combined, %	76.0	62.9	28.9	62.2	57.1	70.8	62.9	57.5
Exclusions, %	24.0	37.1	71.1	37.8	42.9	29.2	37.1	42.5

¹Proportions, or match rates, represent percentage of children who reported or excluded items in comparison to their true intake of those items.

Children excluded 2.36 items on average, and excluded items contributed 172 (range 0–702) kcal and 153 (range 0–577) g, on average per child. Dip was the most frequently excluded item (71.1% did not report the ranch dip), followed by yogurt (42.9%), and juice (42.5%). Girls reported an average of 0.43 intrusions compared to 0.20 among boys (among the 2 students who reported other or prefer not to answer for gender, the mean number of intrusions was 6). A total of 17 children reported intrusions, with 11 reporting 2 or more. Intrusions contributed 42 (range 0–941) kcal and 30 (range 0–873) g, on average per child. The most common intrusion was bottled water (n = 5).

Estimated intakes of energy, protein, and sodium based on ASA24 were higher than true intakes (Table 4) among the full sample. When children for whom details of foods reported were imputed were excluded, the difference was smaller but still significant for calories (data not shown). Linear regression analyses revealed no statistically significant differences in accuracy of energy and nutrient estimates based on reported intake by age, gender, weight perception, eating habits, or family affluence (Supplemental Table 2).

TABLE 4.

Mean amounts of true and reported energy and nutrient intake for the lunch meal among school-age children (n = 100)¹

	Mean amount
Nutrient	True (SE)	Reported (SE)	Difference between true and reported (SE)2
Calories, kcal	542 (10)	755 (46)	−212 (44.8)*
Protein, g	20.3 (0.5)	27.1 (1.8)	−6.8 (1.7)*
Potassium, mg	762 (23)	798 (56)	−36.1 (56.0)
Sodium, mg	696 (12)	1239 (73)	543 (73.9)*

¹In some cases, children (n = 37) did not provide sufficient details on foods or beverages for food codes to be assigned within the ASA24 system. For these foods, FNDDS food codes were assigned to these foods and beverages by simulating children's responses by entering the food item (i.e., pizza) into ASA24 and selecting “don't know” for each subsequent prompt (e.g., “how much was eaten?”). The resulting food codes assigned within ASA24 were applied for the purpose of comparing true and reported energy and nutrient intakes. ASA24, Automated Self-Administered 24-Hour Dietary Assessment Tool; FNDDS, Food and Nutrient Database for Dietary Surveys.

²Values are mean differences between true and reported intakes ± SEs, n = 100. Mean differences reported were estimated with a paired 2-tailed t test. Values with an asterisk (*) were statistically significant at P < 0.05.

Supplemental Figure 2 illustrates true and reported portion sizes, by food and beverage. Reported portion sizes were lower than the true portion sizes for the cookie and juice (Table 5).

TABLE 5.

Mean amounts truly consumed and reported by food and beverage among school-age children (n = 82)¹

		Mean amount, g
Food or beverage2	Number of children who truly consumed the item	True (SE)	Reported (SE)	Difference between true and reported (SE)3
Pizza	100	99.0 (1.7)	104 (10.7)	−5.0 (10.5)
Dip	90	9.8 (1.2)	7.7 (2.0)	2.0 (2.2)
Cookie	98	37.2 (1.2)	17.3 (2.5)	19.7 (2.6)*
Yogurt	84	41.9 (4.2)	51.8 (8.8)	−9.9 (8.5)
Water	20	236 (38.5)	280 (50.3)	−43.9 (85.3)
Milk	35	250 (0.8)	183 (53.6)	67.3 (53.8)
Juice	40	182 (7.4)	56.7 (14.7)	126 (14.7)*

¹Portion size analyses excluded the children who did not report any lunch items (n = 12) and the children who only reported lunch items that were not similar to those served (n = 6), resulting in a final analytic sample of 82 children. ASA24, Automated Self-Administered 24-Hour Dietary Assessment Tool.

²Carrots were excluded from these analyses because there was no standardization of the weight of carrots offered.

³Values are mean differences between true and reported intakes ± SEs, n = 82. Mean differences reported were estimated with a paired 2-tailed t test. Values with an asterisk (*) were statistically significant at P < 0.05.

Discussion

Some children aged 10–13 y were able to report some foods and beverages consumed at a lunchtime meal using ASA24 without assistance; nearly 90% reported at least 1 food item consumed. However, match rates were low for some items and ∼50% of children did not complete the recall in the allotted time. The findings of this study raise several considerations related to the use of technology-enabled tools such as ASA24 without assistance among children.

Three in 4 children reported the main item consumed (pizza), with most reporting at least a “close” match (with a fairly rigid definition of an exact match, considering both the ingredients and crust thickness). Recollection of food details of composite foods such as pizza can be difficult for both children and adults (27, 28). Although the multiple-pass method used in ASA24 asks item-specific detail questions to more accurately capture nuances such as preparation method (9), the ability to recall and identify the distinct qualities of an item may be limited among school-age children (12, 27). This is particularly evident when considering less obvious details of food and beverages; for example, most of the children who reported consuming juice could identify the variety, but few milk-drinkers identified the correct fat level (although milk was served in original containers). The small subset of children who completed ASA24-Kids experienced fewer prompts for additional food details; however, this version of ASA24 has been discontinued. In prior research (16) to assess the accuracy of recalls collected using ASA24-Kids based on observed intake for 9–11-y-old children, match rates were lower than those in the current study; however, this could be because of the lower age range given differences by age observed in the current study.

The current study supports prior findings that some food items, particularly additions to main items, are prone to exclusion. The most commonly excluded item was the ranch dip, omitted by ∼70% of participants who consumed it. Previous research among children and adults has demonstrated exclusion of similar foods, such as salad dressings and condiments (20, 29). Excluded items combined contributed around 170 kcal on average per child for the lunchtime meal only. Portion size misestimation is also common among children (and adults) (11, 12, 30) and true and estimated portion sizes differed significantly for the cookies and juice.

Overall, despite these exclusions and portion size errors, intakes of energy, protein, and sodium were overestimated. As noted, most children reported the main dish, the pizza, and the reported portion was not significantly different from the true portion size. However, many reported close or far rather than exact matches for this food, affecting crust thickness and/or toppings, with implications for estimates of energy, protein, and sodium. Misreporting of the fat level of the yogurt or milk also impacts estimates of energy intake. Further, when imputation was conducted for children who did not provide sufficient detail for ASA24 to assign codes, the resulting codes were not identical to the items truly consumed (e.g., the default for pizza when details were not entered into ASA24 was a variety with meat, with higher values per 100 g for energy and sodium than cheese pizza). Underestimation of portion sizes for the cookie and juice results in underestimation of energy and nutrient intake, but the cookie was less of a contributor to overall intake than the pizza and <50% of the children consumed juice, dampening the overall effect of these errors. Finally, intrusions, including some influential observations based on the observed range of kcals and grams, somewhat counteract the exclusions, contributing to overall overestimation of energy and nutrient intake.

No significant differences in the accuracy of reporting of items consumed at the lunch meal in relation to sociodemographic characteristics were observed, although the statistical power for such analyses was limited. Previous studies investigating accuracy in reporting by gender, ethnicity, and body weight among children have yielded mixed results (10, 29, 31, 32), although the impact of these characteristics on reporting accuracy seems to be more pronounced as children enter adolescence (10, 28). Interestingly, the match rate was lower among 12-y-olds than other children—given our sample size, it is challenging to understand the reasons underlying this observation. The schools in the study were mixed as far as socioeconomic status and it is possible that 1 or more classes including 12-y-old children were affected by socioeconomic disadvantage or other factors that may have impacted their ability to complete ASA24, although our regression models did not suggest an effect of family affluence.

The median length of time for children to complete ASA24 was 41 min and, on average, younger children required more time to complete the recall than did older children (52 min for 10-y-olds and 31 min for 13-y-olds). In comparison to previous studies which used interviewer-administered recalls among this age group, ASA24 required longer to complete (16) (as elaborated upon below, the inclusion of the optional “source” module in ASA24 added to the prompts encountered by children—and thus the length of the recall task—and may have negatively impacted usability by asking children questions they may not have been able to answer). Research assistants anecdotally observed that younger children seemed to find ASA24 less intuitive than did their older peers. This may be attributable to age-dependent cognitive abilities that make use of ASA24 more challenging for younger children. For example, limited literacy may impact children's abilities to read instructions and prompts, or to accurately identify food details such as cooking methods (11, 12). In addition, concepts of time and memory may not be fully developed among younger children (11, 33), limiting their capacity to recall information from the previous day (34). These cognitive abilities develop rapidly among school-age children, can be distinguished in children who are even 1 y apart in age, and vary between individuals (28). This rapid and inconsistent development across children may account for differences in median completion time and rates of accuracy observed in this study among children whose ages were just 1–3 y apart.

Overall, although it is not realistic to expect perfect capture of intake with self-report methods such as ASA24, strategies are needed to continue to improve the quality of data collected from children. Children in this study were not provided with assistance to complete recalls using ASA24. When they asked questions about ASA24 or about the meal consumed the prior day, research assistants advised them to “try their best” and “continue answering the questions.” Training may support independent completion of ASA24, as has been suggested elsewhere (35). For example, a quick start tutorial designed to engage children and/or a practice recall completed in the company of a research assistant may be useful in improving the completeness and accuracy of recalled intake, as well as reducing respondent burden (it is also pertinent to consider the exclusion of optional modules that may increase burden). Although the implementation of training may result in greater researcher burden, it is likely to be much more cost-effective than interviewer-administered recalls. Nonetheless, further research is needed to examine the most appropriate strategies to improve the validity of data collected with tools such as ASA24. For younger children, it may be necessary to conduct proxy-assisted recalls [which are used in NHANES for children up to the age of 12 (19)]. One study has shown good accuracy (with match rates close to 80%) of parent-completed recalls through the use of ASA24 (36); however, it was conducted with toddlers and the findings cannot be generalized to older children, who may exert more independence over their food and beverage consumption (37).

The findings of this study should be interpreted in light of several considerations. Although the initial aim was to collect data from children aged 11 and 12 (grades 6 and 7), the inclusion of some mixed-grade classes resulted in recruitment of 10- and 13-y-olds, who are represented in small numbers in the final sample, posing a challenge to generalizability. In addition, sociodemographic data were collected on Day 2 and it is not possible to examine differences between children who were absent on 1 or both days and those who participated. True intake data were collected for a single meal, and it is not possible to assess the accuracy of reported intakes for other eating occasions. Further, although standardization by weight was requested, slight variability across servings of the same food may have contributed to errors in examinations of accuracy of portion size reporting. This is not an issue for beverages, which were served in cartons or bottles. Finally, the ASA24 source module, which queries the source or purchasing point of foods (9), was activated for consistency with a companion feasibility study. This undoubtedly added to the time to complete (with an extra question for most food and beverage items reported) and may have caused confusion for children who could not easily provide this information.

Overall, although most of the children in this study were able to independently engage with the ASA24 online interface and reported at least 1 item truly consumed, overall accuracy of reporting was suboptimal. Additional research is needed to identify strategies to support the successful implementation of tools such as ASA24 into research with children. Further, the results suggest that ASA24 may not be appropriate for independent administration to children in the lower range of our age group, highlighting the potential need for proxy-assisted reporting.

Notes

Funded by a Public Health Ontario Project Initiation Fund award held by EPH and SIK, and a Canadian Cancer Society Research Institute Capacity Development Award (702855) held by SIK.

Author disclosures: AR, EPH, JES, KML, JH, PJR, KWD, and SIK, no conflicts of interest.

Supplemental Figures 1–2 and Tables 1–2 are available from the “Supplementary data” link in the online posting of the article and from the same link in the online table of contents at https://academic.oup.com/jn.

Abbreviations used: ASA24, Automated Self-Administered 24-Hour Dietary Assessment Tool; FNDDS, Food and Nutrient Database for Dietary Surveys; 24HR, 24-h recall.