A validation study of the Automated Self-Administered 24-Hour Dietary Recall for Children (ASA24-Kids) among 9 to 11-year-old youth

Abstract

Background

Valid methods of diet assessment are important for nutrition research and practice but can be difficult with children.

Objective

To validate ASA24-Kids-2012, a self-administered web-based 24-hour dietary recall (24hDR) among 9-11-year-old children, in two sites.

Design

Quasi-experimental

Participants/setting

In one site, trained staff observed and recorded foods and drinks consumed by children (n=38) during school lunch. The next day, the observed children completed both ASA24-Kids-2012 and an interviewer-administered 24hDR in a randomized order. Procedures in a second site (n=31) were similar, except observations occurred during dinner in a community location.

Statistical analyses

Foods were classified as matches (reported and consumed), intrusions (reported, but not consumed), or omissions (not reported, but consumed) for each participant. Rates of matches, intrusions, and omissions were calculated. Rates were compared between each recall method using repeated measures analysis of covariance. For matched foods, the authors determined correlation coefficients between observed and reported serving sizes.

Results

Match, intrusion, and omission rates between ASA24-Kids-2012 and observed intakes in site 1 were 37%, 27%, and 35%, respectively. Comparable rates for interviewer-administered 24hDRs were 57%, 20%, and 23%, respectively. In site 2, match, intrusion, and omission rates between ASA24-Kids-2012 and observed intakes were 53%, 12%, and 36%, respectively, vs. 76% matches, 9% intrusions, and 15% omissions for interviewer-administered 24hDRs. The relationship strength between reported and observed serving sizes for matched foods was 0.18 in site 1 and 0.09 in site 2 for ASA24-Kids-2012, and 0.46 in site 1 and 0.11 in site 2 for interviewer-administered 24hDRs.

Conclusions

ASA24-Kids-2012 was less accurate than interviewer-administered 24hDRs when compared to observed intakes, but both performed poorly. Additional research should assess the age at which children can complete recalls without the help of a parent or guardian, as well as to elucidate under which circumstances recalls can reasonably be used among children.

Introduction

Valid methods of diet assessment are important for use in diet-related research, evaluations of dietary behavior change programs, and nutrition education.^1-3 Common dietary assessment methods include food frequency questionnaires, 24-hour dietary recalls (24hDRs), and dietary records.^4,5 The interviewer-administered 24hDR is considered the preferred method and is commonly used to assess children's diets.⁶ This method requires a trained interviewer asking the child and/or adult caregiver (depending on the child's age) about the child's diet over the past 24 hours often using dietary intake software, and thus, can be expensive (both the interviewer and software involve cost) and logistically difficult to administer.⁵

Computer-based 24hDRs offer a cost-effective and efficient means for collecting diet measures.^7,8 For children, having child-generated food categories,^9-11 pictures of foods,¹² and different sizes of food images on the same screen¹³ may help improve recall speed and/or accuracy.¹⁴ Examples of computerized 24hDRs include the Food Intake Recording Software System (FIRSST),^15,16 the Young Adolescents’ Nutrition Assessment on Computer (YANA-C),^8,17,18 and the Automated Self-Administered 24-hour Dietary Recall for children (ASA24-Kids).¹⁴ The last is available free of charge through the U.S. National Cancer Institute (NCI) website.¹⁹ ASA24-Kids was adapted from an adult version, ASA24,²⁰ by a collaborative team of researchers knowledgeable in diet assessment,^14,16 and collects data for all foods and drinks consumed the previous day using a web-based interface. Both the number of foods and probes in the adult version of ASA24 were reduced by approximately 50% to simplify responses and burden in ASA24-Kids.²¹ Removals included: 1) foods not likely to be consumed by children based on data from 8-15-year-olds who completed the interviewer-administered 24hDRs in the 2003-2004 and 2005-2006 National Health and Nutrition Examination Survey (NHANES), and 2) probes (such as those regarding food preparation) to which children were unlikely to know the answers based on the experiences of six dietitians.²¹ In ASA24-Kids, children are guided through the program with the aid of food images,²² an animated talking penguin, and multiple passes.²³

As with most self-reported diet assessment methods, there are limits on knowledge and memory, especially for children.^5,24 Children under eight years of age usually require a proxy reporter, such as a parent; older children and adolescents are more able to report their own intake, although the ages and level of assistance vary based on the study.^6,25 In NHANES, children aged 6-11 years provide their own dietary intake information with the assistance of an adult household member.²⁶ Other complicating factors include retention interval (i.e., elapsed time between meal and recall), reporting order (i.e., evening-to-morning vs. morning-to-evening), and number of days of recall.^5,27 Aside from reporting what was eaten, reporting portion sizes is also challenging²⁸ and may be confounded by participant characteristics (e.g., body weight, sex) and food characteristics (e.g., type of food, size).²⁸ Formative research collected using the 2009 Beta version of ASA24 revealed that 8 and 9-year-olds had difficulties completing it and required assistance,¹⁴ so ASA24-Kids is currently recommended for use by children 10 years of age or older, or by proxy reporters for younger children.¹⁹ However, because modifications were made to ASA24 to adapt it for children (e.g., reducing the food list and number of probes), further evaluation is needed to determine if children younger than 10 years of age can independently complete a recall using ASA24-Kids.

The availability of ASA24-Kids-2012, the version available at the time the study was conducted, provided an opportunity to evaluate whether, at what age, and with what level of assistance, children can use ASA24-Kids, and the accuracy of their reported dietary intakes. The primary aims of the current study were to evaluate the accuracy of dietary intakes reported using ASA24-Kids-2012, the first available version of ASA24-Kids (ASA24-Kids-2014 is the current version), among children aged 9-11 years and to compare its performance to a standardized interviewer-administered 24hDR. We also assessed differences by age, sex, race (white vs. non-white), recall method order, and site (for one site, the focus was lunch, and the other site, dinner). The authors hypothesized that, compared to observed food intake, the accuracy of recalls collected using ASA24-Kids-2012 would be comparable to the accuracy of recalls collected via an interviewer-administered 24hDR.

Methods

Study Sample

This was a quasi-experimental study conducted from January to May 2013 with 69 children in two sites: 38 in site 1 and 31 in site 2. Participants were 9-11-year-olds; both the children and their parent provided written informed assent and consent, respectively. Study procedures were approved by Baylor College of Medicine and University of Arizona Institutional Review Boards.

In site 1, participants were fourth or fifth graders, aged 9-11 years, enrolled in an elementary school in Texas. Exclusion criteria included having a physical, mental, or visual limitation that would inhibit dietary recall or computer use; not being able to read or answer questions; not being able to speak, read, and write in English; and not eating school lunch. Information packets were sent home to parents of all fourth and fifth graders in that school; the packets included a letter to parents, flyer, information sheet, consent form, and family demographics form. Students who returned packets to school during the study period, with a family demographics form and signed consent form with both parent and child signatures, were screened and if eligible, included in the study.

In site 2, children were identified and recruited through information sessions at Cooperative Extension nutrition education programs (e.g., Garden Kitchen, 4-H, and Expanded Food and Nutrition Education Program) in one Arizona county. Exclusion criteria were similar to site 1, but instead of school lunch, children had to be available to eat a study-provided dinner. Children who provided assent, whose parents provided consent, and who completed the family demographics form were enrolled in the study.

This design enabled an assessment of the validity of ASA24-Kids-2012 and an interviewer-administered 24hDR over different meals (lunch in site 1 vs. dinner in site 2), which vary in time from occurrence to time of recall.

Procedures & Measures

In site 1, trained staff (n=3) unobtrusively observed children during school lunch, recording on a standardized form what food children selected, food portions obtained and consumed, and any food spilled or exchanged. Each observer watched no more than two children at a time over 19 days. At the end of the meal, observers recorded the amount of food left on the tray. Before lunch started each day, study staff looked through the lunch menu and walked through the lunch line to verify foods served. Over the course of the study, there were 16 different lunch menus, with an average of 13 available foods and drinks per meal, not including snack items (e.g., chips or ice cream). An example lunch meal was fresh garden salad, tostada, applesauce, milk, and pinto beans. School recipes were obtained directly from the school district, and all staff members were trained in meal observations by the principal investigator before data collection began. These procedures have been employed as the criterion method in several studies.⁶

Procedures in site 2 were similar, except observations occurred during a dinner in a community teaching kitchen. Dinner always consisted of a customizable pizza (for which participants chose their toppings), mixed fresh fruit (watermelon and grapes), and water or iced tea (sweetened or unsweetened). All foods were prepared by employees of the teaching kitchen. Study staff (n=5) observed no more than 11 families each night for a total of four nights.

The next day in both sites, children completed both ASA24-Kids-2012 and an interviewer-administered 24hDR in a randomized order, which was preassigned using a random number generator. Both types of recalls queried total intakes, not just the observed lunch or dinner, from midnight to midnight the previous day, and both were based on the U.S. Department of Agriculture's Automated Multiple-Pass Method.²⁹ Specifically, ASA24-Kids-2012 included a meal-based quick list, meal gap review, detail cycle (e.g., pizza type, additions, and amounts), forgotten foods, and final probe, while the interviewer-administered 24hDR included a quick list, review of quick list, collection of food detail (e.g., toppings and additions) and amounts, and final review. After completing one recall method, the child completed the other within the next hour.

ASA24-Kids-2012 was completed on a laptop computer connected to the NCI website, and a staff member was present to observe the child. The observer noted any difficulties encountered (e.g., problems when searching for foods, editing meals and snacks, etc.), troubleshot technical malfunctions, solved major difficulties that impeded further progress (e.g., if the child could not progress after multiple attempts), and answered questions the child had. Observations were recorded using a standard form that was structured by sequence of events, and included areas for the observer to rate and make note of the child's experience in various activities (e.g., selecting a meal, responding to questions about foods that were completely forgotten). The form was created by the authors and previously used in another study.¹⁴

Following completion of ASA24-Kids-2012, the observer interviewed the child regarding his/her ASA24-Kids-2012 experience, including usability (e.g., how easy/difficult it was to search for foods, estimate portion sizes, and report additions), what the child did and did not like, and how easy or hard it was to use the program. All interviewers used a semi-structured interview guide and were trained in qualitative data collection methods.

The second method, an interviewer-administered 24hDR, was completed on a laptop computer with a trained interviewer using Nutrition Data System for Research software (NDSR version 2012, University of Minnesota, Minneapolis, MN). Interviewers were trained and certified in 24hDRs and NDSR: in site 1, interviewers were dietitians, and in site 2, interviewers were research assistants trained by a university-provided behavioral measurement service. Following standard protocol for conducting 24hDRs, the interviewer asked a series of questions related to all foods consumed in the past 24 hours, and prompted the participant for information on brands, portion sizes (with food models and household measuring instruments), recipe ingredients, and cooking methods. At the end of the recall, the interviewer made note of any difficulties with the recall or if, in his/her judgment, the data were not of high quality (e.g., because the child did not pay attention or did not understand). The interviewer was not the same person who observed the child eating the day before nor who observed the child completing ASA24-Kids-2012, and did not have access to the child's observation data. After completion of both recalls, the child was asked which method (ASA24-Kids-2012, interviewer-administered 24hDR, or both) the child liked better.

On the same day that dietary recalls were collected, staff members trained in anthropometrics measured each child's height and weight using standardized procedures. Height was measured twice to the nearest 0.1 cm using a stadiometer, and the mean of the two recordings calculated. Body weight was measured twice to the nearest 0.1 kg using a calibrated scale, and the mean of the two recordings calculated. Each child removed any excess clothing (e.g., shoes, jackets, sweaters) or items from pockets beforehand, and an adjustment was made if there was anything in the hair (e.g., high ponytail, hair bows) that interfered with height measurements. After completion of study procedures, participants received $20 in compensation.

Statistical Analyses

The authors obtained output analysis files from both ASA24-Kids-2012 and NDSR for site 1 and site 2. A coauthor with training in dietetics classified all foods from both sites as matches (observed and recalled), intrusions (not observed but recalled), or omissions (observed but not recalled) for 1) ASA24-Kids-2012 vs. observation and 2) interviewer-administered 24hDR vs. observation for each participant. Another coauthor independently coded the comparisons, and discrepancies were discussed and resolved by the entire study team. Coding was completed within loose categories (e.g., if a child recalled eating a meat pizza but was observed eating a cheese pizza, that was classified as a match).

In order to compare both methods of 24hDRs, the primary analysis involved a repeated measures analysis of covariance (ANCOVA) for percentage of matches between a) observation and ASA24-Kids-2012 and b) observation and an NDSR interviewer-administered 24hDR, while controlling for recall method order. These analyses were repeated for percentages of intrusions and omissions. For matched foods, study staff calculated Spearman correlation coefficients and conducted signed rank tests (due to non-normal distributions). Analyses were completed separately for sites 1 and 2.

We then combined data for sites 1 and 2 and conducted subgroup analyses to assess for differences in match, intrusion, or omission rates based on age (9, 10, or 11), sex (male vs. female), race (white vs. non-white), recall method order (ASA24-Kids-2012 first vs. interviewer-administered recall first), or site (site 1 vs. site 2). Analyses were conducted using SAS^® software (version 9.3, 2011, SAS Institute Inc, Cary, NC).

Qualitative data from interviews and ASA24-Kids-2012 observations from both sites were combined and analyzed using thematic analysis³⁰ to determine problems and obtain feedback about ASA24-Kids-2012. Paired samples t-tests were conducted to determine differences in completion times between ASA24-Kids-2012 and the interviewer-administered 24hDR. Additionally, independent samples t-tests, ANOVA, and chi-squared tests were conducted to test for differences between ages (9, 10, and 11), sex, site, and recall method order in ASA24-Kids-2012 completion time, interviewer-administered 24hDR completion time, ASA24-Kids-2012 ease of use, and method preference. Differences in demographic characteristics between sites 1 and 2 were investigated using either chi-squared tests or Fisher's exact tests, as appropriate. Study staff also calculated BMI-for-age based on measured height and weight using the Centers for Disease Control and Prevention BMI calculator program.³¹

Results

Study Sample

There were 69 children in two sites: 38 in site 1 and 31 in site 2. In site 1, twelve additional students expressed interest, but four had incomplete or missing consent forms, seven did not eat school lunch, and one was eight years of age, so they were not included in the final study. All participants who were eligible were included in and completed the study; there was no non-participation or missing data.

Table 1 illustrates further details about the two samples, including age, sex, ethnicity, race, weight status, highest household educational level, and household income. Over half of the sample in site 1 was non-Hispanic, of healthy weight, and had higher socioeconomic status (SES) (as evidenced by household educational level and household income). Over half of the sample in site 2 was female, white, healthy weight, and of lower SES. The two sites differed significantly in ethnicity (P<0.01), race (P<0.001), highest household educational level (P<0.001), and household income (P<0.01).

Table 1.

Demographics characteristics of 69 children participating in ASA24-Kids-2012^a study in site 1 (Texas) and site 2 (Arizona)

	Site 1 (n=38)		Site 2 (n=31)
	n	%	n	%
Age (years)
9	9	24%	11	36%
10	18	47%	10	32%
11	11	29%	10	32%
Sex
Male	19	50%	12	39%
Female	19	50%	19	61%
Ethnicity**
Hispanic	6	16%	14	45%
Non-Hispanic	32	84%	14	45%
Did not report	0	0%	3	10%
Race***
White	14	37%	24	77%
Asian	12	32%	0	0%
Black	10	26%	1	3%
Otherb	2	5%	3	10%
Did not report	0	0%	3	10%
Weight statusc
Underweight	1	3%	1	3%
Healthy weight	26	68%	20	65%
Overweight	8	21%	3	10%
Obese	3	8%	7	23%
Highest household educational level***
Some college or less	3	8%	15	48%
College graduate	21	55%	6	19%
Post graduate study	14	37%	9	29%
Did not report	0	0%	1	3%
Household income**
Less than $40,000	6	16%	17	55%
$40,000 - $69,999	5	13%	4	13%
$70,000 - $99,999	12	32%	3	10%
$100,000 or more	14	37%	6	19%
Did not report	1	3%	1	3%

^aASA24-Kids-2012 = Automated Self-Administered 24-Hour Dietary Recall for children, 2012 version.

^bOther races included mixed race, American Indian or Alaska Native, and Hispanic.

^cWeight status was defined using the Centers for Disease Control and Prevention BMI calculator program: underweight as less than the 5th percentile for BMI-for-age, healthy weight as between the 5th and 85th percentile, overweight as between the 85th to 95th percentile, and obese as equal to or greater than the 95th percentile.

^**Statistically different between sites 1 and 2, P<0.01

^***P<0.001

ASA24-Kids-2012 vs. Interviewer-Administered 24hDR

Participants spent between 12-58 minutes completing ASA24-Kids-2012 (mean=34 minutes). Less time was required for the interviewer-administered recall (mean=20 minutes; range=6-35 minutes). The difference between ASA24-Kids-2012 completion time and the interviewer-administered 24hDR completion time was statistically significant (P<0.001). However, there were no differences in ASA24-Kids-2012 completion time or in interviewer-administered 24hDR completion time between sites 1 and 2, males and females, recall method order, or ages.

Table 2 reports percent matches, intrusions, and omissions in sites 1 and 2. For both sites, repeated measures ANCOVA indicated significantly higher match rates and lower omission rates for interviewer-administered 24hDRs than ASA24-Kids-2012 (P<0.001 to P=0.0158). There were no significant differences in intrusion rates, while controlling for recall method order.

Table 2.

Differences in percent matches, intrusions, and omissions in site 1 (Texas) and site 2 (Arizona) for children participating in ASA24-Kids-2012 study

Comparison	Percent Matchesa	Percent Intrusionsb	Percent Omissionsc	Portion-size Correlation for Matched Foods
Site 1d (38 9-11-year-olds)
ASA24-Kids-2012e vs. observation	37%	27%	35%	0.18
24hDRf vs. observation	57%	20%	23%	0.46***
P-valueg	0.0013	0.1082	0.0158
Site 2h (31 9-11-year-olds)
ASA24-Kids-2012 vs. observation	53%	12%	36%	0.09
24hDR vs. observation	76%	9%	15%	0.11**
P-valueg	0.0004	0.5233	0.0016

^aMatches are foods that were observed consumed and recalled the next day by the child.

^bIntrusions are foods that were not observed consumed but recalled the next day by the child.

^cOmissions are foods that were observed consumed but not recalled the next day by the child.

^dIn site 1, observations were of a lunch meal.

^eASA24-Kids-2012 = Automated Self-Administered 24-Hour Dietary Recall for children, 2012 version.

^f24hDR = interviewer-administered 24-hour dietary recall.

^gRepeated measures analysis of covariance assessed differences between interviewer-administered 24hDRs and ASA24-Kids-2012.

^hIn site 2, observations were of a dinner meal.

^***24hDR portion size and observation portion size were correlated, P<0.001.

^**24hDR portion size and observation portion size were statistically different, P<0.01.

Table 2 also presents portion-size correlations for matched foods. When comparing matched foods between ASA24-Kids-2012 and meal observations, there was no significant correlation between observation and child-reported serving sizes in site 1 or in site 2. However, when comparing the interviewer-administered 24hDRs to meal observations, the correlation between the serving sizes for matched foods was positive and significant (P<0.001) in site 1 but not significant in site 2. In addition, according to signed rank tests, the mean serving sizes (observation compared to child-reported) were statistically different (P<0.01) between interviewer-administered 24hDR and observation in site 2; the mean serving sizes were not significantly different between interviewer-administered 24hDR and observation in site 1, ASA24-Kids-2012 and observation in site 1, or ASA24-Kids-2012 and observation in site 2.

Based on subgroup analyses to assess differences in accuracy by various characteristics, match rates were significantly higher for site 2 than for site 1 for both ASA24-Kids-2012 (P<0.01) and the interviewer-administered 24hDR (P<0.01). In addition, ASA24-Kids-2012 match rates were significantly lower for participants completing ASA24-Kids-2012 first (P<0.05). Significant site differences in intrusion rates were also detected for both ASA24-Kids-2012 (P<0.01) and the interviewer-administered 24hDR (P<0.001), for which site 1 had higher intrusion rates. No other differences were found in match, intrusion, or omission rates based on age, sex, race (white vs. non-white), recall method order, or site.

Qualitative Findings

Based on observations of ASA24-Kids-2012 sessions, common problems with ASA24-Kids-2012 included the child not knowing what to do at a given point in time (e.g., how to proceed after the first tutorial), difficulties moving foods to “My Selections” in the right panel (e.g., not double clicking on an item), not finding certain foods (e.g., foods searched for were not in the food list database), and difficulties with the “Forgotten Foods” part (e.g., the child may click yes to having forgotten foods but then did not add any foods or did not add all the foods indicated as forgotten).

During interviews with children regarding their ASA24-Kids-2012 experiences, all reported finding the tutorial video in the beginning either ‘okay’ or ‘very helpful.’ When asked to elaborate, the participants who found the tutorial ‘very helpful’ said it explained “what to do and how to do it,” whereas the participants who found the tutorial ‘okay’ said the explanation was too long, the tutorial was confusing, or they could have figured out what to do on their own. A majority of participants reported finding it easy to spell foods (71%), find foods (54%), answer questions about what they ate (71%), report how much they ate (58%), and use the program (71%). There were no differences in how easy or difficult participants found the program between sites, age groups, sexes, or recall method order. When asked what they liked the most about ASA24-Kids-2012, the most common answer (26%) was the penguin avatar. When asked what they liked the least, the most common answer (22%) related to the details asked (e.g., food types, amounts, etc.) and length of time to complete the program. Overall, 30% of participants liked the interviewer-administered 24hDR better than ASA24-Kids-2012; 23% liked ASA24- Kids-2012 better; 13% liked both equally; and 33% did not know or did not provide a response. There were no differences in method preference by site, age group, sex, or recall method order.

Discussion

This study assessed the validity of ASA24-Kids-2012 and an interviewer-administered 24hDR against an objective measure of observed food intake for a single meal. Overall, in both sites, ASA24-Kids-2012 was less accurate than an interviewer-administered 24hDR, obtaining significantly lower match rates and higher omission rates when compared to meal observations. A previous validation study of FIRSSt, a similar web-based self-administered 24hDR,^15,16 among 138 fourth-grade children found similar comparisons.¹⁵ In contrast, a validation feeding study with adults using ASA24-2011 showed high agreement with truth³²; thus, it is possible that age was the major determinant of low recall accuracy. Children aged 9-11 years may not be able to complete recalls without help. Recall accuracy improved with age when comparing third and fifth graders in one study,³³ but findings from other studies, including the current one, did not support this relationship in this age group.³⁴ In NHANES, children this age are always asked to complete their 24hDRs with adult assistance, further indicating some lack of confidence in the ability of children this age to complete a recall independently.²⁶ Although ASA24-Kids is currently recommended for use by children aged 10 years or older,¹⁹ the results of the current study suggest that those aged 9-11 years may not be able to complete recalls on their own, so further research is needed to determine if, and under which circumstances, children can complete recalls without adult assistance.

Site 2 had significantly higher match rates and lower intrusion rates for both ASA24-Kids-2012 and interviewer-administered 24hDRs than site 1, whereas the rates in site 1 more closely resembled the rates in the FIRSSt validation study¹⁵ (i.e., FIRSSt vs. observation had 40% match, 23% intrusion, and 36% omission rates, and the 24hDR vs. observation had 53% match, 20% intrusion, and 26% omission rates). This result was unexpected, given the higher SES of the sample in site 1: a previous, unpublished study has shown recall inaccuracy to be higher for lower SES samples.³⁵ Multiple factors could have accounted for better performance on ASA24-Kids-2012 and the interviewer-administered 24hDR in site 2. First, the retention interval, which has been shown to be inversely related to accuracy of recalls,^36-40 was shorter in site 2 than in site 1 (approximately 18-21 hours vs. 22-24 hours, respectively). Second, in site 1, usual school lunches were observed and analyzed, whereas a single special meal was offered in site 2. The special meal in site 2 was customized by the children (i.e., they chose their pizza toppings) and included only three foods, whereas the meals in site 1 were pre-made by school cafeteria staff and included more variety overall and more foods to report at each meal. The differences in novelty, customizability, and complexity between meals in the two sites (i.e., which may be memorable) likely accounted for differences in recall.

The strengths of relationship (i.e., correlations) between reported and observed serving sizes for matched foods were lower for ASA24-Kids-2012 than interviewer-administered 24hDR in both sites. Only in site 2, when comparing the interviewer-administered 24hDR to observation, were the serving sizes significantly different from each other, which appears to have been a result of inconsistent and unclear reporting by interviewers of serving sizes for pizza in NDSR. For example, some pizzas were entered into NDSR as portions of a 14-inch diameter (the reference amount), whereas others were entered as 12-inch diameter, making it difficult to compare sizes. Overall, children were generally able to report serving sizes using ASA24-Kids-2012 and the interviewer-administered 24hDR, although the correlations between reported and observed portion sizes were low. Other studies have found that when children were able to recall the correct food (i.e., matches), recalled amounts were fairly accurate compared to observed amounts.^41-45 ASA24-Kids-2012, in particular, utilizes multiple images of successively larger portion sizes on one computer screen, which is the preferred method for portion size estimation on computers.¹³

Based on ASA24-Kids-2012 observations and interviews, the problems encountered and participants’ feedback were similar to the problems and obstacles encountered by children when completing the adult version (ASA24) in a previous validation study.¹⁴ Specifically, participants ignored tutorials, did not understand what to do at a given time, and searched for foods that did not appear as typed. The current version, ASA24-Kids-2014, released by the NCI in February 2014, was designed to be more user-friendly, and addresses some of the problems encountered in these validation studies. Further research needs to determine whether the new version is more valid than the 2012 version, which was used in this study.

Limitations of the current study included: first, the sample size was small, so power was limited. This precluded some subgroup analyses to examine differences by other characteristics, such as race/ethnicity or weight status. Other studies have shown differences in self-report accuracy by body mass index and body composition,^46-49 as well as having a computer at home.⁵⁰ Second, the sample included children aged 9-11 years in two cities in the U.S., so findings cannot be generalized to other populations. Relatedly, this age group also prevented the authors from determining whether older children can more accurately complete a 24hDR on their own. Next, the authors coded foods within loose categories (e.g., a salad with tomatoes and a salad without tomatoes were considered matches); this coding rule may have resulted in underestimation of the recall accuracies and may have partly accounted for the higher match rates in site 2 (for which, coding of pizzas and their toppings may have been affected). However, given the study's focus on accuracy of recalled foods, rather than nutrients, coding within loose categories was appropriate. Other dietary-reporting methodological studies with children have coded in a similar manner.^{36,37,43-47,51-55} Last, aside from differences in retention intervals, meal novelty, and meal complexity (i.e., number and types of foods) between the two sites, there may have been differences in observation methods or dietary interviewers' methods. Although the principal investigator in site 2 visited site 1 to observe data collection and meet with study staff before beginning the study in site 2 (thus ensuring some consistency between sites), any variation in inter-observer reliability (one limitation of the reference method) could manifest as errors in participants’ self-report of diet.⁵⁶

Conclusion

ASA24-Kids-2012 was less accurate than interviewer-administered 24hDRs when compared to observed intakes, but both reflected substantial error. Further research is needed to determine how to improve children's performance on ASA24-Kids, under what circumstances (e.g., special meals, dinner, or other circumstances resembling those in site 2) recalls may be used, and at what age children may accurately complete recalls on their own. In addition, research is needed on more objective measures, such as biomarkers or a wearable camera,⁵⁷ to assess dietary behaviors,^57,58 which potentially offer more accurate and less burdensome methods by which to capture the dietary intake of children.