A comparison of the Remote Food Photography Method and the Automated Self-Administered 24-hour Dietary Assessment Tool for measuring full day dietary intake among school-aged children

Abstract

The limitations of self-report measures of dietary intake are well known. Novel, technology-based measures of dietary intake may provide a more accurate, less burdensome alternative to existing tools. The first objective of this study was to compare participant burden for two technology-based measures of dietary intake among school-age children: the Automated-Self Administered 24-hour Dietary Assessment Tool-2018 (ASA24–2018) and the Remote Food Photography Method (RFPM). The second objective was to compare reported energy intake for each method to the Estimated Energy Requirement for each child, as a benchmark for actual intake. Forty parent-child dyads participated in 2, 3-day dietary assessments: a parent proxy-reported version of the ASA24 and the RFPM. A parent survey was subsequently administered to compare satisfaction, ease of use and burden with each method. A linear mixed model examined differences in total daily energy intake (TDEI) between assessments, and between each assessment method and the EER. Reported energy intake was 379 kcal higher with the ASA24 than the RFPM (p=0.0002). Reported energy intake with the ASA24 was 231 kcal higher than the EER (p = 0.008). Reported energy intake with the RFPM did not differ significantly from the EER (difference in predicted means = −148 kcal, p = 0.09). Median satisfaction and ease of use scores were 5 out of 6 for both methods. A higher proportion of parents reported that the ASA24 was more time consuming than the RFPM (74.4% vs. 25.6%, p = 0.002). Utilization of both methods is warranted given their high satisfaction among parents.

INTRODUCTION

It is unclear how best to measure full day dietary intake in school-aged children. Two key considerations in the selection of a dietary assessment tool are accuracy and participant burden ^{(1; 2)}. Accurate tools are fundamental to dietary assessment ^{(1; 3)} because estimates of dietary intake are used to characterize diet-disease relationships, develop dietary guidance, inform policy, evaluate interventions and identify groups at risk for poor nutrition. Dietary assessment tools with low participant burden are also essential because multi-day assessments are critical for evaluating habitual intake ⁽⁴⁾, and parents or children may be less likely to complete multi-day studies if the assessment tool is too burdensome ⁽⁵⁾. However, accuracy and burden are intertwined. Improving the accuracy of a dietary assessment tool often requires a concomitant increase in participant burden (e.g., time commitment, degree of invasiveness). The result is that assessment tools with multiple known threats to validity are frequently used to assess what children eat ^{(3; 6)}, presumably because they pose low burden and can be less expensive to administer. Tools are needed that simultaneously maximize accuracy and minimize burden.

Over the last 15 years, advancements in the science of dietary assessment focused on integrating technology into self-report tools. One of the most notable examples is the Automated Self-Administered 24-hour Dietary Assessment Tool (ASA24), a web-based dietary recall developed by the National Institutes of Health in the United States ⁽⁷⁾. Either respondents or a proxy report intake on the preceding day from memory. The respondent website includes features designed to maximize accuracy and minimize burden: images to allow respondents to estimate portion size, a searchable food database with more than 7,000 items, standardized prompts for frequently forgotten foods or meal gaps, branching logic to streamline reporting and the option to create recipes ^{(7; 8)}. Yet, misreporting with the ASA24 is common ^{(9; 10)}, suggesting that poor recall and inaccurate estimates of portions consumed continue to threaten validity. While a few studies indicate that the ASA24 is well-accepted by respondents, opportunities may exist to reduce burden, especially related to technology use and website interface ⁽¹¹⁾.

Another innovation in the field of dietary assessment is the Remote Food Photography Method (RFPM) ⁽¹²⁾. Participants photograph food selection and plate waste with an iOS-based smartphone and the images are converted into estimates of energy and nutrient intake ⁽¹³⁾. The RFPM offers some advantages over the ASA24 and other self-report tools. Approximately 50% of the error in self-report methods is due to participants’ inability to accurately estimate portion size ⁽¹⁴⁾. This limitation is minimized with the RFPM because portion size is estimated from the food images by a trained operator ⁽¹³⁾. Many self-report tools require participants to recall past intake. The RFPM is a prospective record, and thus eliminates measurement error due to poor recall because intake is captured in real time via photographs. The RFPM provides data on foods offered and plate waste, which can provide insight into the environmental conditions that shape what children eat ^{(15; 16; 17)}. Compared to other self-report methods, the RFPM may also be more viable and yield better accuracy if photographing foods is less burdensome.

Nutrition-related diseases are increasingly common among school-age children ^{(18; 19)}. Accurate estimates of intake are needed for this age group, but may be difficult to obtain. School-age children are too young to accurately report their own intake, but may have enough autonomy to consume some meals and snacks unsupervised ^{(20; 21; 22)}. Thus, parents may be proxy-reporters for unobserved intake, such as foods consumed at school ⁽²³⁾. It is unknown how well existing dietary assessments capture unobserved intake. Most studies with the ASA24 and RFPM have been conducted among adults or families with preschoolers ^{(24; 25; 26)}. More work is needed to understand the unique challenges for school-aged children.

This study provides a novel comparison of 2 methods for measuring full-day dietary intake in school-age children: a parent proxy-reported ASA24 ⁽⁷⁾ and the RFPM ⁽²⁷⁾. Parent-child dyads completed 2, 3-day assessments 1 week apart: 1 using an ASA24 and the other using the RFPM. A parent survey evaluated perceived burden with each method. The objective was three-fold: (1) to compare children’s dietary intake across methods; (2) to compare participant burden across methods; and (3) to support the design of future validation studies by providing insight into data completeness, food source, eating location and reported energy intake relative to the calculated EER.

METHODS

Participants.

Dyads were drawn from a cohort in the Environmental influences on Child Health Outcomes (ECHO) Program (http://echochildren.org) and a community-based sample ⁽²⁸⁾. Forty children ages 7 to 8 years and their parents were recruited at 2 sites. Thirty parent-child dyads that were part of the Healthy Start cohort ⁽²⁹⁾ were recruited in the Denver metro area. Healthy Start is a prospective cohort study that recruited over 1400 pregnant women between 2009 and 2014 and is currently following offspring postnatally. The Healthy Start cohort is part of the ECHO Program ⁽²⁸⁾. The goal of ECHO is to investigate the effects of environmental exposures on child health ⁽³⁰⁾. Ten child-caregiver dyads were recruited from a single elementary school in Baton Rouge, Louisiana. The multi-site design was used to recruit the majority of participants from an ECHO cohort while also documenting feasibility among families recruited from a community-based sample. All interested dyads completed a screening questionnaire by phone and were eligible to participate if their school and school district provided institutional approval to have lunch meals photographed in the cafeteria, per the RFPM protocol.

Study design.

Each parent-child dyad (n = 40) participated in 2, 3-day dietary assessments: a parent proxy-reported version of the ASA24 and the RFPM. The order of assessment was randomized to control for the effects of timing and fatigue. For the first assessment, consecutive or non-consecutive dates were selected a priori by the parent based on days perceived to be “typical”. For the second assessment, data collection occurred on the same days of the week as the first assessment. Data were collected in 2019 from January to June and September to October to avoid data collection during the school break. The study was approved by the Institutional Review Boards at the University of Colorado and the Pennington Biomedical Research Center. Parents provided written informed consent. Children provided written or verbal assent. Children attended 21 schools (4 private, 17 public) across 3 school districts. All school districts provided institutional approval. School principals provided written consent for research staff to photograph lunch meals in the school cafeteria, consistent with the RFPM protocol. Parent-child dyads were remunerated $40 for the ASA24, $40 for the RFPM, and $20 for the survey.

Automated Self-Administered 24-hour Dietary Assessment Tool (ASA24).

The ASA24 is a web-based tool for estimating dietary intake in which participants recall all foods and beverages consumed on the preceding day, including food type, quantity, preparation mode, and time of intake. Parents were proxy-respondents for their child. Each parent self-administered the 2018 version of the ASA24 (ASA24–2018) to report their child’s intake on 3 occasions (2 weekdays and 1 weekend day). Parents reported their perceptions what children consumed at school, although intake was not observed, because this is consistent with the ASA24 protocol. The ASA24 provided estimates of energy and nutrient intake for each assessment day. The ASA24, utilized in over 6,000 research studies, has been validated against the USDA’s Automated Multiple Pass Method and weighed food records ^{(5; 9)}. Participants were asked to access the ASA24 website using a computer or mobile device. All parents received a 3-minute training from the research team in how and when to log on to the website. No additional instructions were provided unless the participants asked for help because the ASA24 is designed to be self-administered using step-by-step instructions provided within the online platform.

Remote Food Photography Method (RFPM).

The RFPM and SmartIntake^® smartphone app were developed at the Pennington Biomedical Research Center. The SmartIntake app was compatible with iPhones only at the time this study was conducted. Participants borrowed a “loaner phone” for the study period, if they did not own an iPhone. Participants placed a reference card next to their child’s food and photographed images of the foods offered by the parent or selected by the child (the “before” image) and plate waste (the “after” image). The app allowed participants to record food descriptors that were automatically tagged to the image (e.g., label a glass of milk as low-fat or non-fat). Customized text reminders were sent to participants 30 minutes prior to each child’s usual meal or snack times ⁽³¹⁾. Participants’ responses to the text reminders were tracked in the period surrounding their usual meal and snack times, but not outside those periods. Participants’ responses to the text reminders were used to troubleshoot potential problems in near real-time, when possible (e.g., the family ate in a restaurant and forgot to bring the reference card, or the research staff sent follow-up questions when parents did not provide enough detail in the food descriptor).

The app sent food images and accompanying data (e.g., food descriptions) to a server where the Pennington Biomedical team analyzed the images. The analysis relied on a computer program built at Pennington Biomedical called the Food Photography Application©. The program allowed the operator to identify a match for each food from the Food and Nutrient Database for Dietary Studies ⁽³²⁾ and manufacturer’s information, to calculate the energy and nutrient content of foods offered and plate waste. Intake was calculated as the difference. The operator also used the program and validated methodology ^{(12; 26; 33; 34)} to estimate portion size by visually comparing participants’ images to images of foods with a known portion size. Parents received a 30-min in-person training in best practices for capturing and sending images and entering food descriptors. Parents captured food images on the weekend and during non-school hours on weekdays. For snacks sent from home and consumed during school hours, parents were asked to photograph snacks offered in the morning and plate waste (e.g., empty wrappers) after school. For eating occasions not observed by the parent (e.g., playdates, breakfast meals provided by the school), parents probed children or alternate caregivers for details and provided text descriptions only of foods consumed and plate waste.

Research staff photographed lunchtime intake at schools. Research staff arrived at schools 30 minutes prior to each child’s lunchtime, checked in at the main office, and were either escorted by a school staff member or walked independently to the cafeteria. Once the child arrived, the research staff approached the child, verbally re-confirmed assent and took a “before” photo and entered food descriptors. The research staff waited on the side of the lunchroom while the child consumed their lunch. The “after” image was captured when the child finished eating. Excluding commute time, this process took approximately 1 hour per lunch meal.

Survey.

After completing the dietary assessments, parents participated in an online, 12-question survey. The first 9 questions used a Likert scale. Six questions assessed perceived burden (1 = Always Burdensome to 6 = Never Burdensome), ease of use (1 = Very Difficult to 6 = Very Easy) and satisfaction (1 = Very Dissatisfied to 6 = Very Satisfied) for each method. Three questions assessed satisfaction with the reminder text messages, the app and the in-person training for the RFPM only. Higher scores represented higher satisfaction and ease of use and lower burden. In addition to the 9 Likert scale questions described above, 3 additional questions asked participants to select which method was most preferable with respect to their overall experience, the technology platform (ASA24 website or RFPM app) and time spent documenting intake.

Derived variables

Healthy Eating Index-2015 (HEI-2015).

The HEI-2015 is an index that compared each child’s intake to the Dietary Guidelines for Americans 2015–2020 ⁽³⁵⁾. Individual diets were scored based on the intake of 10 food categories, 2 nutrients and 1 nutrient ratio. The scores for all 13 components were summed to compute an overall score (ranged from 1 to 100). Higher scores indicated greater adherence to the guidelines ⁽³⁶⁾.

Estimated Energy Requirement (EER).

Each child’s EER was calculated based on age, sex, measured body weight and estimated physical activity level according to established methods ^{(37; 38; 39; 40)} to provide context for estimates of reported energy intake and preliminary data for statistical power calculations for future validation studies. The EER was calculated as the product of BMR and estimated physical activity level using two approaches: (1) the Schofield equation⁽³⁹⁾ for calculating BMR and (2) the Henry equation⁽⁴⁰⁾ for calculating BMR. For both approaches, the PAL was set at a constant of 1.3, which reflects low activity levels. This physical activity level was selected based on 3 data sources. The first source was the 2016 Report Card on Physical Activity for Children and Youth which found low levels of sports participation, physical activity at school, health-related fitness and adherence to screen time recommendations in school-aged children ⁽⁴¹⁾. The second source included studies among school-aged children that used doubly labeled water or accelerometry to estimate physical activity and showed levels ranging from 1.1 to 1.6 ^{(42; 43; 44)}. In these studies, PALs of 1.1 to 1.4 were observed among elementary school children in the US⁽⁴²⁾. Physical Activity Levels of 1.5 or 1.6 have also been observed in high income countries, but mostly among older children and adolescents living in Europe and Asia^{(43; 44)}. The third source was the Estimated Calorie Needs reported in the 2015–2020 Dietary Guidelines for Americans for children age 7 to 8; EERs calculated in the present study using a PAL of 1.3 fall within the range of Estimated Calorie Needs for children of similar sex and age⁽⁴⁵⁾.

Type of eating occasion, eating location and food source.

Type of eating occasion, eating location and food source were assessed in both methods using drop down menus with pre-selected answer choices. Each variable was harmonized across methods. Eating occasion was categorized as breakfast, lunch, dinner, or snack. Eating location was categorized as home, school, restaurant or other (e.g., birthday party, sporting event). Food source was categorized as home-prepared, restaurant, other or don’t know. Body mass index (BMI). Standardized anthropometric procedures were used to assess height and weight ⁽⁴⁶⁾. BMI was calculated as weight (kg) / height (m²). Age- and sex-specific BMI z-scores were used to classify children as underweight (less than 5^th percentile), healthy weight (5^th percentile to less than the 85^th percentile), overweight 85^th percentile to less than the 95^th percentile) and obese (95^th percentile or greater) ⁽⁴⁷⁾.

Statistical analysis

Power calculation:

We examined power a priori for a clinically meaningful difference of 200 kcal/day between the ASA24 and RFPM. The standard deviation of the TDEI measurements for children ages 2 to 9 years has been reported to be 500 kcal/day ⁽⁴⁸⁾. Since the range of ages in our study was smaller (7–8 years), we used a standard deviation that was half as large. We assumed that the ASA24 and RFPM would have a correlation of 0.9. With these inputs, at a Type I error rate of 0.05, we had power greater than 0.95.

Descriptive analysis:

We tabulated participant characteristics in the full sample and by site (Table 1). For both the ASA24 and the RFPM, we tabulated the proportion of days with at least 1 eating occasion documented, the proportion of days with no data, and parent explanations for missing data (Table 2). For both the ASA24 and RFPM, we tabulated median (IQR) HEI-2015 total and components scores; macronutrient intake (g); the percent of eating occasions consumed in each location; and the percent of TDEI that was home-prepared, restaurant/convenience foods, other or unknown (Table 4). For survey questions on a 6-point scale, we reported the median and interquartile range for each method (Table 5). A Wilcoxon signed-rank test ⁽⁴⁹⁾ was used to compare median scores for burden, satisfaction and ease of use between methods. For the binary survey outcomes, we reported the proportion of participants who selected the ASA24 or RFPM as the preferred measure. Proportions were compared using McNemar’s test ⁽⁵⁰⁾.

Table 1.

Participant characteristics among children age 7–8 years and their families in Colorado and Louisiana (n = 40 dyads)

Participant characteristics	Full sample	Colorado site	Louisiana site
Sample		Participants of an existing cohort study	Community-based
Number of dyads	40	30	10
Child Sex, % female	55	50	70
Child Ethnicity, % Hispanic	20	27	0
Child Race, % White	65	54	100
Household income, % <$50,000/year	29	37	0
Maternal education, % with college degree	58	54	70
Number of elementary schools in which research staff photographed lunch intake	21	20	1

Table 2.

Data completeness by dietary assessment method (ASA24 vs. RFPM) among children age 7–8 years in Colorado and Louisiana (n = 40 dyads)

Process Measures	ASA24 % (n)	RFPM % (n)
Of 120 assigned days, percent of days with no data	19.2 (23)	10.0 (12)
Reasons provided by parents for missing data*
Problems using the technology platform (ASA24 website or RFPM app)	21.7 (5)	0.0 (0)
Problems with internet access, electricity or the device	30.4 (7)	0.0 (0)
Participant forgot to complete the assessment	0.0 (0)	25.0 (3)
Child was ill and did not consume any food	4.3 (1)	0.0 (0)
No reason provided by the participant	43.5 (10)	75.0 (9)
Of 120 assigned days, percent of days with at least 1 eating occasion documented	80.8 (97)	90.0 (108)

^*Percentages are based on the number of participants who had missing data

Table 4.

Median (IQR) energy intake, macronutrient intake, HEI-2015 score; eating location and food source by dietary assessment method (ASA24 vs. RFPM) among children age 7–8 years in Colorado and Louisiana

Characteristics of foods intake reported with the ASA24 and RFPM	ASA24	RFPM
Energy Intake in kcal, median (IQR)	1570 (728)	1344 (553)
Macronutrient intake in grams, median (IQR)
Protein	57 (35)	48 (36)
Carbohydrate	200 (94)	163 (100)
Fat	62 (40)	51 (39)
Healthy Eating Index-2015, median (IQR)
TOTAL SCORE	50.9 (17.8)	47.2 (17.3)
Total fruits	3.2 (3.7)	2.9 (4.9)
Whole fruits	4.7 (4.3)	3.3 (5.0)
Total vegetables	2.1 (2.1)	1.9 (2.6)
Greens and beans	0.0 (3.7)	0.0 (1.2)
Whole grains	2.1 (5.6)	1.7 (5.0)
Dairy	8.1 (4.5)	7.9 (5.6)
Total protein foods	3.9 (2.9)	4.4 (2.7)
Seafood and plant proteins	1.3 (5.0)	0.1 (4.7)
Fatty acids	2.7 (6.3)	3.9 (6.6)
Refined grains	5.0 (6.7)	4.0 (5.9)
Sodium	5.1 (5.9)	4.7 (6.8)
Added sugars	7.5 (3.8)	7.9 (4.4)
Saturated fats	4.6 (7.0)	5.6 (6.8)
Eating location (% of documented eating occasions)
Home	67.5	68.7
School	13.1	18.3
Restaurant	7.7	6.0
Other	11.7	7.0
Food source (% of dietary intake)
Home prepared	75.6	58.8
Restaurant or convenience store	3.3	13.8
Other (e.g., pre-prepared, school, birthday party)	7.9	27.2
Participant responded “Don’t know”**	12.1	0.0

^*Median (IQR) were reported for energy, macronutrients and HEI scores because these variables were not normally distributed.

^**The “don’t know” response option is only available in the ASA24.

Table 5.

Participant burden, satisfaction and ease of use by dietary assessment method among parents in Colorado and Louisiana (n = 39)

Comparing ASA24 and RFPM	ASA24 % (n)	RFPM % (n)	P*
Which method would you rather use to record what your child ate for 7 days?	38.5 (15)	61.5 (24)	0.15
Which method required more time to complete?	74.4 (29)	25.6 (10)	0.002
Which technology platform did you prefer (ASA24 website or RFPM app)?	38.5 (15)	61.5 (24)	0.15
Satisfaction, Ease of Use and Participant Burden with Each Method	ASA24 Median (IQR)	RFPM Median (IQR)	P**
How satisfied are you with this method for recording what your child ate?	5 (2)	5 (2)	0.77
How easy was it to use this method for recording what your child ate?	5 (1)	5 (2)	0.46
How often was it burdensome to use this method for recording what your child ate?	4 (2)	4 (3)	0.52

^*Proportions compared using McNemar’s test ⁽⁵⁰⁾

^**Median scores compared using Wilcoxon signed-rank test ⁽⁴⁹⁾. Higher scores (maximum score = 6) represent higher satisfaction and ease of use, and lower burden.

Model-based statistics:

For each day of intake, for each person, we computed the TDEI for both ASA24 and for RFPM. We fit a linear mixed model to assess differences between the dietary assessment methods, and differences between the dietary assessment methods and the EER. Each participant contributed 7 outcomes. The outcomes included the EER and 6 measurements of reported TDEI - 3 from the ASA24 and 3 from the RFPM. The predictors included indicators for EER, ASA24, and RFPM. We fit an unstructured covariance structure to account for repeated measurements of reported TDEI within each participant. Model assumptions were tested using jackknife residuals. Using the Wald test with Kenward-Roger degrees of freedom and a Type I error rate of 0.05, we tested a series of three hypotheses. First, we tested if there were differences in reported TDEI between the ASA24 and RFPM. The contrast averaged ASA24 and RFPM measures, and assessed the difference between the averages for each modality. Second, we tested if there were differences in reported TDEI with the ASA24 and the EER by comparing child-specific EER with averaged ASA24 measures. Third, we tested if there were differences between reported TDEI with the RFPM and the EER by comparing child-specific EER with averaged RFPM measures. To assess differences by sex, indicator variables for the between-participant factor of sex were added to the model, and average outcome measures were computed, among girls, and among boys. The same modeling approach was used for 2 different EER calculations: using the Schofield equation and using the Henry equation. To examine differences in reported TDEI and EER by site, the model was further adjusted for site. Tests for differences between sites were conducted across the entire population and in subgroups stratified by site.

To support the design of follow-up studies, this study explored differences by race/ethnicity in the association between reported TDEI and the EER. All participants in Louisiana were non-Hispanic White (NHW), which meant that the effects of race/ethnicity and site could not be disentangled. Data reporting by race/ethnicity was thus restricted to Colorado children. Using the base model described above for the full sample, we added race/ethnicity as a binary predictor (NHW or other). Each hypothesis was tested in subgroups cross-classified by both sex and race/ethnicity. All analyses were conducted in SAS 9.4.

RESULTS

This study included 40 parent-child dyads (Table 1) and 98% of parents were mothers. Twenty six percent of children had a BMI at or above the 85^th percentile.

Of the 80 lunches to be consumed at school during the ASA24 assessments (40 children x 2 weekday ASA24 assessments), parents provided data for 57 lunches. Of the 80 lunches to be photographed in school cafeterias (40 children x 2 weekday RFPM assessments), research staff photographed 61 lunches. Of the 19 weekday lunches not photographed by school staff, 14 were consumed at home because the child was absent, the school was closed (e.g., snow day) or the child was homeschooled. The remaining 5 lunches were consumed at school, but not photographed because the child did not provide assent for his or her school lunch to be photographed (2 lunches), no research staff were available (2 lunches) or there was a miscommunication about the timing of lunch (1 lunch).

Data completeness is shown in Table 2. Of 120 assigned days, the least frequently documented meal was lunch with the ASA24 (71.7%). The most frequently documented meal was dinner with the RFPM (83.8%). On days with any dietary data, the mean number of meals/day was 2.8 (SD = 0.4) in the ASA24 and 2.6 (SD = 0.7) in the RFPM; and the mean number of snacks/day was 1.1 (SD = 0.3) in ASA24 and 1.5 (SD = 0.7) in the RFPM.

In the full sample, reported TDEI was higher in the ASA24 compared to the RFPM (∆ [difference between the predicted means] = 379 kcal, p = 0.0002) (Table 3). In sub-group analyses by sex, reported TDEI was significantly higher in the ASA24 compared to the RFPM for boys (∆ = 567 kcal, p = 0.0002), but not girls (∆ = 190 kcal, p = 0.12). Supplemental Table 1 shows means and standard errors for reported energy intake for the ASA24 and RFPM by child sex and race/ethnicity and for two different EER calculations. Table 3 shows differences in TDEI and the EER using the EER calculated from the Schofield BMR equations. Reported energy intake with the ASA24 was 231 kcal higher than the EER (p = 0.008). Reported energy intake with the RFPM did not differ significantly from the EER (difference in predicted means = −148 kcal, p = 0.09). Findings were mostly similar when the Henry equation was used to calculate EER, although the difference between reported energy intake with the ASA24 and the EER was no longer significant among boys (Supplemental Table 2). Differences in reported TDEI did not vary by site for the ASA24 (p = 0.85) or the RFPM (p = 0.14).

Table 3.

Predicted means and differences in reported energy intake with the ASA24, reported energy intake with the RFPM and the estimated energy requirement by child sex among children age 7–8 years in Colorado and Louisiana

	Overall		Girls		Boys
Predicted means
	Mean (SE)		Mean (SE)		Mean (SE)
ASA24	1675 (70)		1541 (91)		1809 (105)
RFPM	1296 (77)		1351 (102)		1241 (116)
EER	1444 (34)		1361 (46)		1526 (50)
Predicted differences
	Difference (95% CI)	p	Difference (95% CI)	p	Difference (95% CI)	p
ASA24 – RFPM	379 (194, 564)	0.0002	190 (−51, 432)	0.12	567 (287, 848)	0.0002
ASA24 - EER	231 (63, 400)	0.008	179 (−44, 403)	0.11	283 (31, 536)	0.03
RFPM - EER	−148 (−321, 26)	0.09	−11 (−240, 218)	0.92	−284 (−545, −24)	0.03

^*The EER is based on the Schofield equation and a physical activity level of 1.3.

^*All models included child sex.

^*The analytic sample (n=205 days of dietary assessment) excluded days in which the participant did report any dietary data (n=35 days).

Energy intake, macronutrient intake and HEI-2015 scores by dietary assessment method are shown in Table 4. HEI-2015 total score did not vary by assessment method (∆ = 3.0, p = 0.08) after adjustment for sex. After adjustment for sex, ethnicity and BMI category, reported intake, in grams, was significantly higher in the ASA24 compared to the RFPM for protein (∆ = 16, p = 0.0001), carbohydrate (∆ = 52, p < 0.0001) and fat (∆ = 16, p = 0.0003).

Survey results are shown in Table 5. For the ASA24, mean ease of use score was higher in Colorado compared to Louisiana (4.8 vs. 4.0, p = 0.03). No other survey responses differed by site. When asked which method they would rather use for 7 days, the majority of parents preferred the RFPM (Table 5). When asked which method required more time to complete, 74.4% of parents reported that the ASA24 was more time consuming (p = 0.002). Mean completion time for the ASA24 was 26 minutes (SD = 23). The in-person training for the RFPM was ‘very useful’ or ‘absolutely useful’ for 82% of participants. Reminder text messages were ‘very useful’ or ‘absolutely useful’ for 64% of participants. Seventy-two percent of participants were ‘moderately satisfied’ or ‘very satisfied’ with sending photos using the app.

DISCUSSION

This study was the first to conduct full-day dietary assessment with the RFPM among school-age children. We compared the RFPM and ASA24 with respect to estimates of dietary intake and participant burden. Reported TDEI and intake of carbohydrate, fat and protein was higher in the ASA24 compared to the RFPM. HEI-2015 score did not differ by assessment method. Compared to the EER, reported TDEI was overestimated with the ASA24 by 231 kcal. The difference between the EER and reported TDEI with the RFPM was not significant. Satisfaction and ease of use with both methods was high, but most parents reported that the RFPM was less time consuming. Technological barriers to accessing the ASA24 website and parents forgetting to capture images with the RFPM contributed to missing data.

Our finding that energy intake estimates differed between the RFPM and ASA24 among boys only raises questions about whether child sex may affect the accuracy of dietary intake estimates. If future validation studies that are powered to detect differences between reported energy intake and a gold standard by sex replicate the findings observed here, this will provide an impetus for investigating the causes of sex differences in the accuracy of parent proxy-reported dietary assessments. Few studies have examined whether common sources of measurement error in dietary assessment (e.g., recall bias, social desirability bias) may be more pronounced for parents of children of a particular sex ⁽⁵¹⁾. One source of measurement error that may be specific to pediatric populations is unobserved intake, but future studies will need to determine if misreporting due to unobserved intake varies by sex. Additionally, qualitative studies could help provide insight into the causes of sex differences in measurement error, such as sex differences in social desirability bias, which has been observed among adult males versus females completing dietary recalls ⁽⁵²⁾.

Scores on the HEI-2015 were remarkably similar in the ASA24 and RFPM, and also closely mirrored HEI-2015 scores for a nationally representative sample of school-aged children ⁽⁵³⁾. Thus, the type of assessment tool may have minimal impact on estimates of adherence to federal dietary guidance. Our expectation was that children would consume 3 meals/day on each assessment day, yet children consumed, on average, less than 3 meals/day. This may reveal a potential source of misreporting (undocumented intake) or may indicate that children in this age group are not consuming 3 meals every day. Meal skipping, possibly due to increases in snacking, was documented among adolescents in Minnesota whose intake frequencies were 4 times/week for breakfast, 5 times/week for lunch and 6 times/week for dinner ⁽⁵⁴⁾. Nevertheless, it is likely that undocumented intake contributed to missing meal data given that parents reported barriers to accessing the ASA24 website or forgetting to capture RFPM images.

In some previous validation studies, the ASA24 under-reported energy intake when compared against weighed food records or direct observation ^{(1; 9; 21; 55)}. In contrast, other ASA24 validation studies demonstrated overestimation of energy intake when parents were proxy reporters for their children ⁽⁵⁶⁾. Here, we found that reported energy intake with the ASA24 was 1,675 kcal, which was significantly higher than the RFPM by 379 kcal. One possible explanation for higher reported energy intake in the ASA24 compared to the RFPM is parents’ overestimation of portions consumed by children during school lunches. Substantial food waste has been documented during lunch meals at school, but parents may be unaware that some lunch foods are left unconsumed ^{(57; 58)}. Thus, the higher estimated energy intake from the ASA24 may be due to parents’ overestimation of children’s intake, particularly if parents reported intake was based on the school cafeteria menu or they did not account for lunchbox food waste.

The RFPM has not yet been validated for assessing full day dietary intake among school-age children under free living conditions. The present study provides preliminary data on intake estimates for future RFPM validation studies. A sex-specific analysis showed that reported energy intake with the RFPM was lower than the EER among boys for all 3 EER calculations. This is consistent with a study among urban preschoolers from underrepresented racial/ethnic groups, in which the RFPM underestimated energy intake by 16% under free living conditions ⁽²⁴⁾. The validity of the RFPM for estimating energy intake among preschoolers from underrepresented racial/ethnic groups was also tested against directly weighed food on a research unit ⁽⁵⁹⁾. The RFPM overestimated energy intake by 7.5% ⁽⁵⁹⁾, suggesting that underestimation in the real world is likely due to missing photos, not error in the process of converting images to estimates of intake. Efforts to improve the accuracy of the RFPM should identify opportunities to ensure that each meal and snack is photographed. For adults, reliance on Ecological Momentary Assessment (EMA) methods has been shown to address this problem very effectively, yielding accurate estimates of energy intake even in free-living conditions ⁽²⁶⁾.

Median satisfaction and ease of use scores were 5 on a scale of 6 for both tools. Acceptability was similarly high among school-aged children and parents of preschoolers who used the ASA24 in Canada ⁽¹¹⁾. For the RFPM, the only studies that examined parents’ experience with the method focused on the perceived burden of documenting a single type of meal across 7 to 8 days ^{(60; 61)}. One study among 9 families demonstrated that parents found the app easy to use for capturing breakfast meals ⁽⁶⁰⁾. Another study among 30 families found that many participants would be willing to use the RFPM again in a future study ⁽⁶¹⁾. The data presented here build on these findings by showing that satisfaction with the RFPM remains high, even when parents are asked to participate for 3 full days.

When asked which of the 2 methods they would prefer to use in future studies, almost two-thirds of parents preferred the RFPM. This finding may be due to the lower perceived time commitment. Two meals were photographed by research staff in the school cafeteria, and this may have reduced parents’ overall time commitment and burden with the RFPM. Time commitment is an important dimension of participant burden in nutrition research ^{(62; 63)}. Objectively-measured completion time for a single ASA24 in this study was 26 minutes on average, which closely matched the ASA24 completion times reported by others ⁽⁶⁴⁾. Thus, the daily time commitment for the RFPM may be less than 26 minutes, if participant perceptions of the relative time commitment were accurate.

Both the ASA24 and RFPM are technology-based measures that are frequently updated in ways that may improve accuracy and reduce participant burden. For example, the ASA24–2020 was released in the short period since the present study was conducted. The ASA24–2020 has new features that may reduce misreporting and burden (e.g., enhanced capability to find misspelled foods in the database, new portion size food images and an updated food and nutrient database, HTML5 interface that facilitates the use of additional browsers). Similarly, at the time this study was conducted, the SmartIntake app was solely compatible with the iOS platform, which might limit who can use the RFPM without using a device provided by the research team. Yet, the SmartIntake app is expected to be compatible with Android devices in the near term.

This study contributes an innovative comparison of two dietary assessment tools, and shows low participant burden when assessing full day intake among school-aged children with the ASA24 and RFPM under free living conditions. The findings provide intake estimates by sex and race/ethnicity that can be used to design future studies to validate full-day intake with the RFPM and ASA24 against a gold standard for energy expenditure, such as doubly labeled water or accelerometry, among school-age children. The results raise questions about whether the accuracy of dietary assessments varies depending on the population characteristics, highlighting the importance of pilot testing assessment tools. In addition, this study highlighted the need to consider eating location and food source in future validation studies, given that where and what children eat may vary as a function of measurement type. Finally, this study was funded by the ECHO Program ⁽²⁸⁾, which has as an explicit goal the development of new methods to improve the quality of observational data in children. Results from the present study have the potential to inform the selection of dietary assessments in the ECHO Program, and the broader child health community.

This study has several limitations. First, we may have under- or over-estimated each child’s energy requirement because daily physical activity level and change in body weight during the dietary assessment period were not measured. However, EERs were calculated based on other measured variables (sex, age, body weight); thus, error in the estimated EER would be similar across dietary assessment methods. Further, the EERs calculated in the present study fall within the range of Estimated Calorie Needs reported in the 2015–2020 Dietary Guidelines for Americans for children age 7 to 8, which supports the plausibility of our estimates⁽⁴⁵⁾. Second, our population was limited to parents reporting their child’s intake, so findings cannot be generalized to adults or children reporting their own intake. Another limitation is that all participants were drawn from urban areas, which means that findings may not be generalizable to rural populations where broadband internet, which is needed for both the ASA24 and RFPM, may be less accessible. Finally, there are multiple considerations when selecting an assessment tool, and only 2 were considered in this study: accuracy (including missingness) and participant burden. Future studies should compare cost, child burden and researcher burden across methods, and how burden and accuracy vary by individual sociodemographic characteristics, such as sex or race/ethnicity. The present study was not powered to draw conclusions about any differences by sex or race/ethnicity but does provide preliminary estimates. Future studies should also compare potential sources of social desirability bias in the ASA24 and RFPM.

In conclusion, we provided novel data that can inform method selection and interpretation of findings in dietary assessment studies among school-aged children. Continued effort to improve the accuracy of both the ASA24 and RFPM is justified given their high acceptability. Future efforts should focus on reducing under- and over-estimation of intake in the ASA24 and missing images in the RFPM.