Comparison of an interviewer-administered with an automated self-administered 24 h (ASA24) dietary recall in adolescents

Abstract

Objective

The current pilot study aimed to assess whether reporting quality would decline materially in adolescents completing weekly web-based Automated Self-Administered 24-Hour dietary recalls (ASA24-Kids-2014) and interviewer-administered 24 h dietary recalls for six weeks. We also aimed to assess method preference.

Design

We conducted two studies. Study 1 (n 20) randomized participants to complete either one ASA24-Kids-2014 or one interviewer-administered recall weekly, for six weeks. Energy intake and number of foods reported were described for each method over time. Differences between recall methods for each measure were tested using mixed-effects regression. Study 2 (n 10) employed a randomized crossover design to describe method preference.

Setting

Dietary intake was collected either by telephone (interviewer-administered dietary recalls) or via the Internet (ASA24-Kids-2014 dietary recalls).

Subjects

Adolescents aged 12–17 years with no prior diet recording experience were enrolled.

Results

In Study 1, mean (sd) total energy and number of foods reported decreased by 50 (222) kJ (12 (53) kcal) and 0·05 (0·31) items v. 38 (138) kJ (9 (33) kcal) and 0·17 (0·14) items per recall for participants randomized to the ASA24-Kids-2014 v. interviewer-administered recalls, respectively. There was no difference between groups for either measure (P > 0·57). In Study 2, eight of ten participants preferred the interviewer-administered recall over the ASA24-Kids-2014. Overall, seven of twenty participants experienced technical difficulties with the ASA24-Kids-2014.

Conclusions

No appreciable decay in reporting quality was seen for either method. However, participants reported a preference for the interviewer-administered recall. Our findings can help inform and support larger studies to further characterize the performance of the ASA24 in adolescents.

Characterizing dietary intake of individuals is an important element in many research studies. Dietary intake is usually obtained through self-report because objective measures of dietary intake, such as direct observation, are generally not feasible in community-dwelling populations. Several methods for obtaining self-reported intake have been developed with the most appropriate method for a given study depending on multiple factors including sample size, dietary components of interest and cost, as well as age and other population characteristics^{( 1 , 2 )}.

The 24 h dietary recall interview is widely used for collecting intake data for research. In this approach, a trained interviewer asks participants to recall the previous day’s intake using a standardized protocol such as the US Department of Agriculture’s Automated Multiple Pass Method (AMPM)^{( 3 )}. Compared with other methods, such as the participant-recorded food record or FFQ, the recall interview is more likely to capture details about food components, preparation and amounts due to the AMPM standardized interviewing process. This method also minimizes reactivity bias because the recall is typically unannounced and retrospective^{( 4 )}. The 24 h recall interview method does not require participant literacy and it differs from the FFQ in that it queries foods and amounts consumed on a specific day rather than relying on generic memory to estimate usual patterns of intake over a longer time interval such as a month or a year^{( 5 )}. However, staff time spent with a participant either in person or on the telephone to complete the interview and for data entry add to research time and costs, making the traditional interviewer-administered dietary recall impractical for larger studies. To address these issues, the National Cancer Institute developed the web-based Automated Self-Administered 24-Hour (ASA24^®) Dietary Assessment Tool, allowing participants to report their own 24 h intake via a secure website with guides and prompts modelled on the AMPM^{( 6 )}. This innovative system also has automated electronic reminders that researchers can program according to their study’s time frame and data needs, potentially saving significant time and cost over the interviewer-administered method. The initial version of the ASA24 was released in 2009 with updates in 2011, 2014 and 2016. The ASA24-Kids-2014 was adapted from the ASA24 to capture intake of children over 10 years of age by removing foods not typically consumed by children, asking fewer questions about food preparation, and simplifying the wording of food and beverage items^{( 7 )}.

Assessing dietary intake in adolescents is important, yet there is a lack of research on tools and methods that are useful in this population. A small number of studies have compared the ASA24 (adult version) web-based recall with interviewer-administered dietary recalls or expert-coded food records in terms of nutrient output in adults^{( 8 – 12 )}. Results of these studies have been equivocal, some showing no difference in food and nutrient intakes between methods^{( 13 )} and others finding moderate agreement in nutrient intake^{( 8 , 9 )} or a slight superiority of the AMPM in adults estimating true intake over the ASA24^{( 10 )}. While there are few studies assessing the ability of web-based, self-administered 24 h recalls to accurately collect dietary information in adults, there are even fewer in children and adolescents. Baranowski et al. compared the ASA24 (adult version) with the interviewer-administered recall in children aged 8–13 years and found results were poorer for younger (8–9 years old) than for older children^{( 12 )}. Diep et al. found the ASA24-Kids-2012 to be less accurate than interviewer-administered recalls when compared with observed intake in children aged 9–11 years^{( 11 )}. Thus, examining the accuracy of a self-administered 24 h recall to measure dietary intake in adolescents is important to better understand approaches suited to this population.

The ASA24-Kids-2014 requires investigation to inform researchers of this instrument’s ability to measure dietary intake in older children and adolescents. The purpose of the current pilot study was to: (i) assess whether reporting quality would decline materially in adolescents completing weekly ASA24-Kids-2014 and interviewer-administered 24 h dietary recalls for six weeks; and (ii) gather qualitative feedback about each method including method preference. Thus, the present study represents a necessary first step to identify barriers and inform the design of larger studies investigating the performance of web-based dietary recalls in adolescents.

Methods

Participants

A recruitment email was sent to all employees at Cincinnati Children’s Hospital Medical Center (CCHMC) describing the study and seeking adolescents aged 12–17 years to participate. Respondents to the email were contacted by telephone to determine eligibility until recruitment goals were filled (twenty participants for Study 1 and ten participants for Study 2, with approximately equal distribution of males and females in each). Individuals were excluded if they had prior experience completing dietary recalls, did not speak English, had a self-reported BMI<5th or >85th percentile, or had a mental, physical or visual limitation that would preclude them from reporting their dietary intake. Lack of reliable access to a laptop or desktop computer with Internet connection and lack of a valid email address were also exclusion criteria.

Eligible individuals interested in participating were invited to the Clinical Translational Research Center at CCHMC between November 2014 and May 2015 for consent and baseline assessment. Participants and a parent provided written informed assent and consent, respectively. The study was approved by the CCHMC Institutional Review Board and is registered at www.clinicaltrials.gov (NCT02818933).

Procedures and measures

Height and weight were measured by trained research staff after participants removed their outer clothing and shoes. Height was measured using a wall-mounted stadiometer (Holtain Harpenden) to the nearest 0·1 cm. Two height measurements were recorded; if they differed by more than 0·5 cm, a third height was taken and the two closest height measurements were averaged. Weight was measured on a calibrated digital scale (Scaletronix 5002) to the nearest 0·1 kg. Two weights were recorded; if they differed by more than 0·2 kg, a third weight was taken and the two closest weight measurements were averaged. BMI was calculated as [weight (kg)]/[height (m)]² and BMI Z-scores were determined using the Centers for Disease Control and Prevention 2000 growth charts.

Two studies were conducted. Study 1 addressed feasibility in obtaining multiple weekly recalls and decay in reporting quality. In Study 1, participants (n 20) were randomized to complete either one ASA24-Kids-2014 recall or one interviewer-administered recall weekly, for six weeks. Randomization was stratified by age (12–14 years, 15–17 years) and sex to achieve balance on these factors across study groups. Those in the ASA24-Kids-2014 group were contacted by email between Monday and Friday and provided a link to the respondent website to complete the self-administered recall online. These participants were given up to three reminder emails each week. Participants assigned to the interviewer-administered recall group were contacted by telephone between Monday and Friday to complete the 24 h recall. Three attempts were made to contact the participants in this group each week. The ASA24-Kids-2014 recalls were completed by the participant without parental assistance at a location such as their home, a relative’s home or a library where an Internet-accessible computer was available. This web-based program includes visual aids to assist with food and portion identification. The interviewer-administered recalls were conducted by one of two trained experienced interviewers to minimize inter-observer bias. Participants were provided a visual aid packet to use in portion size estimation and interviewers used a computer-assisted multiple-pass approach^{( 3 )} that prompts for complete food descriptions and amounts, detailed food preparation methods, beverages, and additions such as condiments or sweeteners. All participants received $US 20 for the initial study visit plus $US 10 for each recall completed, whether via the web-based system or the interview format.

After 6 weeks, participants were contacted via telephone by a member of the study team for an ‘exit’ interview, where they were asked for feedback on how easy or difficult the method was; how convenient; whether they thought the time taken to complete was the right amount v. too long or too short; and whether the system, or interviewer, could retrieve their reported foods. Questions were tailored to each format to obtain comparable feedback for both methods (see online supplementary material, Supplemental Files 1, 2 and 3).

Study 2 used a crossover design to assess preference between the two recall methods. Participants (n 10) completed one ASA24-Kids-2014 recall and one interviewer-administered recall, one week apart. Half of the participants were randomly assigned to complete the ASA24-Kids-2014 first and the other half to complete the interviewer-administered recall first. Randomization to sequence was achieved via the sealed envelope approach for both studies^{( 14 )}. After completing both recalls, participants completed an exit interview by telephone. This interview included the same feedback questions as in Study 1, plus additional questions to ascertain method preference. Study 2 participants also received $US 20 for the initial visit plus $US 10 for each recall completed.

Data collected from each 24 h recall in both studies, whether web-based or interviewer-administered, included the number of contact attempts or reminder emails per recall; duration of recall; and amount, type and number of foods and beverages reported. Foods were defined and counted as the overall item eaten (e.g. lasagne) whereas the ingredients would be pasta, tomato sauce, cheese, meat. If an ingredient was entered as an addition at the table, or after the food was prepared, that would count as an ingredient, but not as the whole food. Energy intake was computed from all foods and beverages reported. The ASA24-Kids-2014 platform employs the US Department of Agriculture Food and Nutrient Database for Dietary Studies (FNDDS) version 4.1 (2010). The interviewer-administered recalls employed the Nutrition Data System for Research (NDSR; Nutrition Coordinating Center, University of Minnesota), software and database version 2014.

Statistical analysis

Study 1 baseline characteristics by group assignment were compared using Fisher’s exact tests for categorical variables and two-sample t-tests for continuous variables. Random-effects regression models were used to obtain estimates for the difference in energy intake, number of foods reported and duration of recall between groups and over time. The model was chosen a priori and included group assignment, recall number (i.e. time), age, sex, BMI percentile and a group-by-recall number interaction term as fixed effects and subject as a random effect. Counts and proportions were used to compare reporting preference and responses from the post-study questionnaire. Statistical analyses were performed using the statistical software package SAS version 9.3 and R software version 3.2.3.

Results

We screened a total of thirty-two adolescents; two were not enrolled due to having a self-reported BMI >85th percentile. Our final sample size in Study 1 was twenty, and in Study 2 was ten; participants were unique across both studies. Study 1 participants assigned to the interviewer-administered group (n 10) and ASA24-Kids-2014 group (n 10) had a mean (sd) age of 14·3 (2·1) and 14·5 (1·6) years and mean BMI Z-score of 0·1 (0·8) and 0·1 (1·2), respectively. Both groups were 90 % non-Hispanic white. Participants in Study 2 (n 10) had a mean (sd) age of 14·2 (1·5) years, mean BMI Z-score of 0·2 (1·2) and 87 % were non-Hispanic white. There were no material differences in age, sex, race/ethnicity or BMI Z-score between groups in Study 1, or between Study 1 and Study 2.

Two participants randomized to the ASA-Kids-2014 completed only the initial recall due to an inability to re-access the ASA24 system to complete subsequent recalls. All participants in the interviewer-administered group completed all six recalls. Self-reported energy intake, number of foods reported and recall duration for Study 1 are provided in Fig. 1. There was no difference in the response trajectory between recall methods for energy intake (P=0·91) or number of foods reported over the six weeks (P=0·57). The decline over recall administrations for participants randomized to the ASA24-Kids-2014 was –50 (222) kJ (–12 (53) kcal) and −0·05 (0·31) food items per recall. The decline over recall administrations for participants randomized to the interviewer-administered recall was –38 (138) kJ (−9 (33) kcal) and −0·17 (0·14) food items per recall. There was a pronounced decline over recall administrations in the duration of the ASA24-Kids-2014 relative to the interviewer-administered recall (P=0·01). The mean (sd) duration to complete the first and last ASA24-Kids-2014 recall was 25·2 (9·5) and 13·8 (7·9) min, respectively. The mean times to complete the first and last interviewer-administered recall were 15·5 (5·3) and 12·6 (4·8) min, respectively.

Fig. 1.

Box-and whisker plots comparing the Automated Self-Administered 24-Hour dietary recall (ASA24-Kids-2014; ) and the interviewer-administered 24 h dietary recall () at weeks 1 through 6 for adolescents aged 12–17 years (n 20) in Study 1: (a) total energy (to convert to kJ, multiply kcal value by 4·184); (b) food items reported; and (c) duration of recall. The solid line within the box represents the median value; the top and bottom of the box represent the 75th and 25th percentile, respectively; the whiskers represent values falling within 1·5 times the interquartile range; and data points falling outside these limits are plotted as circles

Study 2 participants reported an overall preference for the interviewer-administered over the ASA24-Kids-2014 dietary recall method (eight out of ten). When asked about preferences on specific features of each method, the interviewer-administered recall was considered more user-friendly, more convenient, more detailed and quicker. The ASA24-Kids-2014 was considered to fit better into one’s schedule (Table 1). Additional feedback about both methods from all participants (Study 1 and Study 2) is displayed in Table 2. Participants were also asked about difficulties they had with either system. Web connection problems for the ASA24-Kids-2014, such as the screen freezing up or having to restart, were reported by five of twenty participants. On the interviewer-administered recall, one of twenty participants reported a problem receiving a telephone call. Finally, seven of the twenty participants who used the ASA24-Kids-2014 reported technical difficulties, including not being able to download the required software, having limited access to a computer or laptop, and slow loading time for portion images. No participants reported technical difficulties with the interviewer-administered recall.

Table 1.

Preferences of adolescents (n 10) aged 12–17 years in Study 2 who completed both one Automated Self-Administered 24-Hour dietary recall (ASA-Kids-2014) and one interviewer-administered 24 h dietary recall

	Interviewer-administered	ASA24-Kids-2014	No preference
Overall, which method do you prefer?	8	2	0
Which method would you say is…
More user-friendly?	5	2	3
More convenient?	6	3	1
More detailed?	5	4	1
Quicker?	10	0	0
Easier to fit in your schedule?	3	6	1

Table 2.

Feedback from adolescents aged 12–17 years on the Automated Self-Administered 24-Hour dietary recall (ASA-Kids-2014) and the interviewer-administered 24 h dietary recall

	Study 1* (n 20)				Study 2† (n 10)
	Interviewer-administered		ASA24-Kids-2014		Interviewer-administered		ASA24-Kids-2014
	n	%	n	%	n	%	n	%
How would you rate the ease of reporting what you ate?
Very/somewhat easy	9	90	5	50	8	80	8	80
Neutral	1	10	4	40	2	20	1	10
Somewhat difficult	0	0	1	10	0	0	1	10
Do you think the time it took to complete the recall was…
The right amount of time?	10	100	7	70	10	100	5	50
Too much time?	0	0	3	30	0	0	5	50
About what percentage of foods could not be found?
Less than 10 % of foods	10	10	7	70	9	90	9	90
10–25 % of foods	0	0	2	20	1	10	1	10
25–50 % of foods	0	0	1	10	0	0	0	0

All participants in both studies completed the exit interview (n 30).

^*Responses from participants (n 20) in the 6-week randomized study comparing reported energy intake and number of foods. Each participant responded based on his/her assigned method only, either ASA24-Kids-2014 (n 10) or interviewer-administered recall (n 10).

^†Responses from participants (n 10) who did one of each type of recall, ASA24-Kids-2014 and interviewer-administered. Each participant responded to questions about both methods.

Discussion

In the current pilot studies, adolescents aged 12–17 years were assessed on the performance of, and preference for, the ASA24-Kids-2014 relative to an interviewer-administered dietary recall. In Study 1 there were modest declines in total energy and number of foods reported for both the ASA24-Kids-2014 and interviewer-administered recalls, but no appreciable difference was seen in the response trajectory between the two groups over 6 weeks. Although we hypothesized that adolescents would prefer the ASA24-Kids-2014 over the interviewer-administered recall, our Study 2 findings did not support this. The interviewer-administered recall was the method preferred in this small sample of adolescents, primarily due to less time to complete recalls. However, participants did report that completing a recall using the ASA24-Kids-2014 ‘fit better into their schedule’. Our results are similar to those of Diep et al., who also reported participants preferred the interviewer-administered recall over the ASA24-Kids-2012 for reasons including the ease of telling the interviewer what they ate. However, a large number of intrusions and omitted food items was recorded for both instruments, suggesting poor ability to recall food intake in this age group. Indeed, the National Health and Nutrition Examination Survey (NHANES) selected 12 years as the youngest age for participants to complete a dietary recall interview on their own^{( 15 )}. For this reason, we opted to include only those aged 12 years or older in our study.

Previous studies reported only moderate agreement between ASA24 results and those of an expert-coded multi-day food record^{( 8 – 12 )}. A study by Bjorge-Schohl et al. found that intra-class correlation coefficients between a participant-coded record in ASA24 and the same record coded by an investigator using a commercial database (ESHA) ranged from 0·65 to 0·81 for macronutrients and from 0·50 to 0·66 for micronutrients^{( 8 )}. Similarly, Frankenfeld et al. compared two ASA24 recalls with a 4 d food record and found that the Pearson correlation coefficients for mean nutrient intakes fell largely between 0·4 and 0·6, with the higher correlations seen in some micronutrients such as B vitamins and Ca^{( 9 )}. Kirkpatrick et al. assessed criterion validity for the ASA24 by comparing it with an interviewer-administered recall (using the AMPM) to see which one better estimated energy, nutrients, food groups and portion sizes compared with the true amounts (based on foods provided and measured by the researchers). While the AMPM did slightly better in estimating true intakes and capturing additions to foods (ASA24 had a higher number of intrusions, i.e. foods reported but not actually consumed; P<0·01), most differences were not significant. The authors concluded that the ASA24 performed well enough to be considered a useful resource for some nutrition studies^{( 10 )}. In a large study by Thompson et al., 1081 men and women were asked to complete two 24 h dietary recalls using either the ASA24 or the interviewer-administered recall method, or both, in random order and assignment^{( 13 )}. Comparing the two methods, there were no significant differences in estimated energy and nutrient intakes and there was a high rate of equivalency (87 %) in foods captured. Importantly, 70 % of participants preferred the ASA24 over AMPM, and use of the ASA24 resulted in less attrition.

One challenge in collecting multiple days of diet intake is ‘reporting decay’, a decline in the number of foods reported over repeated dietary recalls. Arab et al. asked 261 white and African-American adults to complete eight self-administered dietary recalls over 2 months on a web-based platform (DietDay)^{( 16 )}. While 92 % completed all eight recalls, the mean and median intakes of energy, fat and carbohydrate declined as the number of recalls increased. Conversely, a study of multiple interviewer-administered recalls administered every 10 d for 6 months in adults showed no trend across recall attempts in reported nutrient intakes^{( 17 )}. Although we hypothesized a decline in energy and number of foods reported with repeat administration of the ASA24-Kids-2014, we found that the mean reported energy intake and number of foods were comparable for both recall types with no material decline over the 6 weeks. These results are similar to those of Thompson et al. who reported that nutrient and food group intakes in adults were equivalent by a 20 % bound across recall types^{( 13 )}. Understanding the extent to which the accuracy of self-reported diet intake declines with the number and timing of repeat administrations is a crucial first step in determining the potential validity of web-based tools for use in children and adolescents.

Given the tech-savvy nature of today’s youth, we expected that our participants would experience minimal difficulty using the web-based ASA24-Kids-2014. However, two participants were able to complete only one recall using ASA24-Kids-2014, not because of their lack of ability, but due to system dysfunction, whereas all participants in the interviewer-administered group completed all recalls. In addition, the ASA24-Kids-2014 recall had to be completed on a desktop or laptop computer, posing a barrier to completion if a participant did not have consistent access to these tools. This issue affected one of our participants who was travelling and resulted in a missed recall. In an effort to address ongoing technical issues, the 2016 version of ASA24 eliminates the need to download special software to complete a recall and now offers a mobile platform.

Collecting dietary intake information from adolescents poses additional challenges due to cognitive difficulties in recalling foods eaten, frequency of foods consumed away from home, unstructured eating patterns, and limited knowledge of food contents and preparation which may influence reported nutrient intakes^{( 18 )}. Together, these factors increase the potential for self-report error in children and adolescents relative to adults, which would be expected to extend to the web-based 24 h recall. A study by Baranowski et al. compared an early version of the ASA24 with the interviewer-administered recall in children aged 8–13 years and found specific food matches occurred less than 50 % of the time, with 18·9 % of foods missing from the ASA24 reports overall; results were poorer for younger (8–9 years old) than for older children^{( 12 )}. Obstacles to completion in that study informed modifications to improve its usefulness in children and the creation of ASA24-Kids-2012. Important modifications included incorporating validity checks and a phonetic spell-checker. Diep et al. compared the ASA24-Kids-2012 and an interviewer-administered 24 h recall accompanied by direct observation in 9- to 11-year-olds^{( 11 )}. They found that although both recall types reflected error, the ASA24-Kids-2012 error rate was higher than that of the interviewer-administered recall. The young age of participants, however, may have been a factor. Another study by Baxter et al. investigated children aged 9–11 years, by observed and unobserved school meals followed by a 24 h dietary recall. They found a failure to report meals/snacks in this age group, also suggesting that young age was a potential factor^{( 19 )}. Results from the abovementioned studies suggest there may be a minimum age at which using a web-based system is a viable option.

Limitations of the present pilot study include the small sample size of adolescents and the duration of the study. In addition, we examined only a single reporting frequency, i.e. once weekly for six weeks, for the dietary recalls in Study 1. A longer study, or a study in which participants did more frequent dietary recalls, might have had a different result in terms of reporting decay. However, the collection of both quantitative data and qualitative feedback in the present study provides a critical first step and unique insight into the benefits and detractions of using the ASA24-Kids-2014 to collect dietary recall information in adolescents that will be useful to other researchers. Future study in this area should include more rigorous validation of the technical functionality of web-based self-administered dietary recalls in older adolescents, specifically designed to determine the structure of measurement error and changes in reporting behaviours when compared with objective biomarkers or direct observation, or differences in the duration and timing of recall administration, respectively.

Conclusion

Total energy and number of foods reported were similar over six weekly repeat recall administrations for adolescents completing the ASA24-Kids-2014 and an intervieweradministered recall in the current pilot study and provided no evidence that the quality of the recalls declined appreciably over time. However, in this small sample of adolescents, the interviewer-administered method was preferred over the ASA24-Kids-2014 for reporting dietary intake. Less time to complete recalls was commonly cited as a positive feature of the interviewer-administered recall, whereas flexibility was a positive feature of the ASA24-Kids-2014. Technical challenges occurred with the use of the ASA24-Kids-2014 and the most recent version is expected to better address these issues. However, the latest release of the ASA24 (2016) will no longer support a ‘Kids’ version and younger participants are recommended to use the adult version without adaptations to the food list and questions specific to children. Thus, further research is warranted regarding how this instrument will perform in children and adolescents.