Performance of the Automated Self-Administered 24-hour Recall relative to a measure of true intakes and to an interviewer-administered 24-h recall

Abstract

Background: The Automated Self-Administered 24-hour Recall (ASA24), a freely available Web-based tool, was developed to enhance the feasibility of collecting high-quality dietary intake data from large samples.

Objective: The purpose of this study was to assess the criterion validity of ASA24 through a feeding study in which the true intake for 3 meals was known.

Design: True intake and plate waste from 3 meals were ascertained for 81 adults by inconspicuously weighing foods and beverages offered at a buffet before and after each participant served him- or herself. Participants were randomly assigned to complete an ASA24 or an interviewer-administered Automated Multiple-Pass Method (AMPM) recall the following day. With the use of linear and Poisson regression analysis, we examined the associations between recall mode and 1) the proportions of items consumed for which a match was reported and that were excluded, 2) the number of intrusions (items reported but not consumed), and 3) differences between energy, nutrient, food group, and portion size estimates based on true and reported intakes.

Results: Respondents completing ASA24 reported 80% of items truly consumed compared with 83% in AMPM (P = 0.07). For both ASA24 and AMPM, additions to or ingredients in multicomponent foods and drinks were more frequently omitted than were main foods or drinks. The number of intrusions was higher in ASA24 (P < 0.01). Little evidence of differences by recall mode was found in the gap between true and reported energy, nutrient, and food group intakes or portion sizes.

Conclusions: Although the interviewer-administered AMPM performed somewhat better relative to true intakes for matches, exclusions, and intrusions, ASA24 performed well. Given the substantial cost savings that ASA24 offers, it has the potential to make important contributions to research aimed at describing the diets of populations, assessing the effect of interventions on diet, and elucidating diet and health relations. This trial was registered at clinicaltrials.gov as NCT00978406.

INTRODUCTION

Given evidence from biomarker-based validation studies that 24-h dietary recalls capture dietary intake with less bias than do food-frequency questionnaires (1–3), there has been a push toward the use of 24-h dietary recalls in epidemiologic studies and other large-scale research. Traditional 24-h dietary recalls are expensive and impractical for large studies because they rely on trained interviewers and multiple administrations to estimate usual intakes (4). The Automated Self-Administered 24-hour Recall (ASA24) ,⁴ a freely available Web-based tool, was developed to address these challenges by eliminating the need for an interviewer and implementing automated coding, with the goal of making it feasible to collect multiple high-quality recalls from large samples (5).

The ASA24 is modeled on the USDA's Automated Multiple-Pass Method (AMPM) (5). The AMPM is the method used to capture dietary intake data in What We Eat in America, the dietary interview component of the NHANES. The AMPM uses 5 steps, or passes, to enhance accuracy and completeness of recalls, including a quick list, forgotten foods pass, time and occasion pass, detail pass, and final review (6). This multiple-pass approach has been shown to reduce bias in the estimation of dietary intake (6–8).

The ASA24 flows as per a modified AMPM, guiding participants through the completion of a 24-h dietary recall by using an online dynamic user interface (5). A meal-based quick list is used, in which respondents select an eating occasion and report the time of the occasion before reporting the foods and drinks consumed. Foods and drinks are selected by browsing food categories or searching from a list of user-friendly terms derived from foods and beverages reported in NHANES. Next, a meal gap review queries the respondent about any foods or drinks consumed between reported eating occasions separated by ≥3 h. This is followed by the detail pass, during which detailed questions are administered about food preparation, portion size (using images to assist with estimation), and additions so that food codes from USDA's Food and Nutrient Database for Dietary Studies (FNDDS) (9) can be assigned. Respondents are then prompted to think about a list of frequently forgotten items and complete a final review of all foods and drinks reported by meal to confirm that they have included everything they ate and drank the previous day from midnight to midnight.

The purpose of this study was to assess the criterion validity of the ASA24 through a feeding study in which true intake for 3 meals was known. This study made use of ASA24-2011 (released September 2011). A comparison of intake data collected through interviewer-administered AMPM recalls to true intake for 3 meals was also conducted.

SUBJECTS AND METHODS

Participants and data collection

This study included 83 men and women aged 20–70 y who lived in the Washington, DC, metropolitan area and were recruited from a database of research volunteers, which contained details on sex, age range, and race-ethnicity, allowing efforts to recruit a diverse range of participants (10). Eligible subjects had not previously participated in a research study, were not currently dieting, and did not have any formal training in nutrition. Data collection was conducted over an 8-wk period in the spring of 2012. Participants were scheduled to visit a study center to consume 3 meals on one day and to return the following day, at which time they completed an unannounced 24-h dietary recall and a brief demographic and health behavior survey. Participants were reimbursed for their travel expenses and given modest remuneration for their time.

On their scheduled feeding day, the participants arrived at the study center in the morning, were provided with a brief introduction to the study (which was described as a study about improving methods regarding what Americans eat), and completed informed consent. They then were invited to select and consume foods and beverages from a buffet of meal-appropriate items and to return to do the same for lunch and dinner. Foods and drinks were offered in communal containers appropriate to the type of food or drink, such as platters (eg, for sandwiches) and bowls (eg, for salads) from which participants served themselves (11, 12). Some single-serve items, such as yogurt and potato chips, were served in their original containers. These serving decisions were intended to mimic the ways in which participants might encounter foods and drinks at home and in other eating environments. Offerings included foods (such as cereal, sandwiches, and lasagna) and drinks (such as coffee, juice, and soda) and potential additions (such as sweeteners and spreads) (see Supplemental Table 1 under “Supplemental data” in the online issue). Several preprepared multicomponent items, such as salads and sandwiches with spread (eg, mayonnaise), filling (eg, tuna salad), and vegetables, were offered. Participants served themselves one at a time, at 10-min intervals, and each container was inconspicuously weighed before and after to determine the amount of each item taken by each person (11, 12). Plate waste was also weighed to enable a calculation of the amount of each food and drink consumed. Weights were taken with the use of Ultra Ship 35 scales, which have a precise accuracy of 0.1 oz or 2.8 g (up to 2 lb or 0.91 kg) and 0.2 oz or 5.7 g (>2 lb or 0.91 kg). Each item was weighed once independently by 2 technicians; if the weights did not match to the gram, a third weight was taken and the average of the 2 closest weights was taken. The weight consumed (ie, true intake) was calculated as the weight taken minus the weight left (ie, plate waste). Meals were consumed in a communal dining area, with a room monitor present to ensure that external foods or drinks were not introduced into the meals and to discourage sharing of foods and drinks among study participants. Participants were invited to spend the time between meals in the study center in a quiet area with Internet access or could leave the center and return for the next meal. No restrictions were placed on eating and drinking of other foods and drinks outside of the study center meals. Participants were unaware that they would be asked to recall the foods and drinks consumed the following day.

Each participant was asked to return to the center on the following day, at which time the participant completed a 24-h dietary recall. After being stratified by sex and age group to ensure matching on these characteristics, participants were randomly assigned into 2 groups. Half of the respondents completed an ASA24 at a computer station, and the other half completed an AMPM recall conducted by a trained interviewer over the telephone. A staff person escorted those completing the ASA24 to the computer station but did not provide any assistance with the completion of the recall; however, a telephone help line was available to mimic a real-world study in which respondents would have contact information for a study coordinator (4 participants made use of this resource). Those completing the AMPM recall had access to the USDA Food Model Booklet, measuring cups and spoons, and a ruler to help estimate portion sizes, which simulated an NHANES interview. A supervisor monitored 10% of the AMPM interviews for quality assurance. Pairs arrived at intervals across the day so that only one participant was completing either the ASA24 or AMPM at a time. After completion of the recall, participants completed the demographic and health behavior survey on a computer at the study center. The questionnaire included questions on income, education, living situation, height, weight, health status, vitamin/mineral supplement use, frequency of fast food consumption, physical activity, smoking, and alcohol use.

The reported intakes for one ASA24 participant and one AMPM participant did not correspond to the study center meal offerings, which suggested that these respondents did not report their previous day's intakes. Data from these participants were excluded, which resulted in a final analytic sample of 81 participants, 40 of whom completed the ASA24 and 41 of whom completed the AMPM. Three participants (2 from the ASA24 group and 1 from the AMPM group) did not consume breakfast at the study center, and 1 participant (from the ASA24 group) did not consume dinner there. Finally, 1 participant in the ASA24 group did not complete the demographic and health behavior questionnaire. This study was approved by the National Cancer Institute Institutional Review Board as well as the Westat Institutional Review Board.

Coding of true and reported intakes

For the foods and drinks offered, trained coders applied food codes from the FNDDS, version 4.1 (9), and merged the file with nutrient values from the FNDDS. Intakes reported via the ASA24-2011 are autocoded by the ASA24 system with the use of the FNDDS, version 4.1. Intakes reported via the AMPM were coded by a trained coder, with 10% of coded intakes undergoing a detail review and verification for quality control by 2 senior coders. This process was completed with the use of the USDA's dietary coding software, SurveyNet version 2.7, according to procedures developed for data entry for What We Eat in America. This version of SurveyNet uses food codes from FNDDS, version 4.1. Quality-control procedures used in other studies making use of the AMPM method were used to screen for possible errors in coding, and identified errors were corrected (10). For all 3 data sets, food group values were obtained from the MyPyramid Equivalents Database, version 2.0 (13), supplemented with the USDA's Center for Nutrition Policy and Promotion Addendum (http://www.cnpp.usda.gov/OtherProjects.htm), which provides Pyramid Equivalents for food codes added to FNDDS since version 2.0.

Comparison of true and reported intakes

In completing either the ASA24 or the interviewer-administered AMPM recall, participants were prompted to report all foods consumed on the previous day from midnight to midnight. However, for the purpose of these analyses, interest was in the recalled intakes for the meals consumed at the study center. Each recall was reviewed by a nutritionist who was blinded to the true intakes (ie, the weighed data) to identify study center meals by the reported name of the eating occasion, the reported time and location of the eating occasion, and the foods and beverages reported. All foods and drinks reported outside of these meal occasions were excluded from the analysis. The excluded meals included 3 breakfasts and 1 dinner not consumed at the study center (one meal for each of 4 participants).

Before the identification of the study center meals, corrections were applied to address known errors in the ASA24 database (http://riskfactor.cancer.gov/tools/instruments/asa24/resources/issues.html). Furthermore, trained coders conducted a review of free text entered by ASA24 participants who selected “other” as a response to a detail question or who indicated that they could not find the food or drink that they consumed (ie, “match not found”), which resulted in a prompt to type in a description. In the case of “other” responses to detail questions (eg, when asked about the flavor of yogurt consumed, a respondent may select “other” and specify “raspberry”), a default food code is assigned based on the primary food being reported (eg, yogurt, type of milk, or flavor not specified). In the case of “match not found,” questions about the type (eg, beverage) and kind (eg, coffee) of food or drink follow the free text entry prompt, and a default food code is assigned based on the participant's answers to these questions. In some cases, a mismatch was identified between the free text entered by the participant and the assigned code, and corrections were made to more closely reflect the text entry. Of 35 instances of “other” and “match not found” items within the study center meals, 11 required correction, representing 1% of all study center foods, drinks, and additions reported with the ASA24. Finally, corrections were made to address ASA24 system-related issues that have since been remedied; this amounted to 22 corrections, which affected 2% of all study center foods, drinks, and additions reported in the ASA24.

A list of all food codes reported by all participants for the study center meals was generated and assessed by 2 nutritionists to determine whether each was a match for any of the foods and drinks offered. Matches were classified as exact, close, or far. For example, low-fat yogurt was considered to be an exact match, whereas regular yogurt was considered a close match for the low-fat yogurt offered (in single-serve labeled containers). An example of a far match is fresh tuna rather than tuna salad as part of a sandwich offered. The identified matches for the items offered were reviewed by the full study team. At this point, the match identification process was conducted by using only the information on the menu of items offered and a list of all items reported by all participants but without consulting the true intake data for each participant. The next step involved a comparison of the true intakes and reported intakes to determine whether each participant reported a match for each of the foods and drinks he or she consumed. Exclusions (ie, items consumed but not reported) and intrusions (ie, items reported but not consumed) were also identified. The resulting files, showing matches, exclusions, and intrusions for each recall, were reviewed by 2 nutritionists for accuracy. This procedure was conducted for all foods and drinks reported for the 3 study center meals combined (to account for the fact that participants may report an item that was actually consumed but at the wrong eating occasion) and for each meal independently (to assess accuracy of reporting at each eating occasion).

Further differentiation was applied to enable a more in-depth examination of exclusions and intrusions. The proportion of main items (eg, oatmeal, sandwich, and coffee) compared with additions to and ingredients in multicomponent foods and drinks (eg, sugar, mayonnaise, and cheese) excluded were examined by recall mode. Intrusions were identified as being either internal confabulations (ie, offered but not consumed by the participant) or external confabulations (ie, not offered) (14).

Statistical analyses

Analyses were conducted with the use of SAS, version 9.2 (SAS Institute). Chi-square tests were used to test for differences in demographic characteristics and health behaviors between the ASA24 and AMPM groups. Preliminary bivariate analyses showed that the 2 groups differed in terms of education, race-ethnicity, and use of vitamin/mineral supplements. Multivariate analysis showed that after race-ethnicity was accounted for, the differences in the other characteristics did not persist; thus, race-ethnicity was included as a covariate in models that compared the 2 groups for all analyses described below.

Linear regression models were used to examine the association between recall mode and the proportion of items truly consumed for which a match was reported. This was conducted first by considering all matches (exact, close, and far) and second by considering only exact and close matches. Linear regression was also used to assess the association between recall mode and the proportion of items truly consumed but not reported (exclusions). This analysis was run for primary foods and drinks compared with additions and ingredients and also for food groups that might be considered more or less socially desirable or more likely to be forgotten and thus reported to a greater or lesser extent of accuracy (fruit and vegetables; snacks, sweets and desserts; and beverages). Poisson regression was used to assess the association between recall mode and the number of items reported but not truly consumed, considering all intrusions first and also internal and external confabulations separately. To account for the fact that participants who consume a greater number of items may have more difficulty accurately recalling all of them, the models were repeated including a variable indicating the number of items truly consumed. These analyses were conducted for all eating occasions combined and by eating occasion.

A comparison of the accuracy of estimated intakes of energy and selected nutrients and food groups by recall mode was conducted based on reported compared with true intake. Linear regression models were used to assess whether the differences between true and reported intakes were significant within each group (ASA24 and AMPM) and also to examine whether there were differences in the accuracy of estimated energy, nutrient, and food group intakes by recall mode.

Finally, the consistency of reported and true portion sizes by recall mode was examined. This entailed the examination of actual differences (considering magnitude and direction of the gap between true and reported) and absolute differences (considering only magnitude) in gram weights between reported and true portion sizes for all foods and drinks. Linear regression was used to assess differences between reported and true portion sizes by recall mode.

RESULTS

An overview of key characteristics of the study sample by recall mode (ASA24 compared with AMPM) is provided in Table 1, which shows that the 2 groups were well matched on sex and age range as per the study design. However, the 2 groups differed in terms of race-ethnicity, education, and use of vitamin and/or mineral supplements, as noted above. No significant differences were found between the groups in income range or BMI (Table 1) nor in living situation, reported health status, reported frequency of fast food consumption, physical activity, smoking behavior, or alcohol use (data not shown).

TABLE 1.

Characteristics of the sample for examination of the performance of the ASA24 and AMPM relative to true (observed) intakes (n = 81)¹

	ASA24(n = 40)	AMPM(n = 41)	P value2
Sex (n)			0.91
Male	19	20
Female	21	21
Age range (n)			0.91
20–34 y	11	13
35–54 y	14	13
55–70 y	15	15
Race-ethnicity (n)			<0.01
White	27	14
Nonwhite	13	27
Income (n)			0.19
<$25,000	7	15
$25,000–$99,999	18	16
≥$100,000	13	10
Education (n)			0.03
<College graduate	11	23
College graduate	14	6
>College graduate	13	12
BMI			0.66
Normal	9	12
Overweight	13	15
Obese	16	13
Used vitamin or mineral supplements in past 12 mo			0.04
Yes	32	25
No	7	16

¹AMPM, Automated Multiple-Pass Method; ASA24, Automated Self-Administered 24-hour Recall.

²P values are for recall mode (ASA24 and AMPM) and were calculated by using a chi-square test.

An overview of exact, close, and far matches for all meals combined by recall mode is shown in Table 2. When all matches were considered, no statistically significant difference was found in the proportion of matches between the ASA24 (79.6%) and AMPM (83.2%) groups (P = 0.07). However, when only exact and close matches were considered, there was a small but significant difference, with a lower match rate among the ASA24 group (76.6% compared with 82.5% for the AMPM group; P = 0.01). Accounting for the number of items truly consumed by each participant did not affect the direction or significance of these findings (data not shown).

TABLE 2.

Mean proportion of exact, close, and far matches and exclusions and number of intrusions for all meals combined, by recall mode (ASA24 and AMPM), in relation to true (observed) intakes (n = 81)¹

	ASA24(n = 40)	AMPM(n = 41)	P value2
Exact matches (%)	66.3	74.4	<0.01
Close matches (%)	10.3	8.1	0.01
Far matches (%)	3.1	0.7	<0.01
All matches combined (%)	79.6	83.2	0.07
Exclusions (%)	20.4	16.8	0.07
Intrusions (n)	2.6	1.2	<0.01
Items reported (n)	22.7	23.0	0.91

¹AMPM, Automated Multiple-Pass Method; ASA24, Automated Self-Administered 24-hour Recall.

²P values are for recall mode (ASA24 and AMPM), adjusted for race-ethnicity, and were calculated by using linear regression for matches and exclusions and Poisson regression for intrusions and number of items reported.

The mean proportions of exact, close, and far matches by recall mode for each eating occasion are shown in Table 3. The proportions of all matches combined were similar by recall mode for lunch and dinner but were significantly higher for the AMPM than for the ASA24 for breakfast. For both the ASA24 and AMPM recall groups, lower match rates were observed for lunch than for breakfast and dinner.

TABLE 3.

Mean proportion of exact, close, and far matches and exclusions and number of intrusions, by eating occasion and recall mode (ASA24 and AMPM), in relation to true (observed) intakes¹

	Breakfast2 (8 main items and 7 additions offered)			Lunch (13 main items and 14 additions offered)			Dinner3 (12 main items and 3 additions offered)
	ASA24(n = 38)	AMPM(n = 40)	P value4	ASA24(n = 40)	AMPM(n = 41)	P value4	ASA24(n = 39)	AMPM(n = 41)	P value4
Exact matches (%)	76.5	83.2	<0.01	53.2	61.8	0.02	72.0	81.0	<0.01
Close matches (%)	8.3	10.8	0.75	8.2	5.3	0.06	15.2	10.6	0.05
Far matches (%)	4.2	1.0	0.02	2.1	0.4	0.01	4.0	0.2	<0.01
All matches combined (%)	89.0	95.0	0.01	63.5	67.5	0.16	91.2	91.8	0.60
Exclusions (%)	11.0	5.0	0.01	36.5	32.4	0.16	8.9	8.2	0.60
Intrusions (n)	0.8	0.6	0.04	1.9	1.2	0.03	1.1	0.5	<0.01
Items reported (n)	5.7	6.2	0.64	11.1	11.0	0.67	6.4	6.0	0.24

¹AMPM, Automated Multiple-Pass Method; ASA24, Automated Self-Administered 24-hour Recall.

²Two participants in the ASA24 group and one in the AMPM group did not consume breakfast at the study center and were excluded from this analysis.

³One participant in the ASA24 group did not consume dinner at the study center and was excluded from this analysis.

⁴P values are for recall mode (ASA24 and AMPM), adjusted for race-ethnicity, and were calculated by using linear regression for matches and exclusions and Poisson regression for intrusions and number of items reported.

The proportion of exclusions for all eating occasions combined did not significantly differ between the 2 groups (Table 2). Significantly fewer exclusions at breakfast were found for the AMPM than for the ASA24 (Table 3). The highest proportion of exclusions for both the ASA24 and AMPM recalls was observed for lunch (Table 3). The most common exclusions were additions to or ingredients in multicomponent foods, as shown in Table 4. In the ASA24, 9.3% of primary foods and drinks were omitted compared with 6.6% in the AMPM recalls (P = 0.04). In the ASA24, 36.1% of additions or ingredients were excluded compared with 29.1% in the AMPM (P = 0.10). In terms of specific types of foods and drinks, the rates of exclusions for fruit and vegetables were 32.9% in the ASA24 and 27.1% in the AMPM (P = 0.10); for sweets, snacks, and desserts, the rates were 12.6% and 2.5% for the ASA24 and AMPM, respectively (P = 0.05); and for drinks, they were 5.4% for the ASA24 and 5.5% for the AMPM (P = 0.69).

TABLE 4.

Counts of most common exclusions, by recall mode (ASA24 and AMPM), in relation to true (observed) intakes¹

Items	ASA24	AMPM
Tomatoes	42	26
Mustard	17	17
Green and/or red pepper	16	19
Cucumber	15	14
Cheddar cheese	14	18
Lettuce	12	17
Mayonnaise	9	12

¹AMPM, Automated Multiple-Pass Method; ASA24, Automated Self-Administered 24-hour Recall.

The mean number of intrusions for all eating occasions combined was 2.6 for the ASA24 and 1.2 for the AMPM (Table 2). The correlations between the number of intrusions and the number of items reported were higher for both the ASA24 (0.43) and the AMPM (0.20) than for the correlations with the number of items actually consumed (0.06 and 0.12 for the ASA24 and AMPM, respectively). The number of intrusions was significantly higher for the ASA24 than for the AMPM for each meal considered separately (Table 3). For the ASA24, the mean number of intrusions categorized as internal confabulations (ie, items that were offered but not selected or consumed by the participant) was 1.9 compared with 1.0 for the AMPM (P < 0.01). The mean number of intrusions categorized as external confabulations (ie, items that were not offered at the study center as part of the feeding study) was 0.6 for the ASA24 and 0.2 for the AMPM (P < 0.01).

For descriptive purposes, mean true and reported intakes and the differences for energy and selected nutrients and food groups within each of the recall groups for all foods and beverages reported are shown in Tables 5 and 6. The data in these tables are stratified by sex, given that intakes differ between men and women. An indication of whether the differences between true and reported intakes are significant within each group is provided in Table 7. For the ASA24, significant differences were found between true and reported intakes for calories from fat and vitamin D. For the AMPM, significant differences were found between true and reported intakes for carbohydrates, fiber, calories from fat, vitamin D, sodium, vegetables, and added sugars. The difference between the ASA24 and AMPM groups in the gap between true and reported intakes was significant for vitamin D (P = 0.047) and vegetables (P < 0.01).

TABLE 5.

Mean true and reported energy, nutrient, and food group intakes, by recall mode (ASA24 and AMPM), in women (n = 42)¹

	ASA24			AMPM
	True	Reported	Difference between true and reported (CI)	True	Reported	Difference between true and reported (CI)
Energy (kcal)	1939	1939	0.52 (−236, 237)	2241	2292	−51.6 (−427, 324)
Carbohydrates (g)	245	260	−15.2 (−46.8, 16.4)	300	325	−24.4 (−77.1, 28.3)
Fiber (g)	16.6	17.5	−0.84 (−3.40, 1.71)	19.0	20.7	−1.75 (−5.15, 1.65)
Fat (g)	78.6	70.7	7.90 (−3.54, 19.3)	83.6	80.1	3.51 (−12.5, 19.5)
Calories from fat (%)	36.3	33.2	3.08 (0.89, 5.27)	33.8	31.5	2.37 (0.61, 4.12)
Saturated fat (g)	22.8	23.1	−0.36 (−4.47, 3.76)	23.7	23.0	0.68 (−3.71, 5.07)
Protein (g)	73.7	73.3	0.49 (−7.56, 8.53)	84.4	81.3	3.17 (−10.7, 17.0)
Vitamin A (RAE)	798	815	−17.0 (−186, 152)	765	913	−148 (−440, 144)
Vitamin C (mg)	124	116	8.21 (−17.7, 34.1)	146	151	−5.04 (−40.9, 30.8)
Vitamin D (mg)	2.47	2.95	−0.48 (−1.07, 0.11)	2.20	3.48	−1.28 (−2.10, −0.46)
Folate (μg)	433	450	−17.2 (−70.8, 36.4)	472	492	−19.8 (−82.0, 42.5)
Iron (mg)	14.3	15.3	−0.95 (−2.79, 0.89)	16.1	17.1	−1.03 (−3.31, 1.26)
Magnesium (mg)	263	262	1.13 (−41.1, 43.3)	291	296	−4.99 (−46.5, 36.5)
Calcium (mg)	744	757	−12.9 (−135, 109)	744	811	−66.9 (−188, 54.2)
Sodium (mg)	2890	3372	−482 (−864, −99.2)	3270	3765	−495 (−1005, 15.6)
Fruit (cup equivalent)	1.26	1.33	−0.06 (−0.34, 0.22)	1.63	1.48	0.15 (−0.30, 0.60)
Vegetables (cup equivalent)	1.93	1.71	0.22 (−0.14, 0.58)	1.91	2.33	−0.42 (−0.91, 0.07)
Milk (cup equivalent)	1.34	1.19	0.15 (−0.22, 0.52)	1.22	1.00	0.22 (0.03, 0.41)
Meat (oz equivalent)	3.99	4.20	−0.22 (−1.03, 0.60)	5.06	4.92	0.14 (−1.09, 1.37)
Added sugars (tsp)	13.7	16.0	−2.31 (−6.08, 1.46)	21.6	24.5	−2.87 (−8.46, 2.73)

¹1 cup equivalent = 237 mL; 1 oz equivalent = 30 mL; and 1 tsp equivalent = 5 mL. AMPM, Automated Multiple-Pass Method; ASA24, Automated Self-Administered 24-hour Recall; RAE, retinol activity equivalent.

TABLE 6.

Mean true and reported energy, nutrient, and food group intakes, by recall mode (ASA24 and AMPM), in men (n = 39)¹

	ASA24			AMPM
	True	Reported	Difference between true and reported (CI)	True	Reported	Difference between true and reported (CI)
Energy (kcal)	2721	2458	263 (−244, 770)	2391	2612	−221 (−511, 68.5)
Carbohydrates (g)	353	336	17.3 (−51.6, 86.1)	311	355	−44.4 (−70.4, −18.3)
Fiber (g)	21.3	21.6	−0.35 (−4.65, 3.95)	20.6	23.4	−2.78 (−5.10, −0.46)
Fat (g)	104	86.0	18.2 (−6.19, 42.7)	90.5	96.5	−6.00 (−24.1, 12.1)
Calories from fat (%)	34.5	30.9	3.62 (1.11, 6.12)	33.8	32.4	1.38 (−1.37, 4.13)
Saturated fat (g)	31.5	27.1	4.43 (−2.55, 11.4)	27.7	29.1	−1.43 (−7.77, 4.91)
Protein (g)	106	96.1	10.3 (−7.11, 27.8)	95.7	94.4	1.25 (−12.9, 15.4)
Vitamin A (RAE)	937	892	44.9 (−230, 320)	932	1133	−201 (−411, 8.23)
Vitamin C (mg)	164	156	8.32 (−38.5, 55.1)	156	180	−24.3 (−63.8, 15.3)
Vitamin D (mg)	2.90	3.48	−0.58 (−1.29, 0.14)	3.02	4.82	−1.80 (−2.87, −0.73)
Folate (μg)	566	550	15.9 (−109, 141)	528	581	−53.0 (−124, 18.2)
Iron (mg)	20.1	19.1	1.02 (−3.16, 5.20)	17.1	18.4	−1.31 (−3.11, 0.50)
Magnesium (mg)	356	320	36.0 (−13.3, 85.2)	316	344	−27.5 (−65.3, 10.4)
Calcium (mg)	996	1055	−58.9 (−234, 117)	874	941	−66.9 (−219, 85.3)
Sodium (mg)	3925	4011	−86.4 (−753, 580)	3623	4307	−684 (−1145, −222)
Fruit (cup equivalent)	1.77	1.94	−0.17 (−0.56, 0.22)	1.82	1.82	0 (−0.39, 0.40)
Vegetables (cup equivalent)	2.20	2.07	0.13 (−0.50, 0.76)	2.08	2.56	−0.47 (−0.84, −0.10)
Milk (cup equivalent)	1.68	1.67	0.01 (−0.28, 0.30)	1.48	1.36	0.12 (−0.22, 0.47)
Meat (oz equivalent)	6.28	5.36	0.92 (−0.58, 2.41)	5.72	5.60	0.12 (−1.20, 1.43)
Added sugars (tsp)	24.7	21.8	2.83 (−8.83, 14.5)	19.2	23.9	−4.69 (−8.58, −0.81)

¹1 cup equivalent = 237 mL; 1 oz equivalent = 30 mL; and 1 tsp equivalent = 5 mL. AMPM, Automated Multiple-Pass Method; ASA24, Automated Self-Administered 24-hour Recall; RAE, retinol activity equivalent.

TABLE 7.

Mean difference between true (observed) and reported energy, nutrient, and food group intakes, by recall mode (ASA24 and AMPM), in men and women combined (n = 81)¹

	ASA24		AMPM
	Difference between true and reported (CI)	P value2	Difference between true and reported (CI)	P value2
Energy (kcal)	125 (−136, 386)	0.34	−134 (−364, 95.4)	0.24
Carbohydrates (g)	0.22 (−35.0, 35.5)	0.99	−34.2 (−62.9, −5.44)	0.02
Fiber (g)	−0.61 (−2.94, 1.72)	0.60	−2.25 (−4.24, −0.26)	0.03
Fat (g)	12.8 (0.27, 25.4)	0.05	−1.13 (−12.7, 10.5)	0.85
Calories from fat (%)	3.34 (1.75, 4.92)	<0.01	1.89 (0.34, 3.44)	0.02
Saturated fat (g)	1.92 (−1.93, 5.76)	0.32	−0.35 (−4.02, 3.32)	0.85
Protein (g)	5.17 (−3.82, 14.2)	0.25	2.24 (−7.22, 11.7)	0.63
Vitamin A (RAE)	12.4 (−138, 163)	0.87	−174 (−347, −0.56)	0.05
Vitamin C (mg)	8.3 (−16.5, 33.0)	0.50	−14.4 (−40.0, 11.2)	0.26
Vitamin D (mg)	−0.53 (−0.97, −0.09)	0.02	−1.53 (−2.18, −0.89)	<0.01
Folate (μg)	−1.44 (−64.0, 61.1)	0.96	−36.0 (−81.3,. 9.4)	0.12
Iron (mg)	−0.01 (−2.14, 2.11)	0.99	−1.16 (−2.56, 0.24)	0.10
Magnesium (mg)	17.7 (−13.6, 48.9)	0.26	−16.0 (−43.1, 11.2)	0.24
Calcium (mg)	−34.7 (−135.2, 65.7)	0.49	−66.9 (−159, 25.5)	0.15
Sodium (mg)	−294 (−656, 68.5)	0.11	−587 (−918, −256)	<0.01
Fruit (cup equivalent)	−0.11 (−0.34, 0.11)	0.32	0.08 (−0.21, 0.37)	0.58
Vegetables (cup equivalent)	0.18 (−0.16, 0.51)	0.29	−0.45 (−0.74, −0.15)	<0.01
Milk (cup equivalent)	0.08 (−0.14, 0.31)	0.46	0.17 (−0.01, 0.36)	0.07
Meat (oz equivalent)	0.32 (−0.49, 1.13)	0.43	0.13 (−0.73, 0.99)	0.76
Added sugars (tsp)	0.13 (−5.52, 5.78)	0.96	−3.76 (−7.05, −0.46)	0.03

¹1 cup equivalent = 237 mL; 1 oz equivalent = 30 mL; and 1 tsp equivalent = 5 mL. AMPM, Automated Multiple-Pass Method; ASA24, Automated Self-Administered 24-hour Recall; RAE, retinol activity equivalent.

²P values were calculated by using linear regression, were adjusted for race-ethnicity, and indicate whether differences between true and reported intakes within each group (ASA24 and AMPM, respectively) are different from zero.

For foods for which a match (exact, close, or far) was reported, the mean differences between true portion size and reported portion size were −3.7 g for the ASA24 and −11.8 g for the AMPM (absolute differences that consider magnitude but not direction of the differences between true and reported portions were 40.5 g for the ASA24 and 39.8 g for the AMPM; P = 0.42). The differences for primary foods and drinks were −5.4 g for the ASA24 and −14.4 g for the AMPM, whereas the differences for additions or ingredients in multicomponent items were 0.60 g for the ASA24 and −6.2 g for the AMPM.

DISCUSSION

ASA24 performed well relative to a measure of true intakes in terms of the proportion of items consumed for which matches were reported and the relations with true energy, nutrient, and food group intakes and portion sizes. Given the recognition of misreporting in self-reported dietary intake instruments (1–3, 7), perfect correspondence was not expected; however, both the ASA24 and AMPM captured ∼80% of the foods and drinks actually consumed. No statistically significant difference in the overall proportion of foods and drinks for which a match was reported was found between the 2 recall modes. When only exact and close matches were considered, the AMPM performed somewhat better.

The overall proportion of exclusions did not differ by recall mode. A high proportion of the exclusions were additions to or ingredients in multi-ingredient foods, many of which have little effect on estimates of overall intake and that were consumed in small amounts. Similarly, previous research with children has shown high rates of omission for condiments (15). This pattern may explain the lower proportion of matches observed for lunch, which consisted of sandwiches and salads with a number of secondary ingredients (eg, cheese, vegetables, and spreads). The high rate of exclusions for additions and ingredients is also consistent with our finding that a relatively high proportion of fruit and vegetables was omitted because these were often part of multicomponent foods (eg, lettuce and tomato on sandwiches). We did not find that sweets, snacks, and desserts were often omitted, in contrast with other research that has suggested a tendency toward underreporting these foods (16, 17), perhaps because we did not assess true intake for eating occasions other than the 3 meals offered.

A greater number of intrusions was observed in the ASA24 than in the AMPM, perhaps reflecting exploration of the software among first-time ASA24 users and items added and not deleted when a better match for what was consumed was found. For both recall modes, intrusions may have been artificially inflated by the study design because respondents may have included foods and drinks consumed outside of the study in combination with those consumed at study center meals.

The examination of differences in the gap between estimates of energy, nutrient, and food group intakes and portion sizes based on true compared with reported consumption suggests that both instruments perform fairly well. However, it must be noted that this study was powered for the examination of the proportions of items consumed for which respondents reported a match. Our statistical power for the analyses of energy, nutrient, and food group intakes and portion size was limited by the sample size, as indicated by the wide CIs around the mean difference estimates for some dietary components. Nonetheless, examination of the mean true and reported intakes suggests reasonable agreement between the two for both recall modes for most dietary components examined. Further insights into the correspondence between true intakes and those reported via the ASA24 will come from a biomarker-based validation study that is now under way.

The elimination of the interviewer in the ASA24 offers substantial cost savings to researchers, and our results indicate that it does not result in a greatly reduced accuracy of recall. However, the lack of an interviewer may pose some challenges. Participants completing an interviewer-administered recall have the opportunity to report what they consumed, including an estimate of portion size, to a trained interviewer, who serves to clarify and interpret reports before data entry; items are recorded as specified to be later coded by trained coders. In the ASA24, respondents search for the items that they consumed or browse through a predetermined list of foods and drinks, with the help of an avatar that provides guidance during transitions from one part of the recall to the next; items are autocoded based on the main food or drink reported and responses to detail questions, including specification of portion size based on a finite set of images.

Examination of the ASA24 data suggests potential issues associated with self-administration. For example, an assessment of close and far matches indicates a tendency for some respondents to choose items closer to the top of the food and drink lists or search results when browsing or searching, respectively. For example, in the version of ASA24 used in this study, a search for “water” returned several results: “carbonated water” appeared toward the top of the list and “water (bottled)” appeared toward the bottom. Although only still bottled water was offered in the study, multiple ASA24 respondents reported carbonated water, which suggested that they chose a readily available option rather than scrolling through the list to find the best possible match. This type of issue may explain the lower rate of exact and close matches in the ASA24 than in the AMPM.

Another potential challenge to ASA24 respondents is not being able to locate the item consumed, either because it does not exist within the FNDDS and thus in the ASA24, or because the respondent's browse or search strategy does not uncover it. As noted above, there is a “match not found” option, which prompts the respondent to enter a description and answer questions about the type and kind of food or drink and results in the assignment of a default code. As an example of the potential implications of the use of the “match not found” pathway, one respondent consumed only the filling of the apple pie offered at dinner, leaving the crust on the plate. This respondent was randomly assigned to the AMPM group and reported apple pie filling to the interviewer, which was later considered an exact match with the true intake. Had this respondent been randomly assigned to complete the ASA24, which does not include apple pie filling as an option, he or she may have turned to the “match not found” pathway, with a default food code assigned depending on the subsequent details (eg, type of food, such as fruit or dessert) provided. This code may or may not have been an exact match for the true intake. In this study, a small number of codes assigned to items entered by using free text, including the “match not found” pathway, were corrected to better reflect the text entries. An examination of the implications of reviewing and recoding free text entries within the ASA24 is forthcoming.

Since the data collection for this study was completed, a new version of ASA24 that includes enhancements to address usability issues suggested by our results and identified by current and previous researcher users has been released. The current editing tools have been moved to be adjacent to each food and drink reported, which makes it easier to delete or edit a prior entry. Furthermore, a smarter search uses data on the eventual selections made by users entering similar search terms to provide suggestions and order search results. A pending adaptation for mobile devices will increase the feasibility of the tool for use in a wide range of studies.

The sample for the current study consisted of paid volunteers, and it is possible that the results are somewhat optimistic for studies of groups that are less motivated to participate or those with low literacy or computer skills. Furthermore, the ASA24 requires access to high-speed Internet, which may limit its use in certain circumstances. An assessment of the feasibility of using ASA24 in a large community-based sample is forthcoming.

As the implications of measurement error in self-reported dietary intake data become better understood (1, 2, 18, 19), the demand for assessment tools that can provide high-quality intake data at low cost is increasing. The ASA24 was designed to meet this challenge. Since a beta version was released in late 2009, the ASA24 has been used to collect more than 120,000 24-h recalls. Preliminary reviews of ASA24 system data suggest acceptable face validity in terms of energy, nutrient, and food group estimates that are consistent with data from NHANES (5). This study provides further evidence of the ASA24’s validity. Given the substantial cost savings that the ASA24 offers, it has the potential to make a significant contribution to a range of research initiatives.