Comparison of Interviewer-Administered and Automated Self-Administered 24-Hour Dietary Recalls in 3 Diverse Integrated Health Systems

Abstract

Twenty-four-hour dietary recalls provide high-quality intake data but have been prohibitively expensive for large epidemiologic studies. This study's goal was to assess whether the web-based Automated Self-Administered 24-Hour Recall (ASA24) performs similarly enough to the standard interviewer-administered, Automated Multiple-Pass Method (AMPM) 24-hour dietary recall to be considered a viable alternative. In 2010–2011, 1,081 adults from 3 integrated health systems in Detroit, Michigan; Marshfield, Wisconsin; and Kaiser-Permanente Northern California participated in a field trial. A quota design ensured a diverse sample by sex, age, and race/ethnicity. Each participant was asked to complete 2 recalls and was randomly assigned to 1 of 4 protocols differing by type of recall and administration order. For energy, the mean intakes were 2,425 versus 2,374 kcal for men and 1,876 versus 1,906 kcal for women by AMPM and ASA24, respectively. Of 20 nutrients/food groups analyzed and controlling for false discovery rate, 87% were judged equivalent at the 20% bound. ASA24 was preferred over AMPM by 70% of the respondents. Attrition was lower in the ASA24/AMPM study group than in the AMPM/ASA24 group, and it was lower in the ASA24/ASA24 group than in the AMPM/AMPM group. ASA24 offers the potential to collect high-quality dietary intake information at low cost with less attrition.

Epidemiologists studying the effects of diet on chronic diseases in large prospective studies have relied mostly on food frequency questionnaires because of their low cost and relatively low participant burden. However, multiple studies have shown that more quantitative methods, such as 24-hour dietary recalls (24HRs) and food records, are more accurate at assessing short-term absolute intakes of energy, protein, potassium, and sodium than are food frequency questionnaires (1). Both traditional interviewer-administered 24HRs and food records have been prohibitively expensive because of the need for trained interviewers and coding of reported foods and drinks, respectively (2). The cost is compounded if multiple administrations to account for day-to-day variation are planned.

The self-administered automated 24HR circumvents the expense of both administration and coding of dietary intake data, while providing a platform for multiple administrations. The web-based Automated Self-Administered 24-Hour Recall (ASA24), developed by the National Cancer Institute under contract with Westat, a research firm in Rockville, Maryland, and first released in 2009 (3), was modeled after the US Department of Agriculture's state-of-the-art, interviewer-administered Automated Multiple-Pass Method (AMPM). The AMPM is used in What We Eat in America, the dietary interview component of the National Health and Nutrition Examination Survey (4).

Briefly, ASA24 guides the respondent through the completion of a 24HR using an online dynamic user interface (3). A meal-based quick list is utilized, in which respondents select an eating occasion and report the time of consumption prior to reporting the foods and drinks consumed. Respondents report foods and drinks by browsing or searching a list of user-friendly terms derived from foods and beverages reported in What We Eat in America. Next, a meal gap review queries the respondent about any foods or drinks consumed between reported eating occasions separated by 3 or more hours. This is followed by a detail pass, which administers detailed questions about food preparation, portion size (using images to assist with estimation), and additions. Respondents then are asked to think about a list of frequently forgotten items and to complete a final review of all foods and drinks reported by meal to confirm that they have included everything they ate and drank the previous day. All foods and beverages reported by the respondent map directly to foods in the US Department of Agriculture's Food and Nutrient Database for Dietary Studies, version 4.1 (5), and its corresponding MyPyramid Equivalents Database (6, 7). Thus, all foods reported are automatically coded.

As of September 2014, ASA24 had been used in over 1,265 studies to collect about 173,000 days of recalls. However, information about the validity and feasibility of the tool is needed to inform its use in and interpretation for research. Early work found that the ASA24 tool produced dietary intake estimates that were consistent with data from the National Health and Nutrition Examination Survey (3). Additionally, a controlled feeding study of 81 participants found that, compared with observed intake, ASA24 performed nearly as well as the interviewer-administered AMPM in terms of matches (i.e., number of foods matching observed intake), exclusions (i.e., number of foods not reported), and intrusions (i.e., number of foods not consumed but reported) (8).

The feasibility study reported here was a large field trial of ASA24 in 3 different regions of the United States conducted among diverse adult participants who were asked to complete two 24HRs over about 2 months. Participants were randomly assigned to 4 protocols consisting of different combinations and orders of administration of 2 types of 24HR, a web-based self-administered ASA24 and an interviewer-administered AMPM recall. The overall goal of the study was to assess whether ASA24 performs similarly enough to AMPM to be considered a viable alternative administration mode. The research questions addressed by this study are as follows: 1) How do reported nutrient and food intakes compare between ASA24 and AMPM? 2) What are the completion and attrition rates for each protocol? 3) Among participants completing both an ASA24 and an AMPM recall, is there a preference for one type of recall over the other?

METHODS

Study design

The Food Reporting Comparison Study (FORCS) aimed to obtain 2 nonconsecutive unannounced 24HRs from each of about 1,000 adult participants of diverse age and race/ethnicity. Enrollees were randomly assigned to 1 of 4 study groups to control for the effect of administration order and to enable an examination of attrition following completion of the first recall: group 1 (2 self-administered ASA24 recalls); group 2 (2 telephone interviewer–administered AMPM recalls); group 3 (1 ASA24 followed by 1 AMPM); and group 4 (1 AMPM followed by 1 ASA24).

Sample

The sample was drawn from 3 integrated health systems in both urban and rural regions of the country and with different demographics: Marshfield Clinic, Security Health Plan in Wisconsin; Health Alliance Plan of the Henry Ford Health System in Detroit, Michigan; and Kaiser-Permanente Northern California in California. Each site was asked to identify current users of its Internet services and to draw a pool of age-eligible users into sampling strata defined by sex, age (20–34 years, 35–54 years, and 55–70 years), and 3 race/ethnicity categories (non-Hispanic white, non-Hispanic black, Hispanic). A quota-sampling plan, which varied by site, was designed to provide similar numbers by age and sex strata across the 3 sites.

Recruitment

Each potential participant was sent an invitation letter on a health center–specific letterhead, which explained that the purpose of the study was “to learn how best to collect information on what Americans eat.” The letter included a link to the Food Reporting Comparison Study website that provided additional study information and an online consent form to give permission to study staff to contact the potential participant by telephone to assess eligibility. Participants provided contact information (best time to reach, phone numbers, and e-mail address). Eligibility criteria included age 20–70 years, access to high-speed Internet, ability to read English, not pregnant, not on a weight-loss diet, and no history of bariatric surgery for weight loss.

Survey data collection

All dietary recalls were conducted without prior notification to avoid changes in diet on the reporting day (i.e., reactivity). For AMPM recalls, portion size aids (measuring cups and spoons, ruler, food model booklet) identical to those used in What We Eat in America were mailed to participants, and interviewers, trained using the What We Eat in America protocol, phoned participants and administered the recall at that time if possible. For ASA24 recalls, on the assigned day the recall was to be completed, an e-mail was sent asking participants to visit the ASA24 website to complete the recall; 2 automated phone calls (1 in the morning and 1 in the evening) were made the same day to notify participants to check their e-mail. If the participant, for either AMPM or ASA24, was unable to complete the recall on the assigned day, up to 5 additional attempts were made later, again, unannounced. A second unannounced recall was conducted 5–7 weeks after completion of the first recall. Following this, participants were asked to complete an online questionnaire that included demographic, behavioral, and (for those who completed 1 of each type of recall) preference questions.

Respondents received up to $52 in incentives for participation in the Food Reporting Comparison Study. Procedures for incorporating incentives varied by site. At 2 sites (Marshfield Clinic and the Henry Ford Health System), initial invitation letters contained a $2 incentive. Following eligibility determination, those who consented received $5 ($7 for Kaiser-Permanente Northern California), $15 upon completing the first 24HR, and $30 upon completing the second 24HR.

Variables

The Food and Nutrient Database for Dietary Studies, version 4.1, and its corresponding MyPyramid Equivalents Databases, version 2.0 (6, 7), were used to compute reported food and nutrient intakes from all recalls. Of 98 nutrient and food group variables available from AMPM and ASA24 analytical files, 20 nutrients and food groups were selected for analysis, consistent with an earlier validity study (8). These included energy, macronutrients, percentage of energy from fat, fiber, selected micronutrients, and various food groups defined by federal dietary guidance, including fruits, vegetables, dairy, meat, and added sugars. Although supplement use was assessed in both types of recalls, the contributions of supplements to total intakes are not considered in these analyses. Sex, age, and race/ethnicity were self-reported on the initial screening questionnaire and, thus, have no missing data. Other demographic characteristics and behaviors were reported in a final questionnaire, completed by 88% of the participants. For those who completed both an ASA24 and an AMPM, preference questions queried which of the 2 types of recalls was “easier to report,” which they thought “gave more accurate information,” and which “provided them with more control over when they reported.” Response categories were ASA24, AMPM, or both the same. A final question forced a preference judgment between ASA24 and AMPM.

Analysis

The distributions of participants by the design variables of sex, age, and race/ethnicity were examined to determine the success of balancing of participants across the 4 protocols. Additionally, the distributions of these and other (nondesign) variables collected from the demographic and behavioral characteristics questionnaire (education status, household income, use of nutritional supplements, weight, and height) by type of first recall (ASA24 or AMPM) were examined by using χ² statistics, with the significance level set at P ≤ 0.05.

Similarities between nutrient and food intakes by recall type were examined via equivalence testing, using all reports of each type of recall. For superiority testing, the null hypothesis is that 2 means are equal. In contrast to the more commonly used superiority testing, the null hypothesis in equivalence testing is that the 2 means are dissimilar. For statistical reasons involving test power, equivalence testing does not set up the null hypothesis as equality, but rather as a bound on the size of a meaningful difference in means. We used the Two One-Sided Tests Method with a bound equal to 20% of the observed AMPM mean. That is, the 2 recalls are judged equivalent if the null hypothesis (difference ≥20%) is rejected. The observed difference could be less than 20% (e.g., 18%) but still not small enough to reject the hypothesis that the “real” difference is ≥20%. To account for the fact that some individuals contributed an observation to both means, while others contributed 2 recalls to only 1, standard errors of the differences in means were estimated with the delete-one jackknife procedure, operating at the individual level. The Benjamini-Hochberg step-up procedure (9) was used to adjust for multiple comparisons at a 5% false discovery rate. The procedure ordered the nominal P values from the collection of 40 equivalence tests from smallest to largest, and the kth test was judged significant if P ≤ 0.05 × (k/n).

We analyzed preferences for ASA24 versus AMPM and whether these preferences varied by age, race/ethnicity, household income, education, body mass index classification, and use of vitamin/mineral supplements. We examined attrition rates by protocol group. Whether attrition was associated with the type of recall encountered first was also examined. These analyses were performed by using traditional superiority testing, because ASA24 was hypothesized to be less burdensome than AMPM and was therefore expected to perform better on those metrics.

Analyses were conducted by using SAS, version 9.3, software (SAS Institute, Inc., Cary, North Carolina).

All study procedures and informed consent forms were approved by the respective institutional review boards of the National Cancer Institute, Westat, Marshfield Clinic, the Henry Ford Health System, and Kaiser-Permanente Northern California, as well as the US Office of Management and Budget.

RESULTS

Over the 3 sites, about 10% of those invited by letter accessed the study website. Of those who accessed the website, 74% were successfully reached by telephone, were eligible, consented, and enrolled. A total of 1,081 individuals (512 men and 569 women) provided at least 1 complete recall and, of those, 90% provided 2 recalls. For those who reported 2 recalls, the potential effect of administration order was examined for 4 dietary components (energy, total fat, percentage energy from fat, and fruits and vegetables (cup equivalents)) by sex. No significant order effect was found, and thus the order variable was not considered in further analyses.

The distribution of the design variables of age, sex, and race/ethnicity did not differ across protocols (data not shown). Sample characteristics by type of recall first administered are shown in Table 1. There were no significant differences between those completing AMPM and ASA24 for the design variables or other potentially relevant variables (e.g., household income; education status). This indicates that the stratified random assignment worked well in balancing the protocols across the designated variables of interest. Therefore, further analysis did not include these variables as potential confounders.

Table 1.

Frequency Distribution for Demographic and Behavioral Characteristics of Participants^a by Recall Type and Sex, Food Reporting Comparison Study, 2011–2012

	Men				Women
	No.	%	AMPM, no.	ASA24, no.	No.	%	AMPM, no.	ASA24, no.
Total	512		250	262	569		289	280
Site
Henry Ford Health System	188	36.7	90	98	206	36.2	105	101
Kaiser-Permanente Northern California	156	30.5	78	78	176	30.9	93	83
Marshfield Clinic	168	32.8	82	86	187	32.9	91	96
Age, yearsb
20–34	225	43.9	57	68	213	37.4	107	106
35–54	207	40.4	105	102	156	27.4	79	77
55–70	170	33.2	88	92	200	35.1	103	97
Race/ethnicityb,c
White	215	42.4	104	111	258	45.7	126	132
Black	170	33.5	82	88	197	34.9	101	96
Hispanic	122	24.1	61	61	110	19.5	61	49
Assigned study protocold
ASA24/ASA24	127	24.8		127	138	24.3		138
AMPM/AMPM	120	23.4	120		140	24.6	140
ASA24/AMPM	135	26.4		135	142	25.0		142
AMPM/ASA24	130	25.4	130		149	26.2	149
Household incomeb,e
<$25,000	39	9.4	18	21	84	17.5	43	41
$25,000–$99,999	287	69.0	132	155	320	66.8	156	164
≥$100,000	90	21.6	44	46	75	15.7	39	36
Educationb
≤ High school	55	13.0	26	29	49	10.0	24	25
Some college	175	41.4	83	92	192	39.3	88	104
≥ College graduation	193	45.6	89	104	247	50.6	131	116
Body mass indexb,f
Underweight/normal (<25.0)	110	21.5	53	57	193	32.8	96	97
Overweight (25.0–29.9)	224	43.8	116	108	183	31.1	89	74
Obese (≥30.0)	178	34.8	81	97	213	36.2	104	109
Used vitamin or mineral supplements during the past 12 monthsb
Yes	285	67.1	132	153	362	69.9	181	181
No	140	32.9	67	73	156	30.1	68	68

Abbreviations: AMPM, Automated Multiple-Pass Method; ASA24, Automated Self-Administered 24-Hour Recall.

^a Using only the first 24-hour dietary recall.

^b There were no significant differences (χ²; P > 0.05) between the AMPM and ASA24 groups in the distributions for age, race/ethnicity, household income, education, body mass index, or use of vitamin or mineral supplements during the past 12 months.

^c Race/ethnicity from screener. Nine respondents who reported other or combined race/ethnicity do not appear.

^d Protocols: AMPM/ASA24 (AMPM first, ASA24 second); AMPM/AMPM (AMPM both recalls); ASA24/ASA24 (ASA24 both recalls); ASA24/AMPM (ASA24 first, AMPM second).

^e Frequencies for household income, education, and use of supplements do not add up to the total sample because of missing data.

^f Body mass index: weight (kg)/height (m)².

The overall sample was diverse, as intended. Slightly more women (53%) than men participated, and the youngest age group participated less (31%) than the middle (34%) and older (35%) age groups. The total sample included 473 (44%) non-Hispanic whites, 367 (34%) non-Hispanic blacks, and 232 (21%) Hispanics. Of those completing the demographic questionnaire, relatively few had no more than a high school education (11%), and few had household incomes less than $25,000 (14%). Using body mass index standards and self-reported height and weight, about a third (36%) were considered overweight, and a third (36%) obese. Seventy percent reported taking vitamin or mineral supplements in the past year.

Comparison of reported nutrient and food group intakes by recall type for each sex is presented in Table 2. Overall, the estimated values between the 2 recall types were similar; over the 20 nutrients/food groups examined in each sex, the average relative mean difference was 6.0%. Mean energy intakes by recall type for men (2,425 kcal for AMPM vs. 2,374 kcal for ASA24) and women (1,876 kcal vs. 1,906 kcal) and percentage energy from fat for men (34.9% for AMPM vs. 34.5% for ASA24) and women (33.8% for AMPM vs. 35.8% for ASA24) (Table 2) were found equivalent at the 20% bound. Nutrients/food groups with the largest differences were meat (ounce equivalents) and fruit (cup equivalents) among both men and women and saturated fat (g) and sodium (mg) among women. Equivalence testing controlling for the false discovery rate showed that, of all the comparisons made (20 nutrients/food groups and 2 sexes), 87% were judged equivalent at the a priori 20% bound. At this level, only total fruit and total meat (men and women) and vitamin D (men only) were not judged equivalent. Web Table 1 (available at http://aje.oxfordjournals.org/) presents parallel analyses for these variables expressed as nutrient densities. The results are similar to those found in Table 2. Web Tables 2 and 3 provide further descriptive information about the distributions of dietary variables by recall type. Although no statistical testing was performed, the distributions appear to be nearly identical for each recall type, for both men and women.

Table 2.

Comparison of Mean^a Reported Intakes^b Between AMPM and ASA24 by Sex, Food Reporting Comparison Study, 2011–2012

Nutrient/Dietary Component	Males			Females
	Mean AMPM (463 Recalls)	Mean ASA24 (479 Recalls)	Difference as % of AMPM	Mean AMPM (549 Recalls)	Mean ASA24 (532 Recalls)	Difference as % of AMPM
Energy, kcal	2,425	2,374	2.12	1,876	1,906	1.59
Carbohydrates, g	281	264	6.04	228	220	3.34
Fiber, g	20.7	20.3	−2.20	16.5	17.0	2.93
Fat, g	96.2	93.1	−3.16	71.5	76.7	7.27
Energy from fat, %	34.9	34.5	−1.00	33.8	35.8	5.94
Saturated fat, g	31.4	30.0	−4.49	23.0	25.6	11.17
Protein, g	100.0	103.2	3.28	74.5	77.5	4.04
Vitamin A, RAE	764	769	0.66	644	687	6.66
Vitamin C, mg	109	99	−8.91	94	92	−2.53
Vitamin D, IU	5.67	5.18	−8.63c	4.04	4.26	5.65
Folate, µg	495	478	−3.50	392	375	−4.23
Iron, mg	18.7	18.4	−1.94	14.2	13.8	−2.52
Magnesium, mg	371	351	−5.32	291	286	−1.40
Calcium, mg	1,092	1,039	−4.85	907	916	0.94
Sodium, mg	3,932	4,219	7.29	2,966	3,319	11.90
Fruits, cup equivalentsc	1.45	1.26	−12.60d	1.25	1.12	−10.47d
Vegetables, cup equivalentsc	1.96	1.89	−3.60	1.67	1.79	6.97
Dairy, cup equivalentsc	1.73	1.78	2.91	1.49	1.63	9.16
Meat, ounce equivalentsc	2.28	2.72	19.62d	1.25	1.62	29.90d
Added sugars, teaspoonsc	16.5	15.1	−8.66	14.2	14.3	0.50

Abbreviations: AMPM, Automated Multiple-Pass Method; ASA24, Automated Self-Administered 24-Hour Recall; RAE, retinol activity equivalent.

^a Means estimated by using all reports of each type of recall.

^b Values for 2 recalls are excluded because of extreme values.

^c One cup equivalent of fruits and vegetables is 1 cup of raw or cooked fruit or vegetable; ½ cup of dried fruit or vegetables; 1 cup of fruit or vegetable juice; or 2 cups of leafy salad greens. One cup equivalent of dairy is 1 cup of milk or yogurt; 1½ ounces of natural cheese; or 2 ounces of processed cheese. One ounce equivalent of meat (i.e., protein) is 1 ounce of meat, poultry, or seafood; 1 egg; 1 tablespoon of peanut butter; ½ ounce of nuts or seeds; or ¼ cup of cooked dried beans or peas. One teaspoon of added sugar is 1 teaspoon (4 g) of table sugar.

^d Not equivalent at 20% bound.

Preference was reported among those who completed both AMPM and ASA24 (Figures 1 and 2). Overall, when asked to choose between the 2 types of recalls, about 70% preferred ASA24 over AMPM (Figure 1A; Figure 2A). This overall preference was statistically significant for all demographic groups and behavioral variables examined. When preferences were examined across subgroups of these variables, a significant age trend was found among women; preference for ASA24 over AMPM decreased with age (88% vs. 68% for youngest vs. oldest) (Figure 2B). No trends were found for the other variables (Figure 1B and 1C; Figure 2C). ASA24 also was preferred over AMPM for being easier to report food intake, perceived higher accuracy in intake reported, and more control over time of responding (data not shown).

Figure 1.

Preferences for the Automated Multiple-Pass Method (AMPM) versus the Automated Self-Administered 24-Hour Recall (ASA24): men, Food Reporting Comparison Study, 2011–2012. A) Relationship for the total sample; B) relationships by age group; C) relationships by educational level. No differences across demographic groups were significant (χ²; P < 0.05). Black bars, AMSA; hatched bars, ASA24.

Figure 2.

Preferences for the Automated Multiple-Pass Method (AMPM) versus the Automated Self-Administered 24-Hour Recall (ASA24): women, Food Reporting Comparison Study, 2011–2012. A) Relationship for the total sample; B) relationships by age group; C) relationships by educational level. There was a significant difference (χ²; P < 0.05) in preferences between age groups but not educational levels. Black bars, AMSA; hatched bars, ASA24.

Overall, 90% of those who completed a first recall also completed a second recall. This did not vary across sex, race/ethnicity, education status, household income, or weight status (data not shown). However, there is some evidence of differing attrition rates by recall protocol (Table 3). Overall, those randomized to the AMPM first/ASA24 second protocol had the highest attrition (14.7%), and those randomized to the ASA24 first/ASA24 second protocol had the lowest attrition (5.7%). Although not always statistically significant, this trend was seen for most demographic groups.

Table 3.

Attrition Rate^a Among Participants Reporting the First Recall, by Protocol and Demographic Characteristics, Food Reporting Comparison Study, 2011–2012

Demographic Characteristic	Total No.	Protocolb and Attrition Rate, %				P Valuec
		AMPM/ASA24 (n = 272)	AMPM/AMPM (n = 256)	ASA24/ASA24 (n = 262)	ASA24/AMPM (n = 262)	4-Way Comparison	AMPM First vs. ASA24 First	Both AMPM vs. Both ASA24
Total	1,052	14.7	10.6	5.7	9.9	0.0083	0.0093	0.0444
Sex
Male	494	16.7	13.8	6.4	14.2	0.0854	0.0977	0.0555
Female	558	13.0	7.9	5.1	5.9	0.0635	0.0310	NS
Age group, years
20–34	328	15.5	18.4	3.5	17.3	0.0152	0.0727	0.0018
35–54	354	11.7	8.2	5.8	9.0	NS	NS	NS
55–70	370	17.0	6.3	7.9	4.4	0.0124	0.0606	NS
Race/ethnicityd
White	465	11.9	10.3	3.3	12.7	0.0508	NS	0.0326
Black	358	15.9	10.8	6.7	10.3	NS	NS	NS
Hispanic	220	17.2	11.1	10.4	3.7	NS	0.0735	NS

Abbreviations: AMPM, Automated Multiple-Pass Method; ASA24, Automated Self-Administered 24-Hour Recall; NS, not significant; RR, relative risk.

^a Attrition rate = 1 − (response rate) or [no. (first recall) – no. (second recall)]/no. (first recall).

^b Protocols: AMPM/ASA24 (AMPM first, ASA24 second); AMPM/AMPM (AMPM both recalls); ASA24/ASA24 (ASA24 both recalls); ASA24/AMPM (ASA24 first, AMPM second).

^c Significance tests using χ². Four-way comparison: test differences in attrition over all 4 groups. AMPM first and ASA24 first: tests difference in attrition rates between the groups that had AMPM first and the groups that had ASA24 first. AMPM/AMPM vs. ASA24/ASA24: tests difference in attrition rates between the group that had AMPM twice and the group that had ASA24 twice.

^d Race/ethnicity defined from screener. Those with multiple or other race are excluded.

Overall, and for women, experiencing AMPM first was associated with higher attrition than experiencing ASA24 first. This association was not statistically significant for men and for subgroups defined by age and race/ethnicity. Finally, attrition in the groups assigned to “pure” protocols (ASA24/ASA24 vs. AMPM/AMPM) was compared. Overall, the group assigned to the ASA24/ASA24 protocol had significantly lower attrition than the group assigned to the AMPM/AMPM protocol (6% vs. 11%). This difference was borderline significant for men (6% vs. 14%) and was statistically significant among participants aged 20–34 years (4% vs. 18%) and among whites (3% vs. 10%). Notably, the difference in attrition rates between the ASA24/ASA24 and the AMPM/AMPM protocols was smaller for blacks (7% vs. 11%) and nonexistent for Hispanics (10% vs. 11%).

DISCUSSION

Numerous studies have shown that short-term dietary assessment methods, the 24HR and in some studies the food record (10), provide more accurate dietary intake data for energy, protein, potassium, and sodium than does the food frequency questionnaire (1). The principal barriers to adopting these short-term methods in large-scale studies have been the cost of interviewer administration for the 24HR and the cost of coding for the food record. ASA24 addresses both of these barriers by eliminating the need for interviewers and coders.

The interviewer-administered AMPM is considered the optimal method for obtaining a 24HR because of its numerous probes and standardization of interviewer administration and has been validated against recovery biomarkers (11). We compared ASA24 with AMPM to learn whether this self-administered automated 24HR produces comparable dietary intake estimates. In a diverse sample of adults 20–70 years of age from 3 different geographical areas, we found that energy intake estimates between the 2 recall methods for men or women were equivalent. In addition, the intake estimates for nearly all nutrients and food groups examined were equivalent between ASA24 and AMPM at a 20% bound. We also found that participants reported feeling their responses were more accurate using theASA24, compared with using the AMPM.

Feasibility is an important consideration in the choice of dietary assessment method in research. In our study, response to an initial invitation letter, even with incentive payments, was about 10%. This response rate would be unacceptable in a survey aiming to provide representative estimates regarding the diets of a population but is not atypical in the establishment of large cohort studies (12). Attrition also can be a concern, especially when considering collection of multiple 24HRs. In our study, which included financial incentives, attrition between a first and second recall completed 5–7 weeks later was only 10%.

Another feasibility consideration is acceptance of the tool. A substantial proportion of study participants preferred ASA24 over the telephone interviewer–administered 24HR. This preference is reflected in an overall lower attrition rate for a second ASA24 recall compared with a second AMPM recall. In addition, those completing ASA24 first had higher completion rates of the second recall than those completing AMPM first. It should be noted, however, that an eligibility requirement for participation in this study was prior provision of e-mail contact information to the health plans from which participants were recruited, as well as access to a high-speed Internet connection. Thus, the noted preference for ASA24 among Food Reporting Comparison Study participants may be less apparent for studies with less “connected” participants.

Although ASA24 offers a feasible means of collecting 24HRs in large-scale research, there are several potential constraints. Reading and computer literacy are required; thus, like other studies requiring self-administration of tools, low-literacy populations will likely be underrepresented in studies using only ASA24. An assisted or interviewer-administered ASA24 is one possible remedy. The tool does, however, meet minimum federal compliance standards for accessibility to those with disabilities.

Currently, ASA24 requires a computer with a high-speed connection to the Internet. The prevalences of households with computers and households accessing the Internet at home were 79% and 75%, respectively, in 2012 (13). However, substantially fewer Hispanic and black households reported computer use at home (14). In addition, the availability of high-speed Internet varies tremendously across the country and is particularly problematic in some rural areas, for example, in Appalachia (14). The ASA24 is being adapted for tablets and smart phones, allowing researchers to deploy the instrument to households without computers, thus ameliorating some of the access disparities. However, disparities due to connectivity issues remain.

Language may be an important consideration in the use of ASA24. Our study was conducted in English; ASA24 is also available in Spanish. However, in the United States, AMPM recalls administered in the National Health and Nutrition Examination Survey are conducted in numerous languages. A 24HR interview can be conducted in any language if resources permit. The use of ASA24 in populations outside the United States requires attention to both language and food supply, as well as to its nutritional composition, commonly used names for the foods and drinks, usual portion sizes, and food preparation methods. The food composition database used in both ASA24 and AMPM is based on foods consumed in the United States. Canadian researchers already have adapted ASA24 for use in Canada, and researchers in Australia are working in collaboration with the National Cancer Institute to adapt ASA24 for that country.

Participant burden is an important consideration in the choice of assessment instrument. The primary burden associated with use of ASA24 is the time required for reporting and completion of each day of intake; collection of multiple days of intake has the potential for a substantial time commitment. In this study, the median time for completion of a single day of food and beverage intake using the ASA24 was 37½ minutes, similar to the 32 minutes for the interviewer-administered AMPM method. On the other hand, because ASA24 can be completed at the participant's convenience, even if assessment is targeted for a specific day, the perceived burden by the participant appeared to be less than for AMPM, as indicated by responses to the preference questions.

Accuracy of the chosen assessment instrument affects both the precision of effect estimates and the statistical power to detect associations for a given sample size. For the recovery biomarkers examined in pooled data from 5 large validation studies, 24HRs generally yielded more accurate data than food frequency questionnaires. For example, the average correlation coefficients for 1 administration of a food frequency questionnaire with true protein intake were 0.28 for men and 0.29 for women; the average correlation coefficients for 3 administrations of a 24HR were 0.48 for men and 0.49 for women (15). By use of these estimates, a true relative risk of 2.0 would be observed as 1.2 with a food frequency questionnaire and 1.4 with three 24HRs. However, with energy adjustment, a procedure commonly used to reduce measurement error, correlation coefficients for the food frequency questionnaire increased, approaching those of multiple 24HRs. On average, for a single administration of a food frequency questionnaire and 3 of the 24HR, the correlation coefficients for protein density were 0.38 (men) and 0.43 (women) for the food frequency questionnaire and 0.45 (men) and 0.47 (women) for the 24HRs. Unfortunately, we do not know how other dietary constituents behave, because only a few recovery biomarkers are known. However, given that day-to-day variation in protein intake is smaller than for many other dietary constituents of interest in nutritional epidemiology (16), comparable gains in power or reductions in sample size may require more than 3 days of 24HR data collection. Furthermore, some dietary components, and in particular some food groups, may be more accurately measured with a food frequency questionnaire. Emerging research suggests that a combined approach of multiple 24HRs and a food frequency questionnaire may provide data superior to use of either method alone, resulting in more precise risk estimates, especially for episodically consumed foods and nutrients, such as whole grains and vitamin A (17). In such a scenario, 24HRs would serve as the main instrument, with frequency estimates from the food frequency questionnaire included as covariates in statistical models to estimate diet and health associations. Carroll et al. (17) suggest that the optimal approach may be from four to six 24HRs and a food frequency questionnaire from all participants over a period of time, such as a year.

In a large nutritional epidemiologic study enrolling tens or hundreds of thousands of individuals, cost and feasibility are major factors in selecting a dietary assessment instrument. Prior to ASA24, a food frequency questionnaire was considered the only feasible and cost-effective instrument. Recognition of the limitations of food frequency questionnaires and the costs associated with traditional 24HRs and food records may have caused some true associations to be missed and some studies to forgo collection of dietary data altogether. Now, however, given that ASA24 is a freely available web-based tool, cost and feasibility are no longer barriers.

The implications of these study results for nutritional epidemiology are vast. Although issues such as literacy, language, Internet access, computer proficiency, and participant burden still need to be considered, ASA24 makes practical the use of quantitative dietary assessment methods on a large scale. As lifestyle behaviors continue to be considered major contributors of chronic diseases, the addition of ASA24 to existing assessment methods in large epidemiologic studies has the potential to enhance our understanding of the role of diet in disease.