The Provision of Assistance Does Not Substantially Impact the Accuracy of 24-Hour Dietary Recalls Completed Using the Automated Self-Administered 24-H Dietary Assessment Tool among Women with Low Incomes

ABSTRACT

Background

Evidence is lacking informing the use of the Automated Self-Administered 24-h Dietary Assessment Tool (ASA24) with populations characterized by low income.

Objective

This study was conducted among women with low incomes to evaluate the accuracy of ASA24 recalls completed independently and with assistance.

Methods

Three hundred and two women, aged ≥18 y and with incomes below the Supplemental Nutrition Assistance Program thresholds, served themselves from a buffet; amounts taken as well as plate waste were unobtrusively weighed to enable calculation of true intake for 3 meals. The following day, women completed ASA24-2016 independently (n = 148) or with assistance from a trained paraprofessional in a small group (n = 154). Regression modeling examined differences by condition in agreement between true and reported foods; energy, nutrient, and food group intakes; and portion sizes.

Results

Participants who completed ASA24 independently and those who received assistance reported matches for 71.9% and 73.5% (P = 0.56) of items truly consumed, respectively. Exclusions (consumed but not reported) were highest for lunch (at which participants consumed approximately 2 times the number of distinct foods and beverages compared with breakfast and dinner). Commonly excluded foods were additions to main dishes (e.g., tomatoes in salad). On average, excluded foods contributed 43.6 g (46.2 kcal) and 40.1 g (43.2 kcal) among those in the independent and assisted conditions, respectively. Gaps between true and reported intake were different between conditions for folate and iron. Within conditions, significant gaps were observed for protein, vitamin D, and meat (both conditions); vitamin A, iron, and magnesium (independent); and folate, calcium, and vegetables (assisted). For foods and beverages for which matches were reported, no difference in the gap between true and reported portion sizes was observed by condition (P = 0.22).

Conclusions

ASA24 performed relatively well among women with low incomes; however, accuracy was somewhat lower than previously observed among adults with a range of incomes. The provision of assistance did not significantly impact accuracy.

Introduction

Effective nutrition education has the potential to reduce health disparities between higher- and lower-income Americans by promoting healthy diets and thereby reducing the risk of chronic disease (1). For instance, the Expanded Food and Nutrition Education Program (EFNEP) and the Supplemental Nutrition Assistance Program (SNAP) Education are 2 large USDA nutrition education programs that aim to improve health-related behaviors among populations affected by low socioeconomic status through nutrition education, as well as policy, system, and environmental changes (2, 3). These programs educate participants on nutrition, physical activity, and food resource management. To inform program improvement and reinforce sustainability, rigorous tools are needed to enable evaluation of impacts on dietary intake and other outcomes. High-quality dietary intake data are valuable for program evaluation because they provide a lens into dietary behaviors, as opposed to knowledge or intentions, that can be compared to the Dietary Guidelines for Americans (1).

The collection of high-quality dietary intake data has been burdensome and costly in the context of many projects (4, 5), including program evaluations, but these barriers have been eased in recent years by technological innovations (6). For example, the Automated Self-Administered 24-h Dietary Assessment Tool (ASA24) is a web-based system developed by the National Cancer Institute that enables self-administered 24-h dietary recalls (7). ASA24 is freely available and uses multiple passes, adapted from the Automated Multiple-Pass Method (AMPM) (8–10) developed by the USDA for interviewer-administered recalls carried out in the NHANES. ASA24 was designed to make it possible to collect multiple recalls per participant in large studies by eliminating the need for an interviewer and implementing automated coding (6). ASA24 has been shown to be feasible (11), to collect intake data comparable with those from interviewer-administered recalls (12–14), and to perform well in terms of capturing true intake in samples of adults (12, 13). In particular, in prior research, the accuracy of dietary recalls collected using ASA24 was compared with the accuracy of those collected using AMPM interviews among a sample of adults; ASA24 performed nearly as well as the AMPM (12, 13) but with considerable cost savings.

A barrier to the use of ASA24 to evaluate nutrition education and other programming is that evidence to inform use with populations affected by low socioeconomic status is lacking. This is relevant because socioeconomic status may be associated with factors such as literacy, which can influence the accuracy of dietary data (15). Furthermore, technology, albeit useful for overcoming some limitations of traditional dietary assessment methods, has the potential to introduce new challenges, including the need for computer literacy (16). The objective of the current study was thus to assess the criterion validity of recalls completed using ASA24 among women who were eligible for SNAP based on their reported household income and household size. Two modes of administering ASA24, or study conditions, were tested: independently (with assistance available by telephone, if needed) and assisted by a paraprofessional in a small-group setting. It was hypothesized that the accuracy of recalled intakes would be higher among those in the assisted condition.

Methods

This study was designed to replicate a prior validation study (12, 13), with 3 main differences. In the original study, recruitment was intended to result in a diverse sample of adults based on age, sex, and race/ethnicity, with no income threshold applied. In the current study, income thresholds used for SNAP were applied to determine eligibility for inclusion, and only women were included because SNAP Education and EFNEP participants are primarily women (17). Second, in the prior study, recalls completed using ASA24 were compared with those collected using interviewer-administered AMPM recalls (12, 13), whereas in the current research, the 2 conditions tested were independent completion of ASA24 and completion of ASA24 in a small-group setting, led by a paraprofessional and designed to mimic the EFNEP educational environment (2). Finally, a larger sample was sought in the current study to provide sufficient power to assess differences between the 2 conditions and factors associated with any such differences. The target sample size was 300 women, calculated to allow detection of a 5% difference in the proportions of food and beverage items that were truly consumed and accurately reported between the 2 groups, assuming they were matched on characteristics related to food consumption based on random assignment (resulting in a correlation between samples of 0.50).

Aside from these differences in sampling and the conditions tested, the methods and protocols used for data collection and analysis for the current research followed closely those from the original study (12, 13), with the exception of the version of ASA24 used. The prior study (12, 13) used ASA24-2011, which incorporated an avatar that provided written and audio instructions to participants and could be completed on desktops and laptops. The current study used ASA24-2016, which offers a more streamlined interface without an avatar, as well as the capacity to complete recalls on tablets and smartphones, in addition to laptop and desktop computers. Any additional deviations from the protocol used in the original study are noted below and typically related to management of the larger volume of data in the current study.

This study was approved by the Institutional Review Board at Utah State University and Westat. Written informed consent was provided by all participants.

Recruitment and sample

Participants were recruited from a database of research volunteers living in the Washington, DC area. Eligible participants were women between the ages of 18 and 82 y who met the income requirements for SNAP. For example, for a household of 4 persons, the 2017 gross monthly income threshold was $2665 (18). Quota sampling was employed to recruit a sample that was racially/ethnically diverse, and an effort was made to oversample individuals with less than a high school education because of concerns about computer literacy in this audience. Potential participants were excluded if they were unable to read and understand English or Spanish (the 2 languages in which ASA24 is available); had dietary allergies, practices, or preferences that would interfere with the study protocol; were pregnant; or had previously had bariatric surgery. A total of 377 participants were eligible, enrolled, and mailed a welcome package that included information on appointment dates and location. The study was described to participants as research to improve our understanding of methods to collect food intake information from individuals. Three-hundred and six women participated in the study, with an overall response rate of 81%.

Data collection

Data were collected from May through July, 2016. Participants were scheduled to visit the study center on 2 consecutive days. On the first day, after being provided with a brief introduction to the study and completing informed consent, women were invited to select and consume foods and beverages from a buffet of breakfast-appropriate items and to return to do the same for lunch and dinner (12, 13). As with the prior study, efforts were made to simulate a conventional eating environment (19, 20), with a variety of food types offered (Supplemental Table 1), with respect to both perceived healthfulness (e.g., fresh fruit compared with brownies) and potential capacity to accurately estimate amounts consumed (e.g., amorphous compared with shaped foods) (12, 13). When possible, original containers were used so participants had the opportunity to note details about the foods and beverages they were choosing from and potentially consuming. Items served in unmarked containers were labeled in both English and Spanish. Participants served themselves one at a time at 8- to 10-min intervals and were then escorted to a communal dining area. Each food container was inconspicuously weighed before and after each participant served herself to determine the amount of each item taken. Plate waste was weighed by trained research staff at the conclusion of the meal to enable a calculation of the amount of each food and drink consumed. Weights were taken with Ultra Ship 35 scales (MyWeigh), which have a precise accuracy of 0.1 ounces (2.8 g) for items weighing ≤2 pounds (0.91 kg) and an accuracy of 0.2 ounces (5.7 g) for items weighing >2 pounds (0.91 kg). Each item was weighed independently by 2 technicians; if the 2 weights did not match to the gram, a third weight was taken and the mean of the 2 closest weights was used. The weight consumed (i.e., true intake) was calculated as the weight of the food taken minus the weight of the food left (i.e., plate waste).

At the end of the first day, participants received an honorarium ($80 for 3 meals and $60 for 2 meals) and were reminded to return the following day to provide information about their diet and health (they were not specifically told that they would be asked to report what they had eaten). On the second day, participants completed an unannounced 24-h dietary recall on iPads (Apple Inc.), with half randomly assigned to complete ASA24 independently and half randomly assigned to complete ASA24 in a small-group setting with assistance from a paraprofessional. ASA24 provides respondents with the option for completion in either English or Spanish; participants made this selection without assistance. Those completing ASA24 independently were provided with assistance in getting started on an iPad and told that they would be completing a program called ASA24 to report everything they had had to eat and drink yesterday and that they could do so in either English or Spanish. They had the option of calling a telephone helpdesk as needed, but received no other assistance with ASA24. Those in the group setting received a 15-min PowerPoint presentation providing an overview of ASA24 and were assisted with logging in to ASA24 and entering their first eating occasion by the paraprofessional, who was then available for questions but did not offer assistance in recalling foods and beverages offered or consumed. All participants wore headphones playing white noise so that they would not overhear questions or comments from others that could have aided them in remembering foods and beverages served and potentially consumed. Telephone helpdesk attendants and paraprofessionals were trained using a manual developed by Utah State University (21), based on training created by Westat. They were briefed on the study protocol and instructed to not lead participants in completing ASA24.

After completing ASA24, participants were asked to complete a brief demographic and health behavior questionnaire. Questions were based on the Behavioral Risk Factor Surveillance Survey (22). Additional items querying methods for Internet access and frequency of accessing email were developed for this study. Participants were then debriefed on the true purpose of the study and provided with an honorarium for their time (an additional $120 for completion of ASA24 and the demographic and health behavior questionnaire).

Two women did not complete ASA24 and another 2 did not complete the demographic survey. These women were excluded, resulting in an analytic sample of 302 women (148 in the independent condition and 154 in the assisted condition). Of the 302 women, 6 did not attend the study center for breakfast but did attend for lunch and dinner; their data for these meals were included in the analyses.

Coding of true and reported consumption

Each food and beverage offered was coded using the Food and Nutrient Database for Dietary Studies, 2011–2012 (23). Together with the weights consumed, these codes were used to generate estimates of “true” energy and nutrient intakes and linked to the Food Patterns Equivalents Database (FPED), version 2.0 (24), to provide estimates of amounts of fruits, vegetables, and other food groups actually consumed. The FPED disaggregates each food and beverage reported into ingredients that are then assigned to food groups (24).

ASA24 recalls, whether completed in English or Spanish, are automatically coded by the ASA24 system, also using the Food and Nutrient Database for Dietary Studies and the FPED. Output is downloaded in English. Before analysis, corrections were applied to address known errors in the ASA24 database, affecting 4 reports of water and 1 report of a tuna salad sandwich.

Comparison of true and reported consumption

ASA24 data from each participant were reviewed by 2 members of the research team, who were blinded to the true intake data, to identify eating occasions that were not part of the study meals (i.e., nonstudy meals and snacks). These eating occasions may have occurred before attending the study center for breakfast, after dinner, or between meals, as participants were not required to remain at the study center for the full day. A total of 134 respondents reported nonstudy eating occasions (meals, snacks, or drinks), which were identified by considering the reported name, time, and location of each eating occasion, and the foods and beverages reported. All nonstudy eating occasions were excluded from the analysis.

A list of codes assigned by ASA24 to foods and beverages reported by participants for the study center meals was then generated. This was assessed by 2 members of the team, blinded to the true intake data, to determine whether each was an exact, close, or far match for any foods and drinks offered at any study meal. For example, for the tuna salad component of the sandwich, tuna salad made with mayonnaise was considered an exact match whereas water-packed canned tuna was considered a close match and baked or broiled tuna a far match. For meatless lasagna, vegetable lasagna was an exact match whereas lasagna with meat was a close match and other pasta dishes (e.g., noodles with cheese) were a far match. Items that were not a match (i.e., not offered) were considered intrusions. The result was a “match key” that was subsequently used to link the true and reported intake data. The match key developed for the original study (12, 13) was used as a starting point; however, given the larger sample size in the present study and the fact that new foods and beverages had emerged on the market since the original study was conducted, a number of foods and drinks were reported that had not been considered in the original study. For these foods and drinks, the rationale underlying whether an item was a match and the degree of matching (exact, close, or far) was as consistent as possible with the original study; this involved considering what participants could be expected to know about a food or beverage item (e.g., served in a container labeled with the level of fat compared with within a preprepared sandwich for which the ingredients were not listed). The resulting match key was reviewed by the full study team, and true and reported intakes were then compared using the key to determine whether each participant reported a match for each of the foods and drinks consumed.

Demographic and health variables

Data from the demographic questionnaire were double entered and any discrepancies reconciled by referring back to the paper survey. BMI (in kg/m²) was calculated based on self-reported weight and height (25).

Statistical analyses

All analyses were carried out using SAS, version 9.4 (SAS Institute).

The proportions of foods and beverages consumed for which participants reported a match (exact, close, or far) or that were excluded were examined, by study condition. The average number of intrusions (reported but not consumed) was also estimated. Intrusions were considered to be internal (offered but not consumed at study center meals) or external (not offered as part of study center meals) (26). These analyses were conducted for all eating occasions combined and separately for each eating occasion. Further, they were conducted for all foods and beverages; for main items compared with additions to or ingredients in main items using the categorization used previously (12); and for fruits and vegetables, beverages, and desserts to assess potential differential reporting accuracy in relation to perceived healthfulness.

Linear regression was used to examine differences by condition (independent compared with assisted recalls) in the proportions of matches and exclusions. Poisson regression was used to examine the number of intrusions by condition. Models were adjusted for the number of items truly consumed because this was hypothesized to potentially affect the accuracy of recall. Further, linear regression models, stratified by condition, were used to examine associations between variables hypothesized a priori to influence accuracy of recall in terms of the proportion of matches. These variables included BMI, education level, and whether or not the participant reported accessing email every day. As with the models above, a covariate was included to adjust for the number of items truly consumed.

Additionally, linear regression modeling was used to compare whether gaps between energy, nutrient, and food group intakes based on true compared with reported intake were significant within each condition and whether these gaps differed between conditions. Adjustment for multiple comparisons was not conducted such that the analyses would identify the maximum number of dietary components for which differences between true and reported intake were observed. Linear regression was also employed to examine whether the gaps between overall true and reported portion sizes for items for which an exact, close, or far match was reported differed by condition.

Preliminary analyses indicated that the participants in the 2 conditions (independent compared with assisted) differed in relation to participation in the Special Supplemental Nutrition Program for Women, Infants, and Children (WIC), accessing the Internet/email on a phone, and accessing the Internet/email at home. In a regression model including both of the variables related to Internet access, only accessing the Internet on a phone was significantly different by condition. Regression models were thus adjusted for WIC participation and whether or not the participant typically accessed the Internet on a phone.

In addition, the observed rate of matches (items truly consumed for which a match was reported) was higher for recalls completed in English compared with Spanish (73% and 65%, respectively, P = 0.01, adjusted for condition and the number of items consumed); however, for the analyses reported below, recall data were combined because few were completed in Spanish and individuals in the 2 conditions did not differ in terms of language of completion.

Results

The characteristics of the sample are described in Table 1. There were no statistically significant differences between the 2 groups in terms of age grouping, race/ethnicity, preferred language (English or Spanish), income level, educational attainment, employment status, BMI, or participation in SNAP. However, those in the assisted group were more likely to receive WIC benefits. Table 1 also provides an overview of participants’

TABLE 1.

Characteristics of participants by study condition¹

	Independent	Assisted
	(n = 148)	(n = 154)	P value2
Age range,3 y			0.85
18–34	49	51
35–54	70	69
55–82	29	34
Race/ethnicity			0.36
Hispanic	72	62
Black, non-Hispanic	44	59
White, non-Hispanic	18	21
Other	14	12
Preferred language			0.26
English	124	136
Spanish	24	18
Household income			0.45
<$10,000	34	45
$10,000-$19,999	44	43
$20,000-$39,999	43	36
≥$40,000	20	26
Education			0.11
Less than High school	41	48
Some college	69	54
College graduate	37	51
Employment			0.43
Employed	74	63
Out of work	33	35
Unable to work	9	15
Other	32	37
BMI, kg/m2			0.27
Under/normal weight (<25)	43	58
Overweight (25.0–29.9)	33	38
Obese (≥30)	63	55
Currently receiving SNAP			0.13
No	115	108
Yes	33	46
Currently participating in WIC			<0.01
No	143	135
Yes	5	19
Frequency of accessing email			0.54
Every day	116	117
Not every day	31	37
Devices used to access the Internet or email (respondents could select multiple options)
Computer	58	57	0.70
Laptop	74	71	0.50
Tablet	52	54	0.99
Phone	127	110	<0.01
Other device	6	7	0.83
No access	1	4	0.19
Locations at which Internet and/or email are accessed (respondents could select multiple options)
Home	138	133	0.05
Work	50	50	0.81
Library	37	36	0.74
Friend's home	45	39	0.32
Coffee shop	44	35	0.17
Other	10	14	0.45
Nowhere	2	3	0.68

¹ n = 302 women with low incomes. For some variables, numbers may not add up to totals owing to missing data for a small number of participants. SNAP, Supplemental Nutrition Assistance Program; WIC, Special Supplemental Nutrition Program for Women, Infants, and Children.

² P values were estimated using chi-square tests.

³One person in the independent group and 2 in the assisted group were >70 y of age.

reported methods and frequency of accessing the Internet and email. Those in the assisted group were less likely than those in the independent group to use a phone to access the Internet. Subsequent analyses included variables to account for WIC status and accessing the Internet on a phone, as aforementioned.

Participants who completed ASA24 recalls independently reported 71.9% of items truly consumed, compared with 73.5% among those who completed ASA24 with assistance (P = 0.56) (Table 2). At the level of meals, the highest match rates were observed for breakfast (85.1% for independent and 82.6% for assisted condition) and the lowest for lunch (64.1% for independent and 67.0% for assisted) (Table 3).

TABLE 2.

Proportions of foods and beverages truly consumed for which matches were reported or that were excluded, and numbers of intrusions, for all meals combined by study condition (independent or assisted 24-h recall)¹

	Independent (n = 148 participants and 4487 foods and beverages)	Assisted (n = 154 participants and 4539 foods and beverages)	P value for between-group comparison
Exact matches, %	58.5	60.9	0.262
Close matches, %	9.6	8.1	0.032
Far matches, %	3.8	4.4	0.622
All matches combined, %	71.9	73.5	0.562
Exclusions, %	28.1	26.5	0.562
Total, %	100	100
Intrusions, n	2.4	2.5	0.573
Items reported, n	21.5	21.4	0.304

¹ n = 302 women with low incomes. WIC, Special Supplemental Nutrition Program for Women, Infants, and Children.

² P values were obtained from linear regression models accounting for participation in WIC, accessing the Internet using a phone, and the number of items truly consumed.

³ P values were obtained from Poisson regression models accounting for participation in WIC, accessing the Internet using a phone, and the number of items truly consumed.

⁴ P values were obtained from Poisson regression models accounting for participation in WIC and accessing the Internet using a phone.

TABLE 3.

Proportions of foods and beverages truly consumed for which matches were reported or that were excluded and numbers of intrusions, by meal and study condition (independent or assisted 24-h recall)

	Breakfast1			Lunch			Dinner
	Independent (n = 146 participants and 1119 foods and beverages)	Assisted (n = 150 participants and 1042 foods and beverages)	P value for between-group comparison	Independent (n = 148 participants and 2338 foods and beverages)	Assisted (n = 154 participants and 2452 foods and beverages)	P value for between-group comparison	Independent (n = 148 participants and 1030 foods and beverages)	Assisted (n = 154 participants and 1045 foods and beverages)	P value for between-group comparison
Exact matches, %	67.4	64.7	0.262	54.9	58.8	0.112	56.6	61.5	0.062
Close matches, %	11.0	8.6	0.112	7.5	6.1	0.132	13.2	11.6	0.182
Far matches, %	6.6	9.3	0.252	1.7	2.1	0.692	5.2	5.4	0.922
All matches combined, %	85.1	82.6	0.092	64.1	67.0	0.242	75.0	78.4	0.172
Exclusions, %	14.9	17.4	0.092	36.0	33.0	0.242	25.0	21.6	0.172
Total, %	100	100		100	100		100	100
Intrusions, n	0.6	0.7	0.413	1.4	1.4	0.823	0.6	0.6	0.063
Items reported, n	5.6	5.1	0.024	10.3	10.6	0.564	5.8	5.9	0.834

¹Sample sizes are lower for breakfast owing to participants who did not attend that meal but did attend lunch and dinner.

² P values were obtained from linear regression models accounting for participation in the Special Supplemental Nutrition Program for Women, Infants, and Children (WIC), accessing the Internet using a phone, and the number of items truly consumed.

³ P values were obtained from Poisson regression models accounting for participation in WIC, accessing the Internet using a phone, and the number of items truly consumed.

⁴ P values were obtained from Poisson regression models accounting for participation in WIC and accessing the Internet using a phone.

Overall, participants who completed ASA24 independently excluded 28.1% of foods and drinks (mean: 7.6 items), whereas those in the assisted condition excluded 26.5% (mean: 7.2 items). Common exclusions were often additions or ingredients, such as tomatoes, cucumber, or cheese that were part of a salad or sandwich (Table 4). Excluded foods and beverages contributed an average of 43.6 g and 46.2 kcal/person among those in the independent condition and 40.1 g and 43.2 kcal/person among those in the assisted condition. Women in the independent condition reported matches for 80.3% of main foods and 56.6% of additions, compared with 83% of main foods and 57% of additions among those in the assisted group. Among participants in the independent condition, 38.8% of fruits and vegetables were excluded, 23.3% of sweets/snacks/desserts were excluded, and 12.4% of beverages were excluded. Among those in the assisted condition, the exclusion rates were 36.4% for fruits and vegetables, 18.7% for sweets/snacks/desserts, and 13.3% for beverages. There were no significant differences in match rates by condition for any of these categories (data not shown).

TABLE 4.

Counts of most commonly excluded food and beverage items, by study condition, and the true mean grams consumed and energy contributed by these items among women who excluded them¹

	Number of times excluded/number of times consumed		Means ± SEs
Item	Independent (n = 148 participants)	Assisted (n = 154 participants)	Mass consumed (g) among women excluding item	Energy consumed (kcal) among women excluding item
Tomatoes	151/281	148/288	12.3 ± 0.37	2.35 ± 0.07
Red or green peppers	85/120	96/124	14.7 ± 0.55	1.47 ± 0.05
Cucumber	83/120	85/124	5.75 ± 0.22	0.69 ± 0.03
Cheddar cheese	57/126	60/150	12.1 ± 0.73	51.1 ± 3.10
Lettuce	56/288	61/299	7.32 ± 0.80	1.14 ± 0.11
Garlic bread	53/84	54/98	17.6 ± 1.19	66.9 ± 4.51
Mustard	53/69	48/83	2.19 ± 0.07	1.47 ± 0.05
Mayonnaise	34/69	44/83	6.25 ± 0.25	42.5 ± 1.73
Rice pilaf	32/125	39/135	64.3 ± 4.72	87.4 ± 6.42
Broccoli, cooked	39/122	33/122	45.9 ± 3.24	22.9 ± 1.62
Pasta with pesto sauce	44/109	31/108	62.0 ± 4.49	179 ± 13.0
Carrots, cooked	38/97	21/104	44.8 ± 3.50	22.4 ± 1.75
Cookie	23/92	26/93	30.7 ± 2.64	124 ± 10.7
Vegetable lasagna	25/129	25/123	108 ± 10.3	139 ± 13.3
White potato chips	24/86	17/75	22.7 ± 1.14	123 ± 6.16
Bread, white	22/168	16/174	29.0 ± 2.51	40.0 ± 3.19
Sugar	24/62	22/60	11.1 ± 1.37	43.1 ± 5.27
Water, bottled, unsweetened	30/161	20/179	342 ± 22.4	0 ± 0

¹Exclusion and consumption values are frequencies. Some foods were present in multiple items offered (e.g., tomatoes in salad and multiple sandwiches). Thus, the number of times an item was consumed and excluded may exceed the number of participants. Mass and energy consumption values are means ± SEs. The number of observations for the means for each food or beverage item is the sum of the number of times excluded for the independent and assisted participants combined. For example, the n for the calculation of the mean grams and mean energy contributed among women who excluded tomatoes is 151 + 148 = 299.

The average numbers of intrusions were 2.4 and 2.5 (P = 0.57) for independent and assisted recalls, respectively. The average numbers of internal and external confabulations rounded to 1.8 and 0.6, respectively, for both conditions.

Among those in the independent condition, none of the factors assessed, including BMI status, accessing email every day, and education status, were associated with the match rate (data not shown). Within the assisted condition, those who were college graduates had higher match rates (B: 6.94; 95% CI: 1.31, 12.56, P = 0.02) compared with those who had some college (reference group). The total number of items consumed was negatively associated with the match rate (B: −0.69; 95% CI: −1.12, −0.27, P = 0.04) among those in the assisted condition.

In comparing gaps in intake of energy, nutrients, and food groups based on true versus reported consumption, significant differences between conditions were observed for folate and iron (Table 5). For the independent condition, the gap between true intake and that estimated from reported food consumption was significantly different from zero for protein, vitamin A, vitamin D, iron, magnesium, and meat. For the assisted condition, the gap was significantly different from zero for protein, vitamin D, folate, calcium, vegetables, and meat. Significant differences were not observed for energy, which was underestimated by 4% in the independent condition and 1% in the assisted condition based on reported intake.

TABLE 5.

True and reported energy, nutrient, and food group intakes by study condition (independent or assisted 24-h recall)¹

	Independent (n = 148 participants)			Assisted (n = 154 participants)
	Mean true value	Mean reported value	Mean difference (95% CI)	Mean true value	Mean reported value	Mean difference (95% CI)	P value for between-group comparison2
Energy, kcal	1957	1878	79.8 (−31.3, 191)	1834	1811	23.2 (−79.3, 126)	0.38
Carbohydrate, g	254	241	12.7 (−0.67, 26.0)	235	230	4.27 (−9.06, 17.6)	0.35
Fiber, g	16.7	15.7	1.00 (−0.00, 2.00)	15.8	16.2	−0.31 (−1.31, 0.70)	0.06
Fat, g	74.8	75.6	−0.78 (−6.53, 4.97)	70.2	72.3	−2.07 (−7.15, 3.02)	0.60
Potassium, mg	2415	2321	94.8 (−47.2, 237)	2325	2365	−39.9 (−159, 79.2)	0.15
Saturated fat, g	21.8	22.1	−0.25 (−1.94, 1.45)	20.3	20.9	−0.54 (−2.10, 1.01)	0.73
Protein, g	76.4	66.9	9.49 (4.96, 14.0)	74.4	68.2	6.20 (2.25, 10.2)	0.20
Vitamin A, RAE	733	660	73.0 (0.39, 146)	716	731	−15.0 (−91.3, 61.4)	0.11
Vitamin C, mg	118	124	−5.93 (−15.6, 3.68)	119	126	−6.98 (−16.3, 2.30)	0.88
Vitamin D, μg	1.92	2.50	−0.58 (−0.82, −0.34)	1.68	2.46	−0.78 (−1.06, −0.50)	0.45
Folate, μg	430	424	5.54 (−21.6, 32.7)	389	421	−32.5 (−56.0, −8.99)	0.02
Iron, mg	14.8	13.7	1.15 (0.30, 2.00)	13.2	13.4	−0.18 (−0.99, 0.62)	0.03
Magnesium, mg	251	236	15.1 (1.41, 28.9)	242	243	−0.81 (−13.1, 11.5)	0.08
Calcium, mg	728	749	−21.1 (−75.2, 33.0)	694	751	−56.6 (−112, −1.09)	0.39
Sodium, mg	3051	2883	167 (−25.9, 361)	2929	2874	55.0 (−117, 227)	0.25
Fruit, cup equivalent3	1.58	1.50	0.08 (−0.05, 0.22)	1.73	1.62	0.11 (−0.02, 0.25)	0.72
Vegetables, cup equivalent3	1.78	1.93	−0.15 (−0.32, 0.02)	1.63	1.89	−0.26 (−0.42, −0.10)	0.43
Milk, cup equivalent3	1.18	1.13	0.05 (−0.07, 0.16)	1.11	1.13	−0.03 (−0.14, 0.09)	0.41
Meat, ounce equivalent3	4.62	3.97	0.65 (0.26, 1.04)	4.87	4.36	0.50 (0.12, 0.89)	0.45
Added sugars, tsp	16.1	16.9	−0.82 (−2.18, 0.54)	14.5	15.0	−0.53 (−1.95, 0.90)	0.67

¹Values are means unless otherwise stated. RAE, Retinol Activity Equivalent; tsp, teaspoon.

² P values were estimated using multivariable linear regression models, accounting for participation in the Special Supplemental Nutrition Program for Women, Infants, and Children and accessing the Internet using a smart phone.

³Units are based on the Food Patterns Equivalents Database.

The average differences between true and reported portion sizes were 7.3 g for the independent and 6.5 g for the assisted condition (P = 0.22). Absolute differences, which do not account for the direction of the misestimation, were 38.9 g for the independent and 37.8 g for the assisted condition (P = 0.11).

The help desk received 20 calls from participants in the independent group. Calls lasted an average of 7.3 min (range: 1–26.5 min). The most frequent reason for calling was because ASA24 crashed or was slow (7 calls). The second most frequent reason was because the participant completed the recall before adding all the meals or items (6 calls). Three people called for issues related to logging into ASA24. Two called asking about how to change the language and another 2 called with general questions about how to complete the recall.

Discussion

This study suggests that women with low incomes can recall and report their dietary intake relatively well using ASA24-2016.

Assistance in using ASA24 did not make a substantial difference in terms of accuracy. The degree of accuracy observed was somewhat lower than previously reported (72–74% matches for foods and beverages truly consumed compared with 80%) among a sample of adults with a wider range of incomes (12, 13). However, many excluded items were consumed in relatively small amounts and contributed small amounts of energy. Accuracy was lowest for lunch, as observed in the prior study using the same menu (12, 13). Participants consumed approximately twice the number of distinct foods and beverages at this eating occasion compared with breakfast and dinner. Furthermore, the lunch meal included many items that were additions to or ingredients in main dishes, such as cheese, tomatoes, and cucumber in salads or sandwiches. These items were frequently excluded from reporting by participants.

Significant gaps in intake based on true compared with reported consumption were observed for a few nutrients and food groups, which is not surprising given that ∼1 in 4 foods and beverages consumed was not reported, on average. Nonetheless, the differences are generally relatively small, again likely due to the fact that items excluded were often consumed in small amounts and did not contribute large amounts of energy (and other nutrients). There are some exceptions; thus, the accuracy of data at the level of a given individual could be lower, given exclusion of an item such as pasta salad, lasagna, or potato chips. At the group level, overall, amounts consumed were misestimated by ∼7 g, also contributing to differences between true and reported values for energy and nutrients. Portion size estimation has been identified as an ongoing challenge in dietary assessment research (27–29), with various strategies under investigation to improve upon this source of error. In the prior comparison of ASA24 with interviewer-administered recalls among a general sample of adults, the use of digital images tailored to different types of foods in ASA24 may have offered some advantage in supporting accurate estimation over conventional aids such as photos of plates and measuring cups (13). Detailed analyses of portion size reporting in the current study will be presented elsewhere; this will include an examination of accuracy for meat products given differences observed for this food group as well as protein.

Few self-administered dietary assessment tools have been validated specifically for use in populations with low socioeconomic status, and existing studies have generally focused on checklists (30) or FFQs (31, 32). Scott et al. (33) used observation of 1 meal consumed by food-service workers to assess the accuracy of data from interviewer-administered 24-h dietary recalls (n = 19) and group-administered participant-recorded 24-h dietary recalls (n = 23). They found no significant differences for energy and selected nutrients based on observed compared with reported intake for either dietary assessment method; however, as the authors noted, this may have been due to the small sample size (i.e., lack of power). In addition, food-service workers may be more skilled than others at recalling portion sizes and participants were aware that they were being observed. Scott et al. (33) identified higher correlations between observed and reported intakes for energy, carbohydrate, and fat for the group than for the individual method, and suggested that the group method may have yielded more accurate results by dampening social desirability bias. In the current study, in which measures were put in place to ensure that participants in the group setting did not aid each other in recalling what was offered and potentially consumed, the provision of assistance in a group setting did not significantly impact accuracy. The current study did not include a measure of social desirability, so it is not possible to compare this source of bias between conditions; however, given the random assignment, it is unlikely that it would differ greatly between groups. Future validation research incorporating measures of social desirability (34) could shed further light on its contribution to measurement error in dietary data among various populations. Body weight status is also recognized as having an influence on the accuracy of dietary reporting (35). In this study, close to two-thirds of the women in each group were affected by overweight or obesity (based on self-reported weights and heights), suggesting that differences in body weight did not confound the comparisons between groups and that the findings may be generalizable to other populations in which rates of overweight and obesity are relatively high.

In addition to the lack of validation data, the use of 24-h dietary recalls in program evaluation and research has been limited by the fact that they have traditionally been time-consuming to administer and expensive to code and analyze, and can be burdensome for participants (30). Using the ASA24 assists in overcoming some of these barriers by removing the need for highly trained interviewers and introducing automatic coding (7), but introduces the need for computer literacy. However, research indicates that the majority of low-income populations have access to technology, including the Internet, and use it daily (36, 37). Further, nutrition education programs are beginning to use technology to reach their participants through websites (38) and Facebook (39). Smartphones are also an increasingly important way for people of all income levels to communicate and navigate life activities. The proportion of Americans with smartphones has almost doubled since Pew Research Center began tracking it in 2011; in 2014, approximately half of Americans living in households with incomes <$30,000/y and half of those with less than a high school education had smartphones (40). These trends suggest that the incorporation of technology such as ASA24, which can be completed on mobile devices, is feasible for use with populations affected by low socioeconomic status, opening up possibilities to enhance the nature of data available on the impacts of programs.

A telephone help line was maintained for participants who were asked to complete ASA24 independently; >10% of these participants accessed the help line with questions related to technical or navigational aspects. This finding is in line with prior suggestions that, although ASA24 and other technology-enabled dietary assessment tools play a role in reducing researcher burden and making it feasible to collect high-quality and comprehensive dietary data, it is good practice to provide respondents with support to complete assessments (16). Such support could include quick-start guides or videos that orient participants to the ASA24 interface and the tasks involved in completing a recall. The National Cancer Institute (41) maintains resources that can be tailored to participants, as well as study staff.

This study is not without limitations. The sample consisted of paid volunteers and selection biases cannot be ruled out. Participants were required to have the ability to read and understand English or Spanish, potentially limiting the implications of our findings for nutrition education programs that reach participants with low literacy (42). In addition, it was observed that accuracy was lower among participants who completed ASA24 in Spanish; however, this represented a small number of participants and, thus, it is difficult to draw conclusions as to whether this reduced accuracy was a function of the Spanish compared with the English interface or of characteristics of participants. Further, though the age range spanned from 18 to 82 y, very few participants were older than 70 y; older adults may have limited computer literacy compared with other age groups (16) and are also less likely to own smartphones (40). Further, participants were not asked to refrain from consuming foods and beverages aside from the study meals. To the extent that additional items consumed were reported alongside study meals, they may have contributed to intrusions, particularly to external confabulations.

In conclusion, ASA24 performed relatively well among a sample of women with low incomes. Continued innovation in dietary assessment is likely to continue to add to the possibilities for dietary assessment with diverse populations, including those with low incomes. However, ASA24 and similar web-based recalls offer a feasible approach that can yield high-quality data to provide insights into the impacts of nutrition education and other policies and programs.

Notes

Supported by USDA National Institute of Food and Agriculture grant 2015-09151 (to CD). At the time that analyses were conducted, SIK was supported by Canadian Cancer Society Research Institute (CCSRI) Capacity Development Award grant 702855 and MM was supported by funding from Alberta's Tomorrow Project and the aforementioned CCSRI award.

Author disclosures: SIK, PMG, DD, TZ, LLK, AA, MM, MRS-R, KWD, and CD, no conflicts of interest.

Supplemental Table 1 is available from the “Supplementary data” link in the online posting of the article and from the same link in the online table of contents at https://academic.oup.com/jn/.

Abbreviations used: AMPM, Automated Multiple-Pass Method; ASA24, Automated Self-Administered 24-h Dietary Assessment Tool; EFNEP, Expanded Food and Nutrition Education Program; FPED, Food Patterns Equivalents Database; SNAP, Supplemental Nutrition Assistance Program; WIC, Special Supplemental Nutrition Program for Women, Infants, and Children.