Measurement Errors in Dietary Assessment Using Self-Reported 24-Hour Recalls in Low-Income Countries and Strategies for Their Prevention

Abstract

Securing accurate measurements of dietary intake across populations is challenging. Of the methods, self-reported 24-h recalls are often used in low-income countries (LICs) because they are quick, culturally sensitive, do not require high cognitive ability, and provide quantitative data on both foods and nutrients. Measuring intakes via 24-h recalls involves 1) collecting data on food intakes, 2) the appropriate use of relevant food-composition data for calculating nutrient intakes, and 3) statistically converting observed intakes to “usual intakes” for evaluating nutrient adequacy or relations between foods and nutrients and health outcomes. Like all dietary methods, 24-h recalls are subject to random errors that lower the precision and systematic errors that can reduce accuracy at each stage of the measurement protocol. Research has identified the potential sources of measurement errors in 24-h recall protocols and emphasized that sources of random error can be reduced by incorporating standardized quality-control procedures and collecting more than one 24-h recall per person, with the number depending on the study objective. Careful design of the initial 24-h recall protocol can take into account potential sources of systematic error, such as day of the week, season, age, etc. Other sources of systematic error (e.g., energy underreporting) can best be detected by including a reference measure (e.g., doubly labeled water to measure energy expenditure). Alternatively, 24-h recall intakes of energy can be compared with same-day weighed intakes. Nevertheless, very few studies in LICs have assessed the validity of 24-h recalls in their study settings or adopted recommended standardized protocols to mitigate random errors. Hence, efforts should be made to improve the assessment, analysis, and interpretation of self-reported 24-h recall data for population studies in LICs. Accurate and precise dietary intake data at the national level can play an essential role in informing food, nutrition, and agricultural policies; food fortification planning; and compliance to food-based dietary guidelines.

Introduction

Accurate measurements of dietary intakes for populations are essential to assess and monitor the content and quality of diets, to investigate diet-health relations, and to inform food, nutrition, and agriculture programs and policies. However, assessing dietary intakes is a complex and challenging task that involves 3 critical stages. In the first stage, data on food intakes from a representative subsample of the population are collected by using an appropriate and accurate dietary assessment instrument. In the second stage, the food intake data are converted to nutrients by using appropriate nutrient-composition data. In the final stage, the observed intakes can be adjusted statistically to give “usual” intakes of nutrients or foods. This allows comparison with appropriate dietary recommendations and compliance with food-based dietary guidelines; nutrient adequacy and excess in a population can also be assessed. Relations between foods and nutrients and nutrition-related diseases may also be examined at this stage.

Of the available dietary assessment methods, the 24-h recall is most frequently used for population assessment in low-income countries (LICs). The method can be designed to be culturally sensitive and cognitively easy, making it especially suitable for respondents with limited education. In addition, with 24-h recalls, details about the sources, preparation, and processing of foods and timing and location of meals together with quantitative data on all food sources of energy and nutrients can be captured (1).

Nevertheless, like all self-reported dietary assessment instruments, 24-h recalls are subject to random and systematic errors during the dietary assessment process. Random errors reduce the precision of the measurements, resulting in a loss in statistical power. In contrast, systematic errors generate bias, reducing the accuracy of measurements and yielding potentially erroneous conclusions with regard to the absolute amount of foods and nutrients consumed and the relation between intakes of foods or nutrients and nutrition-related diseases (1). The nature, direction, and magnitude of these errors vary with the recall protocol used and the study group, setting, and nutrients of interest. Most of the research on measurement errors inherent in the use of 24-h recalls has been conducted in developed countries. However, special adaptations to avoid additional errors may be needed in 24-h recall protocols for LICs. Such adaptations may include the following: interviewing practices that take into account cultural attitudes, limited literacy and numeracy, food insecurity, food-procurement practices, food sharing, and differences in food preparation, processing, and storage practices. Seasonal fluctuations in both the quantity and quality of foods consumed may create additional uncertainties not always experienced at the same magnitude in developed countries (2, 3).

In this review, we examine the sources of error that may occur during the collection and analysis of dietary data by self-reported 24-h recalls for populations or groups in LICs and highlight their potential impact on the interpretation of the data. In addition, we review possible strategies that can, if implemented, assist in minimizing measurement errors at each stage of the 24-h recall protocol. Sampling errors are not addressed, and readers may need to seek advice on this important issue elsewhere.

Potential Errors When Measuring Food Intakes by Using 24-h Recalls in LICs and Strategies for Their Prevention

The goal of this first stage is to ensure that the recorded daily food intakes collected by the self-reported 24-h recalls are the best-possible unbiased record of the actual daily intakes. The measurement errors that may occur when implementing this first stage together with possible strategies that could be used to minimize them in LICs are summarized below. Some of these errors may be random, and others systematic, as noted earlier.

Random errors may arise during the measurement of food intakes on any given day. An additional source of random error is true day-to-day variation in food intake for a group that arises from differences in food intake both between persons (between- or inter-person variation) and within one person (within- or intra-person variation). Random errors during measurement on any given day can be prevented or minimized by incorporating standardized quality-control procedures in an LIC 24-h recall protocol. The use of a standardized 24-h recall protocol across LICs is rare, however efforts are being made by the International Dietary Data Expansion Project and others to develop a standardized dietary assessment platform. In a recent review of protocols used in Africa, no standardized method was applied across the continent (4), even though the use of a standardized 24-h recall protocol has been achieved in 22 European centers. The Europeans adopted a computerized multiple-pass 24-h recall software (5). This system (GloboDiet) was subsequently extensively modified for use in Korea (6) and other countries. The multiple-pass format was originally developed by the USDA in 1999 (7) and consists of 4 steps (passes) designed to minimize forgotten food items and correctly estimate portion sizes of foods and beverages consumed. Details are given in Gibson and Ferguson (8). The recall period should cover the preceding 24-h period and commence with the first food or drink consumed after waking and end with the last food or drink taken before going to sleep or during the night.

The effect of random measurement error in any given day can also be reduced by repeating the 24-h recalls on each respondent. Importantly, this practice also permits the estimation of usual intakes, taking into account within-person variation in food intake. The number of days over which the 24-h recalls should be repeated depends on the study objective, setting, and the nutrients of interest (9). In population surveys in LICs, repeating the 24-h recall on each person is often not feasible. Instead, repeat recalls can be collected on a random subset of the population on nonconsecutive days, as practiced in national surveys in the Cameroon (10) and Uganda (11). The subset should consist of ≥30–40 individuals who are representative of each life-stage group in the survey to calculate the prevalence of inadequate intakes (8).

Both cultural and environmental factors (e.g., season and food security) can markedly influence the magnitude of within-person variability in food and nutrient intakes in LICs and hence the size of the subsample repeats required to accurately characterize usual intakes. Only a few studies in LICs have estimated the ratio of within- to between-person variability needed to estimate usual intakes. This ratio appears to be lower than in developed countries because there is less variety in food intake (12–15). Hence, fewer repeats may be needed to capture “usual intake.” Statistical methods exist to account for the effect of within-person variation on usual intakes (16, 17), although their use in LICs is limited.

Systematic errors or bias in self-reported 24-h recalls cannot be mitigated by averaging data on the basis of more than one 24-h recall per person. Nevertheless, some potential sources of systematic errors such as day of the week, season, region, rural or urban location, age, and sex of respondents can be taken into account by carefully designing the initial 24-h recall protocol and accounting for them in the data analyses. These are often termed “nuisance effects.” In the national food-consumption survey in Uganda, all days of the week, including the weekend, were proportionately represented to negate any day-of-the-week effects on dietary intakes (11). Seasonal effects on food intakes can be accounted for by administering the survey over a longer period of time and including randomly selected days, preferably representative of all seasons of the year. Feast days should be avoided because they often coincide with unusual dietary practices (1, 9).

Some systematic errors can be detected by including a reference measurement, free from error or with only random error, at the same time as the 24-h recall (1). Of the reference measures, doubly labeled water (DLW) has been used most frequently in LICs to assess underreporting of energy intakes (18), allowing the correction of the 24-h recall data for this systematic error. Other suitable reference measures include urinary nitrogen for protein intake and urinary potassium and sodium for potassium and sodium, respectively (19). Other investigators have used weighed records—completed on the same day as the recall—to identify systematic errors such as underreporting in 24-h recalls. Details of studies in LICs utilizing validation approaches that used both DLW and weighed records completed on the same day as the recall and DLW are summarized in Table 1.

TABLE 1.

Review of validity studies of self-reported 24-HRs in low-income countries¹

Study (ref), location (setting)	Study population; method	Results	Conclusions
Alemayehu et al. (20), Ethiopia (rural)	Women (n = 58; aged 15–49 y); compared same-day WR with 24-HR	Median 24-HR intakes were lower (P < 0.05) for energy and most nutrients compared with WRs. A negative bias for daily energy and nutrient intakes with 24-HRs vs. WRs from Bland-Altman plots was due mainly to inaccurate portion sizes.	Underreporting for 24-HRs at the group and individual levels for energy and most nutrients
Dop et al. (21), Senegal (periurban)	Children (n = 45; aged 11–18 mo); compared 2 same-day WRs with two 24-HRs	Intertechnique mean differences for intakes of selected foods (g/d) were small and for energy and macronutrients were <11% of mean intakes. Rank order correlations ranged from 0.60 to 0.81 for energy and macronutrients. Measuring rice intakes from a common pot via number of handfuls for 24-HR was not satisfactory.	No underreporting for 24-HRs at the group level for intakes of energy and macronutrients; the use of handfuls to estimate food intakes from a common pot in 24-HRs was unsatisfactory
Ferguson et al. (22), Ghana (rural)	Children (n = 72; mean age: 56.2 mo); compared 2 same-day WRs with two 24-HRs in 2 villages	Median intakes of energy, protein, iron, and zinc were lower for 24-HRs vs. WRs (P < 0.05) in one village but only lower for iron (P < 0.05) in a second village. Of energy and nutrient intakes via 24-HR, <35% were within ±10% of the WR. Overall agreement for number of foods reported in 24-HRs vs. WRs was 42%.	24-HRs may be appropriate at the group level with some modification but not at the individual level; portion-size estimates by 24-HR for main staples, soups, and fruit were poor
Ferguson et al. (23), Malawi (rural)	Children (n = 29; aged 4–6 y); compared same-day WR with 24-HR	No differences (P > 0.05) in median intakes of energy, protein, vitamin C, calcium, iron, and zinc between 24-HR and WR. Intraclass correlation coefficients ranged from 0.14 (vitamin C) to 0.55 (zinc). Percentages of energy and nutrients via 24-HR within ±10% of WRs ranged from 12% (calcium and vitamin C) to 33% (protein).	No significant underreporting at the group level for 24-HRs but some modifications needed for individual-level data
Ferguson et al. (15), Malawi (rural)	Pregnant women (n = 60; mean ± SD age: 27 ± 8 y); compared 2 same-day WRs with two 24-HRs	Median 24-HR intakes between 2 methods were comparable for calcium, iron, zinc, and manganese, but energy and protein were underestimated by 24-HR (P ≤ 0.05). Of intakes via 24-HR, <37% were within ±10% of the WR. Recall portion sizes were underestimated for cereal staples but overestimated for relishes.	No significant underreporting at the group level for minerals; need improvements in cereal staple portion-size estimates before using 24-HRs at the individual level
Gewa et al. (24), Kenya (rural)	Children (n = 42; aged 6–8 y); compared same-day WR with 24-HR between 0800 and 1700	Mean daytime 24-HR intakes were lower than daytime WRs (9% for energy) but only significant (P < 0.05) for vitamins A and C. Of daytime 24-HRs, 50% omitted ≥1 food item in the daytime WR. Spearman correlations ranged from 0.49 to 0.64. Nutrient intakes classified within the same quartile ranged from 29% to 48%. There was underreporting of food amounts by 24-HR, especially for sugar, sweets, and fats.	Intertechnique agreement at the group level was relatively high; gross misclassification was low for all nutrients; discrepancies were attributed to misreporting and inaccuracies in 24-HR portion-size estimates
Gewa et al. (24), Kenya (rural)	Mothers (n = 42); compared same-day WR with 24-HR between 0800 and 1700	Mean 24-HR intakes were lower than in the WR (6% for energy) but only significant (P < 0.05) for vitamins A and C. Of 24-HRs, 31% omitted ≥1 food item in the WR. Spearman correlations ranged from 0.43 to 0.65. Nutrient intakes classified within same quartile ranged from 31% to 48%. There was underreporting of food amounts by 24-HR, especially for dairy and beverages.	Intertechnique agreement at group level was higher than for schoolchildren; gross misclassification was low for all nutrients; discrepancies were attributed to misreporting (especially fruit) and some inaccuracies in 24-HR portion-size estimates
Nightingale et al. (25), Uganda (rural)	Children (n = 19; aged 6 mo–12 y); compared same-day WR against same-day 24-HR conducted by 2 interviewers	Intertechnique mean differences for energy, protein, and iron by Bland-Altman plots were low. Intraclass correlation coefficients for combined 24-HRs and WRs were high (>0.90). Nutrient intakes classified within same quartile were 79% for energy and 89% for protein and iron.	No significant differences at the group level, and good agreement at the individual level for energy, protein, and iron intakes
Thakwalakwa et al. (26), Malawi (rural)	Children (n = 169; aged 15 mo); compared same-day WR with 24-HR plus food calendar	Mean 24-HR intakes of energy, protein, fat, iron, zinc, and vitamin A were higher (P = 0.001) than with WRs. Intraclass correlation coefficients ranged from 0.42 (zinc and iron) and 0.73 (energy) to 0.83 (vitamin A).	Overreporting at the group level and disagreement at the individual level
Orcholski et al. (18); Ghana, Jamaica, South Africa, Seychelles, United States (rural and urban)	Adults (n = 324; mean age: 34 y); compared DLW to two 24-HRs in 5 countries in individuals of African ancestry	Energy underreporting across 5 countries, with the highest levels in South Africa (52.1%), compared with Seychelles (25.0%), Ghana (22.5%), the United States (18.5%), and Jamaica (17.9%). BMI was a predictor of underreporting in all countries.	Energy underreporting ranged from 52.1% in South Africa to 17.9% in Jamaica
Pfrimer et al. (27), Brazil (urban)	Older adults (n = 41; aged 60–70 y); compared DLW to three 24-HRs and assessed body composition with DXA	There was energy underreporting in 31.0%. Persons with higher BMI and females were more likely to underreport energy (P < 0.05).	31% energy underreporting with 24-HRs
Scagliusi et al. (28), Brazil (urban)	Women (n = 65; aged 18–57 y); compared DLW with three 24-HRs	There was energy underreporting in 24.6%. Mean energy intake via DLW: 2622 vs. 2078 kcal via 24-HR.	24.6% energy underreporting with 24-HRs

¹DLW, doubly labeled water; ref, reference; WR, weighed food record; 24-HR, 24-h recall.

Respondent bias may arise if respondents think they will receive food or financial aid if they report low food intakes. An inappropriate manner of questioning by the interviewer, such as leading questions or judgmental comments, may also lead to 2 sources of bias: social desirability (the tendency to respond in such a way as to avoid criticism) and social approval (the tendency to seek praise). Examples may include the underreporting of “bad” foods and overreporting of “good” foods, or the tendency for overweight respondents to underreport food intakes (18, 28). There is some evidence that sex and sometimes educational level influence social desirability and social approval biases (29). The inclusion of a social desirability scale such as the Marlow-Crowne Social Desirability Scale (29, 30) could be used to identify social desirability variables in 24-h recall protocols in LICs. Responses could be used as covariates in multiple regression models to control for social desirability. Respondent bias can be minimized by using a multiple-pass recall and standardized probes for descriptive details for each food and beverage. Interviews should be scheduled on randomly selected, nonconsecutive days, including weekdays, weekend days, and market days, preferably in private with no distractions. Probes for wild foods should be included, especially during the hungry season when these foods may contribute substantially to food intakes.

Nonresponse or differential response bias may occur if the respondent burden is high, interviewers make judgmental comments, or language and cultural barriers exist. The result is a biased unrepresentative sample of the studied population. Differential response bias may be of particular concern when examining associations between an independent variable and a diet-dependent variable (e.g., race/ethnicity and diet). This bias can be minimized by reducing respondent burden, offering incentives or material rewards, and training the interviewers, preferably women, who should always interview in the first language of the respondent and convey warmth, understanding, and trust.

Very few investigators have addressed nonresponse bias in 24-h recall studies in LICs. This is unfortunate because individuals who refuse to take part in the 24-h recall study or who drop out may have characteristics that differ from those of the participants. Ndekha et al. (31) compared the sociodemographic and anthropometric data of participants and nonparticipants in a study in pregnant women in rural Malawi and concluded that their study sample represented >95% of all eligible pregnant women in the area. However, in an Indonesian study of underreporting, women who participated were more urban, with fewer working in agriculture than the nonparticipants (32). Attempts should always be made to identify the reasons for nonresponse and to adjust for any nonresponse bias by reweighting the sample (33).

Interviewer biases may occur if different interviewers fail to develop rapport before initiating questioning, probe for information to varying degrees and intentionally omit certain questions, record responses incorrectly, or respond differentially to unusual food practices. Value judgements by the interviewers must always be avoided (9). Interviewer bias can be random across days and respondents, systematic for a specific interviewer, or may exist between certain interviewers and certain respondents (34). Such biases are best reduced by assigning interviewers randomly to respondents, days of the week, and areas within a country; standardizing the 24-h recall with computerized 24-h recall interviews where feasible; and training female interviewers with the use of role-playing and field practice. Women generally have the best knowledge of local food taboos, food patterns, and practices.

Conducting a 24-h recall interview is often a challenge in LIC settings because of the paucity of trained dieticians or nutritionists (4, 35). In some African studies, research assistants with a secondary-school level of education have been trained to conduct 24-h recalls (15, 20). However, only a few studies in LICs have assessed the effect of training on interviewer bias (4): 2 validation studies in Malawi and Uganda showed no evidence of interviewer bias when responses from same-day 24-h recalls by 2 trained interviewers were compared (15, 25).

Misreporting may arise from interviewer or respondent biases, respondent memory lapses, or incorrect estimation of portion sizes and can introduce severe errors not only in energy but in nutrient intakes as well. The magnitude of misreporting can be expressed as the prevalence (i.e., percentage of misreporters in the study sample) or as the extent of under- or overestimation of intake expressed as a percentage (36). Only a few studies in LICs have attempted to identify misreporting, investigate its source, and correct for it statistically by including a reference measure (Table 1). As a result, the extent to which individual foods, energy, or nutrients are affected by misreporting in LICs is uncertain. In general, the prevalence of underreporting in LICs appears to be greater than that of overreporting (36–38). Estimates for energy underreporting range, for example, from a mean percentage of −17.9% in Jamaica to −52.1% in South Africa when based on the DLW method to measure energy expenditure, as shown in Table 1 (18). Energy underreporting in 24-h recalls in LICs has also been identified by comparing 24-h recall energy intakes with same-day weighed energy intakes (15, 20, 24) (Table 1), as well as with the Goldberg cutoff method (37, 39).

Nonetheless, not all studies in LICs have documented energy underreporting. On the basis of the Goldberg criteria, only 17% of pregnant Indonesian women (32) and 10% of Egyptian women (40) reportedly had implausible energy intakes. Risk factors for energy underreporting in LICs are comparable to those documented in more-developed countries (36) and include high BMIs (18, 32, 40) and a low standard of education (32).

In general, whenever energy intake is underreported, then absolute intakes (i.e., milligrams per day) of micronutrients such as iron, calcium, and vitamin C are also lower, although rural Malawian women represented an exception because they overestimated intakes of mineral-dense relishes (15). Some investigators report micronutrient intakes as per 1000 kcal (i.e., energy adjusted), which results in less inconsistency in the direction and magnitude of the discrepancies observed (15, 20, 36).

Respondent memory lapses can affect 24-h recall protocols in 2 ways: the respondent may fail to recall foods actually consumed (errors of omission) or may report foods that were not consumed during the recalled day (errors of intrusion). These may occur because of age, pace of interview, inappropriate probes for describing foods, or the inadequate use of memory aids. The way to minimize errors generated by memory lapses and distortions is to use standardized multiple-pass interviewing techniques and “probing” questions, together with memory aids such as photographs to identify the correct species consumed (e.g., beans, fish). Selecting recall days that represent the usual eating pattern is also essential.

Several validation studies in LICs that used same-day weighed intakes as the reference method examined omissions and intrusions of foods in 24-h recalls. In LICs, errors of omission are more frequent than intrusions (15, 20, 24), with foods such as snacks, fruit, and beverages omitted more frequently than foods habitually consumed in the main meals (20, 24). To avoid omissions and intrusions in LICs, picture charts depicting the most common local foods have been introduced as a modification to the multiple-pass 24-h recall to verify the recall of foods consumed (11, 15, 26, 41, 42).

Omissions may result from the consumption of “out of home” or gathered wild foods. In Mexico, foods eaten between meals by preschoolers were not classified as foods by the mothers and were omitted from the recall (43). In some Kenyan schoolchildren, “out of home foods” such as fruit and starchy foods accounted for 77% of “out of home” energy intake and were most likely not recalled by the mothers or caretakers (24). Discrepancies have also been observed in reports by mothers of children fed by multiple caregivers (44) and have led to the practice of consensus recalls whereby both the caretakers and the children conduct the recalls to overcome this problem (45).

Incorrect estimation of portion size occurs when respondents fail to quantify accurately the amount of food consumed and is probably the largest measurement error in 24-h recalls. In studies in rural Malawi, Ethiopia, and Senegal, for example, underestimates in the portion size of the cereal staples were the major factor in the underestimate of energy (15, 20, 21). Portion-size errors can arise from poor memory, limited quantitative skills, or the incorrect use of portion-size measurement aids by interviewers. The ability of respondents to accurately estimate portion sizes visually varies with the type and size of food. Large errors may occur for estimates of foods high in volume but low in weight (e.g., leaves) and for intact cuts of meat of irregular shape. Several types of portion-size measurement aids are used in 24-h recalls in LICs, including local household utensils, drawings, graduated measuring jugs or cylinders, tape measures, and modeling clay or playdough molded into the correct size and shape of the food (8, 25). The measurement aids to be used for quantifying portion sizes should be prespecified for each food and standardized across interviewers.

More recently in LICs, context-specific graduated photographs or digital images have been used to estimate portion sizes because they can be easily carried by interviewers in an atlas or displayed on a computer or tablet (46–49). In general, validation studies have involved showing weighed portions of common foods to the respondents, followed by a display of an appropriate set of graduated, colored photographs or computer images. The ability of the respondent to choose the correct image from the set is then assessed (46, 47). In general, results have been better for clearly defined, single-unit solid foods than for amorphous foods (47, 50, 51). Memory, ability to relate a food actually present to a portion-size aid (perception), and ability to develop a mental picture of a food not present and relate it to a portion-size aid (conceptualization) may all influence the accuracy of portion-size estimates from photographs or digital images (52). Practical guidelines on how to develop a photographic atlas are given in Nelson and Haraldsdóttir (53), although there are no clear guidelines on acceptable levels of accuracy (50). The photographs should be standardized with respect to size, number, and range of portion sizes depicted; order of presentation; angle at which the photograph is taken; background; use of reference objects for scale; and the interval between the portion sizes and provide the possibility to choose fractions of each portion size. In LICs in which photo-literacy rates are often very poor, there is an urgent need for additional research on this aspect of dietary assessment. The limited research to date suggests that photographs or digital images may be a valuable tool for quantifying food-portion size at a group level (46, 54).

Several strategies have been proposed to reduce errors in estimating portion sizes in self-reported 24-h recalls in LICs. Training respondents to estimate portion sizes before conducting the 24-h recalls has been practiced among subsistence-farming women in rural Malawi (41) in view of the improvements noted after training in more-developed countries (55), although this practice is not feasible for national surveys. Calibrating local utensils in the home and weighing salted replicas of actual staple foods consumed are also recommended strategies (8). In some settings in LICs, eating from a communal plate is the normal practice, making it difficult to estimate the amount consumed. Consequently, in some studies, respondents have been supplied with plates and requested to eat their food on the recall day from the plates provided to improve the portion-size estimates (11, 15, 42). Whether these practices improve the accuracy of portion-size estimates or modify habitual dietary practices in LICs is uncertain and warrants more investigation.

Seasonality often results in marked changes in both the quantity and quality of foods consumed, especially in subsistence-farming settings in LICs where agriculture is dependent on temperature and rainfall. Large seasonal changes in energy (2, 23, 31, 56) and nutrient (3, 57–60) intakes in LICs have been reported. In rural Malawi, energy intakes of rural pregnant women were much lower during the rainy season than during the postharvest season (1520 compared with 2250 kcal) due to the almost complete disappearance of roots and tubers, fruit, legumes, and vegetables at this time (31). In rural Burkina Faso, despite no difference in energy intakes for women and children between the lean and postharvest season, most of the micronutrient intakes, with the exception of vitamins A and B-12 and zinc, were significantly higher in the postharvest season (59). Such seasonal effects can be minimized when designing the 24-recall protocol by ensuring recalls include both the lean and postharvest seasons.

The omission of information on fortified foods or vitamin and mineral supplement usage in 24-h recalls in LICs will underestimate micronutrient intakes. Interviewers must have knowledge of the locally available, fortified foods and supplements and include specific questions and probes for these items in the 24-h recall protocols. Accurate information on the brand names of the fortified foods and dietary supplements and chemical form of the supplements should be recorded during the recall interview, preferably directly from the packages in the home. Alternatively, photographs of locally available, fortified foods and dietary supplements should be carried by the interviewer who should be trained to question and record the frequency of dietary supplement use over a 1-wk period and not just on the recall day. In South Africa, despite the introduction of mandatory national wheat and maize flour fortification in 2004, some reports of micronutrient intakes postfortification have failed to include the contribution of the fortified micronutrients in their calculated micronutrient intakes from 24-h recalls (61, 62), because these fortified foods do not appear in the national food-composition database (FCDB). Similarly, in Malaysian children aged 1–3 y, supplement usage was not included in their micronutrient intakes, even though ∼56% received dietary supplements (63). Even in some European national surveys, the nutrient contribution from dietary supplements has been ignored (64).

The omission of biodiverse foods may also result in errors in nutrient intakes. Wild or underutilized foods together with details on the specific varieties of foods are often not recorded in 24-h recalls, despite the nutrient composition of varieties of the same food varying as much as the differences across foods (65, 66). Guidelines on how to assess biodiversity in dietary assessment methods have been published by FAO and Bioversity International (67) in an effort to emphasize its importance.

Errors Associated with Conversion of Food Intakes to Nutrient Intakes

There are 4 potential sources of error that may occur when converting food intakes from 24-h recalls to nutrients. These are described below together with strategies to minimize these errors.

Inaccuracies in converting portion sizes to weight equivalents will occur if investigators fail to do the following: 1) prespecify methods for estimating portion size for each food item, 2) weigh portion sizes of staple foods directly by using dietary scales, 3) calibrate local household utensils in the home, and 4) compile specific gravity factors for locally prepared staple foods. Accurate estimates of the portion sizes of the major dietary staples consumed in LICs are essential because they frequently contribute ≥50% of total dietary energy (15, 18, 20). As a result, replicas of the major cereal staples preserved with salt (i.e., salted replicas) are often used in 24-h recall interviews in LICs so that respondents can place the amount they consumed into their own dish, which is then weighed directly with the use of dietary scales. For thinner cereal porridges, the volume consumed can be converted to weight equivalents, preferably by applying a locally derived specific gravity factor or a factor accessed from the FAO/International Network of Food Data Systems (INFOODS) Density Database (68). Local specific gravity factors can also be used to convert the volume of playdough replicas of food items to weight equivalents. Weighing foods equivalent to the monetary value of purchased food or applying weight-equivalent factors derived from food-composition tables (FCTs) or FCDBs can also be used. Increasingly, graduated photographs or digital images of portion sizes with associated weights in grams are being used from which the respondent can choose a specific portion or a fraction of each portion size [see Gibson and Ferguson (8) for more details].

Inaccuracies for nutrient contents of mixed dishes may arise because of failure to record accurately all of the ingredients and their amounts, the weight of the final cooked dish, or both. In addition, because no comprehensive database for yield and retention factors of indigenous mixed dishes in LICs exists, with the exception of Thailand (69), the use of yield and retention factors is often ignored or, when applied, they are derived from Western countries (70–74). Hence, such factors may be inappropriate for local preparation and cooking methods (10, 11, 41), resulting in incorrect estimates for the nutrient content of cooked mixed dishes. The recommended approach for calculating the nutrient composition of mixed dishes from recipes is the FAO mixed-method approach because it takes into account the weight changes at the total-recipe level but nutrient retention at the ingredient level (68). Care must be taken when using this approach to include all of the ingredients (especially water or fat) and to match each ingredient with the appropriate food item in the FCDB. Details of the correct method for calculating the nutrient contents of mixed dishes are available in the FAO/INFOODS E-learning course on food-composition data (75) and the Web-based interface FoodBasket, developed by the EuroFIR (European Food Information Resource) (76).

Collecting accurate recipe data during 24-h recalls can be challenging, especially when respondents are not very numerate or not responsible for the meal preparation. Instead, the nutrient composition of local mixed dishes can be based on indigenous recipes compiled earlier (20, 22, 24), as described by Gibson and Ferguson (8). To enhance precision, multiple entries for each local mixed dish containing varying proportions of ingredients can be included in the FCDB (77), although this practice is rarely done. Few studies have compared intakes on the basis of standard recipe data compared with individual recipes collected via 24-h recalls in the households, and more research on this practice is warranted.

The quality of FCTs and FCDBs can be compromised by both systematic and random errors, generating additional uncertainty in calculated nutrient intakes. In many LICs, the FCT or FCDB consists of a very restricted number of foods (78), often based on nonrepresentative samples, and with nutrient values borrowed from other sources that have not necessarily been analyzed by methods accredited by the International Association of Official Analytical Chemists (79). In an effort to address these challenges, the INFOODS was created in 1984 (79).

Systematic errors may arise from inconsistencies in the terminology used for certain nutrients (e.g., niacin equivalents compared with niacin expressed in mg) or inappropriate analytical methods, such as those that assay crude fiber on the basis of cellulose and lignin only instead of total dietary fiber. Most of the other errors in FCTs and FCDBs are random, arising from poorly documented or missing values for a range of foods or nutrients. Increasingly, in many LICs, a wide range of biodiverse foods are consumed for which nutrient composition data are not available, which necessitates substitution with nutrient values from the USDA (70) or European FCDBs (80). This practice may lead to underestimates because traditional varieties often have a lower water content (i.e., higher nutrient density) than high-yielding varieties used in Western countries (81). Commercially produced snack foods are also increasingly available in LICs (82). Some of these products are fortified, but their micronutrient content is not necessarily displayed on the label or available from the manufacturer. As a result, appropriate adjustments for the fortificants may not be included in national FCDBs (61, 83).

Several guidelines and tools on the production, management, and use of food-composition data are available (68, 75, 84–93). The adoption of these guidelines is essential to facilitate the publication of harmonized and well-documented food-composition data that can be exchanged across countries for comparison of nutrient intakes. Several LICs, including Armenia, Lesotho, Bangladesh, and India, have collaborated with FAO/INFOODS to produce revised FCDBs with the use of these published guidelines. However, in view of the expenses involved in creating a national FCDB, the compilation of regional FCDBs, such as the West African Food Composition Table (94), may be a more feasible option.

Currently, the WHO and FAO are collaborating with the Food Monitoring Group and the George Institute for Global Health in Australia to create a database for commercially produced branded foods. To date, 24 countries are participating, including several LICs, with the use of an online data entry system to collect information from nutrition labels on the products, supplemented by information obtained from websites or the manufacturer, where necessary. The data in Australia have been collected by using smartphones and barcode scanning, which are procedures planned in the future for other countries (95). Nevertheless, nutrients on labels are limited in number (often 5) and not always reliable.

Coding errors may occur if descriptions of foods are inadequate or ambiguous in the survey instrument, if the FCDB used is not comprehensive, or both. Coding of reported foods and beverages during 24-h recall interviews can be automated where feasible by the interviewers by selecting food items from computer-based pull-down menus. Nonspecified generic foods should be included in the pull-down menus for use when respondents are unable to describe foods adequately. Food matching between reported foods and those in an FCDB can be a large source of error (96), especially when performed by inexperienced professionals who lack knowledge of the foods under investigation. To reduce this error and harmonize food matching, FAO has published food-matching guidelines (91). Supplying the interviewers with pictures, photographs, or samples of actual foods (e.g., varieties of beans or leaves or dried fish) is a strategy that facilitates the identification and matching of the actual foods consumed with the correct food items listed in the database. More recently, the European Food Safety Authority (97) developed the FoodEx2 food classification and description system to facilitate food matching and comparison of dietary and exposure data across countries and over time.

At the end of each 24-h recall interview, the energy intake of each respondent should be calculated, where feasible, so that any outliers can be identified against reference energy requirements on the basis of age, sex, weight, and height and the 24-h recall data rechecked with the respondent before finalizing the 24-h recall interview. The capacity to adopt this strategy will be enhanced as the application of new technology for collecting 24-h recalls in LICs increases.

Implications of Measurement Errors in 24-h Recalls when Evaluating Nutrient Intakes

The recognition of both random and systematic measurement errors in self-reported 24-h recalls is important because their existence can have serious consequences for the interpretation of the dietary data unless methods are used to mitigate the errors. As noted earlier, an appropriate study design and the standardization of the 24-h recall protocols can reduce errors. Nevertheless, in some LICs, underreporting of energy intakes may persist even after implementing these strategies. Indeed, the existence of energy underreporting has led some investigators to suggest that energy intakes from self-reported 24-h recalls should not be used to evaluate energy balance in large-scale surveys. Instead, body weight together with BMI and waist circumference are the recommended measurements. In view of these uncertainties, investigators working in LICs are urged to recognize the limitations of the data from 24-h recalls and to analyze and interpret them appropriately (98). Here, we outline the methods that have been developed to reduce the impact of measurement error on assessing nutrient adequacy from the distribution of usual intakes and relations between diet and health on the basis of estimates of usual intakes at the individual level.

Implications for assessing nutrient inadequacy and excess

In rural settings in LICs, dietary diversity is often low, so within-person variation in food intake is often assumed to be lower than between-person variation. As a result, only a few studies in LICs have adjusted the observed distribution of intakes to remove the variability introduced by day-to-day variation in intakes within an individual (i.e., to remove the within-person variation) from the total variation (15, 26, 59, 99–101). However, without such an adjustment, the variance of the intake distribution based on single 24-h recalls will be larger, yielding biased estimates of the proportion of individuals with intakes below or above a given cutoff. In rural Malawi, for example, the prevalence of inadequate intakes of protein among rural pregnant women was 88% when based on 24-h recalls without variance adjustment compared with 61% after variance adjustment (15). Such an adjustment can be performed, provided some repeat 24-h recalls are available on at least a representative subsample of the study population. Several adjustment methods have been reviewed by Souverein et al. (16) and Laureano et al. (17). For episodically consumed foods or nutrients (e.g., vitamin A), the statistical modeling method developed by the US National Cancer Institute (102) or the Multiple Source Method (103) is recommended. Recently, these adjustment methods were used in LICs to provide estimates of the usual intakes of nutrients from national food-consumption surveys for specified age- and sex-specific subgroups (11, 42, 104, 105). In the absence of replicate observations from which to calculate within-person variance estimates for the population group under study, external variance estimates of within-person variation can be used (106). For the national food-consumption survey in Ethiopia (105), variance estimates from the Ugandan national food-consumption survey (11) were applied for the adjustment in conjunction with the WHO software IMAPP (Intake, Monitoring, Assessment, and Planning Program) (107).

After adjustment, the distribution of “usual” nutrient intakes can be used to predict the proportion of the population at risk of nutrient inadequacy or excess. For the former, the Estimated Average Requirement (EAR) cutoff method or the full-probability approach should be used. The EAR cutoff method is applied when the distribution of nutrient requirements in the group is symmetrical about the EAR, whereas the full-probability approach is used for the iron intakes of children and menstruating women because their iron requirements are not symmetrical about the EAR. To predict the risk of excessive intakes, the proportion of the population with intakes above the Tolerable Upper Intake Level can be assessed. For further details on these approaches, see Gibson and Ferguson (8), the Institute of Medicine (108), and Otten et al. (109).

Underreporting of energy intakes will bias the distribution of intakes and affect the mean, median, percentiles, and the percentage of persons with nutrient intakes below the EAR (or above the Tolerable Upper Intake Level). Although this bias can be adjusted statistically by measuring energy expenditure with the use of DLW on at least a subset, the DLW method is too costly and impractical for large-scale national surveys. As a result, alternative approaches have been proposed to identify underreporting in community-based surveys. They include the Goldberg cutoff method, whereby the ratio of reported energy intake to estimated basal metabolic rate is calculated, preferably selecting a cutoff assigned to the physical activity level (PAL) that reflects the population under study. The approach is based on the principle that an individual of a given age, sex, and body weight has a certain minimum energy requirement. Intakes below this level are considered to be an unacceptable representation of the habitual intake and incompatible with long-term survival. The method can be used at the group level to determine the probable degree of overall bias to reported intake, provided the comparison is made with a PAL value suitable for the study population. However, the Goldberg cutoff method has limited sensitivity for identifying underreporters at the individual level, unless a short questionnaire designed to assign participants to a low, medium, or high PAL is used to calculate the appropriate Goldberg cutoff. More details are given in Black (39). Some investigators have excluded underreporters identified by this method from the data set. However, such an approach introduces an unknown bias and is not recommended. An alternative is to compare the prevalence of inadequate intakes with and without underreporters and use the difference as part of an uncertainty evaluation (36).

Implications for assessing diet-health relations

Failure to account for measurement errors in 24-h recalls can also profoundly affect the analysis and interpretation of studies designed to investigate diet-health relations. Correlation or regression coefficients between nutrient intake and the outcome variables are attenuated if day-to-day within-person variation in nutrient intake is ignored. This attenuation bias may obscure important associations between diet and health (110). Tables of attenuation factors are available in Anderson (34) and can be used to correct both simple correlation and linear regression coefficients as long as the sample size is >100 and the variance ratio (i.e., ratio of within-person to between-person variation) has been calculated (108).

Complex statistical techniques have also been developed to correct RR estimates for both continuous and categorical variables to better assess diet-health associations (110, 111). The selected technique used depends on the study objective and may include a standard multivariate model (112), an energy-partition model (113), and a residual model (114). Some of these techniques combine data from multiple 24-h recalls with an FFQ that focuses on less frequently consumed foods (115). However, to date, few studies in LICs have used these complex statistical techniques to correct for attenuation bias or combined the data from multiple 24-h recalls and an FFQ by using regression techniques. For further details on these approaches, readers are advised to consult a statistician.

Conclusions

This review underscores the urgent need to adopt standardized protocols not only for the collection of dietary intakes by 24-h recalls for population-based studies in LICs but also for the analysis and evaluation of the nutrient intake data generated. Strategies are available to address the errors inherent in dietary assessment based on self-reported 24-h recalls, but to date, their use in LICs has been limited. Increasingly, with the application of new technology, the capacity to standardize the collection and analysis of 24-h-recall data in LICs will be enhanced. Nevertheless, such technology will not overcome all of the sources of measurement error inherent in dietary studies. There will continue to remain a need to assess and evaluate the impact of measurement errors on self-reported 24-h recalls collected by using these new technologies. Such data play an essential role in informing national food policy, planning food fortification, and examining compliance with national food-based dietary guidelines. Food-consumption patterns associated with inadequate nutrient intakes can also be identified and used to design food-assistance programs, improve nutrition education and agricultural policy for improved nutrition outcomes, and to elucidate associations between diet and health.