ABSTRACT
Background
Few low-burden indicators of diet quality exist to track trends over time at low cost and with low technical expertise requirements.
Objective
The aim was to develop and validate a suite of low-burden indicators to reflect adherence to global dietary recommendations.
Methods
Using nationally representative, cross-sectional, quantitative dietary intake datasets from Brazil and the United States, we tested the association of food-group scores with quantitative consumption aligned with 11 global dietary recommendations. We updated the Healthy Diet Indicator (HDI) to include current quantifiable recommendations of the WHO (HDI-2020). We developed 3 food-group–based scores—an overall Global Dietary Recommendations (GDR) score as an indicator of all 11 recommendations composed of 2 subcomponents: GDR-Healthy, an indicator of the recommendations on “healthy” foods, and GDR-Limit, an indicator of the recommendations on dietary components to limit. We tested associations between these scores and the HDI-2020 and its respective subcomponents. We developed 9 dichotomous food-group–based indicators to reflect adherence to the global recommendations for fruits and vegetables, dietary fiber, free sugars, saturated fat, total fat, legumes, nuts and seeds, whole grains, and processed meats. We conducted receiver operating characteristic and sensitivity-specificity analyses to determine whether the dichotomous indicators were valid to predict adherence to the recommendations in both countries.
Results
The GDR score and its subcomponents were moderately to strongly associated with the HDI-2020 and its respective subcomponents (absolute values of rank correlation coefficients ranged from 0.55 to 0.66). Of the 9 dichotomous indicators, 8 largely met the criteria for predicting individual global dietary recommendations in both countries; 1 indicator (total fat) did not perform satisfactorily.
Conclusions
Food-group consumption data can be used to indicate adherence to quantitative global dietary recommendations at population level. These indicators may be used to track progress of countries and populations toward meeting WHO guidance on healthy diets.
Keywords: World Health Organization, dietary diversity scores, food groups, global dietary recommendations, diet quality monitoring, diet quality questionnaire, DQ-Q
Introduction
Diet quality is a multifaceted construct. According to the WHO, a healthy diet “helps protect against malnutrition in all its forms, as well as noncommunicable diseases (NCDs), including diabetes, heart disease, stroke and cancer” (1). Metrics of diet quality are important to monitor one of the largest public health risk factors and should reflect protection of health against diet-related NCDs, as well as nutrient adequacy.
Low-burden metrics to measure diet quality are critical to enable monitoring of diet quality, as many governments and survey and research efforts do not have financial and/or technical capacity for quantitative dietary intake surveys. Low-burden metrics can be easily calculated using minimal data needs, such as food-group-level data. Currently, there is only 1 low-burden diet-quality indicator with widespread use, the Minimum Dietary Diversity indicator for Women (MDD-W) (2, 3). The MDD-W is useful for the micronutrient adequacy facet of diet quality, suggested for global use in low- and middle-income countries, but it is not strongly correlated with NCD risk (4) and therefore is insufficient as an indicator of total diet quality. Previous research has shown that it is possible to identify specific foods that are associated with healthy or unhealthy diet patterns overall (5).
Many diet-quality indicators have been created to summarize diet quality; however, most require quantitative dietary intake data as an input. Those that have been used most extensively include the Diet Quality Index (DQI) (6, 7) and DQI-International (8), the Healthy Eating Index (HEI; HEI-2005, HEI-2015) (9–11) and Alternative HEI (12), the Mediterranean Diet Score (13, 14), and the Healthy Diet Indicator (HDI) (15–17). All of these indicators are based on quantitative intakes of foods and/or nutrients. All have undergone 1 or more revisions to incorporate updated dietary guidance. The latter, the HDI, was originally created to reflect WHO 1990 dietary recommendations (18), updated to reflect WHO 2003 (19) and 2015 dietary recommendations, and validated as an indicator predictive of morbidity and mortality in prospective cohort studies (15, 17, 20) and also associated with micronutrient intake (16).
We sought to develop a suite of low-burden, food-group–based indicators to reflect healthy diets that could be used globally. WHO dietary recommendations are expressly intended for global application among all member states, and therefore form an appropriate basis for construction of a suite of diet-quality indicators for use across countries. Current WHO guidance (1) includes recommendations to consume fruits and vegetables, legumes, nuts and seeds, and whole grains, and to limit consumption of salt, free sugars, saturated and trans fats, and total fat (1). A previous recommendation on fiber intake remains in line with the current guidance (19). Additionally, the World Cancer Research Fund has made recommendations on limiting red meat and avoiding processed meat based on evidence published by the WHO International Agency for Research on Cancer (21, 22). These global recommendations are generally based on evidence related to risk of diet-related NCDs (19, 21, 23–26). We aimed to develop indicators to reflect each global recommendation individually and also to reflect all global recommendations together in a combined score, using an updated HDI as the quantitative diet-quality standard.
Similar to the MDD-W, the suite of diet indicators developed here uses food-group consumption data based on a low-burden survey questionnaire asking about foods consumed in the previous day, with “yes” or “no” questions for the intake of relevant food groups. The indicators were developed simultaneously with a Diet Quality Questionnaire module (DQ-Q) designed for incorporation into multitopic surveys such as the Gallup World Poll (27, 28). The intent is that the resulting suite of indicators will be highly feasible to collect and calculate and can be used to monitor adherence to dietary recommendations at the population level to provide information about the nature of diet quality in a population and change over time.
Methods
We developed 12 food-group based proxy indicators and tested their validity at population level against outcome indicators of diet quality:
A Global Dietary Recommendations (GDR) score, which was tested against an updated HDI (HDI-2020) that reflects 11 global dietary recommendations.
GDR-Healthy, a subcomponent of the GDR score that reflects 5 global recommendations on “healthy” foods; we tested it against the respective subindex of the HDI-2020.
GDR-Limit, a subcomponent of the GDR score that reflects 6 global recommendations on dietary components to limit; we tested it against the respective subindex of the HDI-2020.
Nine food-group-based indicators to reflect adherence to individual global recommendations for 1) fruits and vegetables, 2) dietary fiber, 3) free sugars, 4) saturated fat, 5) total fat, 6) legumes, 7) nuts and seeds, 8) whole grains, and 9) processed meats; each of these 9 food-group–based indicators was tested against the respective individual global recommendation.
The methodology for updating the HDI, food-group indicator construction, the datasets, analytical strategy and statistical methods are described in detail in the following sections.
Updating the HDI (HDI-2020)
To validate low-burden proxy indicators against a standard of diet quality, we used the HDI, which is an index of WHO global dietary recommendations for the prevention of chronic disease from 1990 (15, 18), 2003 (17, 19), and 2015 (16), and updated it to include current WHO recommendations (1) and 2 other current global recommendations on red and processed meat based on the WHO International Agency for Research on Cancer (21, 22, 23), as shown in Table 1. The resulting updated HDI, which we call HDI-2020, is shown in Table 2. The recommendations shown in Table 1 are included in the index, except for the recommendation on industrial trans fats. It is difficult to estimate industrial trans fat consumption using dietary data due to differences in food composition; it is not contained in the datasets used in this paper. It is more appropriately monitored via presence of national policy/bans (29). Every dietary component is equally weighted and expressed as a simple dichotomous score (0/1) for whether each dietary recommendation was met, as was done in the original HDI (15). We initially created and tested an index with continuous components for the plant foods and fiber, where scoring was continuous between 0 and 1 [following the method of (17)]; the results of all analyses were similar to the 0/1 scoring, so we retained the simple scoring. It is unknown whether the individual recommendations are equally important for health; in the absence of a strong and consistent evidence base for unequal weights, we used equal weights by default. We also created 2 HDI-2020 subindexes: one for meeting the recommendations for healthy dietary components (1–5 in Table 2) and one for meeting the recommendations for dietary components to limit (6–11 in Table 2). The HDI-2020 and subindexes for adherence to global dietary recommendations provide a standard against which to test our candidate indicators for the GDR score, GDR-Healthy, and GDR-Limit.
TABLE 1.
Dietary element | Global recommendation (quoted from the WHO Healthy Diet Fact Sheet 2018, except where noted) | |
---|---|---|
1 | Fruits and vegetables | ≥400 g/d |
2 | Beans and other legumes | “A healthy diet contains fruits, vegetables, legumes (e.g., lentils, beans),… |
3 | Nuts and seeds | … nuts… |
4 | Whole grains | … and whole grains (e.g., unprocessed maize, millet, oats, wheat, brown rice).” |
5 | Dietary fiber | >25 g/d (or 12.5 g/1000 kcal, considering the recommendation of 2000 kcal/d for adults) (19) |
6 | Total fat | <30% total energy |
7 | Saturated fat | <10% total energy |
Unsaturated fats | Replacing saturated fats and trans fats with unsaturated fats | |
8 | Salt | <5 g/d (<2000 mg sodium/d) |
9 | Free sugars | <10% total energy |
[Limit] the consumption of foods and drinks containing high amounts of sugars (e.g., sugar-sweetened beverages, sugary snacks, and candies) | ||
10 | Processed meat | Consume very little, if any, processed meat: 0 g/d (22) |
11 | Unprocessed red meat | ≤350–500 g/wk [(22), supported by (21, 23)] |
trans Fats | <1% total energy; eliminate industrially produced trans fats |
TABLE 2.
Dietary element | Criteria for scoring (quantitative intake in one day) | Scoring2 | |
---|---|---|---|
1 | Fruits, vegetables | ≥400 g | 0/1 |
2 | Beans and other legumes | >0 g | 0/1 |
3 | Nuts and seeds | >0 g | 0/1 |
4 | Whole grains | >0 g | 0/1 |
5 | Dietary fiber | >25 g | 0/1 |
6 | Total fat | <30% total energy | 0/1 |
7 | Saturated fat | <10% total energy | 0/1 |
8 | Dietary sodium | <2g sodium | 0/1 |
9 | Free sugars | <10% total energy | 0/1 |
10 | Processed meat | 0 g | 0/1 |
11 | Unprocessed red meat | ≤71 g3 | 0/1 |
Elements 1–5 are used in an HDI-2020 subindex for healthy dietary components (maximum score, 5), and elements 6–10 are used in an HDI-2020 subindex for dietary components to limit (maximum score, 6).
Total index score: minimum, 0; maximum, 11.
Upper end of the recommendation to consume no more than 350–500 g/wk (22), divided by 7 d.
Indicator construction
We constructed a number of candidate indicators to reflect 9 out of 11 recommendations individually, and also to reflect all 11 global recommendations together in a combined score (the GDR score) and 2 subcomponents (GDR-Healthy for healthy foods and GDR-Limit for foods to limit) (Table 2). We validated this set of healthy diet indicators against quantitative intakes aligned with global dietary recommendations for 11 different foods and nutrients (Table 2). Each indicator candidate is a variant including 1 or more food groups related to the recommendation(s) of interest, where some food groups may be combined and/or others dropped, according to 1) normative considerations and 2) analysis of the associations of each food group with the outcome of interest (see Supplemental Box 1 for details). Within the candidate indicators for the GDR score, the food groups with the highest positive and negative correlations with the HDI-2020 were prioritized (see the highlighted coefficients for correlations of food groups with the HDI-2020 in Supplemental Table A1). The same strategy was followed for constructing scores that are supposed to reflect individual global dietary recommendations (correlations of food groups with each individual dietary recommendation can be found in Supplemental Table A2).
We also experimented with complex weighting schemes, such as fractional or integer weights based on the strength of association of each food group with the outcome indicator of interest (for examples, see Supplemental Table A3). We found that using such weights generally increased the strength of association of the food-group scores with the respective outcome indicators, but by only a small margin. Weighted food-group scores also provided more finely graduated scales that made it easier to find suitable cutoff points for dichotomizing proxy indicators. However, we concluded that these potential advantages did not justify the use of complex weighting schemes in simple food-group–based indicators. To preserve simplicity while reaping some benefits of weighting, we opted for a simpler approach: We tested candidate indicators with double weights for food groups that were, on average, most highly correlated with the outcome of interest if we expected to find similar associations in other countries (Supplemental Table A3).
Indicators of individual recommendations
We constructed dichotomous indicators for each of 9 individual global dietary recommendations designed to track population adherence to specific WHO dietary recommendations. Four of the individual recommendations can be assessed directly from food-group consumption (dietary elements 2–4, and 9 below), and the others may be predicted from multiple food groups (candidates for each indicator are listed in Supplemental Tables A4–A8):
WHO-Fruits and Vegetables (WHO-FV) indicator: based on the intake of up to 7 groups of fruits and vegetables, to create a dichotomous score that predicts consumption of ≥400 g of fruits and vegetables (Supplemental Table A4).
Consumption of any legumes.
Consumption of any nuts/seeds.
Consumption of any whole grains.
WHO-Fiber indicator: based on the intake of foods rich in fiber (e.g., legumes, whole grains, fruits, vegetables), to create a dichotomous score that predicts consumption of >25 g of dietary fiber (Supplemental Table A5).
WHO-Fat indicator: based on the intake of foods rich in fat (e.g., processed meat, red meat, deep-fried food, fast food, baked sweets and other sweets, and possibly other food groups), to create a dichotomous score that predicts consumption of <30% dietary energy from fat (Supplemental Table A6).
WHO-Saturated Fat (WHO-SatFat) indicator: based on the intake of foods rich in saturated fat (e.g., processed meat, red meat, cheese, baked sweets and other sweets, fast food, and possibly other food groups), to create a dichotomous score that predicts consumption of <10% of energy from saturated fat (Supplemental Table A7).
WHO-Sugar indicator: based on the intake of up to 6 groups of sugary foods, to create a dichotomous score that predicts consumption of <10% of energy from free sugars (Supplemental Table A8).
Consumption of any processed meat.
Two global recommendations for which we did not attempt to construct dichotomous indicators are those on sodium intake and red meat intake. We did not construct a metric for the WHO target for sodium (25) because sodium content of food is not well predicted by quantitative dietary intake data, due to variations in products, limitations in food-composition data, and additions “to taste” during cooking or at the table, which is variable by culinary/cultural context (30, 31). We did not construct a metric to reflect the maximum weekly range for red meat because the exposure information is insufficient: a simple yes/no answer about red meat consumption does not give adequate information to determine whether the amount consumed was below or within the maximum recommended range of 350–500 g/wk.
Indicators of the combined global dietary recommendations
We constructed several candidates for a proxy indicator (the GDR score), a continuous score designed to reflect global dietary recommendations altogether. The GDR score is composed of 2 subscales, as follows:
The GDR-Healthy score: based on 5 global recommendations on nutritious foods for healthy diets (when used on its own, this indicator is also called FLAVOURS: Fruits, Legumes and Vegetables; Orange produce; Un-Refined grains; Seeds and nuts). These include fruits and vegetables, whole grains, legumes, and nuts and seeds. The main consideration for GDR-Healthy indicator candidates was whether predictive power was improved by grouping some of the 7 fruit and vegetable groups together or keeping them all separate (Supplemental Table A9).
The GDR-Limit score: based on 6 global recommendations on dietary components to limit [when used on its own, this indicator is also called FAD (Foods to Avoid or limit)]. These included foods that are directly named in the global recommendations (processed meat, red meat) and other food groups that are high in sugar, salt, total fat, or saturated fat (such as sugar-sweetened beverages, baked or other sweets, salty packaged snacks, deep-fried food, instant noodles/soup, and fast food). The main consideration for the GDR-Limit indicator variants was whether predictive power was improved from including, excluding, or combining food groups (Supplemental Table A10). In addition, we tested an indicator where processed meats received a double weight, based on the following considerations: 1) among the food groups included in the candidate indicators for the GDR score, processed meat has the highest average correlation coefficient with the HDI-2020 (Supplemental Table A3); 2) in both Brazil and the United States, the relatively strong association of processed meat with the HDI-2020 can be traced back to correlations of processed meat with individual global recommendations for total fat, saturated fat, and sodium intake (and, of course, processed meat) (Supplemental Table A2); we expect to find similar patterns in other countries; 3) considering the difficulty to measure sodium intake in dietary intake studies, processed meat may contribute more to high sodium intakes than our results imply.
In the absence of a strong and consistent evidence base for unequal weights, we used simple subtraction of GDR-Healthy minus GDR-Limit to compute the GDR score. The GDR score candidates tested were composed of the most promising 2 to 3 GDR-Healthy and GDR-Limit candidates (Supplemental Table A11). The “GDR-Healthy 9e − GDR-Limit 8aW” candidate indicator, for example, has a theoretical minimum of −9 and a theoretical maximum of +9. The theoretical maximum would be achieved by consumption of each of 9 food groups (dark-green leafy vegetables, vitamin A–rich orange-colored vegetables, other vegetables, vitamin A–rich fruits, citrus fruits, other fruits, legumes, nuts and seeds, whole grains) and by no consumption of each of 8 food groups [sugar-sweetened beverages, grain-based sweets, other sweets, unprocessed red meat, processed meat (double weighted), deep-fried foods, fast food or instant noodles, packaged salty snacks].
Exploring a dichotomous GDR score
Dichotomous indicators allow the calculation of population prevalence rates and can be helpful for advocacy purposes. We explored whether it was possible to construct a dichotomous GDR score that was meaningful against cutoffs of the HDI-2020. Before we could test a dichotomous indicator derived from the GDR score, we needed to decide how to dichotomize the HDI-2020. To our knowledge, no cutoffs that are associated with morbidity and mortality have been established, although the associations of the HDI and other diet quality indexes with NCDs have been extensively studied (32). The cutoffs of the HDI-2020 that we tested should therefore be regarded as experimental and are based on the distributions of the index in the study countries. These considerations are described in the Results section.
Datasets
The indicators were validated by analyses of cross-sectional, nationally representative quantitative dietary data from the US NHANES (pooled data from the NHANES cycles 2009–2010, 2011–2012, and 2013–2014) and the Brazil National Dietary Survey (2008–2009). Food-group consumption (yes/no) and quantities of food groups and nutrients consumed were determined using quantitative 24-h recall data in the United States and quantitative self-reported food record data in the Brazil dataset (33) (see Supplemental Box 2 for further information on the data collection, data quality control, and the food-composition tables that were used). Ages <15 y were excluded, resulting in total sample sizes of 17,887 individuals in the United States and 30,062 individuals in Brazil. These datasets are freely available.
Analytical strategy and statistical methods
We analyzed the performance of metrics against the HDI-2020 and individual WHO recommendations, using rank correlations, ORs, receiver operating characteristic (ROC) analysis, and sensitivity-specificity analysis. In line with the approach in the WDDP (34), sampling weights were not applied when examining the associations of indicators (correlations, ORs, ROC, and sensitivity-specificity analysis), but they were used when generating descriptive statistics, such as prevalence rates and mean values. All analyses were performed using Stata version 13 (StataCorp). For any statistical test, a P value <0.05 was considered significant.
In addition, we examined the associations of the GDR score, the GDR-Healthy, and GDR-Limit with other measures of diet quality, including the Food Group Diversity Score (FGDS), which is the continuous score that is used to derive the dichotomous MDD-W, and percentage of energy from ultra-processed foods (UPFs), as well as dietary energy intakes and BMI. We also determined the association of the FGDS with the HDI-2020 and its subindexes. In doing so, we sought to answer the following questions:
How strongly do the GDR score, GDR-Healthy, and GDR-Limit correlate with the FGDS? How large is the difference between the correlation of the FGDS with the HDI-2020, and the correlation of the GDR score with the HDI-2020?
How well do the GDR score, GDR-Healthy, and GDR-Limit reflect the presence of UPFs in the diet?
Does the GDR score have a positive correlation with energy intakes and BMI?
We expect that the GDR score correlates more strongly with the HDI-2020 than the FGDS because the FGDS was validated as a measure of micronutrient adequacy (34, 35) and is not meant to reflect global dietary recommendations. Yet, since both the FGDS and the GDR score are simple, food-group–based measures that include many of the same food groups, we need to answer the question of redundancy: Do the GDR score, GDR-Healthy, and GDR-Limit provide enough added value to justify their introduction as new measures of diet quality that would complement (but not replace) the FGDS and MDD-W?
UPFs are defined as industrial formulations that, in addition to salt, sugar, oils, and fats, include substances that are not used in culinary preparations, especially additives used to imitate sensorial qualities of minimally processed foods and their culinary preparations (36, 37). The percentage of energy from UPFs is not the yardstick against which we validate the GDR score, but a robust negative association of our score with this measure would be desirable because high UPF consumption indicates poor-quality diets (31, 38, 39). In contrast to constructing simple food-group–based measures, computing the percentage of energy from UPFs requires detailed data on quantitative dietary intakes.
Dietary energy intakes tend to be positively associated with dietary diversity (34, 35). Considering the health risks of excessive energy intakes, overweight, and obesity, we would prefer to find no association, or a negative association, of the GDR score with energy intakes. In the same vein, we investigated the association of the GDR score with BMI.
Rank correlations
The rank correlation analysis helps to construct candidate indicators (see Supplemental Box 1) and identify the candidate indicators that are most strongly associated with the outcomes of interest. No threshold has been specified for an acceptable minimum size of rank correlation coefficients of candidate and outcome indicators for this type of analysis. Based on the precedent in the WDDP (3), we were looking for correlation coefficients of ≥0.40 for the relation between the GDR score and HDI-2020. For the associations of GDR-Healthy and GDR-Limit with the HDI-2020, and for the associations of additional submetrics with individual global recommendations, we expected that the size of the correlation coefficients would reach absolute values of at least 0.30, based on our own preliminary analyses of data from the United States and Brazil. Rank correlation analysis was also used to examine the associations of the GDR score and its components with the FGDS, percentage of energy from UPFs, dietary energy intakes, and BMI.
Odds ratios
We compute ORs for the food groups and candidate indicators, in order to corroborate the direction and strength of their associations with the outcomes of interest that were found in the rank correlation analysis. As in rank correlation analysis, there are no thresholds for ORs that indicate an acceptable performance of candidate indicators.
ROC analysis and sensitivity-specificity analysis
ROC analysis was performed to evaluate overall indicator performance and derive the AUC, a test statistic that summarizes the predictive power of each metric across all of its cutoffs. The AUC should be significantly different from 0.50 (a neutral value with no predictive power) and amount to at least 0.70 to indicate some promise for the metric (34, 35, 40). With regard to sensitivity-specificity analysis, we applied largely the same criteria and definitions that have been used in the WDDP (35), adapting them to our analysis (see Supplemental Box 3 for details). We aimed for a balance of sensitivity and specificity when selecting cutoffs, with both measures at 60% or higher, and considered a level of misclassification of up to 30% (i.e., the sum of false negatives and false positives expressed as a proportion of all observations) acceptable (34, 35). Following the example of the WDDP, we relaxed our requirements somewhat if no cutoffs could be found for the candidate indicator where both sensitivity and specificity are at 60% or higher: we also accepted combinations of 50% sensitivity and 60% specificity and of 60% sensitivity and 50% specificity (34). These criteria seem appropriate for indicators that are developed for population-level assessments and have lower requirements than indicators designed for individual targeting or screening (35).
Results
Descriptive statistics
Descriptive statistics show the results of the analysis of national survey data. We present descriptive statistics for food groups consumed, the HDI-2020, the individual global dietary recommendations, the FGDS, MDD-W, percentage of energy from UPFs, dietary energy intakes, BMI, overweight, and obesity to characterize the diets and nutritional status of the population in the 2 study countries.
The most commonly consumed food groups in Brazil are foods made from grains (99% consumed), sweetened coffee/tea/milk (81%), beans and other legumes (79%), unprocessed red meat (57%), milk (55%), and other vegetables (40%); the most commonly consumed food groups in the United States are foods made from grains (97%), cheese and yogurt (58%), other vegetables (51%), unprocessed red meat (47%), poultry (46%), and milk (41%) (Table 3). Mean values of FGDS, which can also be applied to the general population, were <5 for the adult population in both countries (mean ± SD: 4.32 ± 1.22 in Brazil and 4.58 ± 1.51 in the United States), and 42% and 47% of women achieved the MDD-W in Brazil and the United States, respectively (Table 4).
TABLE 3.
Food group | Brazil | United States | |
---|---|---|---|
1 | Foods made from grains | 99 | 97 |
2 | Whole grains | 19 | 38 |
3 | White roots and tubers/plantains | 28 | 40 |
4 | Beans and other legumes | 79 | 20 |
5 | Nuts and seeds | <1 | 20 |
6 | Vitamin A–rich orange-colored vegetables, roots, and tubers | 5 | 17 |
7 | Dark-green leafy vegetables | 3 | 16 |
8 | Other vegetables | 40 | 51 |
9 | Vitamin A–rich fruits | 6 | 5 |
10 | Citrus | 9 | 6 |
11 | Red/purple/blue fruits | 3 | 13 |
12 | Other fruits | 25 | 33 |
13 | Milk | 55 | 41 |
14 | Cheese and yogurt | 19 | 58 |
15 | Eggs | 16 | 24 |
16 | Poultry | 30 | 46 |
17 | Fish and seafood | 9 | 17 |
18 | Unprocessed red meat | 57 | 47 |
19 | Processed meats (sausages, luncheon meats, etc.) | 24 | 39 |
20 | Packaged salty snacks | 3 | 30 |
21 | Instant dry soup/noodles | <1 | <1 |
22 | Deep-fried foods | 11 | 19 |
23 | Food from a fast-food restaurant1 | 3 | 34 |
24 | Baked/grain-based sweets | 24 | 33 |
25 | Other sweets | 14 | 38 |
26 | Sodas/sugar-sweetened beverages | 29 | 38 |
27 | Fruit drinks/juice | 33 | 17 |
28 | Sweetened coffee/tea/milk | 81 | 28 |
For Brazil, the consumption of pizza and hamburgers was used as a proxy for “fast food” consumption, because those items are rarely consumed outside of a fast-food restaurant/delivery. Other types of fast foods were not captured.
TABLE 4.
Brazil | United States | |
---|---|---|
HDI-2020 (0–11, mean score) | 4.56 ± 1.67 | 4.04 ± 1.87 |
FGDS (0–10, mean score) | 4.32 ± 1.22 | 4.58 ± 1.51 |
Percentage of women aged 15–49 y achieving MDD-W | 41.5 | 46.6 |
Total dietary energy, mean kcal | 1896 ± 828 | 2160 ± 988 |
Percentage of energy from UPFs, mean % | 19.0 ± 18.5 | 56.9 ± 21.2 |
BMI, mean kg/m2 | 25.1 ± 4.7 | 28.6 ± 6.9 |
Percentage of the population aged ≥15 y who are overweight or obese2 | 46.0 | 66.7 |
Values are means ± SD or %. Number of observations: 30,062 for Brazil and 17,887 for the United States for all indicators except for BMI, percentage overweight or obese, and MDD-W. For BMI and percentage overweight or obese, the number of observations was 29,675 for Brazil and 17,703 for the United States. For MDD-W, statistics were computed for women of reproductive age only, and the number of observations was 11,978 for Brazil and 5311 for the United States. FGDS, Food Group Diversity Score; HDI-2020, Healthy Diet Indicator 2020; MDD-W, Minimum Dietary Diversity for Women of reproductive age; UPF, ultra-processed food.
Adults aged ≥20 y with a BMI (kg/m2) >25 and adolescents aged 15–19 y with a BMI-for-age z score >1 SD according to the WHO growth reference (41) were defined as overweight or obese.
Table 5 shows which global dietary recommendations are met. In both countries, diets are low in fruits and vegetables and high in salt. Adequate fruit and vegetable consumption is very low (8% in Brazil, 12% in United States), and few meet the recommendation on dietary sodium (21% in Brazil, 16% in the United States). Only 1 out of 5 Americans include legumes in the diet, and nuts and seeds in the diet, and less consume adequate dietary fiber; in Brazil, the vast majority do not include nuts in the diet. On the positive side, the majority of Brazilians consume legumes (79%), over half of Americans consume some whole grains (58%), and in both countries, over half of the population meets the recommendations on avoiding processed meat and limiting red meat to <71 g/d. Altogether, most people in both countries meet between 3 and 6 of the 11 global dietary recommendations (Table 6). On average, adults in Brazil meet 4.6 global dietary recommendations and adults in the United States meet 4.0 global dietary recommendations (Table 4). It is notable that no individuals in Brazil and only 2 individuals in the United States met all 11 dietary recommendations.
TABLE 5.
Dietary element | Brazil | United States | |
---|---|---|---|
1 | Fruits, vegetables (≥400 g/d) | 8.4 | 11.5 |
2 | Beans and other legumes (>0 g/d) | 78.6 | 19.9 |
3 | Nuts and seeds (>0 g/d) | 0.4 | 20.2 |
4 | Whole grains (>0 g/d) | 18.5 | 57.8 |
5 | Dietary fiber (>25 g/d) | 27.7 | 18.1 |
6 | Total fat (<30% of total energy) | 69.7 | 36.2 |
7 | Saturated fat (<10% of total energy) | 63.4 | 45.8 |
8 | Dietary sodium (<2 g/d) | 20.6 | 16.1 |
9 | Free sugars (<10% of total energy)1 | 35.1 | 43.4 |
10 | Processed meat (0 g/d) | 75.8 | 61.0 |
11 | Unprocessed red meat (≤71 g/d)2 | 57.3 | 74.0 |
For the United States, added sugars are used as a proxy for free sugars.
Amount per day corresponds to the upper end of the recommendation to consume ≤300–500 g/wk.
TABLE 6.
Brazil | United States | |||||
---|---|---|---|---|---|---|
Number of recommendations | n 1 | Percentage meeting × number | Percentage meeting × number or more | n 1 | Percentage meeting × number | Percentage meeting × number or more |
0 | 159.8 | 0.5 | 100.0 | 191.9 | 1.1 | 100.0 |
1 | 953.3 | 3.2 | 99.5 | 1159.4 | 6.5 | 98.9 |
2 | 2466.1 | 8.2 | 96.3 | 2587.3 | 14.5 | 92.4 |
3 | 4456.8 | 14.8 | 88.1 | 3449.5 | 19.3 | 78.0 |
42 | 5966.1 | 19.8 | 73.3 | 3675.9 | 20.6 | 58.7 |
52 | 6876.3 | 22.9 | 53.4 | 2914.0 | 16.3 | 38.1 |
62 | 5728.7 | 19.1 | 30.5 | 2008.8 | 11.2 | 21.9 |
7 | 2677.6 | 8.9 | 11.5 | 1162.3 | 6.5 | 10.6 |
8 | 677.4 | 2.3 | 2.6 | 545.9 | 3.1 | 4.1 |
9 | 93.3 | 0.3 | 0.3 | 151.4 | 0.8 | 1.1 |
10 | 6.6 | 0.0 | 0.0 | 38.4 | 0.2 | 0.2 |
11 | 0.0 | 0.0 | 0.0 | 2.3 | 0.0 | 0.0 |
Total | 30,062 | 100.0 | 17,887 | 100.0 |
The number of observations is survey-weighted.
Potential cutoffs for a dichotomous indicator that are based on the distribution of the number of recommendations being met, in the absence of a normative value for a cutoff.
The percentage of energy from UPFs is very high in the United States (57% of dietary energy; total mean energy intake: 2160 kcal) and also quite high in Brazil (19% of dietary energy; total mean energy intake: 1896 kcal) (Table 4). Mean BMI (in kg/m2) in this national sample is 28.6 in the United States and 25.1 in Brazil; nearly half of the population in Brazil and two-thirds of the population in the United States are overweight or obese (Table 4).
Results informing construction of indicators
In order to construct candidate indicators, we used a combination of normative considerations and quantitative findings on the associations of food groups with the outcome indicators (Supplemental Table A1). We noted that many of the same food groups most highly correlated with the HDI-2020 and its subindexes are also more strongly correlated with UPF consumption. Several food groups that were negatively associated with the HDI-2020 subindex of meeting the recommendations for “dietary components to limit” were positively associated with dietary energy intake. No food groups were strongly correlated with BMI.
To inform construction of a dichotomous GDR score, we used the descriptive results. There were either no individuals meeting all WHO recommendations or the proportion was very small (<0.1%) (Table 6). The distributions of the HDI-2020 in the US and Brazil datasets (Supplemental Figure 1) suggest that cutoffs of 4, 5, and 6 out of 11 WHO dietary recommendations would be adequate for ROC and sensitivity-specificity analysis. [The lower than desirable number of dietary recommendations typically achieved in these populations presented a similar challenge as the distribution of mean probability of adequacy (MPA) of micronutrients in the study sites of the WDDP, where none of the women in the sample—or only a very small proportion of the women—achieved an MPA of 90% (let alone 100%), so that lower levels of MPA that were reached by nonnegligible proportions of women had to be selected for the analysis (34).] A cutoff of 6 has the advantage of being easy to communicate as “meeting more than half the global dietary recommendations.” We therefore tested indicator candidates to predict ≥6 of the recommendations.
Validation results
Table 7 shows the main results for the best variants of the new indicators; all following correlation coefficients cited are highly significant (P < 0.001). The correlation between the GDR score and the HDI-2020 (our overall quantitative standard for meeting dietary recommendations) is 0.55 in Brazil and 0.66 in the United States. The correlations of GDR-Healthy with the HDI-2020 subindex on healthy foods and of GDR-Limit with the HDI-2020 subindex on dietary components to limit are of a similar magnitude.
TABLE 7.
HDI-2020 subindexes | |||||||
---|---|---|---|---|---|---|---|
HDI-20202 | Healthy foods3 | Dietary components to limit4 | Percentage of energy from UPFs | Total dietary energy intake | BMI | FGDS | |
Brazil | |||||||
GDR-Healthy | 0.29*** | 0.58*** | −0.03*** | 0.00 | 0.14*** | 0.05*** | 0.75*** |
GDR-Limit | −0.45*** | 0.02*** | −0.57*** | 0.50*** | 0.30*** | −0.03*** | 0.16*** |
GDR score | 0.55*** | 0.36*** | 0.43*** | −0.40*** | −0.15*** | 0.06*** | 0.35*** |
FGDS | 0.04*** | 0.35*** | −0.18*** | 0.06*** | 0.24*** | 0.05*** | 1.00*** |
United States | |||||||
GDR-Healthy | 0.45*** | 0.66*** | 0.12*** | −0.33*** | 0.04*** | −0.03*** | 0.83*** |
GDR-Limit | −0.54*** | −0.12*** | −0.61*** | 0.42*** | 0.38*** | 0.02** | −0.01 |
GDR score | 0.66*** | 0.47*** | 0.51*** | −0.49*** | −0.25*** | −0.03*** | 0.49*** |
FGDS | 0.25*** | 0.51*** | −0.04*** | −0.29*** | 0.14*** | −0.02* | 1.00*** |
Values are Spearman rank correlation coefficients. *Significant at P < 0.05; **significant at P < 0.01; ***significant at P < 0.001. FGDS, Food Group Diversity Score; GDR, Global Dietary Recommendations; HDI-2020, Healthy Diet Indicator 2020; UPF, ultra-processed food.
Index of all 11 global dietary recommendations.
Index of 5 global dietary recommendations encouraging consumption of 1) fruits and vegetables, 2) beans and other legumes, 3) nuts and seeds, 4) whole grains, and 5) dietary fiber.
Index of 6 global dietary recommendations about limiting consumption of 1) total fat, 2) saturated fat, 3) dietary sodium, 4) free sugars, 5) processed meat, and 6) unprocessed red meat.
We examined other associations of interest. The GDR score is negatively correlated with percentage of energy from UPFs (−0.40 in Brazil, −0.49 in the United States), while GDR-Limit is positively correlated (0.50 in Brazil, 0.42 in the United States); and in the United States, GDR-Healthy is also negatively correlated with UPFs (−0.33). The GDR score has a low negative association with energy intake and GDR-Limit has a moderate positive association with energy intake. None of the indicators are strongly correlated with BMI. The FGDS, in contrast, has a low positive correlation with total energy intake and is less strongly associated with UPF intake than GDR-Limit and the GDR score.
The combined GDR score has a much stronger correlation than the FGDS with the HDI-2020 (0.55 vs. 0.04 in Brazil, 0.66 vs. 0.25 in the United States), GDR-Healthy has a much stronger correlation than the FGDS with the HDI-2020 subindex on healthy foods (0.58 vs. 0.35 in Brazil, 0.66 vs. 0.51 in the United States), and GDR-Limit has a much stronger negative correlation than the FGDS with the HDI-2020 subindex on dietary components to limit (−0.57 vs. −0.18 in Brazil, −0.61 vs. −0.04 in the United States).
The results of validity tests of continuous and dichotomous indicators to predict quantitative outcomes are shown in Tables 8 and 9, respectively. Tables 8 and 9 show the best-performing variant of each indicator; Supplemental Tables A12–A25 show the performance of all candidate indicators, from which the preferred indicator variant was selected. The ORs show strong relations between the indicators and outcomes, with the exception of the WHO-Fat indicator. For example, in Brazil, every unit increase in the WHO-FV indicator was associated with 6 times the odds of meeting the quantitative fruit and vegetable recommendation (of ≥400 g/d), and every unit increase in the saturated fat indicator is associated with reduced odds of meeting the quantitative saturated fat recommendation (<10% of dietary energy) (Table 8). The Spearman rank correlations also show that the indicators are moderately to strongly associated with the continuous and dichotomous outcomes in the expected directions, except for the WHO-Fat indicator in both countries and the WHO-Fiber indicator in Brazil (Table 8). The AUC is acceptable (>0.70) for all indicators except for WHO-Fat (0.62 in Brazil, 0.64 in the United States); WHO-Fiber in Brazil, which is slightly below the desired AUC (0.66); and WHO-SatFat in the United States, which has a borderline AUC (0.69) (Table 8). All indicators except for WHO-Fat had acceptable specificity, and most of them also met the criteria for acceptable sensitivity for the selected cutoffs (Table 9). The WHO-FV indicator has lower sensitivity than desired in Brazil, but very high specificity in both Brazil and the United States, which is the favored measure for a valid indicator in this case, because WHO-FV correlates positively with the outcome measure. (For scores that correlate positively with the outcome measure, specificity is preferred over sensitivity, and vice versa for scores that correlate negatively with the outcome measure; see Supplemental Box 3 for detailed explanations.) All indicators result in an acceptable percentage misclassified (<30%), except for WHO-SatFat and WHO-Fat. All indicators met our criterion for closely matching actual population prevalence (<10 percentage points difference)—except for the whole-grains food group in the United States—with most indicators (WHO-FV, WHO-Fiber, WHO-Sugar) showing a population prevalence <5 percentage points different from the actual prevalence. Finally, we note that simple weights generally improved measures of association for WHO-Sugar, WHO-Fiber, GDR-Limit, and GDR score; for the unweighted candidate indicators of WHO-Fiber and the GDR score, it was not possible to find a cutoff in sensitivity-specificity analysis that worked well for both countries. The weighted candidate indicators did not have this limitation.
TABLE 8.
Candidate indicator | Spearman rank correlation (continuous) | Spearman rank correlation (dichotomous) | OR2 (dichotomous) | AUC (dichotomous) |
---|---|---|---|---|
Brazil | ||||
1) WHO-FV | 0.92 | 0.40 | 6.04 | 0.88 |
2) WHO-Fiber | 0.42 | 0.25 | 1.65 | 0.66 |
3) WHO-Sugar3 | 0.62 | −0.54 | 0.26 | 0.81 |
4) WHO-SatFat3 | 0.48 | −0.37 | 0.50 | 0.72 |
5) WHO-Fat3 | 0.29 | −0.19 | 0.65 | 0.62 |
6) Legumes | 0.72 | 1.00 | N/A | 1.00 |
7) Nuts and seeds | 1.00 | 1.00 | N/A | 1.00 |
8) Whole grains | 0.99 | 1.00 | N/A | 1.00 |
9) Processed meat3 | 0.99 | −1.00 | N/A | 1.00 |
GDR score4 | 0.55 | 0.45 | 2.09 | 0.77 |
United States | ||||
1) WHO-FV | 0.85 | 0.39 | 3.50 | 0.86 |
2) WHO-Fiber | 0.55 | 0.35 | 1.79 | 0.77 |
3) WHO-Sugar3,5 | 0.62 | −0.55 | 0.33 | 0.81 |
4) WHO-SatFat3 | 0.40 | −0.34 | 0.59 | 0.69 |
5) WHO-Fat3 | 0.26 | −0.24 | 0.63 | 0.64 |
6) Legumes | 0.99 | 1.00 | N/A | 1.00 |
7) Nuts and seeds | 0.99 | 1.00 | N/A | 1.00 |
8) Whole grains6 | 0.72 | 0.70 | 1.00 | 0.84 |
9) Processed meat3 | 0.96 | −1.00 | N/A | 1.00 |
GDR score4 | 0.66 | 0.51 | 2.03 | 0.86 |
Values are Spearman rank correlation coefficients, ORs, or AUCs. The amount of the food group or nutrient consumed (grams of fruits and vegetables, fiber, legumes, nuts and seeds, whole grains, and processed meat, respectively; and percentage of free sugars, saturated fat, and fat in total dietary energy intake, respectively) or the HDI-2020 was used as a continuous outcome variable. The variable for meeting the respective recommendation (1 = recommendation met, 0 = recommendation not met) was used as a dichotomous outcome variable. All associations are significant at P < 0.001. GDR, Global Dietary Recommendations; HDI-2020, Healthy Diet Indicator 2020; WHO-FV, WHO-Fruits and Vegetables (score); WHO-SatFat, WHO-Saturated Fat (score).
ORs cannot be computed if 2 variables are perfectly (negatively or positively) associated. These cases are denoted as not applicable (N/A).
Higher scores for this candidate indicator indicate a lower probability that the respective recommendation will be met; therefore, the variable for meeting the recommendation was inverted in order to obtain test statistics that are comparable across all candidate indicators.
For the GDR score, the dichotomized HDI-2020 (1 = 6–11 global dietary recommendations met, 0 = 0–5 global dietary recommendations met) was used as the outcome variable.
For the United States, added sugars were used as a proxy for free sugars.
Intake of whole grains is calculated differently in the United States than in Brazil survey data; in the United States, the definition of whole-grain foods excludes some foods that may contain a small amount of whole grain, such as unspecified breads, cereals, chips, crackers, and muffins. This explains why whole-grain consumption >0 based on the quantitative intake data does not match perfectly with the yes/no determination based on consumption of “whole grain foods."
TABLE 9.
Candidate indicator | Cutoff | Percentage of observations ≥ cutoff | Sensitivity | Specificity | Estimated minus actual population prevalence | Percentage of false positives | Percentage of false negatives | Total percentage misclassified |
---|---|---|---|---|---|---|---|---|
Brazil | ||||||||
WHO-FV | ≥3 | 5.2 | 38.42 | 97.9 | −3.3 | 1.9 | 5.2 | 7.2 |
WHO-Fiber | ≥4 | 23.0 | 67.8 | 81.1 | −4.7 | 17.3 | 2.7 | 20.0 |
WHO-Sugar3 | ≥2 | 59.1 | 79.7 | 75.1 | −3.4 | 9.4 | 12.7 | 22.1 |
WHO-SatFat3 | ≥2 | 42.1 | 64.0 | 69.1 | 8.2 | 20.4 | 12.2 | 32.62 |
WHO-Fat3 | ≥2 | 36.3 | 47.52 | 68.1 | 7.9 | 22.8 | 14.9 | 37.72 |
Legumes | ≥1 | 78.5 | 100.0 | 100.0 | 0.0 | 0.0 | 0.0 | 0.0 |
Nuts and seeds | ≥1 | 0.3 | 100.0 | 100.0 | 0.0 | 0.0 | 0.0 | 0.0 |
Whole grains | ≥1 | 21.7 | 100.0 | 100.0 | 0.0 | 0.0 | 0.0 | 0.0 |
Processed meat3 | ≥1 | 21.8 | 100.0 | 100.0 | 0.0 | 0.0 | 0.0 | 0.0 |
GDR score4 | ≥1 | 42.1 | 69.4 | 71.3 | 9.1 | 19.2 | 10.1 | 29.3 |
United States | ||||||||
WHO-FV | ≥3 | 14.5 | 56.4 | 90.3 | 4.2 | 8.7 | 4.5 | 13.2 |
WHO-Fiber | ≥4 | 21.8 | 57.6 | 82.4 | 4.8 | 15.8 | 4.4 | 20.2 |
WHO-Sugar3,5 | ≥2 | 60.9 | 81.9 | 67.4 | 3.4 | 13.8 | 10.4 | 24.3 |
WHO-SatFat3 | ≥2 | 59.1 | 72.7 | 56.0 | 6.6 | 20.9 | 14.3 | 35.22 |
WHO-Fat 3 | ≥2 | 71.7 | 79.2 | 40.72 | 9.4 | 22.3 | 13.0 | 35.32 |
Legumes | ≥1 | 21.4 | 100.0 | 100.0 | 0.0 | 0.0 | 0.0 | 0.0 |
Nuts and seeds | ≥1 | 16.7 | 100.0 | 100.0 | 0.0 | 0.0 | 0.0 | 0.0 |
Whole grains6 | ≥1 | 37.1 | 68.0 | 100.0 | −17.42 | 0.0 | 17.4 | 17.4 |
Processed meat3 | ≥1 | 38.6 | 100.0 | 100.0 | 0.0 | 0.0 | 0.0 | 0.0 |
GDR score4 | ≥1 | 29.4 | 72.3 | 82.3 | 8.0 | 13.9 | 5.9 | 19.8 |
Values are percentage points for the estimated minus actual population prevalence and percentages for all other statistics. All values are based on unweighted data. GDR, Global Dietary Recommendations; HDI-2020, Healthy Diet Indicator 2020; WHO-FV, WHO-Fruits and Vegetables (score); WHO-SatFat, WHO-Saturated Fat (score).
Indicates results that fall outside the predefined criteria for acceptable results.
Higher scores for this candidate indicator indicate a lower probability that the respective recommendation will be met; therefore, the variable for meeting the recommendation was inverted in order to obtain test statistics that are comparable across all candidate indicators.
For the GDR score, the dichotomized HDI-2020 (1 = 6–11 global dietary recommendations met, 0 = 0–5 global dietary recommendations met) was used as the outcome variable.
For the United States, added sugars were used as a proxy for free sugars.
Intake of whole grains is calculated differently in the United States than in Brazil survey data; in the United States, the definition of whole-grain foods excludes some foods that may contain a small amount of whole grain, such as unspecified breads, cereals, chips, crackers, and muffins. This explains why whole-grain consumption based on the quantitative intake data does not match perfectly with the yes/no determination based on consumption of “whole grain foods.”
The development of questionnaires to collect the required data for these indicators suggests that it may be most feasible to collect data on sentinel foods rather than all foods—that is, the most commonly consumed items that capture a majority of consumption of the food group in each context (27). Supplemental Tables S8andS9 show the results corresponding to Tables 8 and 9, if only sentinel foods are included rather than all foods in each food group. The results using sentinel foods only are almost the same as those using all foods, showing that the indicators are valid whether using sentinel foods or all foods.
The indicator candidates that performed best across both countries are described in Table 10, including how to construct the indicator from food group data, the range of scores, and the cutoff for meeting the respective recommendation(s).
TABLE 10.
Name of indicator | Predicted recommendation/outcome | Selected candidate indicator composition: food groups and weights2 | Range (units) | Selected cutoff (units) |
---|---|---|---|---|
WHO-FV | ≥400 g/d of fruits and vegetables | (1) Dark-green leafy vegetables; (2) Vitamin A–rich orange-colored vegetables, roots, and tubers; (3) Other vegetables; (4) Vitamin A-rich fruits; (5) Citrus fruits; (6) Other fruits (including red/purple/blue fruits) | 0–6 | ≥3 |
WHO-Fiber |
>25 g/d dietary fiber | (1) Dark-green leafy vegetables; (2) Vitamin A–rich orange-colored vegetables, roots, and tubers3; (3) Other vegetables; (4) Vitamin A–rich fruits3; (5) Citrus fruits; (6) Other fruits (including red/purple/blue fruits); (7) Legumes (double weighted); (8) Nuts/seeds; (9) Whole grains | 0–10 | ≥4 |
WHO-Sugar | <10% of dietary energy from free sugars4 | (1) Sodas/sugar-sweetened beverages (double weight); (2) Fruit drinks/juice; (3) Sweetened coffee/tea/milk drinks; (4) Baked/grain-based sweets; (5) Other sweets | 0–6 | <2 |
WHO-SatFat |
<10% of dietary energy from saturated fat |
(1) Processed meat; (2) Unprocessed red meat; (3) Food from a fast-food restaurant; (4) Cheese and yogurt; (5) Milk; (6) Other sweets; minus (7) Fish and seafood; (8) Poultry | −2 to 6 |
<2 |
WHO-Fat | <30% of dietary energy from total fat | (1) Processed meat; (2) Unprocessed red meat; (3) Deep-fried foods; (4) Food from a fast-food restaurant; (5) Packaged salty snacks; (6) Baked/grain-based sweets; (7) Other sweets | 0–7 | N/A |
Legumes | >0 g/d | Legumes | ||
Nuts and seeds | >0 g/d | Nuts and seeds | 0–1 | =1 |
Whole grains | >0 g/d | Whole grains | 0–1 | =1 |
Processed meat | 0 g/d | Processed meat | 0–1 | =0 |
GDR score | ≥6 out of 11 global dietary recommendations met | (1) Dark-green leafy vegetables; (2) Vitamin A–rich orange-colored vegetables, roots, and tubers; (3) Other vegetables; (4) Vitamin A–rich fruits; (5) Citrus fruits; (6) Other fruits (including red/purple/blue fruits); (7) Legumes; (8) Nuts/seeds; (9) Whole grains minus(1) Sodas/sugar-sweetened beverages; (2) Baked/grain-based sweets; (3) Other sweets; (4) Processed meat (double weight); (5) Unprocessed red meat; (6) Deep-fried foods; (7) Food from a fast-food restaurant, or Instant noodles; (8) Packaged salty snacks | −9 to 9 | ≥1 |
GDR, Global Dietary Recommendations; WHO-FV, WHO-Fruits and Vegetables (score); WHO-SatFat, WHO-Saturated Fat (score); N/A, not applicable.
The weights are single weights unless indicated otherwise.
The results for this indicator were almost identical if vitamin A–rich fruits and vegetables were combined into a single category.
For the United States, added sugars are used as a proxy for free sugars.
Discussion
Contribution
This is the first work that attempts to translate global dietary recommendations into a low-burden set of indicators. Because diet quality is multifaceted, we have created a suite of indicators to reflect facets of diet quality for which there is global agreement on importance to health. We developed 12 indicators and tested their validity at population level in 2 large, diverse countries with high variability in the aspects of diets related to global recommendations.
While MDD-W is valid for predicting micronutrient intakes in some settings, alone it is not a sufficient indicator of overall diet quality. GDR-Healthy, GDR-Limit, and the GDR score add value as indicators of healthy diet patterns that adhere to global dietary recommendations, thereby reflecting another important aspect of diet quality that is not captured by MDD-W. In our 2 study countries, the GDR score is a much better predictor of meeting global dietary recommendations than the FGDS (the 10-food-group continuous score from which the MDD-W is derived), GDR-Healthy is a much better predictor of health-protective food intake than the FGDS, and GDR-Limit introduces an entirely new ability to predict unhealthy food intake. Measured against the yardstick of the validation results for the MDD-W (3, 34), the GDR score and 8 food-group–based indicators of individual dietary recommendations performed similarly or better in predicting the outcomes of interest in sensitivity-specificity and correlation analyses. The observed strength of association (0.55–0.66) is very acceptable for this type of indicator; the rank correlation coefficients match or exceed the Pearson's correlation coefficients reported for the MDD-W and micronutrient adequacy, which ranged from 0.25 to 0.56 across WDDP study sites (3). Additionally, the associations between the GDR score and UPFs are of moderate size (−0.40 to −0.49 for the GDR score and 0.42 to 0.50 for the GDR-Limit) and are practically important, because UPF consumption is associated with negative dietary quality and health outcomes (37, 42, 43). In all, these new indicators alongside the FGDS and MDD-W create a suite of indicators useful for monitoring diet quality holistically because they predict healthy diet patterns that minimize NCD risk, while the MDD-W reflects micronutrient adequacy.
Limitations
In this study, the indicators are validated in only 2 countries, 1 middle-income country and 1 high-income country. Therefore, the main importance of this work is the approach taken; the indicators cannot yet be considered globally validated. A next step will be to replicate these results in additional nationally representative quantitative datasets from other regions, including low-income and other middle-income countries. Our expectation is that the GDR score will be positively correlated with the HDI-2020 in other datasets, while the cutoffs of dichotomous indicators may be more vulnerable to differences in context. Second, the indicators are valid at population level, and not at an individual level. Like the MDD-W and any 24-h recall–based approach, they are appropriate for use to characterize diet quality in populations and are not meant for clinical application for assessment of an individual's diet quality. Third, although several of these new indicators are proxies for quantitative intakes of the general adult population, they are not quantitative intake data. For information on average quantitative (grams) intakes of foods or nutrients in a population, other techniques of dietary assessment are necessary (such as quantitative 24-h recalls). Fourth, the indicators are not validated as indicators of energy intake, obesity, or NCD morbidity or mortality. Our results show that the GDR score and its components are not strongly correlated with energy intake or BMI. We expect that they reflect NCD risk, because the WHO dietary recommendations were in large part developed based on the association of each dietary risk factor with NCDs (19), and because previous versions of the HDI were correlated with NCD risk (15, 17). It may be inappropriate to attempt validation of population-level indicators against individual NCD outcomes in prospective cohort studies, but future studies could validate the HDI-2020 against NCD outcomes within individuals, as has been done with previous HDI versions.
Finally, like the MDD-W validation (3), our validation uses information derived from detailed quantitative data, while, in practice, the way food-group data are collected can influence the performance of the indicators (44). While the feasibility of accurate data collection is a potential limitation, in this case the data-collection tool for these indicators, the DQ-Q, has undergone extensive cognitive testing in multiple languages and cultures, validation against a reference method, and a quantitative pilot test in the Gallup World Poll (27). Our results here show that mimicking sentinel food data that would be collected by the DQ-Q produces almost exactly the same results as those using all foods (see Supplemental Tables S8 and S9).
While we report the MDD-W in our results, we suggest caution in using or interpreting MDD-W as an indicator of micronutrient adequacy in high-income or upper-middle-income countries. The MDD-W was validated only in low- and lower-middle-income settings (3). A majority of women aged 15–49 y in the United States and Brazil do not achieve minimum diet diversity in the nationally representative datasets we analyzed, which raises the question whether their level of micronutrient adequacy actually falls below 60% (the threshold against which the MDD-W was validated) or whether the MDD-W underestimates it in these countries.
Applications
The new indicators are an important advance for measuring and monitoring diet quality at population level. We present a suite of diet-quality indicators that can be calculated easily from food-group–level data derived from a 5-min questionnaire, the DQ-Q, which also can be used to collect MDD-W data (27). The indicators could be used in national surveillance systems, national multitopic surveys, or nutrition-sensitive projects such as agricultural projects that aim to improve diet quality. They may be particularly applicable for the Gallup World Poll, the Demographic and Health Surveys, and the WHO STEPwise Approach to Noncommunicable Disease Risk-Factor Surveillance (45), as a way to track country progress toward WHO dietary recommendations.
These indicators can be used to track dietary trends—for example, showing the change over time in the overall GDR score, or the GDR-Limit subcomponent versus the GDR-Healthy subcomponent, revealing dietary and nutrition transitions. Furthermore, they can help to identify specific aspects of diets that may be public health threats: for example, to predict the proportion of populations not consuming adequate fruits and vegetables or the proportion consuming excessive free sugars. Even the individual food groups used to construct these indicators will be useful to describe trends related to the nutrition transition, such as the proportion of the population consuming sugar-sweetened beverages, instant noodles, fast food, and salty packaged snacks.
While recognizing the benefit of having a lens into specific aspects of diets, we also recognize that a single metric reflecting total diet quality may be needed for policymakers and advocates. The Sustainable Development Goals (SDGs) lack any diet-quality indicator despite the fundamental importance of diet to several of the SDGs. Global targets and metrics for diets should ideally reflect total diet quality for all people, because SDG2 calls to end all forms of malnutrition. Our results show that the MDD-W does not perform well as an indicator of meeting global dietary recommendations; this is not surprising, as it was not designed to do so. Ultimately, an indicator reflecting both nutrient adequacy and protection against diet-related NCDs would be desirable as a global yardstick of diet quality.
There is potential now to create a combined score that incorporates both the GDR score and the MDD-W/FGDS, and to validate it against both nutrient adequacy and the global dietary recommendations. Such an indicator might meet the need for a single indicator capturing total diet quality. Another recent effort has developed a metric reflective of both nutrient adequacy and protection against diet-related NCDs, the Global Diet Quality Score (46, 47). This metric is based on food groups similar to the GDR score, but in contrast, it requires quantitative or semi-quantitative data. The GDR score is based on yes/no data about consumption of 29 food groups in the previous day, collected from the DQ-Q tool (27). The DQ-Q is currently being adapted for over 90 countries, and will be implemented in the Gallup World Poll beginning in 2021, offering concrete potential for monitoring the suite of indicators presented here across countries, alongside the MDD-W/FGDS, and any subsequently developed indicators of total diet quality.
Conclusions
Diet quality is a critical aspect of public health and sustainable development. The GDR score and suite of indicators reflecting global dietary recommendations adds to the existing indicator of dietary diversity (MDD-W/FGDS), to enable feasible monitoring of diet quality more holistically. It builds our toolbox so that we can understand both healthy and unhealthy aspects of diets and target responses to the specific areas of the diet that most need improvement. The burden of diet-related NCDs is significant in all regions of the world, and these indicators can more fully reflect diet quality relevant to policies and programs. As a suite of metrics that can be calculated simply, using low-burden survey tools, it paves the way for monitoring diet quality globally and in countries.
Supplementary Material
ACKNOWLEDGEMENTS
We are grateful for funds to complete this work from the Swiss Agency for Development and Cooperation (SDC), The Rockefeller Foundation, and the Business Platform for Nutrition Research supported by the Government of Canada through the Global Alliance for Improved Nutrition (GAIN). We thank Francesco Branca for feedback and input about WHO dietary recommendations. We appreciate advisory support and feedback on these indicators from the Global Diet Quality Project Technical Advisory Group (TAG), and Terri Ballard and Andrew Rzepa. We also thank anonymous reviewers for their excellent and helpful comments which improved the manuscript. The authors’ responsibilities were as follows—AWH: designed the study; AWH and DW: wrote the manuscript; DW, EM-S, and GA: analyzed the data; AWH, CAM, and DW: designed the indicators conceptually; and all authors: read and approved the final manuscript.
Notes
Supported by Swiss Agency for Development and Cooperation (SDC), The Rockefeller Foundation, and the Business Platform for Nutrition Research supported by the Government of Canada through the Global Alliance for Improved Nutrition (GAIN).
Author disclosures: The authors report no conflicts of interest.
Supplemental Figure 1, Supplemental Boxes 1–3, Supplemental Tables A1–A25, and Supplemental Tables S8 and S9 are available from the “Supplementary data” link in the online posting of the article and from the same link in the online table of contents at https://academic.oup.com/cdn/.
Abbreviations used: DQI, Diet Quality Index; DQ-Q, Diet Quality Questionnaire; FGDS, Food Group Diversity Score; GDR, Global Dietary Recommendations; HDI, Healthy Diet Indicator; HEI, Healthy Eating Index; MDD-W, Minimum Dietary Diversity for Women of reproductive age; MPA, mean probability of adequacy; NCD, noncommunicable disease; ROC, receiver operating characteristic; SDG, Sustainable Development Goal; UPF, ultra-processed food; WDDP, Women's Dietary Diversity Project; WHO-FV, WHO-Fruits and Vegetables (indicator); WHO-SatFat, WHO-Saturated Fat (indicator).
Contributor Information
Anna W Herforth, Email: aherforth@hsph.harvard.edu, Department of Global Health and Population, Harvard TH Chan School of Public Health, Boston, MA, USA.
Doris Wiesmann, Independent Consultant, Schoenwalde, Germany.
Euridice Martínez-Steele, Department of Nutrition, School of Public Health, University of São Paulo, São Paulo, Brazil.
Giovanna Andrade, Department of Preventive Medicine, School of Medicine, University of São Paulo, São Paulo, Brazil.
Carlos A Monteiro, Department of Nutrition, School of Public Health, University of São Paulo, São Paulo, Brazil.
References
- 1. WHO Healthy diet fact sheet no. 394. Geneva (Switzerland): WHO; 2018; [Internet]. [Cited 2020 Nov 1]. Available from: https://www.who.int/publications/m/item/healthy-diet-factsheet394. [Google Scholar]
- 2. FAO; FHI 360 Minimum dietary diversity for women: a guide for measurement. Rome (Italy): FAO; 2016. [Google Scholar]
- 3. Martin-Prével Y, Arimond M, Allemand P, Wiesmann D, Ballard TJ, Deitchler M, Dop MC, Kennedy G, Lartey A, Lee W, Mourad M. Development of a dichotomous indicator for population-level assessment of dietary diversity in women of reproductive age. Curr Dev Nutr. 2017;1(12):cdn.117.001701. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Fung TT, Isanaka S, Hu FB, Willett WC. International food group-based diet quality and risk of coronary heart disease in men and women. Am J Clin Nutr. 2018;107(1):120–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Darmon N, Vieux F, Maillot M, Volatier JL, Martin A. Nutrient profiles discriminate between foods according to their contribution to nutritionally adequate diets: a validation study using linear programming and the SAIN, LIM system. Am J Clin Nutr. 2009;89(4):1227–36. [DOI] [PubMed] [Google Scholar]
- 6. Haines PS, Siega-Riz AM, Popkin BM. The Diet Quality Index revised: a measurement instrument for populations. J Am Diet Assoc. 1999;99(6):697–704. [DOI] [PubMed] [Google Scholar]
- 7. Patterson RE, Haines PS, Popkin BM. Diet quality index: capturing a multidimensional behavior. J Am Diet Assoc. 1994;94(1):57–64. [DOI] [PubMed] [Google Scholar]
- 8. Kim S, Haines PS, Siega-Riz AM, Popkin BM. The Diet Quality Index-International (DQI-I) provides an effective tool for cross-national comparison of diet quality as illustrated by China and the United States. J Nutr. 2003;133(11):3476–84. [DOI] [PubMed] [Google Scholar]
- 9. Guenther PM, Casavale KO, Reedy J, Kirkpatrick SI, Hiza HA, Kuczynski KJ, Kahle LL, Krebs-Smith SM. Update of the Healthy Eating Index: HEI-2010. J Acad Nutr Diet. 2013;113(4):569–80. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Guenther PM, Reedy J, Krebs-Smith SM. Development of the Healthy Eating Index-2005. J Am Diet Assoc. 2008;108(11):1896–901. [DOI] [PubMed] [Google Scholar]
- 11. Kennedy ET, Ohls J, Carlson S, Fleming K. The Healthy Eating Index: design and applications. J Am Diet Assoc. 1995;95(10):1103–8. [DOI] [PubMed] [Google Scholar]
- 12. McCullough ML, Willett WC. Evaluating adherence to recommended diets in adults: the Alternate Healthy Eating Index. Public Health Nutr. 2006;9(1A):152–7. [DOI] [PubMed] [Google Scholar]
- 13. Trichopoulou A, Costacou T, Bamia C, Trichopoulos D. Adherence to a Mediterranean diet and survival in a Greek population. N Engl J Med. 2003;348(26):2599–608. [DOI] [PubMed] [Google Scholar]
- 14. Trichopoulou A, Kouris-Blazos A, Wahlqvist ML, Gnardellis C, Lagiou P, Polychronopoulos E, Vassilakou T, Lipworth L, Trichopoulos D. Diet and overall survival in elderly people. BMJ. 1995;311(7018):1457–60. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Huijbregts P, Feskens E, Räsänen L, Fidanza F, Nissinen A, Menotti A, Kromhout D. Dietary pattern and 20 year mortality in elderly men in Finland, Italy, and The Netherlands: longitudinal cohort study. BMJ. 1997;315(7099):13–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Kanauchi M, Kanauchi K. The World Health Organization's Healthy Diet Indicator and its associated factors: a cross-sectional study in central Kinki, Japan. Prev Med Rep. 2018;12:198–202. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Stefler D, Pikhart H, Jankovic N, Kubinova R, Pajak A, Malyutina S, Simonova G, Feskens EJM, Peasey A, Bobak M. Healthy diet indicator and mortality in Eastern European populations: prospective evidence from the HAPIEE cohort. Eur J Clin Nutr. 2014;68(12):1346–52. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. WHO Diet, nutrition, and the prevention of chronic diseases. Report of a WHO Study Group. WHO Technical Report Series 797. Geneva (Switzerland): WHO; 1990. [PubMed] [Google Scholar]
- 19. WHO Diet, nutrition and the prevention of chronic diseases. Report of a joint WHO/FAO Expert Consultation. WHO Technical Report Series 916. Geneva (Switzerland): World Health Organization; 2003. [PubMed] [Google Scholar]
- 20. Jankovic N, Geelen A, Streppel MT, de Groot LC, Kiefte-de Jong JC, Orfanos P, Bamia C, Trichopoulou A, Boffetta P, Bobak M et al. . WHO guidelines for a healthy diet and mortality from cardiovascular disease in European and American elderly: the CHANCES project. Am J Clin Nutr. 2015;102(4):745–56. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. International Agency for Research on Cancer (IARC) Red and processed meat. IARC monographs on the evaluation of carcinogenic risks to humans. Vol. 114. Lyon (France): IARC; 2018; [Internet]. [Cited 2020 Nov 1]. Available from: https://monographs.iarc.fr/wp-content/uploads/2018/06/mono114.pdf. [Google Scholar]
- 22. World Cancer Research Fund/American Institute for Cancer Research Continuous Update Project expert report 2018. Recommendations and public health and policy implications [Internet] 2018. [Cited 2020 Oct 28]. Available from: https://www.wcrf.org/sites/default/files/Recommendations.pdf. [Google Scholar]
- 23. Bouvard V, Loomis D, Guyton KZ, Grosse Y, Ghissassi FE, Benbrahim-Tallaa L, Guha N, Mattock H, Straif K; International Agency for Research on Cancer Monograph Working Group. Carcinogenicity of consumption of red and processed meat. Lancet Oncol. 2015;16(16):1599–600. [DOI] [PubMed] [Google Scholar]
- 24. FAO Fats and fatty acids in human nutrition: report of an expert consultation. FAO Food and Nutrition Paper 91. Rome (Italy): FAO; 2010. [PubMed] [Google Scholar]
- 25. WHO Guideline: sodium intake for adults and children. Geneva (Switzerland): WHO; 2012. [PubMed] [Google Scholar]
- 26. WHO Guideline: sugars intake for adults and children. Geneva (Switzerland): WHO; 2015. [PubMed] [Google Scholar]
- 27. Herforth A, Martínez-Steele E, Calixto G, Sattamini I, Olarte D, Ballard T, Monteiro C. Development of a diet quality questionnaire for improved measurement of dietary diversity and other diet quality indicators (P13-018-19). Curr Dev Nutr. 2019;3(Suppl 1):nzz036.P13-018-19. [Google Scholar]
- 28. Herforth A. Seeking indicators of healthy diets. Washington (DC): Gallup, Inc; 2016; [Internet]. [Cited 2020 Nov 1]. Available from: https://www.gallup.com/file/poll/199448/Diet%20Quality_Gallup_white%20paper_FINAL.pdf. [Google Scholar]
- 29. WHO Countdown to 2023: WHO report on global trans-fat elimination 2020, . Geneva (Switzerland): WHO; 2020[Internet]. [Cited 2020 Nov 1]. Available from: https://apps.who.int/iris/handle/10665/334170 [Google Scholar]
- 30. Louzada MLC, Martins APB, Canella DS, Baraldi LG, Levy RB, Claro RM, Moubarac JC, Cannon G, Monteiro CA. Alimentos ultraprocessados e perfil nutricional da dieta no Brasil. Rev Saude Publica. 2015;49(38):2–11.25741658 [Google Scholar]
- 31. Martínez Steele E, Popkin BM, Swinburn B, Monteiro CA. The share of ultra-processed foods and the overall nutritional quality of diets in the US: evidence from a nationally representative cross-sectional study. Popul Health Metrics. 2017;15(1):6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32. Vandevijvere S, Monteiro C, Krebs-Smith SM, Lee A, Swinburn B, Kelly B, Neal B, Snowdon W, Sacks G; INFORMAS . Monitoring and benchmarking population diet quality globally: a step-wise approach. Obes Rev. 2013;14(Suppl 1):135–49. [DOI] [PubMed] [Google Scholar]
- 33. Instituto Brasileiro de Geografia e Estatística (IBGE) Pesquisa de orçamentos familiares 2008–2009: análise do consumo alimentar pessoal no Brasil. Rio de Janeiro (Brazil): IBGE; 2011. [Google Scholar]
- 34. Martin-Prével Y, Allemand P, Wiesmann D, Arimond M, Ballard T, Deitchler M, Dop M, Kennedy G, Lee W, Moursi M. Moving forward on choosing a standard operational indicator of women's dietary diversity. Rome (Italy): FAO; 2015. [Google Scholar]
- 35. Arimond M, Wiesmann D, Becquey E, Carriquiry A, Daniels M, Deitchler M, Fanou N, Ferguson E, Joseph M, Kennedy G et al. . Dietary diversity as a measure of the micronutrient adequacy of women's diets in resource-poor areas: Summary of results from five sites. Washington (DC): FANTA-2 Bridge, FHI 360; 2011. [Google Scholar]
- 36. Martínez-Steele E, Baraldi LG, Louzada ML, Moubarac JC, Mozaffarian D, Monteiro CA. Ultra-processed foods and added sugars in the US diet: evidence from a nationally representative cross-sectional study. BMJ Open. 2016;6(3):e009892. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37. Monteiro CA, Cannon G, Levy RB, Moubarac JC, Louzada ML, Rauber F, Khandpur N, Cediel G, Neri D, Martínez-Steele E et al. . Ultra-processed foods: what they are and how to identify them. Public Health Nutr. 2019;22(5):936–41. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38. Louzada MLC, Ricardo CZ, Martínez-Steele E, Levy RB, Cannon G, Monteiro CA. The share of ultra-processed foods determines the overall nutritional quality of diets in Brazil. Public Health Nutr. 2018;21(1):94–102. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39. Pan American Health Organization (PAHO) Ultra-processed food and drink products in Latin America: trends, impact on obesity, policy implications. Washington (DC): PAHO; 2015. [Google Scholar]
- 40. Arimond M, Wiesmann D, Becquey E, Carriquiry A, Daniels MC, Deitchler M, Fanou-Fogny N, Joseph ML, Kennedy G, Martin-Prevel Y et al. . Simple food group diversity indicators predict micronutrient adequacy of women's diets in 5 diverse, resource-poor settings. J Nutr. 2010;140(11):2059S–69S. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41. WHO Growth reference 5–19 years: BMI-for-age (5–19 years). Geneva (Switzerland):WHO;2007; [Internet]. [Cited 2020 Nov 1]. Available from: https://www.who.int/growthref/who2007_bmi_for_age/en/. [Google Scholar]
- 42. Srour B, Fezeu LK, Kesse-Guyot E, Allès B, Debras C, Druesne-Pecollo N, Chazelas E, Deschasaux M, Hercberg S, Galan P et al. . Ultraprocessed food consumption and risk of type 2 diabetes among participants of the NutriNet-Santé prospective cohort. JAMA Intern Med. 2020;180(2):283–91. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43. Rico-Campà A, Martínez-González MA, Alvarez-Alvarez I, Mendonça RD, de la Fuente-Arrillaga C, Gómez-Donoso C, Bes-Rastrollo M. Association between consumption of ultra-processed foods and all cause mortality: SUN prospective cohort study. BMJ. 2019;365:l1949. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44. Martin-Prével Y, Becquey E, Arimond M. Food group diversity indicators derived from qualitative list-based questionnaire misreported some foods compared to same indicators derived from quantitative 24-hour recall in urban Burkina Faso. J Nutr. 2010;140(11):2086S–93S. [DOI] [PubMed] [Google Scholar]
- 45. Riley L, Guthold R, Cowan M, Savin S, Bhatti L, Armstrong T, Bonita R. The World Health Organization STEPwise approach to noncommunicable disease risk-factor surveillance: methods, challenges, and opportunities. Am J Public Health. 2016;106(1):74–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46. Bromage S, Zhang Y, Holmes M, Fawzi W, Sachs S, Fanzo J, Remans R, Batis C, Bhupathiraju S, Fung T et al. . A novel food-based diet quality score is associated with nutrient adequacy and reduced anemia among rural adults in ten African countries. Curr Dev Nutr. 2020;4(Suppl 2):1381. [Google Scholar]
- 47. Matsuzaki M, Bromage S, Batis C, Fung T, Li Y, Deitchler M, Stampfer M, Willett W, Kinra S, Bhupathiraju S. Validation of a new instrument for assessing diet quality and its association with undernutrition and non-communicable diseases for women in reproductive age in India. Curr Dev Nutr. 2020;4(Suppl 2):1451. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.