A population’s mean Healthy Eating Index-2005 scores are best estimated by the score of the population ratio when one 24-hour recall is available

Abstract³

The U.S. Department of Agriculture’s (USDA) Healthy Eating Index-2005 (HEI-2005) is a tool to quantify and evaluate the quality of diet consumed by the US population. It comprises 12 components, expressed as ratios of a food group or nutrient to energy intake. The components are scored on a scale from 0 to M, where M is 5, 10 or 20. Ideally the HEI-2005 is calculated on the basis of the usual dietary intake of an individual. Intake data, collected via a 24-hour recall, are often available for only one day on each individual. In this paper, we examine how best to estimate a population’s mean usual HEI-2005 component and total scores when one day of dietary information is available for a sample of individuals from the population. Three methods are considered: the mean of individual scores, the score of the mean of individual ratios, and the score of the ratio of total food group or nutrient intake to total energy intake, which we call the population ratio. We investigate via computer simulation which method is the least biased. The simulations are based on statistical modeling of the distributions of intakes reported by 738 women participating in the Eating at America’s Table Study. The results show that overall the score of the population ratio is the preferred method. We therefore recommend that the quality of the US population’s diet be assessed and monitored using this method.

Introduction

The U.S. Department of Agriculture’s (USDA) Healthy Eating Index-2005 (HEI-2005) is a tool to quantify and evaluate the quality of the diet consumed by the US population in terms of its conformity to the 2005 Dietary Guidelines for Americans [1,2]. It comprises 12 components, which are scored on a scale from 0 to M, where M is 5, 10 or 20 according to the component. The maximum total score is 100. Each component is expressed as a ratio of an individual’s intake of a specific food or nutrient to their intake of energy, before scoring.

Ideally, the HEI-2005 should be calculated on the basis of the usual dietary intake of each individual, that is, their mean intake over a specified period (often 1 year). This is in line with the Institute of Medicine’s emphasis on assessing usual diets. Both the Institute of Medicine and the Dietary Guidelines for Americans 2005 point out that recommendations are to be met over the long-term [3,4]. In practice, the usual intake of an individual cannot be observed. Often, only one day of food intake, collected via a 24-hour recall, has been available. In such circumstances, the HEI-2005 component and total scores of each individual’s one-day intake can be calculated, but this will lead to a biased measure of the individual’s HEI-2005 score of usual intake when the individual’s one-day food/nutrient-to-energy ratio is correlated with his/her energy intake. More critically, even in the absence of such a correlation, the HEI-2005 score on a single day can be a biased measure of the mean HEI-2005 score across days because the scoring system is truncated at 0 on one end and at 5, 10, or 20 at the other. As a result, the long-term mean of HEI-2005 scores on single days of intake differs from the score of the long-term mean intake over those days, which is what we want to measure.

USDA’s most important use of the HEI is to monitor the dietary intake of the population over time. For this purpose the natural measure of the quality of the population’s diet is the population’s mean HEI component and total scores, based on the usual intake of each component. We will call these the population’s mean usual HEI component scores. In this report, we examine three ways of estimating the population’s mean usual HEI-2005 component scores from data on a series of individuals, each of whom supplied a single 24-hour recall (24HR). With such limited data, no unbiased estimate is available. Our main concern is to identify which of the three methods has the least bias.

Methods

The three methods that we compare are:

1. For each individual, calculate each HEI-2005 component score on the basis of his/her 24HR. Then, for each component score and for the total score, take the (arithmetic) mean over individuals. We call this the mean score. The HEI-2005 total score is calculated as the sum of these scores over the 12 components.

2. For each individual and each component, calculate the ratio of the reported intake of food group or nutrient (relevant to the HEI component considered) to the reported energy intake. Then take the mean of these ratios over individuals. Finally, calculate the HEI-2005 component score based on this mean ratio. We call this the score of the mean ratio. The HEI-2005 total score is calculated as the sum of these scores over the 12 components.

3. Calculate the population’s total intake of food group or nutrient (relevant to the HEI component considered) and the population’s total energy intake and take the ratio of these. Then calculate the HEI-2005 component score based on this ratio of the totals. We call this the score of the population ratio. The HEI-2005 total score is calculated as the sum of these scores over the 12 components.

The names given to methods (b) and (c) follow those of Krebs-Smith et al [5].

It is not immediately clear which method would be least biased, and one can construct different numerical examples where each one of the three is the superior method. The methods must therefore be tested with data that (a) are realistic and conform to typically reported values, and (b) come from a population with known population mean usual HEI-2005 component scores. Unfortunately, available real datasets do not satisfy condition (b), so we employ instead computer simulations of data generated from a statistical model that is based on real data.

The dataset we used as a basis for our statistical model is drawn from the Eating at America’s Table Study (EATS) [6]. The study was approved by the National Cancer Institute Special Studies Institutional Review Board. The 738 women we studied were part of a nationally representative sample. Participants were asked to complete four 24HRs via telephone over a period of one year (1997–98), with one recall per season. Six hundred and fifty (88%) of these women completed all four recalls. Foods reported on the 24-hour recalls were coded using the Food Intake Analysis System, version 2., which calculated total daily intakes for energy, saturated fat and sodium. The food codes, in turn, were linked to the MyPyramid Equivalents Database, version 1.0, in order to calculate total daily intakes of the food groups of interest.

Summary statistics on the first day’s reported intake of the 12 HEI-2005 components (and energy) were computed (Table 1). Note that the mean ratio is different from the population ratio (final two columns of Table 1). In most cases, the mean ratio has the larger value; but for Oils, Saturated Fat, and Solid Fats, Alcoholic beverages, and Added Sugars (SoFAAS), it has the smaller value.

Table 1.

Summary of distributions of intake of selected foods, nutrients and energy, as reported on first day by 738 women participating in EATS

Dietary component	Mean (SD)	% zeros	Percentiles			Mean ratio to energy ×103	Ratio of mean to mean energy ×103
			10	50	90
Energy, kcala	1753 (777)	0	892	1656	2679	-	-
Total fruit, cup equivb	0.94 (1.15)	20	0	0.56	2.60	0.58	0.54
Whole fruit, cup equivb	0.54 (0.81)	44	0	0.07	1.56	0.34	0.31
Total vegetables, cup equivb	1.62 (1.34)	4	0.22	1.34	3.31	1.00	0.92
Dark green, orange vegetables/legumes, cup equivb	0.26 (0.46)	49	0	0.01	0.86	0.17	0.15
Total grains, oz. equivc	5.82 (3.33)	<1	2.09	5.44	10.02	3.39	3.32
Whole grains, oz.equivc	0.75 (1.18)	43	0	0.12	2.23	0.45	0.43
Milk, cup equivb	1.25 (1.25)	10	0	0.96	3.00	0.72	0.71
Meat/beans, oz. equivc	4.71 (3.77)	3	0.96	4.01	8.83	2.74	2.69
Oils, g	12.38 (14.96)	12	0	8.05	31.96	6.81	7.06
Saturated Fat, g	21.05 (15.11)	0	6.71	18.26	37.37	11.55	12.01
Sodium, g	3.10 (1.61)	0	1.45	2.81	5.06	1.83	1.77
Solid Fats, Alcoholic beverages, and Added Sugars, kcala	651 (453)	0	203	574	1132	359	371

^a1 kcal = 4.184 kJ

^b1 cup equivalent = 240 mL

^c1 oz. equivalent = 28g

The statistical model forming the basis of our computer simulations was constructed under a set of assumptions and calculations. All of the model parameters were estimated from the data on the women participating in EATS. The details of the estimation procedures are contained in on-line Appendix A. A brief description of how the model was formed is given below.

Some food groups are not consumed every day by all individuals. We refer to days on which a given food group is consumed by a given individual as that individual’s “consumption days,” the remaining days being the individual’s “non-consumption days.”

First we made an assumption about the intake distributions. Distributions of intake on consumption days, both between individuals and within individuals, were assumed to be normal after a suitable power transformation. The power transformation for each food/nutrient was individually chosen after inspection of the deciles of the distribution (see second column of on-line Supplemental Table 1).

For food groups (but not for nutrients), there is a probability of non-consumption on a single day. We examined three assumptions regarding this probability, each of increasing complexity.

Assumption (i)

The probability of consumption is the same for all individuals.

Unfortunately, this assumption was not supported by the EATS data, where too many individuals report consuming a particular food group on either no days or on all 4 days. Therefore, we postulated the following.

Assumption (ii)

There are five subclasses of individuals consuming the food on 0%, 25%, 50%, 75% or 100% of days, respectively. In addition, the distribution of intakes on consumption days is independent of the probability of consumption.

The second part of assumption (ii), namely, that the distribution of intakes on consumption days is independent of the individual’s probability of consumption can be readily checked against data. Unfortunately, this independence assumption was also not supported by the EATS data. In fact, reported intakes on consumption days have previously been reported to correlate positively with the probability to consume [7]. This led us to the final assumption.

Assumption (iii)

There are the same five probability of consumption subclasses as in assumption (ii), but each has its own mean intake on consumption days.

Once the statistical model had been formulated, simulation programs were written in S-Plus (S-Plus 2000, Professional Edition for Windows, Release 1, Seattle, 1999) that (a) generated data from the food/nutrient and energy intake distributions under the three different assumptions regarding the probability to consume and (b) computed the three estimates of population mean HEI-2005 component scores (mean score, score of the mean ratio, and score of the population ratio). Each simulation generated a population of 10,000 persons, a single day of intake (both the food/nutrient for the component of interest and energy) for that individual to be used in computing the three estimates, and a true usual intake (both for the food/nutrient and for energy) for each individual. The true usual intakes were used to compute the true population mean HEI-2005 component scores with which the estimates of this population mean based on a one-day report could be compared. Further details of the simulations may be found in on-line Appendix B.

Results

The mean of the true HEI-2005 component scores over individuals and the three estimates (mean score, score of mean ratio, and score of population ratio) were computed (Tables 2–4), from simulations based on the three statistical models described in Section 2, on-line Appendices A and B and on-line Supplemental Tables 1 and 2. Note that all values in Tables 2–4 are based on means of 10,000 simulated individuals; and that, consequently, standard errors are very small and all differences are statistically significant.

Table 2.

Comparison of three estimates with the true mean HEI component scores, based on computer simulation^*: model with constant probability of consumption^#

Component (Range of Score)	True mean score	Mean of individual scores	Score of mean of individual ratios	Score of the ratio of the means
Total Fruit (0–5)	3.30	2.39	4.07	3.49 a
Whole Fruit (0–5)	3.79	2.03	5.00	4.15 a
Total Vegetables (0–5)	3.90	3.30	4.58	4.10 a
Dark Green and Orange Vegetables and Legumes (0–5)	2.02	1.41	2.25	1.90 a
Total Grains (0–5)	4.72	4.21	5.00 a	5.00 a
Whole Grains (0–5)	1.49	1.25	1.68	1.42 a
Milk (0–10)	5.31	4.69	5.61	5.33 a
Meat and Beans (0–10)	9.13	7.42	10.00 a	10.00 a
Oils (0–10)	6.14	4.71	6.29	6.09 a
Saturated Fat (0–10)	6.64	6.57 a	7.70	6.96
Sodium (0–10)	2.45	3.33	1.87	2.23 a
Calories from SoFAASb (0–20)	10.17	10.70	10.58 a	9.67
Total (0–100)	59.06	52.01	64.63	60.34 a
Absolute bias** mean (min, max)	-	0.82 (0.07, 1.76)	0.56 (0.15, 1.21)	0.27 a (0.02, 0.87)

^*10,000 simulations for each component.

^#Parameters of model estimated from data on 738 women participating in EATS

^**Mean of the absolute differences between the estimated mean HEI-2005 component scores and the true mean HEI-2005 component scores, taken over the 12 components.

^aThe best of the three estimates

^bSolid Fats, Alcoholic beverages and Added Sugars

Table 4.

Comparison of three estimates with the true mean HEI component scores, based on computer simulation^*: model with variable probability of consumption, correlated with intake on consumption days^#

Component (Range of Score)	True mean score	Mean of individual scores	Score of mean of individual ratios	Score of the ratio of the means
Total Fruit (0–5)	3.03	2.29	4.01	3.54 a
Whole Fruit (0–5)	3.12	1.95	4.95	4.24 a
Total Vegetables (0–5)	3.86	3.30	4.55	4.06 a
Dark Green and Orange Vegetables and Legumes (0–5)	1.96	1.43	2.27	1.95 a
Total Grains (0–5)	4.70	4.19	5.00 a	5.00 a
Whole Grains (0–5)	1.52	1.29	1.82	1.56 a
Milk (0–10)	5.17	4.67	5.60	5.39 a
Meat and Beans (0–10)	9.09	7.47	10.00 a	10.00 a
Oils (0–10)	6.03	4.61	6.19	6.07 a
Saturated Fat (0–10)	6.64	6.57 a	7.70	6.96
Sodium (0–10)	2.45	3.33	1.87	2.23 a
Calories from SoFAASb (0–20)	10.17	10.70	10.58 a	9.67
Total (0–100)	57.84	51.10	64.54	60.67 a
Absolute bias** mean (min, max)	-	0.73 (0.07, 1.62)	0.66 (0.16, 1.83)	0.37 a (0.01, 1.12)

^*10,000 simulations for each component.

^#Parameters of model estimated from data on 738 women participating in EATS

^**Mean of the absolute differences between the estimated mean HEI-2005 component scores and the true mean HEI-2005 component scores, taken over the 12 components.

^aThe best of the three estimates

^bSolid Fats, Alcoholic beverages and Added Sugars

The results presented in the three tables are qualitatively equivalent; the more sophisticated modeling of the intake distributions seems to have had little impact. The results indicate that of the three estimates, the one that usually comes closest to the true population mean HEI-2005 component scores is the score of the population ratio. The estimate is not guaranteed to be accurate, as can be seen from the tables; but for most of the components, this estimate is better than the alternatives. The exceptions are Saturated Fat and SoFAAS. For Saturated Fat, the mean score was superior; whereas for SoFAAS, the score of the mean ratio was superior. For two other components (Total Grains, Meat and Beans), both the score of the population ratio and the score of the mean ratio attained the maximum, which was closest to the true mean score for these components. It is notable that for food groups, the mean score was consistently too low and the score of the mean ratio was consistently too high. For the other components, the pattern was less clear. For the HEI-2005 total score, the score of the population ratio was clearly superior.

To obtain a summary view of the accuracy of each method we averaged the absolute bias (that is, the absolute difference between the estimate and the true value) over the 12 components (final row of Tables 2–4). The maximum and minimum absolute bias taken over the twelve components is also shown. The mean absolute bias was substantially lower when using the score of the population ratio, than when using either of the other two estimators.

Discussion

The HEI-2005 component scores are based upon ratios of reported intakes of food groups or nutrients to that of total energy. Estimating distribution properties of ratios is always more challenging than estimating those of single variables. Ratios may be complicated by measurement errors and other variation in the denominator values and by correlations between the denominator values with the ratio of the numerator to the denominator [5]. For example, an individual’s energy intake on a specific day is often positively correlated with his/her fat-to-energy ratio on that day.

Such complications are compounded by the HEI-2005 component scores themselves, which are non-linear functions of a ratio, due to the truncation imposed at the minimum and maximum scores. This non-linearity can lead to bias even when the ratio itself is estimated without bias. For example, consider the Whole Fruits component and imagine an individual who consumes exactly 2000 kcal (8368 kJ) consistently each day. Suppose this individual consumes 1 cup-equivalent (240 mL) of whole fruit on half the days, but none on the other half. Then the mean or “usual” ratio for the individual is 0.25 cup-equivalents (60 mL) per 1000 kcal (4184 kJ), leading to a score of 5 × 0.25/0.4 = 3.125, where 0.4 is the truncation point for the maximum achievable score of 5. If we now determine the mean of the ratios over several days, we will obtain over the long-term the correct 0.25 cup-equivalents (60 mL) per 1000 kcal (4184 kJ) (since energy intake is constant). If we determine means of the scores on individual days, however, then over the long-term we will obtain a minimum score of 0 on half the days and a maximum score of 5 on the other half, giving a mean of 2.5, and not the true value of 3.125.

These complications make it impossible to predict analytically which of the three proposed estimates is likely to be the least biased. This suggests that the surest way of investigating the matter is through computer simulation. Based on the results in Tables 2–4, the least biased of the three methods to estimate population’s mean usual HEI-2005 component scores is the score of the population ratio.

Our conclusion is that one should estimate the population’s mean usual HEI-2005 component scores by calculating the score of the population ratio, that is, by taking the score of the ratio of the total food/nutrient intake to energy intake. Nevertheless, this conclusion has some caveats. The conclusion is empirically driven and depends on the US distributions of reported intakes of the components included in the HEI-2005, as well on the standards by which the HEI-2005 component scores are determined.

We have found in a sensitivity analysis that our conclusion is robust to the sampling errors involved when estimating the parameters from the sample of 738 women participating in EATS. The results are reported in on-line Appendix C. We have also examined distributions of intake reported by men in the EATS study and by women in the Continuing Survey of Food Intakes by Individuals, 1994–96 [8]. Although we have not fully modeled these data in the same depth as the data on the women in EATS, we obtained a strong impression that the distributional characteristics were very similar in the three groups (allowing for different levels of absolute intake) and would lead to the same conclusions presented here.

Nevertheless, we are aware that substantial changes in intake distributions or in the scoring standards could change the conclusions. For example, while developing the details of this work, we noticed that changes in the chosen standards for the scores could change the performance of the three methods that we examined.

It is important to check that the data used for calculating the population’s mean usual HEI scores are representative of the usual intake of the population, even if usual intake cannot be assessed in the individual participants. This requires that, in order to make inferences about the US population, the data come from a nationally representative sample, the dietary reports are collected for all seven days of the week with proportional representation weekend and week days and seasons of the year. If probability samples rather than simple random samples are used, then the appropriate weights must be employed when the population ratios of the total food/nutrient intake to total energy intake are estimated. It is also advisable that the sample is quite large, in the order of 1000 individuals or more, to ensure that the standard errors of the estimates are relatively small.

As mentioned above, we are confident that our conclusion holds true for currently available US population data. However, we are not so sanguine with regard to minority subpopulations of the US, nor with regard to populations in other countries. We recommend that researchers interested in HEI-2005 component scores in these populations carry out a similar exercise to that reported here, simulating data that follow intake distributions reported in the population of interest. Until evidence emerges for the superiority of another estimate, the score of the population ratio would seem to be the best choice in such cases. We also recommend that periodic checks be carried out to confirm that this measure remains optimal for the US population because intake distributions may change.

With the caveats mentioned, we recommend estimating the population’s mean usual HEI-2005 component scores by the score of the population ratio. Constructing a (two-sided) 95% confidence interval for this measure is recommended over estimating a standard error, as the sampling distribution may be asymmetric. A 95% confidence interval for a component score can be constructed using standard survey packages in the following manner. First, determine the confidence-interval for the associated population ratio with the package, and then score the end points of the interval. A precision measure for the total HEI-2005 score - the sum of the 12 component scores - is more difficult to develop. An algorithm is given in on-line Appendix D.

Our main comparison of the three estimators was based on their biases and not on their standard errors. We considered the standard error of the estimators to be of secondary importance to the bias, because in the relatively large samples that we envisage the bias will dominate the error of the estimate, especially in this case where the biases are often large. To check this further, we computed from our simulation (under the assumption of a varying probability of consumption that is correlated with amount of intake on consumption days) the standard error of the three estimates that would be expected from a sample of 1000 individuals. The mean of the standard errors taken over the 12 components were 0.09 for the mean score, 0.18 for the score of the mean ratio, and 0.14 for the score of the population ratio, compared to mean absolute biases of 0.73, 0.66 and 0.37 respectively. More details may be found in on-line Appendix E.

Nutritional survey data sometimes include repeated dietary assessments on all or a subset of participants. Such repeat assessments allow statistical modeling of within-person variation and offer the possibility of reducing the bias in estimating the population distribution of usual intakes by using statistical modeling [8,9]. A future research aim will be to extend such methods to estimate the US population distribution of the usual HEI-2005 component scores. It is clearly advantageous to be able to estimate the full distribution rather than just the mean. Furthermore, if this can be implemented successfully, it would be a short further step to estimate the population mean directly from these distributions. In principle, estimates of the population mean derived in this manner should have minimal bias and could, therefore, be an improvement over the best method when one 24HR is available, namely the score of the population ratio. Currently, the score of the population ratio should be regarded as the principal method for estimating the population mean usual HEI-2005 component and total scores.

Supplementary Material

online support

Supplemental Table 1. A: Model parameters of the distribution of power-transformed intake on consumption days for HEI components and energy* (1 kcal = 4.184 kJ; 1 cup equivalent = 236.6 mL; 1 oz. equivalent = 28.35g)

Supplemental Table 2. Probabilities of consumption on a given day, with mean intakes (on power scale) within each probability subclass, for HEI components and energy*

Table 3.

Comparison of three estimates with the true mean HEI component scores, based on computer simulation^*: model with variable probability of consumption model, independent of intake on consumption days^#

Component (Range of Score)	True mean score	Mean of individual scores	Score of mean of individual ratios	Score of the ratio of the means
Total Fruit (0–5)	3.22	2.40	4.09	3.50 a
Whole Fruit (0–5)	3.38	2.02	4.97	4.10 a
Total Vegetables (0–5)	3.86	3.31	4.58	4.10 a
Dark Green and Orange Vegetables and Legumes (0–5)	2.02	1.42	2.28	1.92 a
Total Grains (0–5)	4.70	4.20	5.00 a	5.00 a
Whole Grains (0–5)	1.54	1.30	1.75	1.47 a
Milk (0–10)	5.26	4.64	5.50	5.25 a
Meat and Beans (0–10)	9.05	7.37	10.00 a	10.00 a
Oils (0–10)	6.09	4.65	6.30	6.05 a
Saturated Fat (0–10)	6.64	6.57 a	7.70	6.96
Sodium (0–10)	2.45	3.33	1.87	2.23 a
Calories from SoFAASb (0–20)	10.17	10.70	10.58 a	9.67
Total (0–100)	58.38	51.91	64.62	60.25 a
Absolute bias** mean (min, max)	-	0.77 (0.07, 1.68)	0.62 (0.21, 1.59)	0.31 a (0.01, 0.95)

^*10,000 simulations for each component.

^#Parameters of model estimated from data on 738 women participating in EATS

^**Mean of the absolute differences between the estimated mean HEI-2005 component scores and the true mean HEI-2005 component scores, taken over the 12 components.

^aThe best of the three estimates

^bSolid Fats, Alcoholic beverages and Added Sugars