Sample Size and Repeated Measures Required in Studies of Foods in the Homes of African-American Families^,,

Abstract

Measurement of the home food environment is of interest to researchers because it affects food intake and is a feasible target for nutrition interventions. The objective of this study was to provide estimates to aid the calculation of sample size and number of repeated measures needed in studies of nutrients and foods in the home. We inventoried all foods in the homes of 80 African-American first-time mothers and determined 6 nutrient-related attributes. Sixty-three households were measured 3 times, 11 were measured twice, and 6 were measured once, producing 217 inventories collected at ~2-mo intervals. Following log transformations, number of foods, total energy, dietary fiber, and fat required only one measurement per household to achieve a correlation of 0.8 between the observed and true values. For percent energy from fat and energy density, 3 and 2 repeated measurements, respectively, were needed to achieve a correlation of 0.8. A sample size of 252 was needed to detect a difference of 25% of an SD in total energy with one measurement compared with 213 with 3 repeated measurements. Macronutrient characteristics of household foods appeared relatively stable over a 6-mo period and only 1 or 2 repeated measures of households may be sufficient for an efficient study design.

Introduction

Measurement of the amount of foods in the home and the nutrient content of those foods is of interest to investigators examining food availability as a predictor of dietary intake (1), obesity (2), or food security (3). Investigators developing interventions that target changes in foods in the home to promote healthier diets also need accurate measures of home food availability. Similar to dietary intake, the changing nature of home food availability from day to day must be considered when developing protocols so that valid and reliable estimates of usual availability within a home can be obtained. Variability in measurements is influenced by temporal instability (i.e., true change in availability) as a result of purchasing behaviors and food consumption and by measurement error. It is reasonable to hypothesize that day to day variability of foods in the home may be less than that of daily food intake due to repetition in the foods prepared at home, the periodic nature of shopping, and the storage of the foods purchased in one shopping episode over multiple days, weeks, or months.

Sisk et al. (4) recently presented a pilot study of 9 households in which foods in the home were assessed on up to 5 occasions over 30 d by study staff using a 251-item home observation guide. They found that the presence of foods in some categories tended to vary over time and that fresh fruits and vegetables, milk, canned vegetables, and processed meats were not usually observed during >3 of the 5 measurement episodes. They noted that only a limited number of investigators collected repeated measurements of home food availability (5–9) and that it is not known how many households need to be assessed or how many repeated measures of each household must be obtained to achieve an adequately precise measurement of home food availability.

The purpose of this research was to provide guidance to investigators on the number of households and repeated measurements needed to obtain specified levels of precision in the estimation of nutritional attributes of foods in the home. Results are shown with and without covariates included in models. To our knowledge, this is the first study to examine the study design and sample size implications of within- and between-household variation in the availability of foods.

Materials and Methods

The participants and data collection methods used in the current study were previously described (10–12). Briefly, households were enrolled through the Infant Care Project, a longitudinal study of African American first-time mother/infant dyads who were observed in their home environments (12). Mothers were recruited through clinics for the Supplemental Food Program for Women, Infant and Children in Wake and Durham counties in North Carolina. This study was approved by the University of North Carolina public health institutional review board on research involving human subjects.

The Exhaustive Home Food Inventory was used to measure all foods and drinks in participant homes (10). Staff systematically collected information by scanning Universal Product Codes on packages of food and drink items. Scanned Universal Product Code information was transferred to a laptop computer using a FoxPro data entry program (V6.0 The Sage Group) and linked to a reference database containing food identification and nutrient information. (USDA Food and Nutrient Database for Dietary Studies, 3.0. 2008; Agricultural Research Service, Food Surveys Research Group). Items without barcodes were entered by hand or using surrogate barcodes, permitting entry of all items in the home. Over a period of ~1 y in 2006 and 2007, we aimed to perform 3 assessments in each household, each separated by ~2 mo. Using this methodology, we assessed the number of food items and the household availability of total energy, dietary fiber, fat, percent energy from fat, and energy density (kcal/g).

Mother’s age, presence of maternal grandmother, household size, shopping frequency, and the number of days since the last shopping were assessed by trained staff during the home visits. For analytic purposes, a weighted score was created to indicate household size adjusted for differences in energy needs. To do this, we used the sum of scores generated by each individual in the home using the age- and gender-appropriate energy intake from the Dietary Reference Intakes (13, 14). The score for each household member was their estimated energy allowance relative to (divided by) that of an adult female (2200 kcal/d). Shopping behaviors were measured using 3 items that queried usual shopping frequency and the number of days since the last shopping. Maternal height and weight was measured and BMI [weight (kg)/height (m)²] was calculated.

Statistical methods.

Distributions of the nutrient and food data were skewed for the number of food items, total energy, fiber, and fat. Log transformations created an approximately normal distribution based on the Anderson-Darling test (15). Percent energy from fat and energy density were normally distributed in the original scale.

Mixed models were used to calculate within- and between-household variability and the intra-class correlation (ICC). These quantities were then used to calculate the number of repeated measurements in one household needed for the mean of the observations in that household to lie within a specified percentage of the true value with 95% probability. Standard formulas (16, 17) were used for these and other calculations (Supplemental Table 1). The specified percentages were calculated as 10, 20, and 30% of the true mean. For log-transformed variables, the required number of repeated measurements within a household for 10% deviation from the true mean in the original scale was calculated by designating the relevant interval as log (1.1) unit deviation from the true mean of the log-transformed measurements. Similarly, the calculation for 20 and 30% deviation in the original scale is based on log (1.2) and log (1.3) unit deviation in the long-transformed scale, respectively. We also calculated the number of repeated measures within a household needed to achieve a given correlation coefficient between the observed and true mean. In addition, we estimated the number of households required to detect a mean difference of a given size between 2 groups with α set at 0.05 and β set at 0.20 (power at 80%). Estimates were made based on models with and without adjustment for covariates.

We calculated the number of single measurements of households needed in each of 2 groups to detect an effect size equivalent to 10, 25, 50, and 75% of 1 SD of the food or nutrient. The number of households needed to be measured to detect differences between groups given 2, 3, or 10 repeated measures in each household was also determined with and without adjustment for covariates. Again, α was set at 0.05 and β at 0.20.

Analyses were performed using SAS (version 9.2, SAS Institute).

Results

For the current study, we contacted 112 eligible mothers with 12- to 18-mo-old infants and 80 (71%) agreed to participate. Of the 80 participating households, 63 were successfully measured 3 times, 11 were measured twice, and 6 were measured once, in sum producing 217 inventories. The main reason for not participating in the repeat assessments was change in residence. Descriptive variables were reassessed at each household visit, but because values changed little in repeated visits, only the levels at the first measurement visit are shown (Table 1).

TABLE 1.

Description of households at first measurement¹

	%	Mean (n = 80)	SD	Median	IQR
Maternal age, y		24.7	4.3	23.0	6.0
Grandmother living in home, %	16.3
Adjusted household size, n Shopping frequency, %		3.0	1.4	2.9	2.0
Weekly	12.5
Biweekly	40.0
Monthly	32.5
Other	15.0
Time since last shopping trip, d		7.2	7.3	4.0	11.5
Food items, n		194	97	177	100
Total energy, MJ		990	502	872	595
Total fiber, kg		1.62	0.928	1.41	1.20
Total fat, kg		10.5	6.31	8.54	7.33
Energy from fat, %	38.3		7.55	37.9	11.4
Energy density, kJ/g		9.21	1.47	9.42	1.72

¹Households, = 80. Household inventories, n = 217.

The CV was smallest for log total energy and largest for percent energy from fat (Table 2). As expected, the number of repeated measures needed decreased as the size of the designated deviation around the mean increased. The numbers of observations needed for a specified level of deviation from the mean showed little change when covariates were added.

TABLE 2.

CV within household and number of repeated measurements within a household required for the observed household mean to lie within specified limits of the true value¹

		No covariates				Adjusted for 5 covariates2
Nutrient or index	Mean	Within-household CV	Specified percentage deviation from true mean			Within-household CV	Specified percentage deviation from true mean
10	20	30	10	20	30
		%
Food items,3 n	5.15	4.2	21	6	3	4.1	19	6
Total energy,3 MJ	6.77	3.3	21	6	3	3.1	19	6	3
Total fiber,3 g	7.21	3.6	28	8	4	3.4	26	7	4
Total fat,3 g	9.09	3.2	37	11	5	3.1	35	10	5
Energy from fat, %	38.3	15.3	10	3	2	15.3	10	3	2
Energy density, kJ/g	9.21	10.9	5	2	1	10.8	5	2	1

¹Household inventories, = 217.

²Estimates are presented for the log-transformed data.

³Adjusted for mother's age, presence of maternal grandmother, household size, shopping frequency, days since last shopping.

Percent energy from fat was the only one of 6 outcomes studied for which the variance within households was larger than the variance between households (Table 3). The within- or between-variance ratios and the ICC estimates changed little as a result of covariate adjustment and there was little or no impact on the number of repeated measures needed within households to achieve a correlation between 0.7 and 0.9 between the mean levels assessed and the true values. To obtain a correlation in this range, only one observation per household was required for most outcomes; however, as many as 7 were needed for one outcome (percent energy from fat) to obtain a correlation of 0.9.

TABLE 3.

Ratio of within/between household variances, ICC and number of repeated measurements within a household required to achieve a given correlation (r) with the true household mean¹

	No covariates					Adjusted for 5 covariates2
	Variance ratio within/between		Number of repeated measures for r			Variance ratio within/between		Number of repeated measures for r
Nutrient or index	ICC	0.9	0.8	0.7	ICC	0.9	0.8	0.7
Food items,3 n	0.26	0.79	2	1	1	0.29	0.78	2	1	1
Total energy,3 MJ	0.30	0.77	2	1	1	0.34	0.74	2	1	1
Total fiber,3 g	0.20	0.83	1	1	1	0.25	0.80	2	1	1
Total fat,3 g	0.45	0.69	2	1	1	0.56	0.64	3	1	1
Energy from fat, %	1.43	0.41	7	3	2	1.59	0.39	7	3	2
Energy density, kJ/g	0.91	0.52	4	2	1	0.86	0.54	4	2	1

¹Household inventories, = 217.

²Adjusted for mother‘s age, presence of maternal grandmother, household size, shopping frequency, days since last shopping.

³Estimates are presented for the log-transformed data.

The number of households that need to be studied to detect a difference between 2 groups of equal size was calculated assuming conventional levels of significance and power (Table 4). To detect a very small difference in the log number of food items between groups (10% of total SD) 1570 households would have to be studied if each household was measured once and covariates were not included. In contrast, only 28 households would be needed to detect a difference as large as 75% of 1 SD.

TABLE 4.

ICC and number of households per group required to detect a mean difference of 10, 25, 50k and 75% of SD

	No covariates						Adjusted for 5 covariates 1
Nutrient or index	ICC	Effect size % of SD	Number of households for a fixed number of repeated measures				ICC	Effect size % of SD	Number of households for a fixed number of repeated measures
1	2	3	10	12	2	3	10
Food items,3 n	0.79	10	1570	1408	1354	1279	0.78	10	*	1395	1337	1256
		25	252	226	217	205		25	*	224	214	201
		50	63	57	55	52		50	*	56	54	51
		75	28	26	25	23		75	*	25	24	23
Total energy,3 MJ	0.77	10	1570	1391	1332	1248	0.74	10	*	1370	1303	1210
		25	252	223	213	200		25	*	220	209	194
		50	63	56	54	50		50	*	55	53	49
		75	28	25	24	23		75	*	25	24	22
Total fiber,3 g	0.83	10	1570	1439	1395	1334	0.80	10	*	1413	1361	1287
		25	252	231	224	214		25	*	227	218	206
		50	63	58	56	54		50	*	57	55	52
		75	28	26	25	24		75	*	26	25	23
Total fat,3 g	0.69	10	1570	1326	1245	1131	0.64	10	*	1288	1194	1062
		25	252	213	200	181		25	*	206	191	170
		50	63	54	50	46		50	*	52	48	43
		75	28	24	23	21		75	*	23	22	19
Energy from fat, %	0.41	10	1570	1108	955	739	0.39	10	*	1089	928	704
		25	252	178	153	119		25	*	175	149	113
		50	63	45	39	30		50	*	44	38	29
		75	28	20	17	14		75	*	20	17	13
Energy density, kJ/g	0.52	10	1570	1197	1072	898	0.54	10	*	1206	1085	915
		25	252	192	172	144		25	*	193	174	147
		50	63	48	43	36		50	*	49	44	37
		75	28	22	20	16		75	*	22	20	17

¹Adjusted for mother's age, presence of maternal grandmother, household size, shopping frequency, and days since last shopping.

²Number of households is the same as shown for “1” under No covariates.

³Estimates are presented for the log-transformed data.

Because the assumed effect size was calculated as a percent of the SD of the outcome, the number of repeated measures needed per household was identical for each nutrient or index in calculations that included only one measurement of each household. Increasing the number of repeated measures decreased the number of households needed to detect effects; however, in no case was the total number of measurements required reduced by increasing the number of repeated measures. For instance, with each household studied only once, 63 households in each of 2 groups (a total of 126 measurement episodes) were needed to detect a difference as small as 50% of 1 SD of the log number of food items. To detect the same effect size, 52 households in each of 2 groups need to be studied on 10 occasions (1040 measurement episodes). In this example, covariate adjustment reduced the number of measurement episodes from 63 to 56 to detect this effect with one measurement episode per household and from 1040 to 1020 with each household measured 10 times.

Discussion

We found that over a period of months, the variance between households was larger than the variance within households for several macronutrients and for the total number of food items. The variance ratios were larger for the ratio measures of percent energy from fat and energy density compared with measures of absolute amounts of nutrients. Studies of the within- to between-subject variance in dietary intake from 24-h recalls or food records have shown similar trends. For example, in the study by Beaton et al. (18), the ratio of within-person:between-person variance for grams of fat in women was 1.7, whereas the ratio for percent calories from fat was 2.6. This contrast was even more marked in men, with the respective ratios being 1.2 and 4.8. In the households of the women studied here, the ratios of the within:between variance were 0.45 for grams of fat and 1.43 for percent energy from fat.

The large within-person:between-person ratio in daily dietary intakes in free living individuals is well known (19) and necessitates the collection of multiple days of 24-h recalls to obtain stable estimates of usual intake. Nelson et al. (20) combined dietary intake data from multiple cohorts and showed that the number of days of diet records required to obtain an r of ≥0.9 between the observed and true values in adult women was 5 for energy, 6 for fat, and 5 for fiber. In comparison, we found the estimates of repeated observations needed to obtain the same correlation in foods in the home were 2 for energy, 1 for fat, and 2 for fiber.

Similar to studies of nutrients in daily food intake, we found that the distributions of the amount of nutrients in foods in the home tended to be skewed. In our data, when the distributions were not normal, we applied log transformations. Unfortunately, the interpretation of transformed data is complicated by the fact that the size of the effect of an exposure that can be detected in the untransformed variables between unexposed and exposed groups differs depending on the mean level. For example, if the mean number of food items in one group of households was 127 (4.84 in the log scale), then with 28 households in each group (each measured one time), we would be able to detect a significant difference (P < 0.05) between the 2 groups if the they differed by 56 items. Instead, if the number of food items in one group of households was 244 (5.5 in the log scale), then with the same number of households, we would be able to detect a significant difference between the 2 groups if they differed by 107 items. Thus, investigators estimating the number of households and repeated measures to be collected to obtain a desired level of precision and accuracy will need to estimate both the differences expected and the expected mean in 1 of the 2 groups in order to apply our estimates to a calculation of statistical power for untransformed nutrients. In addition, log transformation will need to be planned in the subsequent data analyses, assuming that the transformation makes the data more normal.

Researchers often use simplifying assumptions in the calculation of power for a planned study. One of the assumptions commonly made is to ignore the covariates that will be used to adjust estimates in the final analyses. Palaniappan et al. (16) compared the within-person:between-person ratio in daily dietary intakes with and without adjustment for 6 covariates. They found that covariate adjustment resulted in a larger ratio for all the nutrients examined and therefore a greater number of days of intake needed to be observed to obtain fixed levels of precision and accuracy. In women, the ratios for energy were 2.04 compared with 1.76 in the adjusted compared with the unadjusted models. The authors noted that the higher ratios were due to a reduction in between-subject variance and cautioned that not adjusting for covariates in a power calculation could result in the study having insufficient power to detect differences in intakes when examined in multivariate analyses. We found that covariate adjustment resulted in the same or a slightly lower within-household CV, ICC, and number of households that needed to be examined for the same power. The list of covariates examined in our work differed from that used by Palaniappan et al. (16) and this difference, differences in daily nutrient intake compared to nutrients in the home, and numerous other differences in the 2 studies may account for the discrepancy. More work needs to be done to better understand the impact of statistical adjustment on the sample size requirement for an adequately powered study of nutrients in the home.

This work was limited to the study of macronutrients pertinent to our interest in obesity and listed on the food facts label. Willett et al. (19) and several others (18, 20–21) have shown that the number of repeated measures needed for a given level of precision and accuracy for estimation of macronutrients is often smaller than that needed for estimation of micronutrients in the diet. This is because in a mixed group of different types of foods (meats, eggs, breads, vegetables, fruits, sweets, etc.), the distribution of micronutrients tends to be more varied than macronutrients. It is possible that a larger number of repeated visits would be required for the examination of micronutrients in the home food supply. Another limitation of this work was that although repeated assessments of households were separated by an average of 2 mo and households were studied over the course of a year, seasonal effects on foods in the home were incompletely evaluated. In addition, caution should be exercised in the application of our results to populations different from the one studied here.

A great strength of home food surveys compared with diet records or recalls is the validity of the data collected. It is well known that self-reported dietary intake is prone to reporting bias, that obese individuals report a lower energy intake than normal-weight participants, and that correlations of reported intake with objective markers such as energy from doubly labeled water are moderate at best (22). It is a refreshing contrast that repeated measures of food in the home show very high validity when assessed using trained examiners. Recent reports on the repeatability and validity of self-reported data on foods in the home from checklists indicate promising levels of precision and accuracy using instruments that are quick and inexpensive to administer compared with the Exhaustive Home Food Inventory (23–26).

Exhaustive surveys of all foods in the home provide a method of validating home food availability checklists as well as providing detailed, precise, and accurate data for analysis of hypotheses concerning the causes and consequences of foods in the home. To our knowledge, studies determining intra- and/or inter-household level food variability have not been conducted previously. This work as well as the work of others (27–30) indicate that direct observation of foods in the home provides a feasible and valid opportunity to gain insights into the influence of the home food environment on diet, obesity, food security, and other diet related factors.