Gains in Statistical Power From Using a Dietary Biomarker in Combination With Self-reported Intake to Strengthen the Analysis of a Diet-Disease Association: An Example From CAREDS

Abstract

A major problem in detecting diet-disease associations in nutritional cohort studies is measurement error in self-reported intakes, which causes loss of statistical power. The authors propose using biomarkers correlated with dietary intake to strengthen analyses of diet-disease hypotheses and to increase statistical power. They consider combining self-reported intakes and biomarker levels using principal components or a sum of ranks and relating the combined measure to disease in conventional regression analyses. They illustrate their method in a study of the inverse association of dietary lutein plus zeaxanthin with nuclear cataracts, using serum lutein plus zeaxanthin as the biomarker, with data from the Carotenoids in Age-Related Eye Disease Study (United States, 2001–2004). This example demonstrates that the combined measure provides higher statistical significance than the dietary measure or the serum measure alone, and it potentially provides sample savings of 8%–53% over analysis with dietary intake alone and of 6%–48% over analysis with serum level alone, depending on the definition of the outcome variable and the choice of confounders entered into the regression model. The authors conclude that combining appropriate biomarkers with dietary data in a cohort can strengthen the investigation of diet-disease associations by increasing the statistical power to detect them.

Measurement error in dietary reporting presents a major challenge to nutritional epidemiology (1). In univariate models, these errors cause underestimation of relative risks and greatly impair the statistical power to detect a diet-disease association. In multivariate models, they cause biases of unpredictable magnitude and direction in the estimation of relative risks (2). Most of the statistical work concerning the effect of dietary measurement error on results in nutritional epidemiology focuses on the validation of questionnaires or the deattenuation of estimated relative risks and odds ratios (3, chapters 6 and 12). However, methods of deattenuation do not increase the statistical power with which diet-disease associations may be detected. On the contrary, the methods usually further decrease, by a small amount, the statistical power (4, p. 60).

We previously described a method for increasing the statistical power of studies of diet-disease associations by combining dietary biomarker levels with usual self-reported dietary intakes (5). We showed, through computer simulations, that such a method potentially leads to an increase in power and a decrease in required sample size in epidemiologic studies (5). In this paper, we illustrate the method using data from the Carotenoids in Age-Related Eye Disease Study (CAREDS) (6) on the inverse association between dietary lutein plus zeaxanthin and nuclear cataracts, and we show that it provides some modest gains in statistical power.

MATERIALS AND METHODS

Study population

CAREDS is an ancillary study of the Women's Health Initiative (WHI) Observational Study, a prospective cohort study of 93,676 postmenopausal women aged 50–79 years at enrollment recruited from 40 sites around the United States (7, 8). The CAREDS population included women enrolled at 3 of the sites: University of Wisconsin (Madison), University of Iowa (Iowa City), and Kaiser Center for Health Research (Portland, Oregon). Women with reliable food frequency questionnaire data who reported total energy ≥600 kcal (2,510 kJ) and ≤5,000 kcal (20,920 kJ) and who reported intakes of lutein plus zeaxanthin above the 78th and below the 28th percentiles, as assessed at the WHI baseline examination in 1994–1998, were considered for recruitment in CAREDS. Of the 3,143 eligible women, 96 had died or were lost to follow-up between WHI recruitment and the time of enrollment in CAREDS in 2001–2004. Of those remaining, 2,005 agreed to participate (66%). Of these women, 194 were excluded; one participant provided unreliable dietary information, 32 reported a history of trauma to both eyes, one reported cataract extraction before the age of 40 years, 132 had missing or ungradable nuclear lens photographs, and 28 had missing covariate data. The final analytical set included 1,811 women. Further details on the study design have already been reported (6). All procedures conformed to the Declaration of Helsinki and were approved by the institutional review board at each university.

Dietary assessment

Dietary intake was assessed at WHI Observational Study baseline (1994–1998) by using the WHI semiquantitative food frequency questionnaire, which was previously pretested (9). Nutrient and food group estimates were computed at the Fred Hutchinson Cancer Research Center, Seattle, Washington.

Serum lutein and zeaxanthin

Serum samples were collected after 10 or more hours of fasting at the WHI baseline examinations (1994–1998) and were analyzed for lutein and zeaxanthin (sum of their trans isomers) (6). Serum lutein and zeaxanthin measurements were available from 1,787 women.

Age-related nuclear cataract

Participants underwent lens photography and eye examinations during the study visits at CAREDS baseline between 2001 and 2004 (6). They also completed a questionnaire querying about time of cataract surgery in each eye; trauma to eyes; physician-diagnosed history of cataract, glaucoma, and macular degeneration; and treatments or lifestyle changes that accompanied these conditions. Information on family history of nuclear cataract was also collected.

Our primary outcome was nuclear cataract, defined as a nuclear sclerosis severity score of 4 or greater in the worst eye and/or a history of cataract extraction in either eye; all women in the data set had information that used this definition (N = 1,811). Women who had a history of cataract extraction, but who lacked evidence of a nuclear sclerosis severity score of 4 or greater, were still included as cases because it has been shown that the most likely indication for cataract surgery is photographically evident cataracts in the nuclear region of the lens (10). We also analyzed a secondary outcome, nuclear sclerosis only with a severity score of 4 or greater, which could be assessed in the 1,580 women who had at least one natural lens for which lens photographs were gradable.

Covariates

The questionnaire completed at CAREDS baseline collected information on age, family history of cataract, ultraviolet B sunlight exposure, and use of dietary supplements. Other lifestyle and medical history data were available from the WHI baseline questionnaire (e.g., height, weight, education, smoking, physical activity, use of hormone replacement therapy, multivitamin use, pulse pressure, alcohol consumption, diabetes, cardiovascular disease, and hypertension). Iris color was determined from photographs taken at the CAREDS eye examination.

Statistical methods

We investigated 3 ways of analyzing reported dietary lutein plus zeaxanthin intake and lutein plus zeaxanthin serum level; the third one combines the self-report and biomarker. Each analysis involved a logistic regression model with a set of confounding variables as well as one of the following exposure measures:

1. Reported dietary intake: food frequency questionnaire–reported lutein plus zeaxanthin.

2. Biomarker level: serum lutein plus zeaxanthin.

3. Howe's score with ranks: participants are ranked from lowest to highest value for reported dietary intake and biomarker level. The 2 ranks are then summed. This is a special case of Howe's method (11) in which subjects are placed in one of k quantiles (1, 2, …, k) of dietary intake and in one of k quantiles (1, 2, …, k) of biomarker level, and the subject is given a score equal to the sum of the 2 quantiles. In the version we used, the number of quantiles k is set equal to the sample size; it is expected to be the most efficient version of Howe's method.

In all analyses, we used the logarithm of the reported intake of lutein plus zeaxanthin, and serum lutein plus zeaxanthin raised to the power 0.2, to achieve approximately normally distributed variables. These transformations were chosen by observing the shape of the histograms after various power transformations.

We ran models for both the primary nuclear cataract outcome and for the secondary nuclear sclerosis severity outcome. We investigated 2 sets of confounders. First, we included only age, the strongest confounder; second, we added confounders used in the CAREDS analyses (6): smoking (current/past/never), iris color (blue/green/light brown/dark brown or black), body mass index (kg/m²), multivitamin use (≥2 years of use/<2 years of use), physical activity (total metabolic equivalents/week), hormone replacement therapy (current/past/never), and pulse pressure (mm Hg).

To compare the distribution of each variable between cases (nuclear cataract) and controls (no nuclear cataract), we used t tests, Wilcoxon tests, or chi-squared tests according to the type of variable. We estimated the logarithm of odds ratios from logistic regression models, and we calculated the associated Wald z statistic. We computed the odds ratio for the 90th percentile versus the 10th percentile of the distribution of each exposure variable to compare the strength of association with disease across the different measures of exposure. We computed sample size savings resulting from use of one exposure measure compared with another as the ratio of the square of the Wald z statistics for their respective coefficients. For example, suppose that the z statistic for self-reported diet was 2.25 and for the combined measure of diet and biomarker was 3.29. Then, the sample size required for the combined measure is estimated to be (2.25/3.29)² = 0.47 times the sample size required for the self-reported diet. Similarly, relative reductions in the detectable effect size were calculated as the ratio of the Wald z statistics for the 2 models.

RESULTS

A total of 738 participants had a nuclear cataract. Table 1 presents the characteristics of these participants and 1,073 controls without nuclear cataract, as defined by the primary outcome measure. Patients with nuclear cataract had a much higher mean age than controls (73 years vs. 67 years). The groups differed with respect to several other variables including serum lutein plus zeaxanthin levels, physical activity, hormone replacement therapy use, pulse pressure, and multivitamin use (Table 1).

Table 1.

Selected Characteristics of the Participants in the CAREDS Study, United States, 2001–2004, by Primary Nuclear Cataract Outcome^a

Variable	Controls (N = 1,073)		Cases (N = 738)		P Valueb
	Mean (SD)	%	Mean (SD)	%
Serum lutein + zeaxanthin, μmol/Lc	0.33 (0.16)		0.30 (0.14)		0.003
Dietary lutein + zeaxanthin, μg/day	1,848 (1,284)		1,788 (1,226)		0.5
Age, years	67 (6)		73 (6)		<0.001
Body mass index, kg/m2	27.3 (5.8)		27.8 (5.9)		0.07
Smoking status					0.6
Never		58		57
Former		38		40
Current		4		4
Physical activity, total METs/week	15.4 (15.1)		14.1 (14.5)		0.04
Hormone replacement therapy use					<0.001
Never		30		39
Former		12		15
Current		58		46
Iris pigmentation					0.14
Blue		42		41
Green		27		25
Light brown		25		25
Dark brown/black		6		10
Pulse pressure, mm Hg	50 (13)		57 (15)		<0.001
Energy intake, kcal/dayd	1,646 (650)		1,603 (595)		0.15
Multivitamin without minerals use		4		2	0.01

Abbreviations: CAREDS, Carotenoids in Age-Related Eye Disease Study; MET, metabolic equivalent; SD, standard deviation.

^aNuclear sclerosis severity score of ≥4 and/or a history of cataract extraction in either eye.

^bTwo-sided P value for test of difference between cases and controls (t, Wilcoxon, or chi-square test as appropriate to each variable).

^cData available for 1,060 controls and 727 cases.

^d1 kcal = 4.184 kJ.

The results of the logistic regression modeling are shown in Table 2. For each outcome definition and each set of covariates, we compared the results of using as the exposure measure 1) food frequency questionnaire–reported intake of lutein plus zeaxanthin, 2) serum lutein plus zeaxanthin, and 3) Howe's score with ranks.

Table 2.

Odds Ratios and Sample Size Savings Estimated From Logistic Regression Relating Nuclear Cataract Outcomes to Dietary and Serum Lutein and Zeaxanthin in the CAREDS Study, United States, 2001–2004

Outcome (No. of Cases/No. of Noncases)	Covariates	Lutein/Zeaxanthin Measure	ORa	95% CI	Wald z Valueb	Sample Size Ratioc	Detectable Effect Size Reduction, %d
Primarye (727/1,060)	Age	Diet	0.73	0.56, 0.96	2.25f	1.0
		Serum	0.64	0.49, 0.84	3.16f	0.50	29
		Diet + serumg	0.61	0.45, 0.82	3.29f	0.47	32
Primarye (724/1,059)	Fullh	Diet	0.77	0.57, 1.02	1.84	1.0
		Serum	0.69	0.51, 0.94	2.38f	0.59	23
		Diet + serumg	0.66	0.48, 0.91	2.56f	0.52	28
Secondaryi (450/1,111)	Age	Diet	0.74	0.54, 1.00	1.95	1.0
		Serum	0.72	0.53, 0.98	2.10f	0.85	7
		Diet + serumg	0.66	0.47, 0.92	2.45f	0.63	20
Secondaryi (447/1,110)	Fullh	Diet	0.74	0.54, 1.01	1.91	1.0
		Serum	0.78	0.57, 1.09	1.44	1.76	−33
		Diet + serumg	0.70	0.50, 0.99	1.99f	0.92	4

Abbreviations: CAREDS, Carotenoids in Age-Related Eye Disease Study; CI, confidence interval; OR, odds ratio.

^aOdds ratio between the 90th and 10th percentile of the exposure distribution.

^bWald z value of the regression coefficient for the continuous variable.

^cThe ratio of sample sizes required to produce the same statistical power as that based on the diet-only analysis.

^dThe reduction in effect size that can be detected with the same statistical power as in the diet-only analysis given the same sample size.

^eExamination + history.

^fP < 0.05 using a 2-sided Wald test.

^gHowe's method based on ranks.

^hIncluding the covariates age (continuous), smoking (never, past, current), iris color (blue, green, light brown, dark brown/black), body mass index (continuous), multivitamin use (yes, no), physical activity (continuous), pulse pressure (continuous), and hormone replacement therapy use (yes, no).

ⁱExamination only.

For the primary nuclear cataract outcome, with both sets of covariates, the odds ratio for the serum level was stronger than for reported dietary intake. In both cases, the odds ratio for the combination of serum and reported intake was slightly stronger than that for serum alone, and the z values were correspondingly larger. When we compared the analysis using reported intake with the analysis using serum level, adjusted for age or in the full model, we estimated that serum level could achieve the same statistical power with only 50% or 59%, respectively, of the sample size. Use of the combined exposure measure reduced this sample size further to 47% or 52%, respectively. When we compared the analysis using serum level with the one using the combined measure, we estimated that the latter could achieve the same statistical power with 94% (47%/50%) or 88% (52%/59%), respectively, of the sample size.

For the secondary nuclear sclerosis severity outcome, with both sets of covariates, the odds ratios for the serum level and the reported dietary intake were quite similar; in one case, the serum-level odds ratio was slightly stronger (0.72 vs. 0.74) and in one case it was weaker (0.78 vs. 0.74). Correspondingly, whereas in the age-adjusted model the serum level provided an estimated 15% sample savings over the dietary report, in the full model, the serum measure required an estimated 76% larger sample to achieve the statistical power of the analysis with reported intake. In both cases, the odds ratio for the combination of serum and reported intake was slightly stronger than the stronger of the 2 separate exposures, and the z values were also correspondingly larger. In the analysis with the full set of covariates, conventional statistical significance was achieved by using the combined measure (z = 1.99) but not the separate measures (z = 1.91 and z = 1.44, respectively). When we compared the analysis with reported intake with that using the combined measure, we estimated that the latter could achieve the same statistical power with only 63% or 92% of the sample size, respectively. Compared with serum level, the combined measure could achieve the same statistical power with 74% (63%/85%) or 52% (92%/176%) of the sample size, respectively. Table 2 also shows the corresponding reductions in detectable effect sizes under equal sample size.

DISCUSSION

In the example presented, we have demonstrated an association of nuclear cataract risk with both reported dietary lutein/zeaxanthin and serum lutein/zeaxanthin. The association with reported dietary lutein/zeaxanthin is somewhat weaker than that reported in the original publication, with odds ratios between the 90th and 10th percentiles of approximately 0.75 compared with approximately 0.65 reported originally (6). The difference is due to our having modeled the risk according to a continuous intake model rather than in quintiles. Nevertheless, both our analysis and that presented previously (6) clearly demonstrate the association.

We have shown in our analysis that combining a biomarker of dietary intake with self-reported dietary intake can increase the statistical power for detecting a diet-disease association. The gains demonstrated in this example are rather similar to those reported previously in computer simulations (5). Besides using Howe's method for combining measures, we also applied a principal components method, described in detail previously (5). Given that the principal components results were very similar to those using Howe's method, we have reported only the latter here.

The gains obtained from combining a marker with a self-report may be considered in one of two ways. Firstly, we may ask what we gain from introducing a marker into a study. To answer this question, the correct comparison is between the combination of marker and self-report versus self-report alone, or between the marker alone and self-report alone. Here, we have demonstrated substantial efficiencies, equivalent to halving the sample size, although the monetary cost of introducing the marker may be high. Secondly, we may ask what we gain from combining a marker and a self-report when both are already included in the study. In this case, the correct comparison is between the combination and the best of either self-report or marker alone. Here, the power gains we have demonstrated are more modest, sometimes a few percent, sometimes a 20%–30% reduction in sample size. However, the monetary cost of performing the combined analysis is essentially nil because both measurements are already available.

There are several limitations to our suggested approach. First, the measure of lutein in either the diet or serum might, to some extent, reflect other aspects of diet that, in addition to the effects of lutein itself, might also lower the risk of nuclear cataract. These, rather than improved estimation of lutein status, may explain stronger associations with cataract. For example, women who have high, compared with low, estimates of dietary lutein are likely to also have lower intakes of fat and higher intakes (and blood levels) of many other micronutrients that may lower risk of cataract (6). Thus, the stronger associations with cataract may reflect broader aspects of diet that are captured by a measure of lutein in the serum rather than less error in measuring lutein status. On the other hand, the gains in power demonstrated here are rather close to those predicted from computer simulations of a simple model in which all gains were due to reductions in measurement error (5).

Second, Howe's score has no recognized units, being the sum of 2 ranks. Nevertheless, as shown in Table 2, odds ratios for the 90th versus the 10th percentile may be estimated as a useful quantity to express the strength of the association with disease. This measure is really no different from the conventional odds ratio between highest and lowest quintiles often used in epidemiologic research reports. The combined approach proposed here could be used as an efficient means of establishing the existence of a nutrition-disease association. Subsequent analyses could explore the associations between dietary intake, biomarker level, and disease risk in more depth, including, for example, whether the dietary effect is mediated by the biomarker.

Use of the combined exposure measure will not always increase statistical power over that obtained by using one of the separate exposures. For example, if the reported intake were to demonstrate no association (estimated odds ratio = 1) with disease while the serum level were found to be associated strongly (estimated odds ratio much greater than 1), then the combined exposure measure would likely have an estimated odds ratio intermediate between those for the separate exposure measures and would have less power than that for the serum-level analysis. The most successful results from the combined exposure measure will arise when the associations for each separate exposure are of similar strength, as occurs in the example presented here.

The markers that should be considered for combination with dietary reports are those that have been shown in controlled feeding studies to be modified by changes in the relevant dietary intake. Then, if changes in the marker causally affect disease risk, it implies that dietary intake will also affect disease risk, thus justifying the combination of the 2 measures. Clearly, situations will arise in which no suitable marker exists for the nutrient or food under study.

It is not essential that there be a high correlation between biomarker and reported dietary intake. More important is the biomarker's correlation with true intake. To be helpful in combination, this correlation should be similar to, or higher than, the correlation between reported intake and true intake (5). Information about these correlations may be available from controlled feeding studies.

It would also be helpful if the biomarker were known not to be affected by other risk factors for the disease. If other risk factors were to affect the biomarker, then the association between biomarker and disease would be at least partly a result of confounding. In the worst case, modifying the biomarker level through diet change might not affect disease. This problem of confounding has been an important consideration in previous studies of biomarkers and disease. If a strong risk factor for the disease is known to affect the marker, that risk factor must be included in the disease risk model to avoid ascribing its effect to nutritional causes. In our study, smoking was included in the full model since it is associated with nuclear cataract and may also lead to depletion of lutein and zeaxanthin in blood, being a source of free radicals and oxidative stress (12).

Including biomarker measurements for all participants in a large study is a considerable challenge and can be extremely expensive. However, collecting biologic samples from participants is no longer uncommon, and their uses are manifold. Thus, the proposed approach will be feasible for studies with an already established “biobank.” Note that, for large prospective studies, the analytic cost of the bioassays need not be prohibitive if analyses are based on a nested case-control design. In view of the potential reductions of approximately 50% in sample size, it seems worth considering, in the design stage of a cohort study, the budgets required for a study with and without biomarkers relevant to the exposures of greatest interest.

The extent to which a dietary intake effect is mediated by the biomarker is often unknown. Methods that perform well under different disease risk models are therefore to be encouraged. We found in previous simulation work (5) that Howe's method seemed to do this. In the scenarios examined, it was uniformly superior to univariate dietary intake analysis. It was also superior to univariate biomarker analysis under the no-mediation model and was not substantially inferior to that analysis under full mediation. Thus, when the extent to which the biomarker mediates the dietary effect is unknown, a combination approach would appear to be a good strategy.

In summary, we have provided an example of how combining a biomarker of dietary intake with self-reported dietary intake somewhat increases the statistical power to detect a diet-disease association. The gains in statistical power are fairly modest if compared with the best of the 2 separate exposure measures but are potentially useful in a research area in which measurement error severely limits our ability to elucidate such associations.

Glossary

Abbreviations

CAREDS

Carotenoids in Age-Related Eye Disease Study

WHI

Women's Health Initiative