Abstract
Objective
To assess the relative validity and reproducibility of the quantitative FFQ used in the Tzu Chi Health Study (TCHS).
Design
The reproducibility was evaluated by comparing the baseline FFQ with the 2-year follow-up FFQ. The validity was evaluated by comparing the baseline FFQ with 3 d dietary records and biomarkers (serum folate and vitamin B12). Median comparison, cross-classification and Spearman correlation with and without energy adjustment and deattenuation for day-to-day variation were assessed.
Setting
TCHS is a prospective cohort containing a high proportion of true vegetarians and part-time vegetarians (regularly consuming a vegetarian diet without completely avoiding meat).
Subject
Subsets of 103, seventy-eight and 1528 TCHS participants were included in the reproducibility, dietary record-validity and biomarker-validity studies, respectively.
Results
Correlations assessing the reproducibility for repeat administrations of the FFQ were in the range of 0·46–0·65 for macronutrients and 0·35–0·67 for micronutrients; the average same quartile agreement was 40%. The correlation between FFQ and biomarkers was 0·41 for both vitamin B12 and folate. Moderate to good correlations between the baseline FFQ and dietary records were found for energy, protein, carbohydrate, saturated and monounsaturated fat, fibre, vitamin C, vitamin A, K, Ca, Mg, P, Fe and Zn (average crude correlation: 0·47 (range: 0·37–0·66); average energy-adjusted correlation: 0·43 (range: 0·38–0·55); average energy-adjusted deattenuated correlation: 0·50 (range: 0·44–0·66)) with same quartile agreement rate of 39% (range: 35–45%), while misclassification to the extreme quartile was rare (average: 4% (range: 0–6%)).
Conclusions
The FFQ is a reliable and valid tool to rank relative intake of major nutrients for TCHS participants.
Keywords: FFQ, Tzu Chi Health Study, FFQ validation, Reliability, Biomarker
To investigate how diet impacts health in epidemiological studies requires assessment tools that are capable of estimating long-term dietary intake, yet inexpensive and simple enough to administer to a large population( 1 ). Although the FFQ has these advantages, the cognitive challenges in filling out the FFQ, the potential omission of important food items and the reliance on generic memory may introduce error, limiting its ability to assess dietary intake( 2 , 3 ). Therefore, the validity and reproducibility of an FFQ need to be confirmed in the population of its intended use, as the quantitative understanding of the tool's performance and limitations will be important in the interpretation of the future diet–health analysis( 4 ).
A vegetarian diet has been shown to protect against heart diseases( 5 ), diabetes( 6 ) and other chronic degenerative diseases( 7 ) in the West. However, relatively little is known about vegetarians in Asia who may consume a different diet from vegetarians in the West and from the non-vegetarian general population in Asia. To study how vegetarian diet affects health, we need a tool to assess the nutritional intake of such a dietary pattern. While several Taiwanese FFQ have been developed and validated( 8 – 10 ), none has been designed with the vegetarian diet in mind. The current FFQ was modified from the FFQ used in the Nutrition and Health Survey in Taiwan (NAHSIT)( 8 ) to include more commonly consumed vegetarian foods. The present study aimed to examine the relative validity and reproducibility of the modified FFQ in a new cohort study with a high proportion of true vegetarians and part-time vegetarians in Taiwan.
Methods
The Tzu Chi Health Study
The Tzu Chi Health Study (TCHS) is the first cohort to look into diet and health outcomes in vegetarians in Taiwan and in Asia. It enrolled 6002 adults that came to the Buddhist Dalin Tzu Chi Hospital for a health examination from 2007 to 2009. The recruitment focused on Tzu Chi commissioners, a devoted group of volunteers to the Buddhist Tzu Chi Foundation who receive a free health examination at Tzu Chi hospitals every 2–3 years. Approximately one-third of participants were vegetarians, and 53% of omnivores may be considered part-time vegetarians as they reported regular consumption of a vegetarian diet without completely avoiding meat and fish. The participants were interviewed on lifestyle, medical history and FFQ. The institutional review board at the Buddhist Dalin Tzu Chi Hospital approved the cohort study and the validation sub-studies.
The FFQ
The TCHS FFQ was modified from the one used in NAHSIT( 8 ). The original FFQ contains only twenty-eight items on top of an open-ended section on staple foods (such as rice, noodles and bread). We modified the FFQ in several ways:
-
1.
Multiple foods grouped into single items in the original FFQ were expanded into separate items.
-
2.
More commonly consumed vegetarian products (such as meat analogues, gluten products and fermented soya products) were added in consultation with dietitians experienced with vegetarian dietary habits.
-
3.
The open-ended staple food section was modified into individual items in the FFQ.
-
4.
A portion size section was added for each item in the FFQ.
The modified FFQ contained sixty-four food items or food groups, and separate sections on cooking methods, dietary habits and the use of condiments and dietary supplements. The FFQ presented items by major food groups in the following order: fish, seafood and meat; eggs and dairy; soya and other vegetarian protein products; vegetables; fruits and nuts; staples (i.e. rice, noodles, bread, oats); beverages; and cooking oils. Each item in the FFQ had on average five accompanying photographs (the photographs were published by the Department of Health in Taiwan for portion-size education( 11 )). The research dietitians asked the participants to quantify the frequency (in terms of per day, per week, per month, or never) they consumed a food or food group, and then to quantify the amount of the food consumed compared with the photographs (i.e. similar portion, half or twice the amount). Prior to administering the FFQ, the dietitians asked questions to determine participants’ vegetarian status (vegetarian defined by not eating any meat, fish and animal flesh) and then skipped non-relevant sections to reduce participant burden.
Reproducibility study
The reproducibility of the FFQ was evaluated by comparing the baseline FFQ (FFQ1) with a second administration of the FFQ (FFQ2). Between March 2009 and February 2010, a total of 103 previously enrolled TCHS participants returned to Dalin Tzu Chi Hospital for a health examination and all of them agreed to be interviewed on the FFQ the second time. Nutrient intakes were calculated for both FFQ and compared to evaluate reproducibility. The mean time interval from FFQ1 to FFQ2 was 23·2 (sd 3·6) months.
Diet record-validity study
The relative validity of the FFQ was assessed by comparison with 3 d of diet records (DR). From February to June of 2011, 101 TCHS participants (not overlapping with the reproducibility study participants) returning to Dalin Tzu Chi Hospital for the second health examination were invited to participate in the DR-validity study. Of these, seventy-eight (77%) agreed to participate and completed the instructed dietary records. The research dietitian instructed the participants on keeping 3 d of DR, including two weekdays and one weekend day over a 2-week period. The participants were taught to record in a structured booklet the details of everything they consumed, including the time and location, the brands for commercial items, the names of the food or the dishes, the ingredients, the food preparation method and the portion size, with reference to standard household measurements such as bowls, cups, spoons and reference objects. A few participants who did not have the ability to record diet by themselves received assistance from their daughters-in-law who lived with the participants and prepared their meals. The research dietitian made follow-up telephone calls to clarify any recording that appeared ambiguous or lacking the required detail description. Nutrient intakes as assessed by the FFQ were compared with those obtained by the DR to evaluate for relative validity. The mean time interval from FFQ1 to DR was 30 (sd 9; range: 21–54) months.
Biomarker-validity study
The FFQ's relative validity for folate and vitamin B12 was further assessed by comparing nutrient intakes calculated from the FFQ with the levels of serum folate and vitamin B12 in a subgroup of 1528 participants. These participants were selected at a ratio of two omnivores to one vegetarian among those enrolled during the early phase of the cohort (October 2007 to June 2008). All participants completed the FFQ at enrolment and had fasting blood drawn the next morning. Concentrations of serum folate and vitamin B12 were measured using a commercial RIA kit (Siemens, Malvern, PA, USA) at the Radioimmunoassay Laboratory, Department of Nuclear Medicine of the Chiu Clinic in Taiwan. As the blood for these biomarkers was drawn one day after the baseline FFQ, there is no time lag between these two measurements.
Computation of nutrients
Nutrient analysis programs were developed to calculate intakes from the FFQ and the DR based on values in Taiwan's Food Composition Database( 12 ), with the addition of folate compiled by experts in Taiwan( 13 ). When food items could not be found within the database, nutrient values from other sources( 14 – 16 ) were adopted. For FFQ items that represent food groups, we computed the average food composition of the relevant foods, weighted by the frequency of consumption based on the 2005–2008 NAHSIT.
Statistical analysis
Demographics of participants in the reproducibility/validity studies were compared with those of the entire TCHS cohort using the independent-samples t test (continuous variables) and the χ
2 test (categorical variables). Reproducibility and validity were assessed by the following methods. First, median nutrient intakes were compared by the Wilcoxon signed-rank test. Second, Spearman correlations were calculated for both crude and energy-adjusted nutrient intakes; we adjusted for energy by (i) the residual model(
17
) for normally distributed nutrients (loge transformation applied when necessary to improve normality) and (ii) the nutrient density model (divided by energy intake) for nutrients that violated normality (defined as P value <0·05 on a one-sample Kolmogorov–Smirnov test) and could not be normalized with loge transformation. For the DR-validity study, as the 3 d of DR were not sufficient to capture the day-to-day variation in diet, we deattenuated for the within-person variation by multiplying the correlation coefficient by the factor
, where n is the average number of repeated DR,
is the within-person variance and
is the between-person variance calculated from the DR(
18
). The deattenuation was carried out only for nutrients with normal distribution after energy adjustment. Third, cross-classification was used to evaluate the ability of the FFQ to rank order participants by quartiles of dietary intake. For the biomarker-validity study, we computed Spearman correlation (with and without energy adjustment by the residual model) and ‘actual values for surrogate categories’(
19
), in which the average of actual values of biomarkers for each quartile of FFQ intakes were computed. All statistical analyses were performed using the statistical software package IBM SPSS Statistics Version 17.
Results
The 103 participants in the reproducibility study represented the total TCHS cohort study well in terms of demographics, dietary patterns and BMI. The seventy-eight DR-validity study participants, in comparison, were slightly younger, had a higher proportion of females and of those with a middle/high-school education, while the dietary pattern and BMI did not differ significantly (Table 1). The 1528 biomarker-validity study participants were 0·1 years younger, with 3·8% more vegetarian compared with the cohort, and no significant difference seen for other variables.
Table 1.
Comparison of baseline characteristics of participants in the reproducibility, diet record (DR)-validity and biomarker-validity studies with those of the cohort study
| Reproducibility study (n 103) | DR-validity study (n 78) | Biomarker-validity study (n 1528) | Tzu Chi Health Study (n 6002) | |||||
|---|---|---|---|---|---|---|---|---|
| Mean or % | sd | Mean or % | sd | Mean or % | sd | Mean or % | sd | |
| Age (years) | 54 | 11 | 51* | 5 | 53* | 11 | 53 | 10 |
| BMI (kg/m2) | 23·9 | 3·1 | 23·5 | 3·1 | 23·5 | 3·3 | 23·6 | 3·3 |
| Female gender (%) | 68·0 | – | 73·1* | – | 59·6 | – | 59·6 | – |
| Diet patterns (%) | ||||||||
| Omnivores | 65·0 | – | 61·5 | – | 66·6* | – | 70·4 | – |
| Vegetarians | 35·0 | – | 38·5 | – | 33·4 | – | 29·6 | – |
| Education (%) | ||||||||
| Elementary or less | 25·2 | – | 15·4* | – | 26·1 | – | 24·6 | – |
| Middle or high school | 48·5 | – | 64·1 | – | 48·2 | – | 48·5 | – |
| College or above | 26·2 | – | 20·5 | – | 25·7 | – | 26·9 | – |
*P value < 0·05 for comparison with Tzu Chi Health Study participants.
For reproducibility (Table 2), FFQ1 tended to show slightly higher intake than FFQ2 for most nutrients but the differences were small (<20% for most nutrients) and statistically insignificant, except for folate and vitamin B12. The energy-adjusted correlations for macronutrients ranged from 0·46 (protein) to 0·65 (fibre), and for micronutrients from 0·35 (riboflavin) to 0·67 (Fe). Inclusion of supplements decreased correlations for most vitamins and minerals but increased correlations for thiamin and riboflavin. The cross-classification showed that (on average) 40% and 81% of participants were classified into the same quartile and the same-and-adjacent quartiles, respectively. Energy adjustment improved the agreement slightly (data not shown). Misclassification into extreme quartiles was about 3–4% for both crude and energy-adjusted nutrients.
Table 2.
Daily intake estimates from, correlation between and cross-classification of the two FFQ in the reproducibility study
| Intake estimates | Spearman correlation | Cross-classification (%) | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| FFQ1 | FFQ2 | Percentage difference | Energy | Same | Adjacent | Extreme | ||||
| Dietary variable | Median | P25, P75 | Median | P25, P75 | in median (%)† | Crude | adjusted | quartile | quartiles | quartiles |
| Energy (kJ) | 7184 | 5335, 9816 | 7184 | 5519, 8727 | 0·0 | 0·49 | – | 44 | 37 | 4 |
| Protein (g) | 54 | 42, 71 | 52 | 40, 66 | 4·4 | 0·46 | 0·46 | 32 | 46 | 5 |
| Total fat (g) | 44 | 30, 70 | 43 | 30, 59 | 2·5 | 0·43 | 0·60 | 34 | 48 | 4 |
| SFA (g) | 11 | 6, 15 | 9 | 6, 12 | 22·2 | 0·41 | 0·56 | 39 | 37 | 4 |
| MUFA (g) | 15 | 9, 21 | 13 | 8, 18 | 15·4* | 0·50 | 0·52 | 44 | 38 | 3 |
| PUFA (g) | 10 | 6, 19 | 10 | 7, 15 | 0·0 | 0·48 | 0·56 | 32 | 51 | 5 |
| Carbohydrate (g) | 265 | 191, 353 | 268 | 200, 319 | −0·8 | 0·56 | 0·61 | 38 | 42 | 3 |
| Dietary fibre (g) | 22 | 16, 29 | 21 | 15, 27 | 7·0 | 0·59 | 0·65 | 42 | 43 | 2 |
| Cholesterol (mg) | 121 | 73, 195 | 119 | 77, 165 | 1·6 | 0·57 | 0·59 | 45 | 40 | 2 |
| K (mg) | 2410 | 1820, 2877 | 2063 | 1652, 2053 | 16·8 | 0·55 | 0·62 | 32 | 50 | 3 |
| K with supplements (mg) | 2433 | 1856, 2943 | 2108 | 1694, 2108 | 15·4* | 0·55 | 0·61 | 34 | 49 | 3 |
| Ca (mg) | 494 | 389, 711 | 466 | 346, 694 | 6·0 | 0·45 | 0·52 | 35 | 44 | 3 |
| Ca with supplements (mg) | 569 | 426, 857 | 602 | 380, 838 | −5·5 | 0·47 | 0·47 | 47 | 32 | 5 |
| Mg (mg) | 271 | 198, 355 | 252 | 187, 364 | 7·5 | 0·60 | 0·65 | 44 | 41 | 2 |
| Mg with supplements (mg) | 316 | 236, 412 | 273 | 207, 380 | 15·7 | 0·56 | 0·61 | 40 | 46 | 1 |
| P (mg) | 862 | 752, 1084 | 768 | 650, 1052 | 12·2 | 0·49 | 0·48 | 35 | 46 | 1 |
| Fe (mg) | 11·3 | 8·8, 14·1 | 10·5 | 8·2, 14·1 | 7·6 | 0·60 | 0·67 | 42 | 43 | 1 |
| Fe with supplements (mg) | 12·9 | 10·3, 21·4 | 11·4 | 8·6, 17·6 | 12·8 | 0·53 | 0·55 | 46 | 38 | 3 |
| Zn (mg) | 8·2 | 6·5, 10·8 | 7·8 | 6·2, 9·8 | 5·1 | 0·59 | 0·57 | 45 | 35 | 0 |
| Zn with supplements (mg) | 10·2 | 7·3, 15·3 | 8·4 | 6·5, 12·7 | 21·6 | 0·49 | 0·44 | 42 | 41 | 6 |
| Thiamin (mg) | 1·0 | 0·9, 1·7 | 0·9 | 0·6, 1·3 | 11·1* | 0·47 | 0·40 | 38 | 47 | 4 |
| Thiamin with supplements (mg) | 1·5 | 0·9, 3·0 | 1·2 | 0·7, 2·3 | 19·4 | 0·50 | 0·52 | 41 | 41 | 4 |
| Riboflavin (mg) | 0·9 | 0·7, 1·1 | 0·9 | 0·7, 1·1 | 0·0 | 0·31 | 0·35 | 32 | 41 | 6 |
| Riboflavin with supplements (mg) | 1·1 | 0·9, 2·3 | 1·1 | 0·8, 21·9 | −8·1 | 0·33 | 0·44‡ | 43 | 38 | 9 |
| Niacin (mg) | 17·5 | 12·6, 22·1 | 15·7 | 11·4, 21·9 | 11·5 | 0·49 | 0·45 | 42 | 38 | 3 |
| Niacin with supplements (mg) | 20·2 | 14·2, 37·8 | 19·3 | 13·8, 28·9 | 4·6 | 0·45 | 0·47 | 39 | 41 | 4 |
| Vitamin B6 (mg) | 1·1 | 0·8, 1·4 | 1·0 | 0·8, 1·3 | 10·0 | 0·51 | 0·52 | 30 | 53 | 2 |
| Vitamin B6 with supplements (mg) | 1·4 | 1·0, 2·7 | 1·3 | 1·0, 2·4 | 10·5* | 0·42 | 0·46‡ | 39 | 43 | 9 |
| Folate (μg) | 407 | 274, 582 | 336 | 254, 515 | 21·1* | 0·60 | 0·61 | 49 | 36 | 3 |
| Folate with supplements (μg) | 456 | 313, 747 | 426 | 283, 642 | 7·1* | 0·52 | 0·57 | 41 | 40 | 3 |
| Vitamin B12 (μg) | 1·6 | 0·9, 3·8 | 1·2 | 0·7, 3·2 | 33·3* | 0·59 | 0·55 | 48 | 37 | 2 |
| Vitamin B12 with supplements (μg) | 3·9 | 1·2, 8·0 | 3·5 | 1·1, 6·0 | 11·2 | 0·40 | 0·34 | 43 | 34 | 5 |
| Vitamin C (mg) | 166 | 111, 223 | 139 | 103, 200 | 19·4* | 0·55 | 0·53 | 37 | 51 | 3 |
| Vitamin C with supplements (mg) | 173 | 138, 242 | 152 | 114, 242 | 14·1 | 0·46 | 0·43 | 41 | 36 | 5 |
| Vitamin A (μg RE) | 2172 | 1528, 3862 | 2063 | 1469, 3414 | 5·3 | 0·52 | 0·49 | 39 | 41 | 1 |
| Vitamin A with supplements (μg RE) | 2235 | 1529, 3856 | 2076 | 1480, 3422 | 7·6 | 0·52 | 0·48 | 39 | 41 | 1 |
| Vitamin E (mg α-TE) | 4·5 | 2·7, 5·8 | 3·9 | 2·8, 5·7 | 15·4 | 0·46 | 0·50 | 40 | 41 | 4 |
| Vitamin E with supplements (mg α-TE) | 6·1 | 3·9, 12·7 | 5·3 | 3·3, 9·2 | 13·9 | 0·36 | 0·37‡ | 42 | 35 | 6 |
P25, 25th percentile; P75, 75th pecentile; RE, retinol equivalents; α-TE, α-tocopherol equivalents.
*P value < 0·05 in Wilcoxon signed-rank test.
†(FFQ1 – FFQ2)/FFQ2.
‡Energy adjustment by the density model due to non-normal distribution.
For the DR-validity study (Table 3), DR and FFQ showed comparable median intakes for most nutrients, except cholesterol, niacin, vitamin B6, vitamin B12 and vitamin E. Correlations were low for polyunsaturated fat, vitamin E and B vitamins, but reasonable for most macronutrients, minerals, vitamin C and vitamin A. Inclusion of supplemental sources of vitamins and minerals resulted in an increased correlation for Fe, decreased correlations for Zn, folate, vitamin B12 and vitamin E, and similar correlations for other nutrients. The overall mean correlation was similar whether supplements were included or not. Since only two participants recorded use of vitamin/mineral supplements in the DR, the median intakes with and without supplements were almost identical. In cross-classification, on average, 36% and 76% of participants were placed in the same quartile and same-and-adjacent quartiles, respectively. Adjustment for energy resulted in similar agreement rates: 35% and 76% for the same quartile and same-and-adjacent quartiles, respectively (data not shown). Misclassification into extreme quartiles was about 5% for both crude and energy-adjusted intakes.
Table 3.
Daily intake estimates from, correlation between and cross-classification of the FFQ and diet records (DR) in the DR-validity study
| Intake estimates | Spearman correlation | Cross-classification (%) | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| FFQ | DR | Percentage difference | Energy | Energy adjusted, | Same | Adjacent | Extreme | ||||
| Dietary variable | Median | P25, P75 | Median | P25, P75 | in median (%)† | Crude | adjusted | deattenuated‡ | quartile | quartiles | quartiles |
| Energy (kJ) | 7117 | 5238, 9602 | 6883 | 5653, 8358 | 3·4 | 0·58 | – | – | 45 | 41 | 3 |
| Protein (g) | 51 | 42, 70 | 54 | 45, 62 | −5·8 | 0·50 | 0·41 | 0·49 | 42 | 38 | 5 |
| Total fat (g) | 48 | 32, 66 | 52 | 42, 63 | −8·9 | 0·30 | 0·38 | 0·44 | 36 | 40 | 9 |
| SFA (g) | 9 | 6, 13 | 10 | 8, 14 | −5·0 | 0·42 | 0·41 | 0·46 | 37 | 40 | 4 |
| MUFA (g) | 15 | 9, 22 | 18 | 13, 24 | −15·7* | 0·41 | 0·41 | 0·44 | 38 | 40 | 6 |
| PUFA (g) | 11 | 7, 16 | 14 | 11, 21 | −17·2* | 0·17 | 0·13 | 0·15 | 28 | 40 | 9 |
| Carbohydrate (g) | 269 | 191, 359 | 249 | 196, 304 | 7·6* | 0·66 | 0·38 | 0·44 | 40 | 45 | 0 |
| Dietary fibre (g) | 19 | 15, 29 | 24 | 18, 29 | −19·4 | 0·47 | 0·50 | 0·55 | 38 | 40 | 6 |
| Cholesterol (mg) | 119 | 50, 171 | 167 | 59, 269 | −28·6* | 0·37 | 0·31 | 0·36 | 37 | 37 | 4 |
| K (mg) | 2161 | 1630, 2979 | 2316 | 1797, 2891 | −6·7 | 0·46 | 0·44 | 0·49 | 38 | 40 | 4 |
| K with supplements (mg) | 2190 | 1657, 3029 | 2316 | 1797, 2891 | −5·5 | 0·46 | 0·46 | 0·51 | 38 | 40 | 4 |
| Ca (mg) | 464 | 337, 666 | 495 | 388, 596 | −6·3 | 0·40 | 0·43 | 0·49 | 29 | 47 | 4 |
| Ca with supplements (mg) | 523 | 359, 851 | 495 | 388, 596 | 5·6* | 0·40 | 0·39 | 0·44 | 37 | 36 | 3 |
| Mg (mg) | 248 | 183, 382 | 275 | 221, 342 | −10·0 | 0·46 | 0·50 | 0·54 | 41 | 36 | 1 |
| Mg with supplements (mg) | 271 | 193, 394 | 275 | 221, 342 | −1·6 | 0·46 | 0·48 | 0·52 | 41 | 40 | 3 |
| P (mg) | 805 | 608, 1084 | 902 | 754, 1030 | −10·8 | 0·43 | 0·46 | 0·53 | 35 | 42 | 4 |
| Fe (mg) | 10·7 | 8·3, 15·3 | 13·2 | 10·7, 18·3 | −18·7 | 0·37 | 0·38 | 0·43 | 45 | 32 | 4 |
| Fe with supplements (mg) | 11·7 | 8·8, 19·1 | 13·2 | 10·7, 18·3 | −11·3 | 0·40 | 0·42 | 0·47 | 49 | 29 | 6 |
| Zn (mg) | 8·1 | 6·0, 10·7 | 7·9 | 6·2, 9·8 | 2·9 | 0·50 | 0·38 | 0·46 | 42 | 31 | 3 |
| Zn with supplements (mg) | 9·0 | 6·5, 12·7 | 7·9 | 6·2, 9·8 | 15·8* | 0·41 | 0·25§ | – | 44 | 33 | 5 |
| Thiamin (mg) | 1·0 | 0·6, 1·6 | 1·1 | 0·7, 2·8 | −6·2* | 0·32 | 0·17 | 0·21 | 31 | 42 | 5 |
| Thiamin with supplements (mg) | 1·2 | 0·66, 2·2 | 1·1 | 0·7, 2·8 | 12·0 | 0·30 | 0·20§ | – | 29 | 45 | 5 |
| Riboflavin (mg) | 0·9 | 0·6, 1·2 | 1·0 | 0·7, 1·4 | −10·7* | 0·31 | 0·36 | 0·41 | 28 | 44 | 3 |
| Riboflavin with supplements (mg) | 1·0 | 0·7, 1·6 | 1·0 | 0·7, 1·4 | −3·9 | 0·27 | 0·35§ | – | 22 | 49 | 5 |
| Niacin (mg) | 16·3 | 11·7, 22·8 | 13·4 | 9·3, 19·0 | 21·4* | 0·24 | 0·24 | 0·26 | 33 | 35 | 6 |
| Niacin with supplements (mg) | 20·2 | 12·1, 28·3 | 13·4 | 9·3, 19·0 | 50·4* | 0·22 | 0·21 | 0·24 | 37 | 29 | 6 |
| Vitamin B6 (mg) | 1·0 | 0·8, 1·3 | 1·0 | 0·9, 1·2 | 4·5 | 0·32 | 0·27 | 0·33 | 41 | 36 | 8 |
| Vitamin B6 with supplements (mg) | 1·2 | 0·9, 1·7 | 1·0 | 0·8, 1·2 | 25·9* | 0·29 | 0·29§ | – | 36 | 37 | 4 |
| Folate (μg) | 344 | 257, 575 | 363 | 273, 476 | −5·2 | 0·29 | 0·28 | 0·32 | 29 | 42 | 4 |
| Folate with supplements (μg) | 395 | 275, 718 | 364 | 274, 476 | 8·8 | 0·24 | 0·23 | 0·27 | 26 | 40 | 4 |
| Vitamin B12 (μg) | 1·3 | 0·7, 2·2 | 1·1 | 0·5, 1·7 | 21·8 | 0·31 | 0·25§ | – | 32 | 42 | 9 |
| Vitamin B12 with supplements (μg) | 1·9 | 1·0, 3·5 | 1·1 | 0·6, 1·7 | 79·6* | 0·22 | 0·18§ | – | 28 | 41 | 8 |
| Vitamin C (mg) | 163 | 105, 213 | 181 | 134, 245 | −9·9 | 0·53 | 0·55 | 0·66 | 41 | 40 | 1 |
| Vitamin C with supplements (mg) | 171 | 107, 227 | 181 | 134, 245 | −5·7 | 0·53 | 0·55 | 0·66 | 37 | 42 | 1 |
| Vitamin A (μg RE) | 2285 | 1191, 3780 | 2135 | 1171, 4408 | 7·0 | 0·35 | 0·40 | 0·46 | 35 | 40 | 6 |
| Vitamin A with supplements (μg RE) | 2318 | 1197, 3785 | 2135 | 1171, 4408 | 8·6 | 0·35 | 0·41 | 0·47 | 35 | 42 | 6 |
| Vitamin E (mg α-TE) | 4·3 | 2·8, 6·1 | 5·6 | 4·0, 8·0 | −23·5* | 0·29 | 0·23 | 0·25 | 23 | 49 | 5 |
| Vitamin E with supplements (mg α-TE) | 5·2 | 3·3, 9·7 | 5·7 | 4·0, 8·0 | −9·4 | 0·27 | 0·16§ | – | 29 | 40 | 6 |
P25, 25th percentile; P75, 75th pecentile; RE, retinol equivalents; α-TE, α-tocopherol equivalents.
*P value < 0·05 in Wilcoxon signed-rank test.
†(FFQ – DR)/DR.
‡Deattenuation factor was calculated from energy-adjusted nutrients from DR.
§Energy adjustment by the density model due to non-normal distribution.
Table 4 shows the FFQ's relative validity to assess folate and vitamin B12 with biomarkers as the reference method. The correlation for FFQ folate and serum folate improved substantially from 0·29 to 0·41 after energy adjustment. Serum folate levels increased by 51·5% and 70·7% across the quartiles for crude and energy-adjusted FFQ folate intake, respectively. For vitamin B12, energy adjustment changed the correlation slightly from 0·40 to 0·41. Serum vitamin B12 levels increased by 71·6% and 86·6% across the quartiles for crude and energy-adjusted FFQ vitamin B12 intake, respectively.
Table 4.
Spearman correlation between FFQ intake and biomarkers, and mean values of biomarkers by quartile of FFQ intake, in the biomarker-validity study
| n | Correlation* | Quartile 1 | Quartile 2 | Quartile 3 | Quartile 4 | |
|---|---|---|---|---|---|---|
| Comparison with crude intake | ||||||
| Serum folate (ng/ml) | 1528 | 0·29 | 11·26 | 12·24 | 14·56 | 17·06 |
| Serum vitamin B12 (pg/ml) | 1524 | 0·40 | 371·2 | 489·0 | 534·5 | 636·9 |
| Comparison with energy-adjusted intake | ||||||
| Serum folate (ng/ml) | 1528 | 0·41 | 10·61 | 11·99 | 14·40 | 18·11 |
| Serum vitamin B12 (pg/ml) | 1524 | 0·41 | 346·2 | 498·7 | 541·3 | 645·9 |
*P < 0·001 for all correlations.
Discussion
The reproducibility of the FFQ is reasonable despite being administered two years apart. Our results are comparable to those observed in the Shanghai Women's Health Study (SWHS)( 20 ), which had crude correlations ranging from 0·48 to 0·51 for macronutrients and from 0·33 to 0·39 for vitamins; and the Shanghai Men's Health Study (SMHS)( 21 ), in which crude correlations were in the range of 0·39–0·53 for macronutrients and 0·38–0·52 for vitamins. Our correlations for most nutrients are lower than those reported in a Taiwanese meal-based FFQ by Lyu et al.( 10 ), which had the two FFQ administered at a much shorter interval (7·5 months). Due to the longer time interval between our two FFQ administrations (2 years), our reported data may be affected by both the actual reproducibility of the questionnaire and the potential change in diet over time. The Dutch arm of the European Prospective Investigation into Cancer and Nutrition (EPIC) also observed a trend of decreasing reproducibility with increasing time lag between the two FFQ for all nutrients and food groups( 22 , 23 ).
The present FFQ has reasonable validity for energy and most macronutrients, except polyunsaturated fat. Our validity for energy, protein, carbohydrates and fibre (correlations: 0·47–0·66, same quartile agreement: 38–45%) are slightly lower than those in the SWHS (correlations: 0·52–0·66, same quartile agreement: 38–46%) but comparable to the SMHS (correlations: 0·48–0·58, same quartile agreement: 35–47%) and Taiwanese FFQ by Lee (correlations: 0·20–0·46)( 9 ) and Lyu (correlations: 0·36–0·67)( 10 ). The low correlation of polyunsaturated fat likely results from challenges in assessing this nutrient. The main predictors for unsaturated fat in the Taiwanese population include several cooking oils such as soyabean oil and peanut oil( 24 ), which could vary considerably in the amount used in typical stir-fried dishes and would be difficult to estimate for those not involved in cooking. In addition, the surface area of the food (cutting into smaller pieces would increase the surface area and result in a coating of more oil) and the amount of juice in a dish eaten may also affect the amount of oil consumed( 24 ). These details are difficult to collect in self-reported data such as FFQ and DR, and thus require considerable assumptions from researchers. Other researchers have used fatty acid composition in adipose tissue as the reference method( 25 ), which may be useful when validating more specific types of fatty acids.
Among micronutrients, our FFQ is expected to provide good relative ranking for most minerals, vitamin C and vitamin A (similar or better correlation and agreement compared with other Chinese FFQ( 9 , 10 , 20 , 21 )), but we had low correlations for B vitamins and vitamin E. The low correlations for B vitamins may have resulted from higher within-person to between-person variation, as found in another Taiwanese study( 9 ). This suggests that more repeats of DR throughout a longer period may be needed to minimize random errors in the reference method. Moreover, the predicting foods for B vitamins may be different for vegetarians. For example, pork is a strong predicting food for thiamin in the Taiwanese population( 26 ), but is not eaten by vegetarians. Further studies to find better predicting foods for vegetarians are needed to improve the FFQ's ability to assess B vitamins. The low correlation for vitamin E may have been caused by missing values in the nutrient database.
Biomarker validation usually shows lower correlation and is considered to be the lower bound of questionnaire validity due to independence in sources of errors( 27 ). Our correlations for folate (crude: 0·29, energy-adjusted: 0·41) are moderate, lower than in an American( 27 ) and a Canadian( 28 ) study, and higher than in two Japanese( 29 , 30 ), two Dutch( 31 , 32 ) and one Jamaican( 33 ) studies. This may be related to bioavailability, as the North American populations may have consumed more bioavailable folic acid from fortified food and supplements that is more readily reflected in serum levels. Our correlations for vitamin B12 (crude: 0·40, energy-adjusted: 0·41) are substantially higher than those reported by others( 27 , 28 , 30 , 32 ). This may have resulted from both good validity and the wide range of nutrient intake and nutrient status due to the high proportion of vegetarians. Similar to other biomarker validation studies( 27 , 34 ), we found energy-adjusted nutrients correlated better with biomarkers, suggesting that energy-adjusted nutrient intake may have greater biological relevance( 35 ).
FFQ validation is usually challenged by the lack of a gold standard reference method. The next best are those with different sources of errors, as non-independent measurement errors would tend to falsely elevate correlation( 36 ). DR may have less correlated errors with FFQ than does 24 h recall, as it allows direct measurement of portion size and does not rely on memory( 36 ). However, it may be difficult for those with lower education, adding to selection bias. The assistance from family members enabled us to gather some data from the less educated, although not to the extent to be representative of the TCHS population. Although not perfect, our result is more generalizable to the present cohort than if we were to use the other Taiwanese FFQ( 9 , 10 ), as our FFQ was validated among our cohort participants, while the other FFQ were validated among nutrition students or university staff/students.
Our DR validation is hampered by the inaccurate reference method used. The DR were obtained for only 3 d within a short period of time, which could hardly capture foods and nutritional supplements consumed episodically and nutrients with a high ratio of within-person to between-person variation, leading to potential underestimation for validity. In addition, comparing baseline FFQ (which assessed previous diet) with DR obtained 30 months post FFQ is of incorrect time frame. Potential dietary changes throughout this long duration would likely decrease the correlations. Although these limitations would both have led to underestimation of the FFQ's performance, our results are still comparable to those from other Chinese FFQ that had more replicates of dietary records or 24 h recalls completed across the span of 6–12 months prior to the compared FFQ(( 9 , 10 , 20 , 21 ).
The additional biomarker validation has enabled us to overcome the limitations of DR and further ascertain the relative validity of folate and vitamin B12, both otherwise shown to be low–moderate in the DR-validity study. The higher correlations for biomarker validation (the supposed lower bound for validity) than in DR validation confirm that our estimation from DR validation is a severe underestimation and a conservative estimation of the real validity of the present FFQ.
Conclusion
Our study suggests that the FFQ has acceptable reproducibility and relative validity for ranking intakes of energy and major nutrients among TCHS participants. For the first time, an FFQ for assessing nutrient intakes of Taiwanese and Asian vegetarians is available.
Acknowledgements
Sources of funding: The TCHS received grants (TCRD-I9605-01 and TCRD-I9605-02) from the Buddhist Tzu Chi Hospital which covered the expenses for baseline data collection of the cohort study and the biomarkers, while the DR-validity study did not receive any extra funding. Conflicts of interest: None of the authors have conflicts of interests. Authors’ contributions: study design by T.H.T.C., M.-N.L., K.-J.C. and C.-L.L.; coordination and data collection by M.-N.L., H.-Y.H. and Y.-R.W.; data computation, analysis and interpretation by T.H.T.C., J.P.C.C., Y.-H.L. and B.C.-H.C.; writing of the manuscript by T.H.T.C. All authors contributed to the final version of this report. Acknowledgements: The authors would like to thank Dr Wen Harn Pan of Academia Sinica in Taiwan for providing the database of food intake in NAHSIT, the consumption frequency of which was used as the basis to assign weight distribution for our FFQ items; Dr Rwei-Fen S. Huang from the Department of Nutrition in Catholic Fu-Jen University, for sharing the folate database; and Dr Meei Shyuan Lee from the School of Public Health, National Defense Medical Center in Taiwan, for consultation on computation of FFQ.
References
- 1. Willett W (2013) Food-frequency methods. In Nutritional Epidemiology, 3rd ed., pp. 70–95 [W Willett, editor]. New York: Oxford University Press. [Google Scholar]
- 2. Kristal AR, Peters U & Potter JD (2005) Is it time to abandon the food frequency questionnaire? Cancer Epidemiol Biomarkers Prev 14, 2826–2828. [DOI] [PubMed] [Google Scholar]
- 3. Smith AF (1991). Cognitive Processes in Long-Term Dietary Recall. Hyattsville, MD: US Department of Health and Human Services, Public Health Service, Centers for Disease Control, National Center for Health Statistics.
- 4. Pisani P, Faggiano F, Krogh V et al. (1997) Relative validity and reproducibility of a food frequency dietary questionnaire for use in the Italian EPIC centres. Int J Epidemiol 26, Suppl. 1, S152–S160. [DOI] [PubMed] [Google Scholar]
- 5. Crowe FL, Appleby PN, Travis RC et al. (2013) Risk of hospitalization or death from ischemic heart disease among British vegetarians and nonvegetarians: results from the EPIC-Oxford cohort study. Am J Clin Nutr 97, 597–603. [DOI] [PubMed] [Google Scholar]
- 6. Tonstad S, Stewart K, Oda K et al et al. (2013) Vegetarian diets and incidence of diabetes in the Adventist Health Study-2. Nutr Metab Cardiovasc Dis 23, 292–299. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Huang T, Yang B, Zheng J et al et al. (2012) Cardiovascular disease mortality and cancer incidence in vegetarians: a meta-analysis and systematic review. Ann Nutr Metab 60, 233–240. [DOI] [PubMed] [Google Scholar]
- 8. Huang YC, Lee MS, Pan WH et al et al. (2011) Validation of a simplified food frequency questionnaire as used in the Nutrition and Health Survey in Taiwan (NAHSIT) for the elderly. Asia Pac J Clin Nutr 20, 134–140. [PubMed] [Google Scholar]
- 9. Lee MS, Pan WH, Liu KL et al et al. (2006) Reproducibility and validity of a Chinese food frequency questionnaire used in Taiwan. Asia Pac J Clin Nutr 15, 161–169. [PubMed] [Google Scholar]
- 10. Lyu LC, Lin CF, Chang FH et al et al. (2007) Meal distribution, relative validity and reproducibility of a meal-based food frequency questionnaire in Taiwan. Asia Pac J Clin Nutr 16, 766–776. [PubMed] [Google Scholar]
- 11. Department of Nutrition Tri-Service General Hospital (1998) Photos of Food in Taiwan. Taipei: Department of Health. [Google Scholar]
- 12. Department of Health of Executive Yuan in Taiwan (1988) Nutrient Composition Data Bank for Food in Taiwan Area. Taipei: Department of Health. [Google Scholar]
- 13. Lee CH, Lee FY, Wong J et al et al. (2003) Design of food frequency questionnaire for assessing dietary folate: its application to study consumption frequency of folate-rich foods in ischemic stroke patients. Nutr Sci J 28, 210–217. [Google Scholar]
- 14. Gebhardt SE, Lemar LE, Pehrsson PR et al et al. (2010) National Nutrient Database for Standard Reference Release 23. Washington, DC: US Department of Agriculture, Agricultural Research Service, Nutrient Data Laboratory. [Google Scholar]
- 15. The Japanese Food Composition Table Committee (2011) Japanese Food Composition New Table 2011. Tokyo: Tokyo Law Publishing. [Google Scholar]
- 16. Yang Y, He M & Pan X (2004) China Food Composition 2004. Beijing: Peking University Medical Press. [Google Scholar]
- 17. Willett W & Stampfer M (2013) Implications of total energy intake for epidemiologic analysis. In Nutritional Epidemiology, 3rd ed., pp. 260–286 [W Willett, editor]. New York: Oxford University Press. [Google Scholar]
- 18. Willett W (2013) Correction for the effects of measurement error. In: Nutritional Epidemiology, 3rd ed., pp. 287–304 [W Willett, editor]. New York: Oxford University Press. [Google Scholar]
- 19. Willett W & Lenart E (2013) Reproducibility and validity of food-frequency questionnaires. In Nutritional Epidemiology, 3rd ed. pp. 96–141 [W Willett, editor]. New York: Oxford University Press. [Google Scholar]
- 20. Shu XO, Yang G, Jin F et al et al. (2004) Validity and reproducibility of the food frequency questionnaire used in the Shanghai Women's Health Study. Eur J Clin Nutr 58, 17–23. [DOI] [PubMed] [Google Scholar]
- 21. Villegas R, Yang G, Liu D et al et al. (2007) Validity and reproducibility of the food-frequency questionnaire used in the Shanghai Men's Health Study. Br J Nutr 97, 993–1000. [DOI] [PubMed] [Google Scholar]
- 22. Ocke MC, Bueno-de-Mesquita HB, Goddijn HE et al et al. (1997) The Dutch EPIC food frequency questionnaire. I. Description of the questionnaire, and relative validity and reproducibility for food groups. Int J Epidemiol 26, Suppl. 1, S37–S48. [DOI] [PubMed] [Google Scholar]
- 23. Ocke MC, Bueno-de-Mesquita HB, Pols MA et al et al. (1997) The Dutch EPIC food frequency questionnaire. II. Relative validity and reproducibility for nutrients. Int J Epidemiol 26, Suppl. 1, S49–S58. [DOI] [PubMed] [Google Scholar]
- 24. Lee MM, Pan WH, Yu SL et al et al. (1992) Foods predictive of nutrient intake in Chinese diet in Taiwan: I. Total calories, protein, fat and fatty acids. Int J Epidemiol 21, 922–928. [DOI] [PubMed] [Google Scholar]
- 25. Knutsen SF, Fraser GE, Beeson WL et al et al. (2003) Comparison of adipose tissue fatty acids with dietary fatty acids as measured by 24-hour recall and food frequency questionnaire in Black and White Adventists: the Adventist Health Study. Ann Epidemiol 13, 119–127. [DOI] [PubMed] [Google Scholar]
- 26. Pan WH, Lee MM, Yu SL et al et al. (1992) Foods predictive of nutrient intake in Chinese diet in Taiwan: II. Vitamin A, vitamin B1, vitamin B2, vitamin C and calcium. Int J Epidemiol 21, 929–934. [DOI] [PubMed] [Google Scholar]
- 27. Jacques PF, Sulsky SI, Sadowski JA et al et al. (1993) Comparison of micronutrient intake measured by a dietary questionnaire and biochemical indicators of micronutrient status. Am J Clin Nutr 57, 182–189. [DOI] [PubMed] [Google Scholar]
- 28. Green TJ, Allen OB & O'Connor DL (1998) A three-day weighed food record and a semiquantitative food-frequency questionnaire are valid measures for assessing the folate and vitamin B-12 intakes of women aged 16 to 19 years. J Nutr 128, 1665–1671. [DOI] [PubMed] [Google Scholar]
- 29. Iso H, Moriyama Y, Yoshino K et al et al. (2003) Validity of the self-administered food frequency questionnaire used in the 5-year follow-up survey for the JPHC Study to assess folate, vitamin B6 and B12 intake: comparison with dietary records and blood level. J Epidemiol 13, 1 Suppl., S98–S101. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Yoshino K, Nishide M, Sankai T et al et al. (2010) Validity of brief food frequency questionnaire for estimation of dietary intakes of folate, vitamins B6 and B12, and their associations with plasma homocysteine concentrations. Int J Food Sci Nutr 61, 61–67. [DOI] [PubMed] [Google Scholar]
- 31. van de Rest O, Durga J, Verhoef P et al et al. (2007) Validation of a food frequency questionnaire to assess folate intake of Dutch elderly people. Br J Nutr 98, 1014–1020. [DOI] [PubMed] [Google Scholar]
- 32. Verkleij-Hagoort AC, de Vries JH, Stegers MP et al et al. (2007) Validation of the assessment of folate and vitamin B12 intake in women of reproductive age: the method of triads. Eur J Clin Nutr 61, 610–615. [DOI] [PubMed] [Google Scholar]
- 33. Jackson MD, Walker SP, Younger NM et al et al. (2012) Use of a food frequency questionnaire to assess diets of Jamaican adults: validation and correlation with biomarkers. Nutr J 10, 28. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34. Bingham SA, Gill C, Welch A et al et al. (1997) Validation of dietary assessment methods in the UK arm of EPIC using weighed records, and 24-hour urinary nitrogen and potassium and serum vitamin C and carotenoids as biomarkers. Int J Epidemiol 26, Suppl. 1, S137–S151. [DOI] [PubMed] [Google Scholar]
- 35. Willett W (2001) Invited commentary: a further look at dietary questionnaire validation. Am J Epidemiol 154, 1100–1102. [DOI] [PubMed] [Google Scholar]
- 36. Willett WC, Sampson L, Stampfer MJ et al et al. (1985) Reproducibility and validity of a semiquantitative food frequency questionnaire. Am J Epidemiol 122, 51–65. [DOI] [PubMed] [Google Scholar]