Validation of the MEDFICTS Dietary Assessment Questionnaire in a Diverse Population

Abstract

Background

The National Cholesterol Education Program (NCEP) recommends MEDFICTS, a rapid screening instrument for dietary fat, to assess adherence to the Adult Treatment Panel (ATP) III Therapeutic Lifestyle Changes (TLC) diet (score <40 points indicates intake of <7% of energy from saturated fat, <30% of energy from total fat, and <200 mg dietary cholesterol/day). MEDFICTS has only been validated in small, select populations and its utility in diverse clinical settings is unknown.

Objective

To evaluate the ability of MEDFICTS to identify individuals who are nonadherent to a TLC diet in an ethnically diverse population that includes both English-and Spanish-speakers.

Design

MEDFICTS was administered concurrently with the Gladys Block Food Frequency Questionnaire to participants (n=501; mean age 48±13.5 years; 36% nonwhite; 66% female) in the National Heart, Lung, and Blood Institute Family Intervention Trial for Heart Health (FIT Heart) at the baseline screening visit. Reliability and validity analyses were conducted overall and by sex, age, and race/ethnicity.

Results

MEDFICTS score correlated significantly with percentage of energy from saturated fat (r=0.52, P< 0.0001), percentage of energy from total fat (r=0.31, P<0.0001), and milligrams per day of dietary cholesterol (r=0.54, P<0.0001). Sensitivity of MEDFICTS to correctly identify TLC diet adherence was 85.7% and did not differ significantly by sex, age, or race/ethnicity. Specificity of MEDFICTS to correctly identify nonadherence to the TLC diet was low (56.9%) and significantly worse for women than men (48.4% vs 72.9%; P<0.0001), but did not differ significantly in older vs younger participants or among white, black, or Hispanic participants.

Conclusion

Our data suggest that sex-specific recalibration of MEDFICTS may improve specificity and clinical utility.

The Therapeutic Lifestyle Changes (TLC) diet, which limits dietary saturated fat and cholesterol, is recommended as the first step to reduce low-density lipoprotein (LDL) cholesterol levels in patients with or at risk for coronary heart disease (CHD) (1). Diet assessment is a key first step before the initiation of any therapeutic diet change. MEDFICTS (Meats, Eggs, Dairy, Fried foods, fat In baked goods, Convenience foods, fats added at the Table, and Snacks), a brief dietary assessment instrument, has been provided as part of the National Cholesterol Education Program (NCEP) Adult Treatment Panel (ATP) guidelines as a free tool to use for proper cardiovascular diet assessment (1,2).

MEDFICTS was designed to assess adherence to NCEP step 1 and step 2 diets (3). The step 1 diet is characterized by less than 10% of energy intake from saturated fat, less than 30% of energy intake from total fat, and less than 300 mg of dietary cholesterol per day. The step 2 diet is characterized by less than 7% of energy intake from saturated fat, less than 30% of energy intake from total fat, and less than 200 mg of dietary cholesterol per day, and until 2001 was recommended for LDL reduction and for all patients with established CHD (3). With the publication of the ATP III guidelines in 2001, NCEP transitioned from the step 2 diet to the TLC diet. The TLC diet contains the same goals for saturated fat and dietary cholesterol reduction as step 2, but also includes additional therapeutic options for LDL reduction if initial saturated fat and cholesterol reductions are not sufficient to achieve target LDL cholesterol levels. MEDFICTS is recommended by ATP III to assess adherence to the TLC diet (1).

Correlation of MEDFICTS score with TLC diet components (4–6) and the sensitivity and specificity of MEDFICTS to assess adherence to the step 1 diet (5,6) have been assessed in small and/or select populations, so generalizing the validity of the instrument in more diverse populations and clinical settings is not possible. More-over, the utility of MEDFICTS to identify nonadherence to the TLC diet has not yet been established. The purpose of this study was to evaluate the ability of MEDFICTS to correctly classify patients as adherent or nonadherent to the TLC diet in a diverse population of participants in the NHLBI Family Intervention Trial for Heart Health (FIT Heart) (n=501; mean age 48±13.5 years; 36% racial/ethnic minority) designed to test the effectiveness of a screening and educational intervention for family members of patients hospitalized with cardiovascular disease (CVD).

SUBJECTS AND METHODS

Study Design and Participants

The design of this validation study was a cross-sectional analysis of baseline data. Baseline data collection was conducted between January 2005 and June 2007. Participants were eligible if they were family members or cohabitants of patients hospitalized with CVD, were 20 to 79 years of age, did not have established CVD or diabetes, and spoke either English or Spanish. All participants signed written informed consent to be a part of the study. The Columbia University Medical Center Institutional Review Board approved the study.

Dietary Assessments

Dietary assessments were completed with all participants at their baseline study visit. Diet was assessed using two instruments administered by trained health educators: a) MEDFICTS, and b) the full-length Block 98 Food Frequency Questionnaire (FFQ).

The MEDFICTS questionnaire was originally developed for and printed in the NCEP ATP II guidelines (2,4). Although originally intended to assess adherence to the NCEP step 2 diet, MEDFICTS was recommended in the ATP III guidelines for assessment of TLC diet adherence. Step 2 and TLC diets share the primary goals of consuming less than 7% of energy from saturated fat, less than 200 mg dietary cholesterol per day, and less than 30% of energy from total fat. However, TLC allows for liberation of total fat up to 35% of energy for individuals with metabolic syndrome. MEDFICTS focuses on foods that are major sources of fat, saturated fat, and dietary cholesterol in the average American diet and provides a quick way to record intake, frequency, and portion size. The questionnaire yields a continuous score, with a cut-point of less than 40 points recommended to define TLC diet adherence.

The Block FFQ was originally developed for research examining the role of diet in health and disease, has been continually updated and improved, and has been extensively studied and validated (7–12). The Block 98 FFQ is a 110-item questionnaire designed to estimate usual and customary dietary intake during the previous 4 weeks. The food list for the questionnaire was developed from dietary recall data from the Third National Health and Nutrition Examination Survey. The questionnaire takes 30 to 40 minutes to administer and records the frequency and portions of foods consumed (13). Completed questionnaires are scanned and analyzed using a nutrient database developed from the US Department of Agriculture Nutrient Database for Standard Reference (14). Output includes daily calorie, macronutrient, and micronutrient totals.

To enlarge the pool of potential study participants by reaching out to the Hispanic community, which is traditionally underrepresented in research studies, all study questionnaires including MEDFICTS and the Block 98 FFQ were translated into Spanish by the Columbia University Medical Center Hispanic Translation Center. The Center employs experts in the translation of research questionnaires with the goals of precision in the transfer of meaning and assurance of the validity and comparability of results across languages. Translated questionnaires were administered by trained, fluent Spanish-speaking health educators using methods identical to English version administration.

Demographic data were collected using standardized questionnaires. All data were double-entered into a secure Microsoft Access 2003 database (Microsoft Corporation, Redmond, WA) and cleaned by a trained data management team before analysis.

Statistical Methods

Participant characteristics were described using means and proportions. Adherence to the TLC diet was defined as a total MEDFICTS score less than 40 points. True TLC adherence was assessed by the Block 98 FFQ and defined as consuming a diet with less than 7% of energy from saturated fat, less than 30% of energy from total fat, and less than 200 mg dietary cholesterol per day. Percentage of energy intake from saturated fat was calculated by multiplying the Block 98 FFQ output variable saturated fat (in grams) by a factor of 9 (calories per gram) and dividing by total calories. Percentage of energy from total fat and milligrams dietary cholesterol per day are standard output variables provided in the Block dataset and did not require calculation.

Pearson correlation coefficients were calculated to assess the relationship between (a) MEDFICTS score and percentage energy from saturated fat, (b) MEDFICTS score and percentage energy from total fat, and (c) MEDFICTS score and milligrams of dietary cholesterol per day. Kappa statistics were used to assess categorical agreement between MEDFICTS and Gladys Block on TLC diet adherence and agreement between MEDFICTS and individual TLC cut-points for saturated fat (<7% of energy intake vs ≥7%), total fat (<30% of energy intake vs ≥30%), and dietary cholesterol (<200 mg/day vs ≥200mg/day).

Sensitivity and specificity were calculated to assess validity of a MEDFICTS score cut-point of less than 40 points to correctly categorize participant adherence to the TLC diet overall, and by sex, age group, and race/ethnic group. Sensitivity was defined as the proportion of participants evaluated as adherent to the TLC diet by the Block 98 FFQ and who were also evaluated as adherent by MEDFICTS. Specificity was defined as the proportion of participants evaluated as nonadherent to the TLC diet by the Block 98 FFQ and who were also evaluated as nonadherent by MEDFICTS. Statistical significance of between-group differences in sensitivity and specificity were assessed by χ² test. Sensitivity analyses were conducted to quantify the effect that higher and lower MEDFICTS score cut-points for TLC adherence would have on sensitivity and specificity. Statistical analysis was done using the SAS System (version 9.1, Cary, NC).

RESULTS

All 501 participants enrolled in the FIT Heart Trial (100%) completed both MEDFICTS and the Block FFQ at their baseline visit and were included in this validation study (Table 1). More than one third of participants were racial/ethnic minorities and more than two thirds were women. Eight percent of participants (n=40) were interviewed in Spanish.

Table 1.

Demographic and diet characteristics of the study population (n=501) in the National Heart, Lung, and Blood Institute Family Intervention Trial for Heart Health (FIT Heart)

Characteristic	n	%
Age ≥65 y	60	11.98
Female	330	65.9
Race/ethnicity
White	323	64.4
Black	30	6.0
Hispanic	120	24.0
Other	28	5.6
Education level
≤High school or equivalent	23	4.6
Marital status
Married/cohabitating with partner	357	71.3
	←mean±SDa→
MEDFICTSb
Score	48.4±28.8
Gladys Block
% energy from total fat	37.8±6.8
% energy from saturated fat	10.7±2.6
Dietary cholesterol (mg/day)	240.0±130.6

^aSD=standard deviation.

^bMEDFICTS=Meats, Eggs, Dairy, Fried foods, fat In baked goods, Convenience foods, fats added at the Table, and Snacks dietary assessment instrument.

Mean MEDFICTS score was 48.4 points. MEDFICTS categorized 44.9% of study participants as adherent to the TLC diet. Based on the Block FFQ, significantly fewer participants (4.2%) were categorized as adherent (P=0.0002).

Pearson correlation coefficients between continuous MEDFICTS score and individual components of the TLC diet were fair to good: (a) 0.52 for percentage of energy from saturated fat/day, (b) 0.54 for milligrams of dietary cholesterol per day, and (c) 0.31 for percentage of energy from total fat per day, and all were statistically significant (P<0.0001).

Categorical agreement between MEDFICTS and the Block FFQ on TLC diet adherence was poor overall (κ=0.08; P<0.0001). Agreement was slightly better between MEDFICTS and individual TLC diet components assessed by the Block FFQ: 1) less than 7% of daily calories from saturated fat (κ=0.13; P<0.0001), 2) less than 30% of calories (κ=0.16; P<0.0001), and 3) cholesterol less than 200 mg/day (κ=0.34; P<0.0001). Because the category for total fat less than 30% of energy is not a major TLC diet goal for all patients, we removed it from our definition of the TLC diet (saturated fat and cholesterol parameters were left in) and reassessed agreement between MEDFICTS and the Block FFQ on TLC diet adherence. This change yielded a slightly higher kappa of 0.10 (P<0.0001), but did not meaningfully improve agreement.

MEDFICTS score dichotomized at less than 40 points had the sensitivity to correctly categorize adherent participants as adherent to the TLC diet 85.7% of the time and specificity to correctly categorize nonadherent participants as nonadherent 56.9% of the time (Table 2).

Table 2.

Number of participants classified as nonadherent to NCEP^a Therapeutic Lifestyle Changes diet by MEDFICTS^b vs Gladys Block food frequency questionnaire^cd

	Gladys Block Food Frequency Questionnaire
	Adherent		Nonadherent
MEDFICTS	n	%	n	%	Totals
Adherent (<40 points)	18	8.0	207	92.0	225
Nonadherent (≥40 points)	3	1.1	273	98.9	276
Totals	21		480		501

^aNCEP=National Cholesterol Education Program.

^bMEDFICTS=Meats, Eggs, Dairy, Fried foods, fat In baked goods, Convenience foods, fats added at the Table, and Snacks dietary assessment instrument.

^cSensitivity=Proportion of persons adherent to Therapeutic Lifestyle Change diet who were identified as adherent by MEDFICTS: 18/21=85.7%.

^dSpecificity=Proportion of persons nonadherent to Therapeutic Lifestyle Change diet who were identified as nonadherent by MEDFICTS: 273/480=56.9%.

Sensitivity did not differ significantly in men vs women (P=0.68), or among white, black, and Hispanic participants (P=0.61). Specificity was significantly lower for women (48.4%) vs men (72.9%) (P<0.0001) and did not differ significantly by age group younger than 65 years vs 65 years and older (P=0.16), or by white (55.8%), black (69.0%), or Hispanic (57.1%) race/ethnicity (P=0.39) (Table 3–Table 5).

Table 3.

Number of participants classified as nonadherent to Therapeutic Lifestyle Changes diet by MEDFICTS^a vs Gladys Block food frequency questionnaire by sex

	Gladys Block Food Frequency Questionnaire
	Adherent				Nonadherent
	Male		Female		Male		Female		Totals
MEDFICTS	n	%b	n	%	n	%	n	%	Male	Female
Adherent	4	2.3	14	4.2	45	26.3	162	49.1	49	176
Nonadherent	1	0.6	2	0.6	121	70.8	152	46.1	122	154
Totals	5		16		166		314		171	330

^aMEDFICTS=Meats, Eggs, Dairy, Fried foods, fat In baked goods, Convenience foods, fats added at the Table, and Snacks dietary assessment instrument.

^bPercent within sex category.

Table 5.

Number of participants classified as nonadherent to Therapeutic Lifestyle Changes diet by MEDFICTS^a vs Gladys Block food frequency questionnaire by race/ethnicity

	Gladys Block Food Frequency Questionnaire
	Adherent						Nonadherent						Totals
	White		Black		Hispanic		White		Black		Hispanic		White	Black	Hispanic
MEDFICTS	n	%b	n	%	n	%	n	%	n	%	n	%	n	n	n
Adherent	10	3.1	1	3.3	6	5.0	138	42.7	9	30.0	48	40.0	148	10	54
Non-Adherent	1	0.3	0	0.0	2	1.7	174	53.9	20	66.7	64	53.3	175	20	66
Totals	11		1		8		312		29		112		323	30	120

^aMEDFICTS=Meats, Eggs, Dairy, Fried foods, fat In baked goods, Convenience foods, fats added at the Table, and Snacks dietary assessment instrument.

^bPercent within race/ethnic category.

Sensitivity analysis imputing a range of MEDFICTS cut-points (from <15 to <60) by which to define TLC adherence yielded values for optimal sensitivity and specificity that differed overall, and for men and women, from the cut-point of less than 40 points that is currently recommended. Although our results support the current adherence-defining cut-point of less than 40 points for sensitivity, our sensitivity analysis did not find this to be an optimal cut-point for specificity to identify nonadherence. Optimal cut-point for defining TLC adherence was selected based on what would provide a maximum (or acceptable) specificity, without appreciable loss in sensitivity. Figure 1 illustrates sensitivity and specificity curves for our dataset. A MEDFICTS score of less than 25 points yielded a specificity improved to 82.5% and a sensitivity of 76.2%.

Figure 1.

Sensitivity and specificity of MEDFICTS (Meats, Eggs, Dairy, Fried foods, fat In baked goods, Convenience foods, fats added at the Table, and Snacks) score to identify Therapeutic Lifestyle Changes diet adherence.

Sex differences in sensitivity and specificity of MEDFICTS score to identify TLC diet adherence are illustrated in Figure 2. For men, a cut-point of less than 37 points was found to be optimal and yielded a specificity of 80% and a sensitivity of 78.3%. For women, the optimal cut-point was less than 20 points, which yielded a specificity of 83.8% and a sensitivity of 75.0%.

Figure 2.

Sensitivity and specificity of MEDFICTS (Meats, Eggs, Dairy, Fried foods, fat In baked goods, Convenience foods, fats added at the Table, and Snacks) score to identify Therapeutic Lifestyle Changes diet adherence by sex.

DISCUSSION

We documented a significant correlation between MEDFICTS score and percentage energy from saturated fat, percentage energy from total fat, and dietary cholesterol, and a sensitivity of 85.7% for MEDFICTS to correctly determine adherence to the TLC diet in an ethnically diverse population. However, we also documented that MEDFICTS has a very low specificity to identify nonadherent patients.

The ability of a test to obtain the same results under the same conditions (reliability) is a prerequisite for validity. Beyond its brief nature and low cost to administer, the major strength of MEDFICTS is the correlation of the MEDFICTS score with TLC diet components. Correlation between MEDFICTS score and percentage energy from saturated fat obtained in our study (r=0.51) was similar to that obtained in a previous MEDFICTS validation study conducted in 22 subjects participating in a diet intervention program at Mary Imogene Bassett Research Institute in Cooperstown, NY (r=0.60) (4). Correlation between MEDFICTS score and total fat intake was lower in our study (r=0.30) compared with that study (r=0.56) and another conducted in 26 participants at the Diet Modification Clinic, Baylor College of Medicine, Houston, TX (r=0.71) (4). However, a more recent study conducted in 184 premenopausal African-American women found the correlation between MEDFICTS score and the percentage of energy from total fat to be the same as we did (r=0.30). That study estimated percentage of energy from total fat using the Arizona FFQ, a modified Block FFQ (6), whereas the prior two studies used 3-day food records. Our findings may be more similar to this study than the ones from Cooperstown and Baylor due to use of similar instruments, larger sample sizes, and the significant proportion of women in our study.

The validation studies from Cooperstown and Baylor reported a correlation between MEDFICTS and milligrams of dietary cholesterol per day to be r=0.39 to 0.54, and a validation study published 2 years later (conducted in a larger [n=164], mostly male sample) found the correlation between MEDFICTS score and dietary cholesterol estimated by an abbreviated Block FFQ to be r=0.55, both estimates almost identical to our results (r=0.53) (4,5).

Kappa statistics for agreement between MEDFICTS and the Block FFQ were low. Kappa is dependent on the percentage chance expected agreement between two rating methods. The higher the chance expected agreement, the lower kappa will be (15). In our study, chance expected agreement between the two measures was 54% based on marginal totals, which is high (Table 2). Observed agreement was 58%, yielding a small difference between them (ie, little agreement beyond what was expected by chance).

Good overall sensitivity lets us know that MEDFICTS will identify patients who are adherent to the TLC diet most of the time. But low specificity tells us that the current scoring system does not tell clinicians what we really need to know: whether a patient is nonadherent and needs further adjustments to his or her diet. Specificity estimates from our study suggest that nonadherent patients would be incorrectly screened as adherent more than 40% of the time. This translates into potential missed opportunities for counseling and diet modification for CHD risk reduction. This may be a limitation of the current tool, because diet modification is the first tier approach in lipid management and it is estimated that the TLC diet can reduce LDL cholesterol by 20% to 30% (16).

Reasons for the observed sex differences in sensitivity and specificity of MEDFICTS deserve further evaluation. Recent research has provided corroborating evidence that accuracy of FFQs may vary by sex and characteristics of the survey, such as length (17). Addressing the underlying cause of the sex difference could lead to improved validity of MEDFICTS and other rapid dietary assessment methods.

A major strength of this study was participation of women and minorities. More than 60% of participants were women and more than 30% were racial/ethnic minorities. Lack of significant difference in specificity or sensitivity of MEDFICTS by race/ethnic group suggests that MEDFICTS is a robust tool to assess TLC adherence in diverse patient populations. Differences in specificity by sex suggest that different cut-points for men and women could improve the clinical utility of the MED-FICTS questionnaire to identify patients in need of diet modification.

A limitation of our study is the small sample size within each age and race strata. This could have limited our power to detect differences between age and racial groups. Our minority study participants were largely English-speaking and most had completed high school, which may affect generalizability of these findings to non–English speaking or less-educated groups. The use of an FFQ as a comparison method has been associated with limitations due to measurement error (18). But alternate diet assessment methods are also considered imperfect (19), and, because the Block FFQ was developed using nationally representative diet recall data and is scored mechanically, bias due to differential scoring errors by group may be less likely.

MEDFICTS is a fast, free diet assessment tool that is easily accessible and recommended in national prevention guidelines. In this diverse population of participants without known CVD we showed a significant correlation between MEDFICTS score and the Block FFQ for dietary intake of saturated fat, total fat, and cholesterol. Our finding that MEDFICTS lacks specificity for TLC nonadherence may be clinically important because diet therapy is indicated for nonadherent patients. Recalibration of MEDFICTS to improve specificity and account for sex differences in scoring may increase its clinical utility.

Table 4.

Number of participants classified as nonadherent to Therapeutic Lifestyle Change diet by MEDFICTS^a vs Gladys Block food frequency questionnaire by age group

	Gladys Block Food Frequency Questionnaire
	Adherent				Nonadherent				Totals
	<65 y		≥65 y		<65 y		≥65 y		<65 y	≥65 y
MEDFICTS	n	%b	n	%	n	%	n	%	n	n
Adherent	14	3.2	4	6.7	178	40.3	29	48.3	192	33
Non-Adherent	3	0.7	0	0.0	246	55.8	27	45.0	249	27
Totals	17		4		424		56		441	60

^aMEDFICTS=Meats, Eggs, Dairy, Fried foods, fat In baked goods, Convenience foods, fats added at the Table, and Snacks dietary assessment instrument.

^bPercent within age category.