Criterion Validity of the Healthy Eating Self-monitoring Tool (HEST) for Black Adolescents

Abstract

The criterion validity of a prototype version of the Healthy Eating Self-Monitoring Tool (HEST), a computer-mediated food record for assessing fruit and vegetable consumption among economically disadvantaged Black adolescents, was examined. A qualitative design employing focus group methodology gathered data for informing the design of the HEST (N = 32) and the selection of screens for inclusion in the measure (N = 6). The criterion validity of the HEST was examined in a cross-sectional design (N = 60) using direct observation of intake as the validation standard. Pearson correlations and paired-samples t tests compared mean observed and mean HEST-recorded servings of fruits and vegetables measured over three consecutive meals. Correlations were significant for seven of the nine HEST items, ranging from r = 0.41 (P < 0.01) to r = 0.65 (P < 0.001); for the overall measure, the correlation was r = 0.51 (P < 0.001). Paired samples t tests confirmed the absence of differences between observed and recorded intake for approximately half of the HEST items. The HEST offers a viable alternative to traditional paper-and-pencil food records. With further development and validation, the HEST can be used to measure fruit and vegetable consumption among economically disadvantaged Black adolescents.

INTRODUCTION

Self-monitoring is the systematic observation and recording of one’s behavior (1). It is among behavioral self-management strategies that have been effectively used to modify adolescent dietary behaviors (2,3). Self-monitoring can actively engage adolescents in the change process, increase their awareness of their current intake relative to recommendations, and promote dietary goal setting and progress monitoring. The success of self-monitoring depends on the availability of reliable and valid self-monitoring tools. Food records are ideal for this purpose: they provide a mechanism for systematically recording eating behaviors; they provide quantitatively accurate food intake data; they can be modified to record intake of certain foods or food groups; and they can be tailored to the cultural and developmental needs of specific populations. However, because they are traditionally used as the validation standard against which other measures are compared, validity data on food records generally (4), among specific population groups (4), and on alternate forms of food records are limited. The purpose of this study was to develop and validate a prototype computer-mediated food record (the Healthy Eating Self-monitoring Tool, or HEST) for measuring fruit and vegetable consumption among economically disadvantaged Black adolescents. The validity of the HEST was determined by comparing HEST-recorded servings of fruits and vegetables with observed intake of these foods as measured by the plate-waste-by-visual-estimate method (5). Direct observation of food intake by visual estimation was used in this study because it is relatively unobtrusive, is well suited to public eating situations, and is a good method for validating other intake assessment methods (6). Validity testing was guided by two hypotheses. The first was that the correlation between observed and HEST-recorded servings of fruits and vegetables would significantly differ from zero. The second was that there would be an absence of significant differences between mean observed and HEST-recorded servings of fruits and vegetables.

METHODS

A qualitative design employing focus group methodology provided data for informing the development of the HEST. The HEST was validated in a cross-sectional design using measures of dietary intake gathered over three consecutive meals. Direct observation of fruit and vegetable intake by visual estimation was the criterion measure. Mean observed and mean HEST-recorded intake were compared to determine the criterion validity of the measure.

Selection criteria for study participation were Black ethnic-racial heritage and between 11 and 14 years of age. Participants were recruited through social service agencies located in the greater New York City area. This geographic area was defined as the Bronx, Brooklyn, Manhattan, Queens, and Staten Island; the Long Island counties of Nassau and Suffolk; and eastern New Jersey (Jersey City, Elizabeth, and Newark east to the Hudson River). Social service agencies were private nonprofit organizations that provided youths with human services such as school dropout prevention, substance abuse prevention, recreation, educational tutoring, computer literacy, and youth club activities. To ensure a homogenous sample, two criteria for agency inclusion were used. First, the agency had to serve a predominantly Black population of youths from communities in which 20% or more of households reported 1999 family incomes below Federal poverty thresholds (7). Second, the agency had to be equipped with Windows PCs or Macintoshes with the minimum hardware specifications required to run the HEST software.

Separate samples of youths were recruited to participate in the successive research activities conducted in this study. Agency administrators from sites located in targeted communities were contacted by telephone to discuss the study and confirm agency eligibility for study participation. Follow-up meetings were held with administrators from eligible sites who expressed interest in participating to review the study protocol and discuss participant recruitment strategies. Because agency administrators and staff had ongoing relationships with potential participants and their families, the decision was made to have representatives from each site inform potential participants about the study and offer them the opportunity to participate. Following Institutional Review Board approval, packets containing a written study description, informed consent document for focus groups or validity testing, and contact information for a member of the investigative team were developed. Agency representatives distributed the packets at each stage of the research and instructed potential participants to review the materials with their caregivers. Data for this study were provided by successive samples of 32, 6, and 60 adolescents, respectively, who returned consent documents containing their written assent and written permission from one of their parents or legal guardians.

The prototype HEST was developed and validated in a three-step process. First, focus groups were conducted with 32 youths to gather data on commonly consumed fruits and vegetables among the target population, examine youths’ preferences for electronically recording their intake of these foods, and explore features of computer programs that were appealing to them. Data provided by focus group participants informed the initial design of the HEST. Second, sample HEST screens were developed. A focus group was conducted with a separate sample of 6 youths to gather feedback on youths’ screen preferences. Feedback provided by participants guided the selection of screens for inclusion in the measure, which was programmed for CD-ROM delivery. Third, the criterion validity of the HEST was examined with a separate sample of 60 adolescents (55% male; mean age 12 years).

The 60 adolescents participated in full-day dietary assessments at participating sites. Youths completed a brief sociodemographic questionnaire and were served breakfast, lunch, and dinner. Following meals, youths recorded their food intake using the HEST. Pairs of trained research staff recruited through graduate nutrition and public health programs at local universities convened assessment sessions. Prior to visiting sites, the staff were required to attend a half-day training session led by the investigators. The training addressed their ethical responsibilities; adherence to human subjects protocols; and procedures for serving meals, manually recording amounts of food left on youths’ plates at the conclusion of meals, and supervising computer-mediated assessment sessions. Because meals and computer assessments were conducted in separate locations at each site, one member of the pair served meals and recorded amounts of food left on youths’ plates after eating. The other helped participants access and advance the HEST as needed and ensured that each participant completed the measure independently. Youths were neither offered nor provided with material guidance when recording their food intake using the HEST.

The HEST guides participants through three screens to record fruit, juice, and vegetable intake. On the first screen, users indicate the item eaten by selecting a picture of it. On the second screen, users view life-sized images of various serving sizes of the item. They indicate the serving size that they received of the item by selecting the picture that corresponds to this amount. On the third screen, users select an image that depicts the amount of the food or juice they ate or drank.

Meals included one-serving units based on 5 A Day criteria (8) of fruit, juice, and vegetable items, as well as other foods. For breakfast, youths were served six ounces of orange juice and a medium-sized apple and were given the choice of a bowl of cereal or oatmeal. For lunch, youths were served six ounces of apple juice, a medium-sized orange, and four ounces of baby carrots and were given the choice of a sandwich or salad. For dinner, youths were served a baked chicken breast, four ounces each of broccoli, corn, and grapes, and six ounces of 100% fruit juice. To reduce reactivity, participants were unaware of the specific foods being measured.

Food items and their serving sizes were recorded in advance of meals. The trained research staff member present at meals manually recorded amounts of food left on youths’ plates at the conclusion of meals, using the plate-waste-by-visual-estimate method (5). From food item, serving size, and consumption data, observed intake was determined.

Completion of the HEST, like other self-report measures of dietary intake, relies on memory, which is subject to errors that are manifest as omissions (failures to report items eaten) and intrusions (reports of items not eaten) (9). Prior to hypothesis testing, we determined the extent of such errors. The 60 records documenting observations of participants’ intake across meals were matched with their HEST entries. The percentages of omissions ranged from 13% for apple and orange juice to 28% for orange, grapes, carrots, and 100% juice blend. For intrusions, the percentages were lower, ranging from zero for apple and 100% juice blend to 8% for carrots and corn.

In a test of our first hypothesis, correlation analyses assessed the degree of correspondence between mean observed and recorded intake for each HEST item and for the overall measure. To determine expected coefficients, we drew from a study comparing adolescent fruit and vegetable intake measured by four different questionnaires with the mean of three 24-hour recalls. Reported correlations ranged from r = 0.28 to r= 0.43 (10). We established a priori that statistically significant Pearson r-values within this range would provide evidence of the criterion validity of the HEST. In a test of our second hypothesis, mean differences between observed and recorded intake were examined with paired-samples t tests. The absence of significant differences would provide further evidence of the validity of the HEST.

RESULTS AND DISCUSSION

Significant correlations between observed and recorded servings of fruits and vegetables were observed for seven of the nine HEST items and ranged from r = 0.41 (P < 0.01) to r = 0.65 (P < 0.001) (Table 1). For the overall measure, this correlation was r = 0.51 (P < 0.001). Paired-samples t tests revealed the absence of significant differences between observed and recorded intake for approximately half of the HEST items. Comparison of mean observed (5.92) and recorded (5.00) servings of fruits and vegetables revealed that youths underestimated their intake using the HEST.

Table 1.

Comparison of Mean Servings of Fruits, Juices, and Vegetables Estimated by Direct Observation and HEST Software

Food item	Correlation between observed and recorded	Mean observed	Mean recorded	Difference ± SE (observed – recorded)	P value
Fruits
Apple	.61***	.40	.38	.02 ±.04	.51
Orange	.52***	.57	.35	.22 ±.06	.00
Grapes	.47***	.79	.77	.02 ±.09	.81
Vegetables
Broccoli	.41***	.51	.40	.11 ±.07	.10
Corn	.52***	.59	.60	− .01 ±.07	.91
Carrots	.65***	.35	.40	−.05 ±.06	.39
Juices
Orange	.57***	.79	.63	.16 ±.05	.00
Apple	.04 *	.96	.79	.17 ±.07	.01
100% juice blend	.24***	.95	.69	.26 ±.06	.00
HEST	.51***	5.92	5.00	.92 ±.33	.01

All comparisons were based on matched observations and HEST entries provided by 60 participants. Values shown in the last column indicate the level of significance for paired-samples t tests comparing mean observed and mean recorded intake.

^**Comparison of mean observed intake with mean recorded intake significant at P = 0.01, determined by correlation analysis.

^***Comparison of mean observed intake with mean recorded intake significant at P = 0.001, determined by correlation analysis.

Findings from correlation and t test analyses supported the criterion validity of the HEST. Although participants’ average fruit and vegetable intake (about six servings) was higher than that found in national survey data (11), participants did not eat all that was served (nine servings). Thus, observed correlations cannot be attributed to ceiling effects. Moreover, observed correlations cannot be an artifact of the institutional setting in which the study took place, given that youths were regularly served meals at these sites. Findings from this study add to the limited validity data on food records generally, among Black adolescents in particular, and on alternate forms of food records.

Our finding that adolescents underestimated their fruit and vegetable intake using the HEST is consistent with previous studies. In this sample of Black adolescents, the mean number of servings of fruits and vegetables measured by the HEST was lower than youths’ observed intake of these foods, even though there was overestimation for some individual items. Data from a study comparing observed and recorded intake of foods measured via food records completed by a predominantly Black sample of fourth-and fifth-grade students revealed that youths under- and overestimated their intake of meal items, with underestimation more common than overestimation (12). Food record validation studies comparing child and adolescent nutrient intake have similarly found underestimation to be common (13–15).

This study is not without limitations. The lack of data on anthropometrics precludes assessment of whether the underreporting of intake observed in this study is attributable to such factors as height, weight, and body mass index (BMI). Prior studies have found that obese adolescents underreport energy intakes derived from food records (16,17). Study participants may have been overrepresented on this characteristic, leading to biased results. The use of a self-selected sample and the small number of items in the HEST may have biased study findings. Youths who completed the measure may have been more motivated to record their intake than typical adolescents. Results may be an artifact of the number and type of foods and juices studied. Although comparison of observed and HEST-recorded intake revealed the underestimation of intake, examination by food item (e.g., apple, corn) and type (e.g., fruits, juices, vegetables) failed to reveal a consistent pattern. Additional research with sizable numbers of foods and juices is needed to determine whether the accuracy of youths’ recording varies by food and type. Because the HEST is designed for use on a frequent basis, more work is needed to determine the test-retest reliability, or stability, of the measure.

CONCLUSIONS

The comparability of validity data observed in this study with that from previous studies suggests that the HEST is a viable alternative to traditional paper-and-pencil food records. With further development and validation, the HEST can be used to measure intake in nutrition education and behavior change programs to increase fruit and vegetable consumption among economically disadvantaged Black adolescents. The HEST can also be used in the same way as conventional paper-and-pencil food records – as the standard against which other dietary assessments are compared. Because the HEST is computer-mediated, the time and expense required to administer and score this tool are reduced. Moreover, all respondents are presented with the same set of instructions and response tasks, thus, fidelity administration is assured. Perhaps most importantly, the measure quantifies fruit and vegetable intake in serving units, thereby eliminating the time and expense associated with coding and analyzing food record data. These advantages implicate increased use of the HEST over conventional food records.