Process Evaluation of a Farm-to-WIC Intervention

Abstract

Background:

Despite the promise of farm-to-institution interventions for addressing limited vegetable access as a barrier to intake, programs designed for the Special Supplemental Nutrition Program for Women, Infants, and Children (WIC) are lacking. As such, little is known about the implementation of and mechanisms of action through which farm-to-WIC interventions affect vegetable intake and participant satisfaction with such programs.

Objective:

To examine whether a farm-to-WIC intervention to promote vegetable intake was implemented as intended, differences between participants who received the intervention relative to those in a usual-care control group in intermediate outcomes of vegetable-related knowledge, attitudes, and behaviors, and secondary outcomes of physical activity and weight status; and participant satisfaction with the intervention.

Design:

A process evaluation encompassing descriptive and comparative analyses of implementation fidelity logs and survey data collected as part of a pilot study was conducted.

Participants/setting:

The setting was a large, New Jersey-based, urban WIC agency. Recruited between June 3, and August 1, 2019 through 3 of the agency’s 17 sites (one intervention and two control sites), participants were 297 primarily Hispanic adults (160 enrolled at the intervention site and 137 at control sites).

Intervention:

The intervention combined behaviorally focused instruction and handouts with the introduction of a WIC-based farmers’ market, field trips to an area farmers’ market, telephone coaching and support, and recipe demonstrations and tastings.

Main outcome measures:

Primary outcomes were vegetable intake (measured via self-report and objectively using dermal carotenoids as a biomarker of intake) and the redemption of vouchers provided by WIC for fruit and vegetable purchases at farmers’ markets (measured objectively using data provided by WIC). For the process evaluation, logs were used to document program activities. Vegetable-related knowledge, attitudes, and behaviors, physical activity, and satisfaction with the intervention were assessed with participant questionnaires. Weight status was assessed with direct measures of height and weight. Data were collected at baseline and at mid- and post-intervention (3 and 6 months post-baseline, respectively).

Statistical analyses performed:

Descriptive statistics were used to characterize implementation fidelity. Associations between intermediate and secondary outcomes and vegetable intake were examined at baseline with Pearson correlations. Post-baseline between-group differences in the outcomes were examined with linear mixed-effects models adjusted for baseline values and covariates. Satisfaction with the intervention was assessed with inferential and thematic analyses.

Results:

Post-intervention, measures of vegetable intake were higher in the intervention relative to the control study group. Receipt of the intervention was also associated with a greater likelihood of voucher redemption. Nearly all participants (≥ 94%) received the intervention as intended at the WIC-based farmers’ market; smaller percentages completed one or more planned trips to the area farmers’ market (28%) and telephone coaching and support calls (88%). Although most intermediate and secondary outcomes were associated with measures of vegetable intake at baseline, post-intervention, the variables did not differ between study groups. Mean satisfaction ratings were ≥ 6.8 on a 7-point scale. Recipe demonstrations, learning about vegetables, field trips, and the rapport with staff were liked most about the program. Although adding days and times for field trips was suggested, limited market days and hours of operation limited the ability to do so.

Conclusions:

Preliminary data highlight the promise of this well-received intervention. Intermediate outcome findings suggest that other potential intervention mechanisms of action should be considered in future large-scale trials of this program. Broad-scale initiatives are needed to improve access to farmers’ markets in underserved communities.

INTRODUCTION

Many low-income communities in the United States lack access to affordable and nutritious foods.¹ Farmers’ markets are an important mechanism for increasing food access in such communities.² Acceptance of Supplemental Nutrition Assistance Program (formerly known as food stamps) benefits at farmers’ markets and Farmers’ Market Nutrition Program (FMNP) vouchers provided to participants of the Special Supplemental Nutrition Program for Women, Infants, and Children (WIC) have made farmers’ market food more affordable to low-income consumers.^2,3 Yet, in low-income populations, lacking or having limited access to transportation and distance to markets are key barriers to farmers’ market use.^4–6 By bringing local fruits and vegetables from regional farms to institutions such as worksites, schools, and faith-based organizations, farm-to-institution interventions can increase the likelihood of fruit and vegetable purchases and consumption. Despite their promise, programs designed for implementation in the WIC setting are lacking. As such, little is known about the outcomes of such programs and the design, implementation, and mechanisms of action through which they achieve their effects.

This paper describes the process evaluation of a novel, theory-driven, farm-to-WIC intervention. WIC serves to safeguard the health of low-income, nutritionally at-risk, pregnant and postpartum women and preschool children by providing nutritious foods to supplement diets, nutrition education, and health care referrals.⁷ In addition to their regular WIC benefits, participants receive seasonal FMNP vouchers for fruit and vegetable purchases at farmers’ markets.⁸ Designed to promote vegetable intake and redemption of the vouchers, the intervention was piloted in three WIC agency sites (one intervention and two control sites) with 297 urban, primarily Hispanic WIC-enrolled adults (160 enrolled at the intervention site and 137 at control sites).⁹ Vegetable intake was measured via self-report and objectively using dermal carotenoids (skin concentrations of plant pigments found in fruits and vegetables) as a biomarker of intake.¹⁰ Post-intervention, objective and self-reported vegetable intake were higher among those who received the intervention relative to those who did not.⁹ Receipt of the intervention was also associated with FMNP voucher redemption.⁹ In exploratory analyses, the relationship between the intervention and vegetable intake was moderated by FMNP voucher redemption, with larger post-intervention between-group differences in vegetable intake found among participants who redeemed the vouchers relative to those who did not.⁹ To inform larger trials of the program, here we examined: 1) intervention fidelity, the extent to which the program was implemented as intended, 2) pre-intervention levels of intermediate outcomes of vegetable-related knowledge, attitudes, and behaviors, variables hypothesized to influence vegetable intake as per the underlying intervention logic model, and secondary outcomes of physical activity and weight status, 3) whether intermediate and secondary outcomes differed by study group post-baseline, and 4) participant satisfaction with the intervention. The aims were to describe challenges and successes during program implementation, provide a more in-depth understanding of outcomes of the pilot, and document participant satisfaction with the intervention.

MATERIALS AND METHODS

Design and Sample

The study design is described elsewhere.⁹ In brief, the program was piloted in a randomized design with WIC agency site as the unit of randomization. The design was subject to the n = 1 confound, which occurs when the intervention or control group contains a single study unit, and therefore does not support causal attribution despite random selection of the intervention site.^11,12

A key inclusion criterion was the receipt of FMNP vouchers. Across sites, trained research staff contacted adults considered eligible to receive the vouchers based on information provided by WIC via telephone prior to forthcoming appointments, provided a description of the study, and obtained verbal consent from those who expressed interest in participating. Consenting adults were orally administered an outcome battery of self-report measures. During appointments, biometric measures (height, weight, and carotenoid levels) were taken. Participants were also recruited from the waiting room of the WIC clinic (adults who had been called in advance but not reached and those who walked in) where they completed all measures onsite. All participants provided informed written consent prior completing in-person assessments. Thereafter, a WIC administrator confirmed whether adults who completed all assessments had received FMNP vouchers. Those who did not were informed that they were ineligible and were thanked for their involvement. The study was approved by the William Paterson University Institutional Review Board for Human Subject Research (2018–339) and registered with ClinicalTrials.gov (NCT04038385).

In total, 297 adults were enrolled (160 at the intervention site and 137 at control sites; mean age = 31.7 ± 7.2 years; 73% Hispanic and 14% non-Hispanic African American; 70% foreign-born; 55% reporting a high school education or less; and 75% overweight or obese [body mass index ≥ 25 kg/m²]).¹³ Adults enrolled at the intervention site received routine services provided by WIC and the farm-to-WIC intervention. Those enrolled at control sites received routine services only. Across sites, research staff contacted participants to complete follow-up measures at mid- and post-intervention (3 and 6 months post-baseline, respectively). As at baseline, the outcome battery was telephone-administered; during WIC appointments, biometric measures were taken.

To enhance retention, research staff mailed appointment reminders to all participants and confirmed appointments via telephone one day in advance, maintained multiple contacts for participants, and updated participants’ contact information at each assessment Honoraria (gift cards redeemable at local supermarkets and discount chain stores) were also provided to participants for completing successive assessments ($10 at baseline, $10 at mid-intervention, and $20 at post-intervention [participants received up to $40 in honoraria in total]).

Intervention

Designed to address individual and broader systems-level influences on vegetable intake, the intervention was conceptually grounded in the Social Ecological Model and Social Cognitive Theory.¹⁴ According to the Social Ecological Model, behavior is shaped by influences operating at multiple levels, i.e., intrapersonal and interpersonal factors, community and organizational factors, and public policies.¹⁵ Social Cognitive Theory emphasizes targeting the environment (factors external to a person), behavioral capacity (knowledge and skills to perform a behavior), and self-efficacy (confidence in the ability to perform a behavior) using self-control strategies such as monitoring and feedback to regulate behavior, observational learning (the acquisition of behaviors by observing outcomes of others’ behavior, ideally, credible and relatable role models); and reinforcement (incentives, rewards, and feedback) to increase the likelihood of a behavior.^16,17 The program consisted of 1) a WIC-based farmers’ market to improve community access to vegetables, and among those purchasing items at the market, home vegetable availability; 2) behaviorally focused instruction to enhance social support for vegetable consumption and build vegetable-related knowledge, skills, and self-efficacy; 3) field trips to an area farmers’ market to further improve vegetable access, provide opportunities for experiential and hands-on learning, and enable participants to apply knowledge and skills learned at the WIC-based market to a real-world setting; 4) telephone coaching and support before and after trips to facilitate plans to incorporate vegetables into daily meals, 5) recipe demonstrations and tastings to build vegetable knowledge, preparation skills, and taste preferences; and 6) handouts to reinforce vegetable knowledge and preparation skills. The program logic model is shown in the figure.

Figure.

Logic model for a farm-to-WIC intervention to promote vegetable intake among urban, WIC-enrolled adults

WIC indicates Special Supplemental Nutrition Program for Women, Infants, and Children; FMNP, Farmers’ Market Nutrition Program. Adapted with permission from Liberato SC, Bailie R, Brimblecombe J. Nutrition interventions at point-of-sale to encourage healthier food purchasing: a systematic review. BMC Public Health. 2014;14: 919.

The WIC-based market was implemented at the intervention site in the summer of 2019 during the FMNP voucher issuance period. Participants presenting for appointments (during which they would receive FMNP vouchers) were directed to wait in a classroom with the market. Nutrition educators provided group-based instruction to participants while waiting for appointments, conducted recipe demonstrations and tastings (in total, 3 recipes were demonstrated), and instructed participants to return to the classroom after appointments to receive additional instruction and a recipe pack containing handouts and the ingredients for one of the recipes to try at home. Participants also learned that they had the option to purchase fruits and vegetables at the WIC-based market with their FMNP vouchers. Nutrition educators provided personalized, 1:1 instruction to participants returning to the classroom after appointments.

Thereafter, participants completed up to three field trips to the area farmers’ market (one each in September, October, and November). The same individuals who delivered nutrition education at the WIC-based market scheduled the trips and provided group-based instruction to participants during trips. At the market, the educators conducted recipe demonstrations and tastings (3 recipes per trip, differing each month), distributed recipe packs containing the ingredients for one of the recipes, and provided 1:1 instruction to participants. The educators also provided telephone coaching and support to participants before and after trips. The 5-month intervention was implemented between July 1, 2019 and November 30, 2019.

The intervention was delivered by bilingual English/Spanish-speaking nutrition educators with undergraduate degrees in nutrition, public health, and related fields, of similar race and Hispanic ethnicity and origin to the participants. The educators completed a half-day training session led by the Principal Investigator (PI) of the project. At the training, standardized protocols and intervention scripts (developed by the PI) for implementing the intervention and documenting program activities (to ensure fidelity implementation) were reviewed.¹⁸ To enhance retention and participant satisfaction with the intervention, the educators were trained in rapport-building and culturally responsive communication strategies. An MA-level supervisor (also trained in the protocols) observed the educators delivering the intervention, provided corrective feedback as needed, and alerted the PI of any problems or issues that arose. Ongoing trainings (led by the PI) were held to discuss problems and how to avoid or address them, re-review protocols and scripts, and engage educators in rehearsals of skills taught to prevent the skills from decaying.¹⁸

Process Measures

Logs completed by nutrition educators (one per participant) were used to document program activities, including problems if any, encountered in implementing the intervention. Information recorded in logs was entered into an electronic database for aggregation and analysis. Intervention components, assessment dimensions, and implementation schedules used to examine the extent to which the intervention was implemented as intended are shown in Table 1.

Table 1.

Implementation of a farm-to-Special Supplemental Nutrition Program for Women, Infants, and Children (WIC) intervention among 160 urban, WIC-enrolled adults who received the program

Component and assessment dimension	Planned implementation	Actual implementation
WIC-based marketa
Schedule of operation	Weekly (Monday-Saturday) during FMNPb voucher issuance period until targeted number participants were enrolled in study	Implemented July 1, 2019 and in operation for 4 weeks (at which point targeted enrollments were achieved)
FV deliveryc	Delivery Monday, Wednesday, and Friday mornings while market was in operation	FV were delivered as scheduled during the 4 weeks the market was in operation
FV quantity and variety	At least 15 FV per delivery [quantity]; at least 15 different FV per delivery [variety]	15 unique FV were provided on 90% of delivery days; more than 15 unique FV were provided on 40% of days
Field trips to area farmers’ market	One trip per month in September, October, and November (3 trips in total)	31 (19%) participants completed one field trip, 9 (6%) completed two field trips, and 5 (3%) completed all three field trips
Behaviorally focused nutrition education
WIC-based market
Group-based instruction	20 minutes of instruction provided to participants while waiting for appointments during which they received FMNP vouchers	160 (100%) participants received group-based instruction while waiting for WIC appointments as planned
Individualized instruction	Up to 10 minutes of instruction per participant provided to those returning to the market after WIC appointments	150 (94%) participants received individualized instruction at the market after WIC appointments as planned
Monthly trips to area farmers’ market
Group-based instruction	20 minutes of instruction per trip	All participants who completed monthly field trips received group-based and individualized instruction as planned
Individualized instruction	Up to 10 minutes per person provided to participants while shopping at the market
Recipe demonstrations and tastings	Demonstrations and tastings of three recipes each at the WIC-based market and area farmers’ market (12 recipes in total)	160 (100%) participants participated in demonstrations and tastings of 3 recipes at the WIC-based market; at the area farmers’ market, 31 (19%) participated in a second set of 3 (6 in total), 9 (6%) participated in a third set (9 in total), and 5 (3%) participants a fourth set (12 in total)
Recipe packs	Packs containing the ingredients for one of the recipes demonstrated at the WIC-based market and area farmers’ market (4 in total)	150 (94%) participants received one recipe pack at the WIC-based market; at the area farmers’ market, 31 (19%) received another pack (2 in total), 9 (6%) received a third pack (3 in total), and 5 (3%) received a fourth pack (4 in total)
Handouts	Vegetable fact sheets (11 in total, each featuring a single vegetable with tips on how to select, store, and prepare the item); recipe steps (12 in total, each depicting the steps [in words and images] for preparing recipes demonstrated in the program); vegetable seasonality chart; and map of the area farmers’ market (with contact information for farmers and information on which farmers accepted FMNP vouchers)	150 (94%) participants received all handouts distributed at the WIC-based market; at the area farmers’ market, 31 (19%), 9 (6%), and 5 (3%) participants, respectively, received all handouts distributed on the first, second, and third field trips to the market
Telephone coaching and support calls	20 minutes of coaching and support before and after field trips (4 calls in total)	46 (29%) participants completed 1 call, 43 (27%) completed 2 calls, 28 (18%) completed 3 calls, and 23 (14%) completed all 4 calls

^aWIC = Special Supplemental Nutrition Program for Women, Infants, and Children.

^bFMNP = Farmers’ Market Nutrition Program.

^cFV = fruit and vegetable.

Primary, Intermediate, and Secondary Outcome Measures

Vegetable intake was objectively measured using dermal carotenoid levels (assessed with a portable device, the Veggie Meter® [Longevity Link Inc., Salt Lake City, UT], by scanning the tip of the finger)¹⁹ and via self-report with one of two items in a brief fruit and vegetable screener developed by the National Cancer Institute (“How many cups of vegetables [including 100% vegetable juice] do you eat or drink each day?”).²⁰ To facilitate the estimation of food portions, participants were informed that a cup was about the size of their fist.²¹ FMNP voucher redemption was objectively assessed once at the end of the voucher redemption period (June 1 to November 30, 2019) using data provided by WIC. WIC reported whether participants redeemed any vouchers (yes/no) over this period. In addition to the overall voucher redemption rate, the rate at the WIC-based market was also computed using vouchers collected onsite.

Intermediate outcomes of vegetable-related knowledge, attitudes, and behaviors were assessed via self-report, as described below. The measures were orally administered (in English or Spanish, depending on the participant’s preference). As national health promotion objectives group nutrition, obesity, and physical activity together,²² physical activity and weight status were also measured and were included as secondary outcomes. Physical activity was assessed via self-report and weight status was measured as body mass index (BMI), calculated as weight in kilograms divided by height in square meters.¹³ Height and weight were measured with participants wearing light clothing without shoes using standardized methods and equipment (Seca 876 scale and Seca 213 stadiometer [Seca Corp., Chino, CA]).²³ Primary, intermediate, and secondary outcomes were assessed at baseline, mid- and post-intervention. At baseline, information was also collected from participants on the following prognostic factors (potential influences on vegetable intake): age, race, breastfeeding status, and exposure to secondhand smoke (obtained via closed-ended, multiple choice items).

Knowledge of Vegetable Intake Recommendations.

Knowledge of vegetable intake recommendations was assessed with an item from the Food Attitudes and Behaviors Survey.²⁴ Based on their responses, participants were classified as knowledgeable of the recommended cups/day of vegetables (yes/no) as per the My Plate amounts for women aged 19 to 50 years.²⁵

Home Vegetable Availability.

The presence of vegetables at home was measured with the Home Health Survey, shown to have moderate reliability and validity.²⁶ Participants responded to items querying whether they had any fresh, canned, and frozen vegetables at home (yes/no). Those responding affirmatively were asked to report which vegetables they had at home. Two measures of availability were constructed based on the responses: 1) the quantity or total number of vegetables at home, and 2) the variety or number of different items at home.

Vegetable Taste Preferences and Social Support for Vegetable Consumption.

Vegetable taste preferences were assessed with a single item, “Overall, how much do you like the taste of vegetables?” Responses were on a 7-point scale ranging from not at all (1) to a lot (7). Higher scores indicated greater taste preferences. Social support was assessed with the item, “How much would you say people in your life support you to eat vegetables?” Responses were on a 7-point scale ranging from not at all (1) to a lot (7). Higher scores indicated greater social support.

Vegetable Preparation Practices and Preparation Skills.

Vegetable preparation practices were assessed with three items from the Project EAT-II survey (Cronbach’s α = 0.80).²⁷ Participants reported how often they performed the following behaviors in the past month: 1) bought fresh vegetables, 2) prepared a green salad, and 3) prepared a dinner with vegetables. Responses were on a 5-point scale ranging from never (1) to more than once a week (5). In the present sample, Cronbach’s alpha was 0.59. Vegetable preparation skills were assessed with the following item: “How would you rate your overall skill in preparing vegetables?” Responses were on a 7-point scale ranging from poor (1) to excellent (7). Higher scores indicated greater preparation skills.

Self-Efficacy for Vegetable Consumption.

Participants completed an adapted version of a 5-item measure of self-efficacy for fruit and vegetable consumption (Cronbach’s α = 0.91).²⁸ The measure was adapted by revising items referencing fruits and vegetables to reference vegetables only (sample item: “I feel I can prepare recipes with vegetables.”).²⁸ Responses were on a 4-point scale ranging from strongly disagree (1) to strongly agree (4); higher scores indicated greater self-efficacy. In the present sample, Cronbach’s alpha was 0.83.

Physical Activity.

Physical activity was operationalized as meeting current physical activity guidelines and was assessed with a validated 2-item measure.²⁹ Participants reported the frequency and duration of moderate and vigorous intensity physical activity, respectively, in a typical week. In accordance with the scoring protocol, item responses were assigned point values and summed to derive a total score. Score ranges were used to classify participants as meeting physical activity guidelines (yes/no].²⁹

Satisfaction with the Intervention.

At the post-intervention assessment, intervention-group participants completed a single-item measure of overall satisfaction with the program and a 5-item satisfaction measure developed by the investigators (sample item: “The program was interesting;” Cronbach’s α = 0.90). Responses were on a 7-point scale ranging from very dissatisfied (1) to very satisfied (7); higher scores indicated greater satisfaction. For the composite measure, ratings were averaged across items to derive a total score. Participants also responded to open-ended items on what was liked most and least about the program, what, if anything, could be done to improve it; and the most important thing learned in the program.

Analysis

Descriptive statistics were used to summarize fidelity data. Relative to what was planned (Table 1), actual dates on which the WIC-based market operated and produce was delivered and the quantity and variety of items were examined. The numbers and percentages of participants who completed successive intervention components as intended were also examined.

Means and percentages were used to characterize baseline values of intermediate and secondary outcomes. The values were screened for a possible ceiling effect. A ceiling effect occurs when scores on a measure are concentrated at or near the possible upper limit.³⁰ Ceiling effects limit the discriminative ability of a measure to detect meaningful change and can be problematic in intervention studies because participants may continue to improve a behavior that is the focus of intervention but the measure will not detect the improvement.³¹ A ceiling effect was considered present when more than 15% of participants scored the highest possible score on a measure.^32,33 Baseline between-group differences in intermediate and secondary outcomes were examined with chi-square analyses and independent samples t tests. Associations between the measures and measures of vegetable intake were examined at baseline with Pearson correlations. The association between baseline measures of objective and self-reported vegetable intake was also examined.

Linear mixed-effects models were used to relate intermediate and secondary outcomes to time, study group, and a time-by-group interaction, while controlling for baseline values of the outcomes (one at a time). Linear mixed-effects models generalize classical analysis of repeated measures and allow for data missing at random and time-varying covariates.³⁴ Under the missing at random assumption, all participants with at least one non-missing post-baseline observation were included in the analyses. While there is no formal statistical test for the missing at random assumption, baseline values of outcomes such as the ones considered here are typically related to later values (observed for completers but not for dropouts). Controlling for baseline values in the analyses therefore provides some control for factors related to missingness to the extent that the factors are reflected in baseline measures of the outcomes. The inclusion of the random effect of the subject accounted for the nesting of repeated measures within a participant.³⁴ Fixed effects (covariates) included baseline measures of each outcome and prognostic factors (age, race, breastfeeding status, and exposure to secondhand smoke). For intermediate and secondary outcomes found to differ between groups at mid- and post-intervention, paired samples t tests were used to explore within-group changes in the variables from baseline to mid- and post-intervention.

Associations between the intervention and binary measures of knowledge of vegetable intake recommendations and meeting physical activity guidelines were examined with generalized linear mixed models with binary error distributions. Generalized linear mixed models extend linear mixed modeling to the error distributions from the exponential family (linear mixed modeling is for approximately normally distributed errors).³⁵ In addition to tests of the significance of differences between least square means by study group at each time point, 95% confidence intervals were estimated for the differences.

Mean ratings on measures of satisfaction with the intervention were computed. Scores at or above 5.0 on the 7-point scale were considered evidence of a high degree of satisfaction. Responses to open-ended items were analyzed by question. Similar responses were grouped together and assigned descriptive titles in accordance with established guidelines.³⁶ As the aim was to identify common responses to the questions, for each, the three most frequently mentioned answers are reported (in descending order of frequency of mention). Quantitative analyses were conducted with SAS statistical software, version 9.4.³⁷ Across analyses, P values < 0.05 indicated statistical significance.

RESULTS

As shown in Table 1, the WIC-based market was implemented in the first week of July and was in operation Monday through Saturday for four weeks (at which point targeted numbers of participants were enrolled). Produce was delivered as scheduled on Mondays, Wednesdays, and Fridays. At least 15 unique items were delivered on 90% of delivery days; more than 15 unique items were delivered on 40% of the days (in total, 18 unique foods were provided; for one item [squash], three varieties were provided [patty pan, yellow summer squash, and zucchini]).

Of the 160 intervention-group participants, 160 (100%) received group-based instruction and participated in recipe demonstrations and tastings at the WIC-based market; 150 (94%) returned to the market after WIC appointments and received individualized instruction, recipe packs, and handouts. Whereas 115 participants (72%) did not complete any field trips to the area farmers’ market, 31 (19%) completed one trip, 9 (6%) completed two trips, and 5 (3%) completed all three trips. All participants who completed field trips received individual and group-based instruction, participated in recipe demonstrations and tastings, and received recipe packs and handouts as planned. Whereas 20 (12%) participants did not complete any coaching calls, 46 (29%) completed 1 call, 43 (27%) completed 2 calls, 28 (18%) completed 3 calls, and 23 (14%) completed all 4 calls. Common reasons participants did not complete field trips and coaching calls (as recorded in logs) were that they could not be reached, lacked childcare and did not want to bring small children on trips, and had to work on Saturdays (the day trips were held).

In total, 134 (84%) participants in the intervention group and 51 (37%) in the control group redeemed FMNP vouchers. Whereas 100 (63%) intervention-group participants redeemed the vouchers onsite, 34 (21%) did so at area farmers’ markets. Of the 34 who redeemed the vouchers at area farmers’ markets, half (17) did so on field trips and half (17) did so on their own.

Baseline values of intermediate and secondary outcomes and Pearson correlations between the outcomes and measures of vegetable intake are shown in Table 2. Both the quantity and variety of vegetables at home and self-efficacy for vegetable consumption differed by study group (values were significantly higher in the control group). Ceiling effects were found for vegetable taste preferences, preparation practices, preparation skills, and social support for vegetable consumption (between 23% and 55% of participants achieved the highest possible scores on these measures). Vegetable taste preferences and preparation practices were positively associated with objectively measured vegetable intake (for both, r = 0.15, P = 0.01), and BMI was negatively associated with objectively measured intake (r = -0.22, P ≤ 0.001). The quantity and variety of vegetables at home, vegetable taste preferences, preparation practices, preparation skills, and self-efficacy for vegetable consumption were positively associated with self-reported vegetable intake (r ≥ 0.20; P ≤ 0.001). Objectively measured vegetable intake was not associated with self-reported intake at baseline (r = 0.05, P = 0.367).

Table 2.

Baseline differences, by study group, in intermediate and secondary outcomes, and Pearson correlations between the outcomes and measures of vegetable intake among 297 urban, WIC-enrolled adults participating in a pilot study of a farm-to-WIC intervention^a

	Mean ± standard deviation or n (%)			Pearson correlation coefficientb
Outcome	Intervention (n=160)	Control (n=137)	P valuec	ObJectively measured intaked	Self-reported intakee
Intermediate
Home vegetable availability (Quantity)f	4.4 ± 2.9	6.2 ± 3.8	≤ .001	−.07	23***
Home vegetable availability (Variety)g	4.2 ± 2.6	5.7 ± 3.3	≤ .001	−.04	21***
Vegetable taste preferencesh	5.8 ± 1.5	6.1 ± 1.4	.160	.15*	23***
Vegetable preparation practicesi	12.0 ± 2.7	12.2 ± 2.5	.524	.15*	.30***
Vegetable preparation skillsj	5.4 ± 1.5	5.4 ± 1.7	.908	.07	20***
Social support for vegetable consumptionk	5.8 ± 1.7	5.9 ± 1.6	.473	.04	.05
Self-efficacy for vegetable consumptionl	14.9 ± 2.1	16.1 ± 2.8	≤ .001	.04	24***
Knowledge of vegetable intake recommendationsm	3 (2%)	3 (2%)	.861	−.01	.02
Secondary
Body mass indexn	29.5 ± 6.8	29.4 ± 6.8	.933	−.19**	.06
Meeting physical activity guidelineso	101 (63%)	73 (53%)	.252	−.05	.11

^aWIC = Special Supplemental Nutrition Program for Women, Infants, and Children.

^bSample sizes ranged from 293 to 297.

^cP values are for tests of differences by study group (assessed with chi-square analyses and independent samples t tests).

^dAssessed using a pressure-mediated reflection spectroscopy device, the Veggie Meter® (Longevity Link Inc., Salt Lake City, UT), to assess dermal carotenoids as a biomarker of intake.¹⁹ Scans of the index finger were taken in triplicate and the average of the three scans was recorded. Scores can range from zero to 800, with higher scores indicating higher dermal carotenoid levels.

^eAssessed with one of two items in a brief fruit and vegetable screener developed by the National Cancer Institute (“How many cups of vegetables [including 100% vegetable juice] do you eat or drink each day?”).²⁰ To facilitate the estimation of food portions, participants were informed that a cup was about the size of their fist.²¹

^fScores ranged from 0 to 17; higher scores indicated greater quantities of vegetables at home.

^gScores ranged from 0 to 16; higher scores indicated greater variety of vegetables at home.

^hScores ranged from 1 to 7; higher scores indicated stronger vegetable taste preferences.

ⁱScores ranged from 3 to 15; higher scores indicated more frequent vegetable preparation practices.

^jScores ranged from 1 to 7; higher scores indicated greater vegetable preparation skills.

^kScores ranged from 1 to 7; higher scores indicated greater social support for vegetable consumption.

^lScores ranged from 6 to 20; higher scores indicated greater self-efficacy for vegetable consumption.

^mAssessed with an item from the Food Attitudes and Behaviors Survey.²⁴ Based on their responses, participants were classified as knowledgeable of the recommended cups/day of vegetables (yes/no) as per the My Plate amounts for women aged 19 to 50 years.

ⁿBody mass index (BMI) was calculated as weight in kilograms divided by height in square meters.¹³ As weight and height were measured at time of study entry, among pregnant participants, weight and height prior to pregnancy and correspondingly, prepregnancy BMI could not be determined. Pregnant women (n = 54) were therefore excluded from analyses of BMI.

^oAssessed with a validated 2-item measure.²⁹ The measure queried the frequency and duration of moderate and vigorous intensity physical activity, respectively, in a typical week. Item responses were assigned point values and summed to derive a total score. Score ranges were used to classify participants as meeting physical activity guidelines (yes/no).

^*Correlation significant at P < .05.

^**Correlation significant at P < .01.

^***Correlation significant at P < .001.

There were no differences, by group, in intermediate and secondary outcomes at mid- and post-intervention with the exception of the quantity of vegetables available at home, which was significantly higher among participants in the control group at mid-intervention, and the variety of vegetables at home, which was significantly higher in this group at mid- and post-intervention (Table 3). Exploratory analyses revealed that in the intervention group, the quantity and variety of vegetables at home did not differ from baseline to mid- and post-intervention; however, in the control group, both measures increased from baseline to mid-intervention but then returned to baseline levels thereafter (i.e., did not differ from baseline to post-intervention). In the longitudinal analysis of knowledge of vegetable intake recommendations, odds ratios at mid- and post-intervention favored prticipants in the intervention group but the confidence intervals were very wide (Table 4). There were no differences by group in the analysis of meeting physical activity guidelines.

Table 3.

Least square means and standard errors for intermediate and secondary outcomes at mid- and post-intervention, by study group, among 297 urban, WIC-enrolled adults participating in a pilot study of a farm-to-WIC intervention^a,b

	Study group, least square mean ± standard error
Outcome	Interventionc	Controld	Mean difference (95% CI)e	P
Intermediate
Home vegetable availability (Quantity)f
Mid-intervention	5.3 ± 0.3	7.1 ± 0.3	−1.7 (−2.5, −1.0)	.001
Post-intervention	5.6 ± 0.3	6.3 ± 0.3	−0.7 (−1.4, 0.1)	.082
Home vegetable availability (Variety)g
Mid-intervention	5.0 ± 0.3	6.5 ± 0.3	−1.5 (−2.2, −0.8)	.001
Post-intervention	5.2 ± 0.3	5.9 ± 0.3	−0.7 (−1.4, −0.0)	.039
Vegetable taste preferencesh
Mid-intervention	6.3 ± 0.1	6.3 ± 0.1	−0.0 (−0.3, 0.3)	.962
Post-intervention	6.3 ± 0.1	6.5 ± 0.1	−0.2 (−0.4, 0.1)	.277
Vegetable preparation practicesi
Mid-intervention	12.6 ± 0.2	13.0 ± 0.2	−0.4 (−0.1, 0.2)	.155
Post-intervention	12.5 ± 0.2	12.4 ± 0.2	0.1 (−0.5, 0.7)	.740
Vegetable preparation skillsj
Mid-intervention	5.7 ± 0.1	6.0 ± 0.1	−0.2 (−0.5, 0.1)	.122
Post-intervention	5.6 ± 0.1	5.8 ± 0.1	−0.2 (−0.5, 0.1)	.177
Social support for vegetable consumptionk
Mid-intervention	5.9 ± 0.2	5.8 ± 0.2	0.1 (−0.3, 0.5)	.514
Post-intervention	6.0 ± 0.2	5.9 ± 0.2	0.2 (−0.2, 0.6)	.335
Self-efficacy for vegetable consumptionl
Mid-intervention	15.4 ± 0.2	15.3 ± 0.2	0.1 (−0.4, 0.6)	.750
Post-intervention	15.8 ± 0.2	16.0 ± 0.2	−0.2 (−0.7, 0.3)	.455
Secondary
Body mass indexm
Mid-intervention	30.3 ± 0.4	29.4 ± 0.4	0.9 (−0.0, 1.8)	.053
Post-intervention	29.9 ± 0.4	29.9 ± 0.4	0.0 (−0.9, 0.9)	.986

^aWIC = Special Supplemental Nutrition Program for Women, Infants, and Children.

^bData were analyzed with linear mixed-effects models. Covariates included baseline measures of the intermediate and secondary outcomes and prognostic factors (age, race, breastfeeding status, and exposure to secondhand smoke).

^cSample sizes ranged from 159 to 160.

^dSample sizes ranged from 132 to 135.

^eCI = confidence interval.

^fScores ranged from 0 to 17; higher scores indicated greater quantities of vegetables at home.

^gScores ranged from 0 to 16; higher scores indicated greater variety of vegetables at home.

^hScores ranged from 1 to 7; higher scores indicated stronger vegetable taste preferences.

ⁱScores ranged from 3 to 15; higher scores indicated more frequent vegetable preparation practices.

^jScores ranged from 1 to 7; higher scores indicated greater vegetable preparation skills.

^kScores ranged from 1 to 7; higher scores indicated greater social support for vegetable consumption.

^lScores ranged from 6 to 20; higher scores indicated greater self-efficacy for vegetable consumption.

^mBody mass index (BMI) was calculated as weight in kilograms divided by height in square meters.¹³ As weight and height were measured at time of study entry, among pregnant participants, weight and height prior to pregnancy and correspondingly, prepregnancy BMI could not be determined. Pregnant women (n = 54) were therefore excluded from analyses of BMI.

Table 4.

Odds ratios and 95% confidence intervals for binary intermediate and secondary outcomes at mid- and post-intervention, by study group, among 297 urban, WIC-enrolled adults participating in a pilot study of a farm-to-WIC intervention^a,b

	Study group, percent
Outcome	Interventionc	Controld	Odds ratio (95% CI)e	P
Intermediate
Knowledge of vegetable intake recommendationsf
Mid-intervention	7%	3%	3.9 (1.0, 16.1)	.056
Post-intervention	7%	3%	4.0 (1.0, 16.3)	.050
Secondary
Meeting physical activity guidelinesg
Mid-intervention	66%	64%	1.1 (0.6, 2.2)	.810
Post-intervention	68%	70%	0.8 (0.4, 1.7)	.591

^aWIC = Special Supplemental Nutrition Program for Women, Infants, and Children.

^bData were analyzed with generalized linear mixed models. Covariates included baseline measures of intermediate and secondary outcomes and prognostic factors (age, race, breastfeeding status, and exposure to secondhand smoke).

^cSample sizes ranged from 158 to 160.

^dSample sizes were 135.

^eCI = confidence interval.

^fAssessed with an item from the Food Attitudes and Behaviors Survey.²⁴ Based on their responses, participants were classified as knowledgeable of the recommended cups/day of vegetables (yes/no) as per the My Plate amounts for women aged 19 to 50 years.²⁵

^gAssessed with a validated 2-item measure.²⁹ The measure queried the frequency and duration of moderate and vigorous intensity physical activity, respectively, in a typical week. Item responses were assigned point values and summed to derive a total score. Score ranges were used to classify participants as meeting physical activity guidelines (yes/no).

In total, 128 (80%) intervention-group participants completed post-intervention measures of satisfaction with the intervention and provided feedback on their reactions to the program. Mean ratings of satisfaction were 6.8 ± 0.6 on the single-item measure and 6.8 ± 0.5 on the composite measure. The ratings were above the a priori criterion establishing a high degree of satisfaction.

In response to the item querying what was liked most about the intervention, frequently reported answers were recipe demonstrations and tastings (reported by 45 or 35% of participants), learning about vegetables (reported by 22 or 17% of participants), trips to the area farmers’ market (reported by 21 or 16% of participants), and the rapport with project staff (also reported by 21 or 16% of participants). When asked what was liked least about the program, participants frequently mentioned nothing (reported by 95 or 74% of participants), the number of calls from project staff (reported by 10 or 8% of participants), and the limited schedule of visits to the area farmers’ market (reported by 6 or 5% of participants). When asked what could be done to improve the program, frequently reported answers were nothing (reported by 79 or 62% of participants), add more days and times for trips to the area farmers’ market (reported by 14 or 11% of participants), not sure (reported by 9 or 7% of participants), and make the program more permanent (also reported by 9 or 7% of participants). When asked about the most important thing learned, participants frequently mentioned how to prepare vegetables (reported by 28 or 22% of participants), the amount of vegetables I should have each day (reported by 23 or 18% of participants), and to eat more vegetables (reported by 14 or 11% of participants).

DISCUSSION

At the WIC-based farmers’ market in this study, ≥ 94% of participants received the planned components and dose of intervention; fewer completed one or more field trips to the area farmers’ market (28%) and telephone coaching and support calls (88%). Although most intermediate and secondary outcomes were associated with vegetable intake at baseline, post-intervention, the outcomes did not differ by study group, and satisfaction with the intervention was high.

The high implementation fidelity found at the WIC-based market was not surprising as this was an opportune time for intervention. Participants were already onsite and had time to spare while waiting for appointments. Directing participants to wait for appointments in the classroom with the market provided a window of opportunity to deliver group-based instruction and generated interest in the market. Further, having participants return to the classroom after appointments afforded opportunities for personalized, 1:1 instruction. A lesson learned was that fidelity can be enhanced by integrating intervention activities into routine clinic visits and structuring activities to fit within participants’ schedules.

As found elsewhere, participants were a transient group and were therefore difficult to reach; phone number and address changes were common, numbers that were in service at time of recruitment were disconnected by the time of the first coaching call and field trip 4 and 8 weeks later, respectively, and some participants traveled to their homelands for weeks at a time during the course of intervention.³⁸ The area farmers’ market was open on weekends only; for this reason, field trips were held on Saturdays. Anecdotally, work schedule conflicts arose because many employed participants worked in service-oriented jobs involving weekend shifts.³⁹ Although field trips to other area markets were considered, the markets were small and operated on limited weekday schedules, limiting the ability to reach more participants with trips. With low-income adults, the need for childcare is often a barrier to research and program participation as found in this study.⁴⁰ As this was the first time trips were offered, not knowing what to expect combined with a lack of referrals may also have affected participation.

Participation rates varied for the different intervention components, highlighting the need to identify effective strategies to address participation challenges in future research. Providing small monetary incentives ($2 to $5) has been shown to improve survey response rates;⁴¹ program planners should therefore examine whether providing modest incentives would also improve the completion of coaching and support calls. Providing participants with phones or phone minutes is also suggested to improve the ability to reach them.⁴² Warranting examination is the utility of more upstream approaches or attempts to address problem behaviors at their source,⁴³ for example, intervention with farmers to expand market days and hours of operation. Expanded hours of operation would improve access to markets and enable more days and times for trips, e.g., when children are in school, obviating the need for childcare. Operating the WIC-based market for the duration of the local growing season (June to November) also warrants consideration, as this would extend the benefits of the market (regular access to fruits and vegetables in a convenient and familiar setting where children are welcome).

Consistent with the underlying logic model, at baseline, all but two of the intermediate outcomes (social support for vegetable consumption and knowledge of vegetable intake recommendations) were significantly related to objectively measured or self-reported vegetable intake. Two (vegetable taste preferences and preparation practices) were significantly associated with both measures. The consistency of these findings suggests that these variables may be important to target in other similar such interventions. Although statistically significant, the magnitude of the correlations was modest.⁴⁴ The higher values on measures of home vegetable availability at mid- and post-intervention favoring participants in the control group and the absence of post-intervention between-group differences in self-efficacy for vegetable consumption were unexpected. There were large between-group differences in the measures at baseline, making these findings difficult to interpret. To minimize the potential for such differences to influence study findings, analyses were adjusted for baseline measures of the variables.⁴⁵ After baseline, the quantity and variety of vegetables at home improved in the control group but stayed the same in the intervention group and remained higher in the control relative to the intervention group at mid- and post-intervention even after adjustment for baseline values.

In previous work, skin carotenoid status as measured by the Veggie Meter was positively associated with intake of vegetables found to be significant sources of carotenoids.⁴⁶ Moreover, Veggie Meter scores were unrelated to self-reported fruit and vegetable intake as measured by a brief screener.⁴⁷ Screening measures are designed for rapid assessment of total (vs. carotenoid-rich) vegetable intake,⁴⁷ possibly explaining why in this study, measures of objective and self-reported vegetable intake were unrelated at baseline. The lack of association may explain the different patterns of association at baseline between the intake measures and measures of intermediate and secondary outcomes. In future studies of this type, the use of alternative approaches to comprehensively assess self-reported vegetable intake is recommended, e.g., 24-hour recalls and food frequency questionnaires, which allow for the assessment of specific foods.⁴⁸ When using the food frequency approach, including high-carotenoid items, e.g., red bell peppers, tomatoes, dark-green vegetables, and such orange vegetables as sweet potatoes and carrots is suggested.⁴⁹

Baseline analyses also revealed the presence of ceiling effects for vegetable taste preferences, preparation practices, and preparation skills; as such, participants had little room to improve their scores. This likely explains why post-intervention values on the measures did not differ by group. Although the investigators pretested study measures, the aim was to examine the clarity and interpretability of items and response formats. Examining response patterns is also recommended to identify ceiling effects early. If found for rating scale measures as used in this study, the number of possible responses can be expanded.³⁰ Alternatively, an extreme anchor stimulus can be used. This involves asking participants to rate a stimulus requiring them to use extreme ends of the distribution before presenting the items of interest. Doing so will encourage respondents to use the middle range of the rating scale, thereby decreasing the likelihood of a ceiling effect.³⁰ With these lessons learned, the modified measures can be formally tested as mediators and secondary outcomes in future large-scale trials not limited by a small number of sites as in this pilot.

The voucher redemption rate at the WIC-based farmers’ market was three times the rate at the area farmers’ market. A lesson learned was that implementing an onsite farmers’ market is a promising approach for improving FMNP voucher redemption. Considering that the relationship between the intervention and vegetable intake was moderated by FMNP voucher redemption, with larger post-intervention between-group differences in intake found among participants who redeemed FMNP vouchers relative to those who did not, a further lesson learned was the importance of targeting FMNP voucher redemption in other similar such interventions as a strategy to promote vegetable intake in the WIC population.

The uniformly high satisfaction ratings indicate that the program was well received by participants. Participant feedback affirmed the role of formative research in planning interventions and the importance of designing programs to be responsive to the cultural values of the groups for whom they are intended.⁵⁰ The intervention built upon research with adults served by the collaborating WIC agency demonstrating that basic knowledge of local vegetables was low and requests for vegetable recipes were common.^4,51 This likely explains why in this study, the information provided about vegetables, recipe demonstrations and tastings, and field trips to the area farmers’ market were liked most about the program. Guidelines on culturally sensitive intervention approaches with Latinos emphasize the value of personalismo, the preference for personalized interactions, and the importance of getting to know participants through la platica (small talk), factors shown to influence satisfaction with the helping relationship.^52,53 Nutrition educators provided 1:1 instruction, were trained in culturally sensitive communication strategies including rapport building, and worked with the same participants throughout the study. Their personalized approach clearly resonated with participants as evidenced by their endorsements of the rapport with staff. An encouraging finding was that “to eat more vegetables,” a key program message, was among the most important things participants reported learning in the program.

Participants’ dislike of the number of calls from staff raises questions regarding how best to achieve the dual goals of completing all planned contacts and sustaining participants’ interest in continuing with an intervention. Although setting a limit on the number of attempts made to reach participants was considered, the investigators were unclear as to what the limit should be. In low-income samples, the number of attempts reported to contact participants varies considerably,^54,55 highlighting the need for research to identify the number of attempts found effective in reaching participants as well as the number considered acceptable to participants.

Limitations and Strengths

Nutrition educators self-reported fidelity data; as such, the data may have been influenced by social desirability and recall biases.⁵⁶ Twenty percent of participants did not complete post-intervention measures of satisfaction with the intervention. Possibly, the responses of the 80% who did respond were not representative of the 20% who did not. Findings from the analysis of intermediate and secondary outcomes should be interpreted with caution given limitations of the study design. The small number of sites, the single site in the intervention group, and the unbalanced design limit the extent to which findings can be causally attributed to the intervention despite the randomized design. Study strengths include the use of both quantitative and qualitative process data, allowing for a more thorough examination of program fidelity and participant satisfaction with the intervention,⁵⁶ and the high (80%) response rate among participants reporting their reactions to the program. The relatively new device for measuring dermal carotenoids is a novel feature of the study.

CONCLUSIONS

In this study, implementation fidelity was high for components of a farm-to-WIC intervention delivered onsite at WIC, suggesting the promise of integrating intervention activities into the WIC setting. The low participation rates for field trips to an area farmers’ market and telephone coaching and support calls highlight the need to identify strategies for strengthening implementation of the components. Although most intermediate and secondary outcomes were associated with measures of vegetable intake at baseline, the means of these variables did not differ between study groups after baseline, possibly owing to design limitations (unbalanced design and small number of study sites) and ceiling effects for some of the measures. Replication studies with stronger designs and early pretesting of measures are therefore needed to confirm these findings and to rigorously test the intervention and its mechanisms of action. Although adding days and times for trips to the area farmers’ market was suggested to improve the program, limited market days and hours of operation limited the ability to do so, underscoring the need for broad-scale initiatives to expand the number and hours of operation of farmers’ markets in underserved communities.

Research Question

Was a farm-to-WIC intervention to promote vegetable intake implemented as intended, and post-intervention, were such intermediate outcomes as home vegetable availability and taste preferences and secondary outcomes, e.g., weight status, higher among participants who received the intervention relative to those who did not? Were recipients satisfied with the intervention?

Key Findings

Whereas nearly all participants (≥ 94%) received the intervention as intended at a WIC-based farmers’ market introduced in the study, smaller percentages completed planned field trips to an area farmers’ market and telephone coaching and support calls. Post-intervention, between-group differences in intermediate and secondary outcomes favoring the intervention group were not found. Quantitative and qualitative data provided by participants supported satisfaction with the program.