Making community-supported agriculture accessible to low-income families: findings from the Farm Fresh Foods for Healthy Kids process evaluation

Abstract

A randomized trial of Farm Fresh Foods for Healthy Kids (F3HK) was initiated across 4 states and 12 farms to test whether cost-offset community-supported agriculture (CO-CSA) could improve diet quality among children in low-income families. Intervention households purchased a 50% subsidized share of local produce and were invited to nine complimentary nutrition classes.

The purpose of this study was to assess F3HK reach, dose, and fidelity via a mixed methods process evaluation. Screening and enrollment records indicated reach; study records and postlesson educator surveys tracked dose delivered; CSA pickup logs, lesson sign-in sheets, postseason participant surveys, and postlesson caregiver surveys assessed dose received; and coordinator audits and educator surveys tracked fidelity. Educator interviews contextualized findings. The results of this study were as follows. Reach: enrolled caregivers (n = 305) were older (p = .005) than eligible nonenrollees (n = 243) and more likely to be female (p < .001). Dose: mean CSA season was 21 weeks (interquartile range [IQR]: 19–23). Median CSA pickup was 88% of the weeks (IQR: 40–100). All sites offered each class at least once. Most adults (77%) and children (54%) attended at least one class; few attended all. Eighty-two percent of caregivers indicated that their household consumed most or all produce. Median lesson activity ratings were 5/5 (“very useful”). Fidelity: CSA locations functioned with integrity to project standards. Educators taught 92% of activities but frequently modified lesson order. This study demonstrates the feasibility of pairing a CO-CSA intervention with nutrition education across geographically dispersed sites. Greater integration of intervention elements and clearer allowance for site-level modifications, particularly for educational elements, may improve intervention dose and, ultimately, impact.

Implications.

Practice: Curricula that emphasize local foods should be designed to address the (greater-than-expected) range of knowledge, attitudes, skills, and behaviors of those who self-select to participate.

Policy: Cost-offset community-supported agriculture (CO-CSA) can be feasibly integrated with a skill-building, health-promoting curriculum, but appeal may be limited to a unique subset of low-income households.

Research: Future studies of multimodal food system interventions should explore whether such models can be implemented with greater flexibility (e.g., varying CSA pickup frequency) and allowance for adaptation (e.g., site-tailored educational content) without compromising participant outcomes and farmer viability.

Background

Community-supported agriculture (CSA) has emerged as a strategy for linking community members directly to their local food system [1]. A growing body of evidence supports the potential of CSA to improve healthy eating habits and produce consumption [1–4]. In a standard CSA program, consumers, also called members or shareholders, pay upfront to receive a share of a local farm’s harvest throughout the growing season. While there is a broader debate regarding the exact definition of “local,” no such definition is typically employed in this model given that potential customers can self-select into a CSA based on their perceptions of the CSA pickup location’s accessibility for their household. CSA membership typically includes households with higher levels of education and income [5]. Less participation in CSA by lower-income households has been attributed, in part, to share cost and upfront payment [6–8]. Cost-offsets—or subsidies—have emerged as a way to improve CSA share affordability [9, 10].

Despite the potential of subsidized CSAs to facilitate improved diet quality in low-income households, there is a paucity of large, rigorously designed studies of the implementation and effectiveness of such programs. Seguin and colleagues designed and tested Farm Fresh Foods for Healthy Kids (F3HK) a cost-offset CSA (CO-CSA) intervention, via a multistate, two-arm randomized controlled trial to address this gap [11, 12]. F3HK is an intervention that provides low-income families with children in New York, Vermont, North Carolina, and Washington state the opportunity to purchase a 50% subsidized CSA summer share (mostly vegetables and some fruit); to pay for their portion of the share weekly (vs. at the beginning of the season) and with their Supplemental Nutrition Assistance Program (SNAP) benefits, if applicable and desired; to receive two to four pieces of free cooking equipment of their choice (from a range of options); and to attend nine CSA-tailored healthy eating classes taught by a community health educator (or to receive the corresponding educational materials for at-home use when attendance was not possible). For those opting to pay for their portion of the share via SNAP benefits, study staff worked with participating farmers to take those payments not more than 14 days in advance of the CSA pickup per U.S. Department of Agriculture policy.

Conducting process evaluations as a part of intervention evaluation provides important data about implementation and specific factors that may impact intervention effectiveness and replicability [13, 14]. Process evaluations may be particularly useful for multistate trials, such as that which was done to test F3HK, given the potential for variable reach, dose, and fidelity across research sites [15]. The purpose of this study was to conduct a process evaluation of F3HK and, more specifically, to evaluate reach, dose delivered, dose received, and fidelity of F3HK from the perspective of caregivers, participating farms, nutrition educators, and intervention coordinators.

Methods

The study protocol and all procedures were reviewed and approved by the Cornell University (protocol ID #1501005266) and University of Vermont (#16393) institutional review boards.

Design

The F3HK study used a longitudinal crossover design with a 1:1 random assignment of participants after baseline data collection (more detail available here [11]). The two components of the intervention—subsidized CSA and CSA-tailored healthy eating classes—were distinct; enrollees were encouraged, but not required, to attend the classes; and the classes were not held in conjunction with CSA pickup at most sites. The CSA-tailored classes followed a study-specific curriculum designed by the first author, a registered dietitian, in consultation with extension educators and researchers. Each class had a theme (e.g., “produce preparation basics” and “storing & preserving produce”) that aligned with the study aims and featured CSA produce in tastings, demonstrations, or hands-on cooking activities. All study educators were provided with an educator guide that dictated lesson spacing and order (to frontload key skills) and included detailed schedules and facilitator notes. A multitude of recipe ideas was provided for each lesson to accommodate differing agricultural calendars and unpredictable farming seasons, and both educators and enrollees were provided a study cookbook from which alternate recipes could be selected. All educators were trained annually.

This manuscript reports on the F3HK process evaluation, which used mixed methods to characterize both intervention components across four constructs commonly assessed for the implementation of health promotion programs: reach, dose delivered, dose received, and fidelity [16]. This study reports findings from data collected in Years 1 (2016) and 2 (2017) of F3HK implementation during which both quantitative and qualitative data were collected from enrolled caregivers, participating farms, educators, and state coordinators.

Sample

In spring 2016 and 2017, study personnel recruited caregivers for F3HK from communities across the four states; across those states and communities, 12 partner CSA farms were engaged to participate in the study. To be eligible, community members had to be 18 years of age or older; English speaking; the parent or legal guardian (i.e., caregiver) of a child between the ages of 2 and 12 years; have a self-reported household income ≤185% of the Federal Poverty Level; not a CSA participant for at least 3 years; willing to pay weekly for a 50% subsidized CSA share (using SNAP benefits or other means of payment); have access to the internet and an active email address; and express willingness to attend the CSA-tailored healthy eating classes in their community. Though not stated explicitly, caregivers also had to have the ability to pick up the weekly CSA share, as there was no delivery option and most sites were not located within close proximity of public transportation. Interested caregivers were made aware of their site’s pickup location(s) before enrolling. Interested and eligible adults enrolled as a dyad with one of their children between the ages of 2 and 12 years. All participating farms, educators, and state study coordinators also contributed data. Eighty percent of educators were affiliated with cooperative extension and were, thus, able to compare their experience teaching the F3HK curriculum to other experiences working with low-income households.

Data sources

This analysis relied upon nine sources of data: eligibility screening data, CSA pickup logs, lesson sign-in sheets, postlesson caregiver surveys, postlesson educator surveys, state coordinator CSA and education quality assurance (QA) audits, postseason educator interviews, and season-end intervention group surveys. Postseason caregiver focus groups were also conducted and are reported on elsewhere, with some findings referenced in this paper’s discussion to contextualize presented data [17].

Measures

Eligibility screeners captured caregiver age, sex, race, ethnicity, and whether or not the caregiver’s household had participated in SNAP, the Special Supplemental Nutrition Program for Women, Infants, and Children (WIC), or Head Start within the prior month. Participating farms maintained CSA pickup logs, which were used to record whether each participant picked up their produce share each week.

Educators maintained lesson sign-in sheets, which captured class attendance, including the names of any enrolled caregivers present, names of any accompanying adults, and the names and ages of any accompanying children. Educators also administered simple, two-page postlesson surveys, which measured caregiver’s perceptions as to the usefulness of lesson content, featured recipes, and featured physical activities on a five-point scale (from very useless to very useful); whether caregivers intended to try any featured recipes or activities (yes or no); and open-ended lesson feedback. Educators adhered a sticker with the attendee’s identification number to each paper questionnaire so that data sources could be linked. Surveys were original to the study and designed by the interdisciplinary study team, including a subset of study educators, to reflect the specific content, recipes, and activities planned for each lesson. A five-point scale was used to balance response option burden and nuance and to avoid forcing a positive or negative response (i.e., allowing neutrality via a score of 3 out of 5). Surveys were edited slightly after the first couple of lessons to reduce response error; labels for any scale questions were bolded to minimize confusion about the scale’s direction.

Postlesson educator surveys were also original to the study and were administered online via Qualtrics (Provo, UT). All educators received a link to the survey at the start of the CSA season and were asked to complete the survey as soon as possible after each lesson. Study staff maintained a current record of completed lessons and followed up with educators, as needed, to ensure survey completion. Surveys captured class date, location, and duration and completion of planned activities (yes or no). Five-point scales were used to measure educators’ perceptions regarding receptivity to each activity (from received very poorly to very well received); difficulty of facilitating each activity (from very difficult to very easy); and difficulty of facilitating each activity with children present (from very difficult to very easy). Surveys asked whether any recipes were featured (yes or no); the names of any recipes; and how each recipe was featured (tasting, demonstration, or group preparation). Five-point scales were also used to measure perceptions regarding difficulty of featuring the recipe (from very difficult to very easy); receptivity of the featured recipe (from received very poorly to very well received); whether the recipe used the produce available in that week’s CSA share (yes or no) and why or why not; whether any physical activities were featured (yes or no) and why or why not; whether any attendees refused to participate in any aspect of the lesson (yes or no) and why or why not; and overall relevance of the lesson to attendee needs (from not very relevant to very relevant). Surveys also asked whether there were any child behavioral issues during the lesson and invited open-ended feedback on lesson design.

State coordinator CSA QA audits used a simple form to capture audit date and location; the types and sizes of CSA shares assessed; a summary of share contents across all available share sizes (e.g., types and amounts of produce provided); and whether (yes or no) the produce was generally of acceptable quality (i.e., not bruised, old looking, overripe in smell or appearance, or spotted), share pickup was available at the advertised time, share pickup was organized well, share pickup area was clean, share pickup area seemed safe, a farm representative was available for questions during pickup, participants were treated respectfully by any available staff, the Electronic Benefit Transfer (EBT) machine for SNAP benefit use was functional, the produce was labeled thoroughly (if a free-choice CSA), and the produce was arranged attractively (if a free-choice CSA). Coordinators also had open-ended opportunities to explain their responses and provide additional comments. Coordinators were trained by the team’s process evaluation coordinator on how to use the form, with an opportunity to ask questions and seek clarity, as needed. Coordinators were asked to audit each CSA pickup location twice per program season, once toward the season’s start and again near its end.

State coordinator education QA audit forms mirrored the content of the postlesson educator surveys and also included eight questions that asked coordinators to evaluate educators on their preparedness, adaptability, knowledge, facilitation of group cohesion, respectfulness, promotion of a safe and trusting environment, promotion of content immediacy (i.e., usefulness), and overall engagement on a five-point scale (from low to high). Coordinators were asked to audit each education location twice per program season, once toward season the start and again near its end.

Following F3HK implementation, study personnel sent an online survey to all enrolled caregivers and invited each F3HK educator (n = 10) to participate in a single, in-depth, phone-based, semistructured interview. To capture CSA utilization data from the entire sample, caregivers were asked to estimate, on average, how they used their CSA shares across the entire season (i.e., how much was consumed, preserved, spoiled, or given away) on a seven-point scale (from none to all). Educator interviews lasted 1–2 h and explored barriers and successes experienced during the implementation of the curriculum and factors perceived by educators as impacting class attendance.

Analyses

Data were analyzed across four constructs: reach, dose delivered, dose received, and fidelity. “Reach” was defined as the degree to which the intervention was taken up by the target demographic. It was assessed by comparing the characteristics of adults who were screened and deemed eligible for F3HK by enrollment status (enrolled or not). “Dose delivered” was defined as the delivery of planned intervention components. It was assessed by characterizing the duration, cost, and composition of the CSA offering and determining whether healthy eating classes were scheduled across sites as planned. “Dose received” was defined as enrollees’ exposure to and acceptance and utilization of intervention components. It was assessed by analyzing the percentage of total weeks that enrolled households purchased the CSA share; utilization of CSA contents; attendance at the healthy eating classes; and caregiver perceptions of lesson utility. “Fidelity” was defined as the degree to which implementation of F3HK met project standards and, for the education classes, was conducted according to the project curriculum. It was assessed by analyzing coordinator-reported CSA produce quality, CSA pickup functionality, and educator teaching techniques, as well as by educator-reported adherence to the F3HK curriculum. A summary of the data collected for the three primary constructs—dose delivered, dose received, and fidelity—is provided in Table 1. Because the definition, measure, source, and collection of “reach” data did not differ across intervention components, it is not included in the table. Postseason educator interview data were used to contextualize findings across all constructs.

Table 1.

F3HK process evaluation data definitions, measures, sources, and collectors by intervention component

	Dose delivered	Dose received	Fidelity
CO-CSA SHARE
Definition	Duration, content, and cost of share	Receipt and use of share	Produce quality, pickup site functionality
Measure	Mean CSA season (in weeks), share size (# of items), and cost ($) per week	% of shares picked up % of CSA share: Consumed Preserved Given away Spoiled	% met standards for: High quality and labeled Pickup on-time and well organized Site clean and safe Staff available and respectful SNAP/EBT working
Data source	Study records	Pickup logs; postseason caregiver surveys	State coordinator QA audits
Data collector	Farms/state coordinators	Farms; participants	State coordinators
HEALTHY EATING CLASSES
Definition	Lesson provision	Lesson attendance and perceived utility	Alignment with educator guide
Measure	% of lessons held as planned	% of lessons attended; lesson ratings (out of 5); % intending to use info	% adherence to written curriculum; educator attributes
Data source	Postlesson educator surveys	Lesson sign-in sheets; postlesson caregiver surveys	Postlesson educator surveys; state coordinator QA audits
Data collector	Educators	Educators; participants	Educators; state coordinators

CO-CSA cost-offset community-supported agriculture; EBT Electronic Benefit Transfer; QA quality assurance; SNAP Supplemental Nutrition Assistance Program.

Sociodemographic variables collected via the study screener and baseline survey were compared between enrollees and eligible nonenrollees using t-tests and chi-square analysis. Quantitative data from pickup logs, sign-in sheets, all surveys, and coordinator audits were entered into Microsoft Excel 2016 and SPSS Statistics (version 25.0, IBM Corp., Armonk, NY) and analyzed using descriptive statistics. For nonnormally distributed variables, medians were reported. Data from 2016 and 2017 were pooled for most analyses; for class attendance trend data, though, data were analyzed and presented separately to provide a quantitative perspective on anecdotal reports from educators regarding differences across intervention years. To limit bias introduced by missing data, CSA pickup rates were calculated only for individuals for whom a preponderance (i.e., 50% or more) of weekly pickup data were available. A sensitivity analysis was completed to understand how CSA pickup rates would have differed if all missing data represented weeks in which the CSA was not picked up. When data were missing for an entire farm, we did not make this assumption. Educator interviews were recorded, transcribed verbatim, and uploaded into NVivo 12 (QSR International, Melbourne, Australia) for coding of key themes via template analysis, a process by which both a priori and emergent codes are used to organize and analyze data [18].

Results

Reach

A total of 685 caregivers were screened for trial eligibility; 548 (80.0%) were deemed eligible and 305 (55.7%) of those eligible enrolled in the study. Enrollees and eligible nonenrollees differed on some sociodemographic characteristics (Table 2). Enrolled caregivers were older (p = .005) and more likely to be female (p < .001). Fewer enrolled households had used Head Start within the month prior to screening (p = .015).

Table 2.

Reach: sociodemographic characteristics of eligible caregivers by enrollment status

Characteristic	Not enrolled (n = 243)	Enrolleda (n = 305)	p
Age, mean (SD)	34.1 (8.2)	36.1 (8.0)	.005
Sex, n (%)			<.001
Female	212 (87.2)	297 (97.4)
Male	31 (12.8)	8 (2.6)
Race, n (%)c			.868b
American Indian/Alaskan Native	2 (0.8)	4 (1.3)
Native Hawaiian/Pacific Islander	2 (0.8)	4 (1.3)
Black/African American	32 (13.5)	42 (13.8)
White	182 (76.8)	232 (76.1)
Multiracial	10 (4.2)	16 (5.2)
Not one of the above	9 (3.8)	7 (2.3)
Hispanic/Latino, n (%)c			.055
Hispanic	26 (10.8)	19 (6.2)
Non-Hispanic	214 (89.2)	286 (93.8)
Program participation, n (%)
SNAP received in past 12 months	152 (62.6)	173 (56.7)	.168
WIC received in past 12 months	105 (43.2)	127 (41.6)	.712
Head Start used in past 12 months	43 (17.7)	32 (10.5)	.015

All data drawn from study screener unless otherwise noted.

^aData drawn from both baseline survey and screener survey to eliminate missing data.

^bLikelihood ratio test used because over 20% of cells had expected counts less than 5.

^cSample sizes vary by up to 6 due to missing data.

Dose delivered—CSA share duration, content, and cost

The mean CSA season was 21 weeks (interquartile range [IQR]: 19–23 weeks), and share sizes typically included 7.5 items (IQR: 7–9.5) for which participants paid $13/week (IQR: 10.42–13.50).

Dose delivered—lesson provision

All sites offered each of the nine healthy eating classes at least once, as planned. Some classes served participants from two proximate farms and some sites combined efforts for field trips or make-up classes. Only one class across both years was canceled in advance (due to weather).

Dose received—share receipt and use

Participants picked up shares a median of 88% of weeks (IQR: 40–100; n = 125) or 84% of weeks (IQR: 26–100; n = 134) depending upon how missing data were treated. More than a third of participants (39%) picked up all available shares, while just 5% opted to pick up none. Most caregivers (82%) reported using most, almost all, or all of their CSA produce across the entire season (n = 115 postseason surveys). They also reported that they preserved “no” (17%) or “a little” CSA produce (56%); had “no” (12%) or “a little” CSA produce spoil (81%); and gifted “no” (31%) or “a little” CSA produce to friends and family (64%; n = 87 postseason surveys).

Dose received—lesson attendance and utility

Sixty-seven percent of caregivers and 54% of children attended one or more lessons (Fig. 1), although few caregivers (30%) and children (20%) attended the majority of lessons (i.e., five or more). Attendance was better among enrolled caregivers than children; greater in 2016 than 2017; and declined over the course of the CSA season (Fig. 2). F3HK participants were allowed to bring other household or family members as a means to facilitate household-wide capacity building and to boost class size. In 2016, attendance averaged eight persons, including a mean of four adults and four children (including three guests). Of 104 lessons scheduled in 2016, there were 3 (3%) for which no one showed. In 2017, attendance declined to six persons, including a mean of three adults and three children (including two guests). Of 116 scheduled lessons in 2017, no one showed for 9 (8%).

Fig 1.

F3HK healthy eating class attendance frequency.

Fig 2.

F3HK healthy eating class attendance across the community-supported agriculture season.

During the postseason interviews, educators expressed frustration with low attendance, which made discussions difficult to facilitate and seemed to hinder group cohesion. Educators identified three primary factors that they perceived as impacting attendance: (a) friends and family; (b) widely diverse levels of baseline nutrition knowledge, attitudes, and behaviors; and (c) schedules.

First, educators noted various ways in which family or friendships may have influenced attendance. Educators postulated that attendance was bolstered by friendships formed between caregivers and between children; the enthusiasm expressed by older children engaged in the lessons, especially, food preparation; and the active participation of enrollees’ guests:

They [caregivers] were actually becoming friends outside of the class as well. And their kids were friends as well… they were so excited to see each other every week. (NY)

The kids would come back every week and be like “what are we gonna cook today, what could we help with today?” …whether the mom’s feeling like coming or not, the kids are excited… I think that can have positive repercussions as far as making sure that everybody’s coming every week. (WA)

We did have some participants who would bring grandmothers or sisters or things like that to the program as well. And it was really people who either lived with their family or they and their kids interacted with a lot. And it seemed like those people were more likely to come to all the sessions. (NC)

Conversely, children—especially, those too young to participate in lesson activities—were seen as a possible deterrent to class attendance:

I think there was confusion around the whole kid issue. Bringing your kids, not bringing your kids… Is it really for the kids, is it really for the adults, who is the participant here? I think there was confusion around that, and think that probably had an impact on whether people could come. (VT)

Second, educators noted the diversity of nutrition-related knowledge, attitudes, and behaviors at the first lesson. Interviews revealed that such diversity made teaching challenging: “that’s the thing, I had three families. And one family, it was new to all of them, but the other two it was like ‘Well, I know all of this’. So that’s kind of ... the hard place because you want to ... you want to teach it to the family who doesn’t know any of it, but…” (NY). Such diversity was perceived as impacting attendance across the spectrum of knowledge, attitudes, and behaviors:

I feel like the people who were at the lower levels who didn’t have the understanding or the knowledge and had maybe some unhealthy habits, I think they felt a little bit, um ... kind of a sense of shame. (WA)

It was kinda hard for me to balance that and still engage the participants that had a lot of nutrition knowledge coming in. And I think that might be why the attendance wasn’t very great throughout the program was because they kind of ... like I said, checked out when they thought, oh, I know all this already. (NY)

Third, attendance was impacted by variable schedules. Because the nine healthy eating classes were spread out across the growing season, educators struggled often to find a time that worked for most participants due to shifting work and school schedules: “We noticed it was very difficult to [find a good class time] once school had started. Um, because there are after school activities and things like that” (NC).

The median lesson activity score across all postlesson participant surveys was 5 or “very useful” (n = 1,691, IQR = 4–5). The median recipe score was also 5 (n = 279, IQR = 4–5). Overall, 96% of responses (n = 262) indicated an intention to try at least one featured recipe.

Fidelity—produce quality and CSA pickup site functionality

All farms had acceptable quality produce with shares available during the scheduled times at a well-organized and safe pickup site. Auditors perceived available farm staff to be respectful of F3HK participants. In the case of “free-choice” shares, produce was labeled adequately at all but one site (though a farmer was available to answer questions). In 2016, one site was deemed to have unsatisfactory cleanliness (though this was for an educational site to which the CSA shares were delivered for participant convenience) and four sites did not have farm staff available to assist study participants (but three had nonfarm staff available instead). Across both years, coordinators only observed or mentioned the verification of a functioning SNAP/EBT payment system for 57% of audits, but this was typically due to a noted lack of demand for payment via EBT (at least while auditor was present).

Fidelity—class alignment with educator guide

Educators taught an average of 92% of planned class activities. Educators often modified lessons to minimize familiar content (e.g., MyPlate) and emphasize content of greater interest to participants (e.g., cooking skills), primarily due to the high level of knowledge and behaviors among attendees:

…the participants in this project were very different from what we typically see… I think because of the way that people were screened, or the length of the research project. The participants came to these classes with a lot more background knowledge and already really valued food, cooking, and physical activity, like all of these healthy habits… (NY)

Educators’ emphasis on cooking-related content (and minimization of other content) was driven, in part, by their perception that hands-on cooking activities successfully engaged both children and adults. In-class cooking and recipe tasting also allowed caregivers to try new recipes and kids to taste new vegetables without risk of waste:

…the cooking and tasting of the recipes was really, really helpful because they don’t have to take the chance on buying this vegetable or cooking this recipe that maybe they don’t like… it opened them up to wanting to eat different things like fennel or fresh corn in a stir fry. (WA)

Lesson modification was also driven by lack of time: “It wasn’t always enough time… we definitely were trying to put a lot of information into a one hour block, which was hard” (VT). This was corroborated by participants, many of whom requested class times longer than 1 h in the open-ended portions of their postlesson surveys.

Two final factors influenced educator decisions regarding class modifications: produce seasonality and lack of equipment, with the former being a more ubiquitous issue than the latter. Unpredictable seasonality of local produce meant that, despite attempted alignment of curriculum-endorsed recipes with local agricultural calendars, week-to-week ambiguity regarding CSA contents was inevitable: “It was a little hard because we didn’t always know what participants were getting [in their CSA] each week. And the farmers didn’t know until the week of, so we couldn’t really plan with them” (NC). This required educators to quickly adapt recipes or spend time discussing potential recipe adaptations to use share contents. A few educators also reported that class locations were not always equipped for intensive food preparation: “I didn’t have a kitchen where I was doing the classes and it turned out we didn’t even have water, only in the bathroom. I probably did end up doing a couple more taste-testings than actual [hands-on] cooking. We cooked probably only three times, I think, the entire series…” (VT).

The majority of lessons (78%) were taught out of the intended order, typically to accommodate scheduling of field trips to farm and grocery store. The farm tour was considered by educators to be a particularly important aspect of the curriculum. It allowed participants and their children to understand how and where their produce is grown and to see the farm in action:

I think for the kids, I think it’s huge. I was telling one of the kids, I was like, “These vegetables came from the farm.” [S/he said] “No they didn’t, they came from the store!” [I said] “Yeah, but first they came from the farm.” (laughs) So many kids are disconnected now… so it’s nice to have that connection… they’re seeing different things that they don’t normally see… They’re trying new things... (WA)

Auditors gave educators a median score of 5 (“excellent”) across all evaluated categories: preparedness, adaptability, knowledge, respectfulness, facilitation of group cohesion, and promotion of a safe and trusting environment, immediacy of content, and engagement.

Discussion

This process evaluation contributes to the growing conversation about successes and challenges associated with administering a CO-CSA and complementary healthy eating classes. Discussions of our findings are organized by process evaluation construct to facilitate reader clarity.

Reach

Our results suggest that F3HK may have reached a specific subset of low-income households with children; when compared to eligible nonparticipants, F3HK participants were older and, more often, white and female. These findings are in line with prior research that found that, among caregivers participating in an existing CO-CSA in Vermont, many were white (87.5%) women (87.2%) with a college education (69.2%) [19, 20]. This pattern is not isolated to subsidized CSA operations; participants of a randomly selected set of CSA sites in Indiana were found to be primarily white (95.3%), women (82.1%), with at least a Bachelor’s degree (85.6%) [5]. Among caregivers deemed eligible for this study, one-fifth chose not to enroll. The F3HK participants’ high baseline knowledge and healthy eating behaviors noted by educators may have played a role in facilitating interest in, or feasibility of, participating in CSA, regardless of income.

Dose—CO-CSA shares

Enrolled households received a high “dose” of the CSA portion of the intervention. On average, participants picked up most shares, though a small percentage made no pickups. Even among those who regularly picked up their share, F3HK focus group data published elsewhere suggest that doing so was challenging due to travel distances and difficulty incorporating pickups (and classes) into their weekly routines [17]. This is consistent with other studies in which inconvenience associated with location (e.g., travel distance and parking) was cited along with work schedules, timing, and transportation issues as reasons for low CSA participation [5, 9]. In a different sample of CSA members, both affordability and convenient pickup or delivery locations were considered more important attributes among low-income households than higher-income households [2]. As with the healthy eating classes, participant schedules and availability during pickup may have also been a factor in their differential rates of pickup. In a CSA employing a “pay-as-you-go” model, 39% of subscribers opted to purchase a share two to three times per month and another 44% purchased one or fewer shares per month [21]. Future studies of the CO-CSA models may consider testing CSA share delivery and share frequency flexibility and evaluating the impact of such changes on participation, produce consumption, and the economic viability of this business model for farmers.

Postseason participant survey data indicated that most, if not all, CSA contents were consumed by members of participating households. Preservation, sharing, and spoilage of CSA contents were minimal, suggesting that participants chose CSA shares of an appropriate size for their households. This is corroborated by focus group reports of some F3HK caregivers that they opted to decrease their share size due to an excess of produce [17].

Dose—healthy eating classes

Enrolled households received a moderate dose of the intervention curriculum. Though a majority of enrolled adults and over half of enrolled children had at least some exposure to the healthy eating classes, total attendance never exceeded 56% of households and dipped to less than 10% of households near the end of the CSA season in 2017. Attendance was not articulated as a requirement for study enrollment in contrast to the financial deposit required for the CO-CSA, which may have contributed to poor attendance. Even so, attendance rates were better than prior intervention attempts in which no classes about cooking and preserving produce were held due to transportation issues and conflicting schedules [6]. Educators in this study did struggle to schedule class times, but children’s enthusiasm and participant friendship formation seemed, in their view, to facilitate attendance. Their observations were corroborated by participants during postseason focus groups in which they reported that their children looked forward to the classes and that they appreciated the comradery fostered by the in-person experience (data unpublished, under review). Future CO-CSA educational efforts might consider tying study compensation to both CSA and class participation; offering classes at multiple times to accommodate more schedules; delivering some educational content online for easier accessibility (while retaining opportunities for in-person engagement); and designing tiered classes and content (e.g., beginner and advanced) to accommodate varied skill sets.

Postlesson survey respondents also reported high satisfaction with lesson activities and featured recipes, though this may simply reflect self-selection of class attendees—that is, those who attended and completed the survey may have also been those who had lesser knowledge or lower self-efficacy at baseline and, thus, gained greater value from the classes.

Fidelity

High fidelity was achieved for the CO-CSA and moderate fidelity for the educational component. QA audits of CSA pickup sites revealed no major issues that could have detracted from the participants’ experiences. This is in slight contrast to some caregiver reports that poor organization of CSA pickup sites caused delays and confusion at times [17]. While the vast majority of educators taught the nine lessons out of their intended order, they typically did so for logistical reasons. Such decisions compromised fidelity as technically defined but may have promoted participant engagement and implementation success; indeed, some scholars recommend a hybrid or responsive approach to intervention design that values fidelity while allowing for “built-in” adaptations to improve program fit and, potentially, effectiveness [22, 23]. Lack of time did prevent 100% completion of planned activities at all classes. Given these issues, future implementation of the F3HK curriculum may benefit from clearer allowances for adaptation; longer time allotments; and use of online resources to substitute for some class activities.

Limitations

Limitations of this study included reliance upon self-report participant data regarding CSA share utilization and missing CSA pickup data for 16% of enrolled households. These limitations may have resulted in some upward bias of results related to these data sources, though sensitivity analyses were provided.

Conclusions

Taken together, the results of this process evaluation demonstrate the feasibility of implementing a CO-CSA with supplementary healthy eating classes while also elucidating key factors that influenced its reach and could improve implementation and uptake. Our data suggest that CO-CSA enrollees would welcome share delivery services and greater logistical and topical integration with any accompanying educational opportunities. Curricula that emphasize local foods should be designed to address the range of knowledge, attitudes, skills, and behaviors of those who self-select to participate. Our data also demonstrate the value of the in-person educational experience and demand from both educators and participants for longer class times. Given discussed scheduling issues, complementary online content may be a welcome strategy for reaching those whose schedules or circumstances preclude consistent attendance. As there are ongoing discussions regarding the definition of fidelity and how to accommodate potentially valuable site-level intervention adaptations, interventionists should endeavor to document site-specific adaptations and to assess their perceived and testable impacts on reach and effectiveness. Differential engagement across multimodal intervention components should also be explored in relation to outcome data in order to understand their relative contributions to observed outcomes.

Funding

Research reported in this paper was supported by the National Institutes of Health under award T32‐DK007158 and the National Institute of Food and Agriculture, U.S. Department of Agriculture under award number 2015-68001-23230. Any opinions, findings, conclusions, or recommendations expressed are those of the author and do not necessarily reflect the views of the National Institute of Diabetes and Digestive and Kidney Diseases, the National Institutes of Health, or the U.S. Department of Agriculture. Funders had no role in designing the study, collecting, analyzing, interpreting the data, or writing the manuscript.

Compliance with Ethical Standards

Conflicts of Interest: The authors declare that they have no conflicts of interest.

Authors’ Contributions: All authors conceived of the study collaboratively; JAG led the design of data collection protocols with support from SBJP and feedback from all authors; JAG coordinated data collection, performed analyses, and prepared the manuscript; all authors provided critical feedback and approved of the final submission.

Ethical Approval: All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. All study protocols were reviewed and approved by institutional review boards at both the University of Vermont and Cornell University (protocol ID # 1501005266). This article does not contain any studies with animals performed by any of the authors.

Informed Consent: All participating adults (18+ years) provided written informed consent and written permission for their child to participate. Participating children aged 7–12 years provided oral assent.