Small improvements in an urban food environment resulted in no changes in diet among residents

Abstract

Living in a low-income neighborhood with low access to healthy food retailers is associated with increased risk for chronic disease. The U.S. Healthy Food Financing Initiative (HFFI) provides resources to support the development of infrastructure to improve neighborhood food environments. This natural experiment examined a HFFI funded food hub that was designed to be implemented by a community development corporation in an urban neighborhood in Cleveland, Ohio. It was intended to increase access to affordable, local, and healthy foods; establish programs to increase social connections and support for healthy eating; and create job opportunities for residents. We used a quasi-experimental, longitudinal design to externally evaluate food hub implementation and its impact on changes to the built and social environment and dietary patterns among residents living in the intervention neighborhood (n=179) versus those in a comparison (n=150) neighborhood. Overall, many of the food hub components were not implemented fully, and dose and reach of the executed food hub components was low. There were statistically significant improvements in observed availability of healthy foods in the intervention neighborhood versus the comparison neighborhood. There were no changes over time in diet quality scores, total caloric intake, or fruit and vegetable intake in the intervention neighborhood. In conclusion, low dose implementation of a food hub led to small improvements in availability of healthy foods but not in dietary patterns. Findings highlight challenges to implementing a food hub in neighborhoods with low access to healthy food retailers.

Introduction

Access to healthy foods is limited in low-income communities in the US, especially in racial and ethnic minority communities.[1] Areas that are both low-income (LI) and have low-access (LA) to full-service supermarkets, supercenters, or larger grocery stores have been defined as “food deserts.”[1] From 2010–15, the number of LILA census tracts decreased by 5% in rural areas yet increased by 5% in urban areas.[2] Many observational studies have found living in a LILA area is associated with poorer diet quality and higher body mass index assumed to be influenced, in part, by limited neighborhood access to healthy foods and greater access to high-calorie, low-nutrition foods.[3–9] These findings heighten the need for additional strategies to improve healthy food retail options in low-income, urban communities.

There is growing support for implementation of multicomponent food environment interventions in LILA urban areas.[10–13] As a result, community organizations are increasingly looking to food environment change as a strategy for community health development. This research is focused on a community development corporation and its partners that received funding to implement a multicomponent food environment intervention (i.e., food hub) in a LILA neighborhood. Food hubs are an alternative wholesale and retail model focused on aggregating, distributing, and selling foods with the goal of better linking supply and demand actors in the local food system.[14] Food hubs with a focus on consumer retail are similar to supermarkets in the sense that they have the potential to offer a wide variety of foods for consumers to purchase.[15] Unlike supermarkets, however, most retail food hubs are multi-vendor food retailers. Food hubs with different vendors (i.e., meat, cheese, produce) selling their products in a shared space allow for infrastructure costs to be distributed across vendors.[15] The ability of a food hub to distribute costs is particularly relevant in LILA neighborhoods since the financial risks associated with opening and sustaining a supermarket is a barrier to supermarket development.

Other factors make retail food hubs a potential solution in LILA urban neighborhoods. They are often values-based, many with the goal of bringing together business with social and community services.[16] Accordingly, food hubs may have greater flexibility to address social dynamics influencing food habits through food-related entrepreneurship and the creation of a public space for social interaction. In contrast to more commonly available alternative food retailers such as farmers’ markets, food hubs are novel retail settings. They represent a food shopping space that may be free of existing norms, such as who belongs in the space. Accordingly, consumers have the opportunity to construct meaning and develop new shopping patterns within this space.[17]

Study Goals

In this natural experiment, we sought to achieve three goals. First, we monitored real world implementation and reach of the food hub. Second, we conducted a quasi-experimental, longitudinal study to evaluate if the food hub resulted in neighborhood-level changes in the built and social food environment over time. Third, we evaluated if improvements in the food environment were related to improvements in individual-level dietary outcomes among residents.

Methods

Design and Intervention

The natural experiment involved external evaluation of the implementation of a proposed food hub, which was supported by a grant from the federal Healthy Food Financing Initiative (HFFI) awarded to a community development corporation in Cleveland, Ohio, USA. The HFFI grant was leveraged to secure additional funding through multiple public-private partnerships to support food hub development and implementation over a five-year timeframe (2014–2019) with an intensive planning phase during the first two years. A separate three-year (2015–2018) grant was awarded to our research team through a rapid-response funding mechanism to support external evaluation of food hub implementation. We used a quasi-experimental, longitudinal design for the evaluation with data collection at three time points: Baseline (2015–16), 12-month (2016–17), 24-month (2017–18).

The food hub grant proposal was developed by a community development corporation and included proposed efforts to improve built and social food environments in a targeted neighborhood. The proposed food hub had five goals and related implementation components described in Table 1. It was organized around the planned formation of a large (42,000 square feet) space that would serve as the “hub” for the components. Over the three-year natural experiment timeline both implementation of food hub components and resident self-reported use of the hub were documented as well as following a group of residents over time to examine the potential impact of exposure to the proposed food hub.

Table 1.

Food hub goals, proposed and actual components implemented, and reach by study participants, Cleveland, OH, 2015–2018.

Domain of Nutritious Food Access	Proposed Food Hub Goalsa	Components of Food Hub Implementation		Reach of Food Hub Componentsc
		Proposeda	Actualb
Built Food Environment	1. Increase access to affordable, local, healthy food.	• Develop 42,000 square foot food hub space with year-round multi-vendor marketplace open 6 days per week • Open seasonal farmers’ market Ensure new food retail venues accept SNAP and WIC and offer healthy food incentives • Open healthy food café	• Multi-vendor marketplace – Not developed during timeframe	• N/A
			• 19 seasonal local produce market days (open Saturdays summer/fall) – market accepted SNAP and 2 fruit and vegetable food incentive programs	• Year 2: 17%, Year 3: 20% visited market at least once
			• Healthy food café (open year-round for 30 months; open 5–6 days per week)	• Year 2: 12%, Year 3: 14% went to café at least once
	2. Develop a healthy food distribution system to service small food retail outlets.	• Aggregate fresh and healthy foods in food hub to support affordable distribution to small food retail stores	• Not developed during timeframe.	• N/A
Social Food Environment	3. Implement strategies to promote healthy food education and consumption.	• Offer cooking classes and demonstrations	• 6 cooking classes	• Year 2: 12%, Year 3: 5%
			• 7 cooking demonstrations at seasonal local produce market	• No data available
	4. Promote community building.	• Develop community garden mentorship program and neighborhood cooking circles • Conduct community outreach to promote connectedness and networking	• 3 community garden mentorship events	• Year 2: 7%, Year 3: 5%
			• 6 cooking circles • 83 community outreach activities	• Year 2: 3%, Year 3: 1% • Year 2: 45%, Year 3: 27%
	5. Create sustainable food and beverage related employment and business opportunities.	• Create new jobs such as food hub manager, food-related entrepreneurs, café staff, general management • Create new business such as café, brewery, community supported agriculture, food hub vendors	• Jobs created: project manager, general manager, seasonal local produce market manager, café and brewery staff, community organizer, community ambassadors	• Year 2: 1%, Year 3: 1%
			• New businesses created/ maintained (openclose dates): Café (July 2015December 2017), Brewery (March 2016-present), Seasonal local produce market (June 2016-August 2017)	• N/A

^aFor overall 5-Year Project (September 2014-September 2019)

^bEfforts during Study 3-Year Study Timeframe (July 2015-June 2018)

^cReach of Food Hub Components by Study Participants in Intervention Neighborhood. Year 2 Data Collection (August 2016-June 2017); Year 3 Data Collection (August 2017-June 2018)

Context

The food hub was implemented in one neighborhood in Cleveland. A comparison neighborhood with similar racial and economic composition and access to healthy food retailers was selected in Columbus, OH.[18] The intervention and comparison neighborhoods, respectively, had a resident population of 7,088 and 11,214, 42.9% and 37% of the households received Supplemental Nutrition Assistance Program (SNAP) benefits, and 72.2% and 62.5% of the residents were African American.[19] The intervention neighborhood was targeted for food hub implementation because most of its census tracts (5 of 7 tracts) were defined as LILA; all census tracts (4 of 4 tracts) were LILA in the comparison neighborhood.[18, 20]

Participant Recruitment and Retention

Participants (i.e., neighborhood residents) were recruited from contiguous census tracts in the intervention (n=7 tracts) and comparison neighborhoods (n=4 tracts). A community-based approach was used to identify, screen, and enroll potential participants from August 2015 to July 2016 using mailings, flyers, public presentations, and word-of-mouth. A total of 1,395 people expressed interest and were screened to determine eligibility. Eligibility criteria included: at least 18 years old, English speaking, living in targeted census tracts, plan to live in current neighborhood at least 12 months, and responsible for more than half of household food shopping. Study enrollment was limited to one person per household. Ineligibility was primarily due to living outside the geographic target area. All individuals who were eligible reviewed the consent form verbally in person or over the phone, had an opportunity to ask questions, and then, if interested, provided written or verbal (over the phone) informed consent. A copy of the consent form was provided to all participants.

The CONSORT diagram in Figure 1 summarizes recruitment and retention processes. Among those eligible and who provided informed consent (n=655), 516 completed three 24-hour dietary recalls at baseline (78.8%), which was required for inclusion in the longitudinal study. The retention rate of the baseline sample was 87.2% (n=450) at 12-month and 85.7% (n=442) at 24-month follow-ups. Those not retained could not be reached after multiple attempts.

Figure 1.

CONSORT Diagram

Data Collection

Data collection entailed structured observations of food hub implementation, objective measures of the food environment, and participant surveys. Data collection occurred from August 2015 to September 2018.

Food hub implementation observations occurred at all social events, once per season at the produce market, and monthly at the healthy café. We did not systematically observe the brewery since it was not directly connected to study goals related to diet. For social events, either research staff or programmatic staff from the community development corporation recorded the number of people present and activities. The produce market and café were assessed by a trained research assistant using an audit form to systematically capture products available, prices, and customers present.

To examine the influence of the hub on the overall food environment, food retail data collection occurred once per year at all stores within the intervention and comparison neighborhoods. Subsequent data collection occurred in the same month on the same day of week and time of day as baseline store observations to limit variability due to store operations and stocking practices. Two trained auditors completed each store observation with high inter-rater reliability (97% agreement).

Survey data collection for each participant involved three phone surveys conducted during a one-month timeframe each year for three years for a total of up to nine surveys. All calls included a 24-hour dietary recall. Additionally, the first call each year included a psychosocial survey. Materials needed to support data collection (e.g., food amounts booklet with standardized information about reporting portion sizes, response options for Likert scales) were mailed to participants yearly prior to surveys. Surveys were administered by trained research assistants and took about 60 minutes. Each year of data collection, participants were compensated $35 after calls 1 and 2 and $40 after call 3. The study protocol was reviewed and approved by the Case Western Reserve University Institutional Review Board. The authors have no conflicts of interest related to this research.

Measures

Primary Outcomes: Dietary Behaviors

We examined changes in three dietary outcomes over time: (a) diet quality scores assessed using the Healthy Eating Index score (HEI-2010),[21] (b) daily fruit and vegetable intake, and (c) daily energy intake (kcal). Outcomes were calculated based on data collected and processed using the Nutrition Data System for Research, a standardized and comprehensive 24-hour dietary recall method.[22] Each year we aimed to have participants complete three dietary recalls on two nonconsecutive weekdays and one weekend day within a one month window and subsequent data collection occurred within 37 days of the baseline date to limit within subject seasonal variability.[23, 24] Outcomes were calculated based on the average of the yearly 24-hour dietary recalls.

Reach of Food Hub Components

Reach of the food hub components was assessed at the 12- and 24-month follow-up surveys using the following items; participants were asked to respond based on the past 12 months: (a) how frequently did you visit the [name of market]?, (b) how frequently did you visit [name of café]?, (c) did you take part in any of the following activities in your neighborhood (e.g., cooking classes, community gardening, cooking circles, and community engagement activities)?, and (d) have you worked at or been a vendor at any of the following places: [name of produce market, name of healthy café, name of brewery]?

Built Food Environment

An objective measure of the retail availability of healthy foods was assessed using an adapted Nutrition Environment Measures Survey in Convenience Stores (NEMS-CS), a standardized tool for evaluating availability, price, and quality of healthy food options.[25] Data collection focused on all food retail outlet types including supermarkets, grocery stores, convenience stores, gas stations, pharmacies, dollar stores, and ethnic and specialty food stores located in and on the periphery (i.e., directly across the street) of the boundaries of the intervention and comparison neighborhoods. The nearest supermarket to each neighborhood commonly used by residents was included since these stores are part of the food retail choice context for residents who cross neighborhood boundaries to access a larger food retailer.[26] Data included 53 food stores located within the study area, plus the two nearest supermarkets resulting in 55 stores (Intervention = 34, Comparison = 21). Each census block was assigned an average healthy food availability score based on scores of stores within 0.5 mile network distance from the block’s centroid (score range: 0–38).[26] Observed healthy food availability scores were calculated for all study area census blocks and then assigned to participants based on the census block of their home address.

Perceptions of healthy food availability was assessed using the mean score of a 3-item measure (α=0.78).[27] Participants were asked to think about the area within a 20-minute walk or one mile from their home and rate agreement with three statements: (a) A large selection of fruits and vegetables is available in your neighborhood, (b) The fresh fruits and vegetables in your neighborhood are of high quality, and (c) A large selection of low-fat products is available in your neighborhood. Responses were coded on a 4-point Likert scale (1=strongly disagree; 4=strongly agree). Mean scores were calculated for participants who had data on at least two of the three items.

Social Food Environment

Social support for healthy eating was measured using the mean score of an 8-item measure (α=0.87) adapted from two existing measures.[28, 29] Participants were asked, separately for family and neighbors, how often in the past 12 months they: (a) ate fresh and healthy foods with you, (b) encouraged you to eat fresh and healthy foods, (c) told you about fresh and healthy foods and how to prepare them, and (d) prepared fresh and healthy foods with you. Responses were coded on a 5-point Likert scale (1=never; 5=most of the time). Mean scores were calculated for participants who had data on at least five of the eight items.

Sociodemographics

Sociodemographics were measured based on self-report using questions adapted from the Behavioral Risk Factor Surveillance System.[30] Questions assessed race, age, sex, level of education, annual household income, and receipt of Supplemental Nutrition Assistance Program (SNAP) benefits in the past 12 months.

Statistical Analysis

All analyses were limited to participants (N = 329) who (a) resided in the study catchment area all three years, (b) had complete data on all predictors and covariates, and (c) had complete dietary data. Complete dietary data was defined as having at least two dietary recalls within a 37-day window at baseline and having two recalls during at least one other year. Missing data were examined, no patterns were noted, therefore, listwise deletion was used.[31]

Data management and all analyses were conducted using SAS v9.4. Descriptive statistics were estimated to examine the distribution of variables and a series of growth curve models (GCM) were estimated to examine changes in the built and social food environment as well as changes in dietary outcomes. For each of the GCMs, we examined the main effect of time and the interaction between time and intervention to determine if there was statistically significant change over time and if the change differed between the intervention and comparison neighborhoods.

For the food environment GCMs, when the interaction was statistically significant, we included that variable as a time-varying predictor in the dietary outcome GCMs. Using this process, the food retail environment was the only variable included as a predictor in the outcome GCMs. For all GCMs, time was coded as baseline = 0, 12-month = 1, and 24-month = 2.

We determined which covariates to include in our analyses following Shaddish, Cook, and Campbell’s (2002) process for quasi-experimental research studies (i.e., that covariates need to be theoretically and empirically [r ≥ .20] related to the study’s primary independent variable and the dependent variables).[32] Following these guidelines, we first identified variables that are theoretically related to dietary outcomes; this led us to consider age, education, income, receipt of SNAP, and food security as possible covariates. Next, we examined the correlation between each of the potential covariates, neighborhood, and dietary outcomes. We followed the same process for determining which covariates to include in the food environment GCMs. Income (1 = less than $10,000, 2 = between $10,001 and $20,000, 3 = between $20,001 and $30,000, and 4 = $30,000 +) was the only variable that met the inclusion criteria for the dietary outcome models whereas none of the variables met the inclusion criteria for the food environment models.

Results

Study Sample

There were no significant differences between the intervention and comparison groups on gender, age, and Hispanic ethnicity (Table 2). Overall, more than half of the study population had less than 12 years of education, an annual household income of $20,000 or less, received SNAP benefits, and identified as African American. Participants from the intervention neighborhood had significantly lower levels of education and income, higher receipt of SNAP, and were more likely to be African American.

Table 2.

Baseline description of study population from self-reported data collected in 2015–16 (N=329).

	Intervention Neighborhood: Cleveland, OH (N = 179)	Comparison Neighborhood: Columbus, OH (N = 150)	p-valuea
Demographic Variables
Female (n, %)	134 (74.9)	120 (80.0)	0.33
Age (mean, SD)	52.08 (13.42)	51.95 (13.03)	0.929
Education (n, %)			0.005
Less than 12 years of education	107 (59.8)	90 (60.0)
12 years or GED	58 (32.4)	32 (21.3)
At least 1 year of college Education	13 (7.3)	28 (18.7)
Income (n, %)			0.001
Less than $10,000	71 (40.6)	37 (24.7)
Between $10,001 and $20,000	60 (34.3)	47 (31.3)
Between $20,001 and $30,000	25 (14.3)	27 (18.0)
$30,001 or higher	19 (10.9)	39 (26.0)
Receives SNAP (n, %)	130 (72.6)	84 (56.0)	0.005
Race (n, %)			0.001
Black	123 (68.7)	92 (61.3)
White	37 (20.7)	53 (35.3)
Other1	19 (10.6)	5 (3.3)
Non-Hispanic (n, %)	174 (97.2)	148 (98.7)	0.544
Food Environment Variables
Perception of Healthy Food Availability (mean, SD)	2.67 (0.86)	2.91 (0.98)	0.018
Social Support (mean, SD)	2.34 (0.84)	2.63 (0.83)	0.002
Observed Healthy Food Availability (mean, SD)	7.07(2.04)	8.12(2.48)	<0.001
Dietary Outcomes
Healthy Eating Index –11 2010 Score (mean, SD)	49.63 (10.74)	51.05 (12.19)	0.264
Daily Fruit & Vegetable Consumption (mean, SD)	3.23 (2.25)	3.88 (2.87)	0.021
Daily Energy Intake (Kcal) (mean, SD)	1823.98 (865.20)	1845.06 (739.39)	0.814

SD standard deviation, SNAP Supplemental Nutrition Assistance Program

^aThe test statistic is a t statistic or χ-squared statistic comparing baseline continuous and categorical variables, respectively

^bOther Race = American Indian or Alaskan Native, Asian, Pacific Islander

Implementation and Reach of the Food Hub Model

Results of structured observations revealed that actual implementation of the food hub components differed substantially from the proposed plan during the three-year timeframe of the natural experiment evaluation (Table 1). Importantly, the large space designed to be the “hub” of the components was not developed, resulting in neither implementation of a multi-vendor marketplace nor the healthy food distribution system that was designed to service small food retail outlets such as corner stores.

Actual implementation included 19 seasonal local produce market days where SNAP was accepted and two fruit and vegetable incentive programs were available, a healthy food café that operated year-round for 30 months, six cooking classes organized in the community, seven cooking demonstrations at the seasonal local produce market, three community garden mentoring events, six cooking circles (i.e., community building centered around shared meal preparation), 83 community outreach events organized by a full-time community organizer and three community ambassadors (i.e., neighborhood-based outreach to connect residents to food resources and each other), and the brewery. By the end of the natural experiment timeframe, most of the components of the food hub model were not maintained except for the brewery, which was designed to support the creation of food and beverage related jobs.

Based on self-reported data from participants living in the intervention neighborhood, reach of the food hub components varied. The highest reach was related to community outreach, which also had the highest dose with 83 outreach events. At the 12-month and 24-month follow-up surveys, 45% and 27%, respectively, of the study participants in the intervention neighborhood reported they participated in at least one community event during the past 12 months. Seventeen and 20% of the participants at 12- and 24-month follow-up, respectively, reported they went to the local produce market operated by the food hub at least once in the past 12 months. Only one percent of the study population reported they had employment related to any of the food hub components.

Changes in Food Environment over Time

As shown in Model 2 of Table 3, there were statistically significant changes in the three measures of the built and social food environment over time. Perceptions of healthy food availability and observed availability of healthy foods improved while social support for healthy eating declined (b = 0.09, 0.58, −0.06, respectively). However, observed availability of healthy foods was the only factor where change over time differed between the two communities (Model 4, b = 0.68). In year 1, the comparison neighborhood (score = 7.9) had slightly better scores for observed availability of healthy foods than the intervention neighborhood (7.3). By year 3, the intervention neighborhood (9.1) had better scores than the comparison neighborhood (8.3).

Table 3.

Estimates from growth curve models examining change in built and social food environment over three years (N = 329).

	Fixed Effects (SE)				Error Variance (SE)		Model Fit
	Intercept	INT	Time (Years)	INT × Time	Intercept	Level-1	AIC	BIC
Observed Healthy Food Availability
Model 1a	8.15 (0.10)				1.56 (0.24)	4.24 (0.23)	4477	4485
Model 2b	7.57 (0.16)	0.58 (0.08)			1.67 (0.24)	3.91 (0.22)	4425	4433
Model 3c	7.54 (0.16)	0.07 (0.19)	0.58 (0.08)		1.68 (0.25)	3.90 (0.22)	4427	4434
Model 4d	7.91 (0.18)	−0.61 (0.24)	0.21 (0.11)	0.68 (0.15)	1.72 (0.24)	3.80 (0.21)	4409	4417
Perceptions of Healthy Food Availability
Model 1a	2.86 (0.04)				0.38 (0.04)	0.43 (0.02)	2384	2392
Model 2b	2.77 (0.05)	0.09 (0.03)			0.39 (0.04)	0.43 (0.02)	2378	2385
Model 3c	2.93 (0.06)	−0.31 (0.08)	0.09 (0.03)		0.36 (0.04)	0.43 (0.02)	2366	2374
Model 4d	2.89 (0.07)	−0.24 (0.09)	0.13 (0.04)	−0.06 (0.05)	0.36 (0.04)	0.43 (0.02)	2369	2376
Social Support for Healthy Eating
Model 1a	2.42 (0.04)				0.33 (0.03)	0.30 (0.02)	1975	1983
Model 2b	2.47 (0.04)	−0.06 (0.02)			0.33 (0.03)	0.29 (0.02)	1974	1981
Model 3c	2.58 (0.06)	−0.20 (0.07)	−0.06 (0.02)		0.32 (0.03)	0.29 (0.02)	1970	1978
Model 4d	2.62 (0.06)	−0.25 (0.08)	−0.09 (0.03)	0.06 (0.04)	0.32 (0.03)	0.29 (0.02)	1973	1980

SE Standard Error, INT Intervention, OAHF Observed Availability of Healthy Foods

^aA Null model estimating the average score of a respective measure of the built and social food environment at time beginning of the study

^bA Model examining if a respective measure of the built and social food environment differs by study sites at baseline

^cA Model examining change over time in a respective measure of the of the built and social food environment

^dA Model examining if change over time in the respective measure of the built and social food environment differs by site

Boldface indicates statistical significance (p < 0.05)

Impact on Dietary Outcomes

As shown in Table 4, there were no statistically significant changes in energy intake and fruit and vegetable intake over time. Conversely, as shown in Model 4 of Table 4, HEI-2010 scores did exhibit statistically significant changes over time (b = 3.48). However, changes over time did not differ between the intervention and comparison neighborhoods (b = −3.88). In addition, even though the intervention neighborhood exhibited greater improvements in observed availability of healthy foods over the course of the study, these changes were not associated with changes in HEI-2010 scores (b = 0.39).

Table 4.

Estimates from growth curve models examining change in dietary outcomes over three years (N = 329).

	Fixed Effects (SE)									Error Variance (SE)		Model Fit
	Intercept	Time (Years)	INT	Time × INT	OAHF	Time × OAHF	INT × OAHF	Time × INT × OAHF	Income	Intercept	Level-1	AIC	BIC
Healthy Eating Index-2010
Model 1a	50.59 (0.51)									70.01 (6.76)	47.59 (2.63)	7165	7172
Model 2b	51.56 (0.78)	0.43 (0.27)	−2.58 (1.01)							68.65 (6.66)	47.48 (2.62)	7155	7162
Model 3c	51.21 (0.85)	0.77 (0.39)	−1.93 (1.15)	−0.64 (0.53)						68.67 (6.67)	47.45 (2.62)	7153	7160
Model 4d	47.88 (2.15)	3.48 (1.46)	1.36 (2.98)	−3.88 (2.21)	0.34 (0.25)	−0.32 (0.17)	−0.31 (0.36)	0.39 (0.26)	1.53 (0.34)	62.21 (6.29)	47.75 (2.67)	7098	7105
Energy (Kcal)
Model 1a	1776.55 (43.42)									366487 (50424)	761622 (41990)	16455	16463
Model 2b	1803.07 (72.83)	−2.67 (34.04)	−43.85 (87.29)							367520 (50610)	762774 (42085)	16436	16443
Model 3c	1792.29 (81.80)	8.12 (50.46)	−24.03 (110.90)	−19.82 (68.41)						367165 (50618)	763839 (42176)	16425	16433
Model 4d	1640.18 (251.72)	39.95 (181.16)	248.09 (350.71)	−140.19 (271.04)	19.32 (29.51)	−4.22 (20.91)	−34.52 (43.20)	15.15 (32.04)	−5.25 (36.16)	373187 (51460)	767575 (42658)	16303	16311
Fruit and Vegetable Intake
Model 1a	3.52 (0.12)									3.93 (0.40)	3.33 (0.18)	4491	4499
Model 2b	3.83 (0.20)	0.05 (0.07)	−0.66 (0.25)							3.83 (0.39)	3.33 (0.18)	4488	4496
Model 3c	3.88 (0.21)	0.001 (0.11)	−0.74 (0.28)	0.08 (0.14)						3.83 (0.39)	3.34 (0.18)	4490	4497
Model 4d	3.46 (0.56)	−0.13 (0.38)	−0.06 (0.78)	−0.41 (0.59)	0.05 (0.07)	0.01 (0.04)	−0.08 (0.09)	0.06 (0.06)	0.19 (0.09)	3.66 (0.39)	3.38 (0.19)	4481	4488

SE Standard Error, INT Intervention, OAHF Observed Availability of Healthy Foods

^aA null model estimating the average value of a respective study outcome at baseline study

^bA model examining if a respective outcome differs by study site at baseline

^cA Model examining change over time in a respective study outcome

^dA Model examining effect of observed availability of healthy food (OAHF) on a respective outcome and if it modifies the effect of time or intervention on the outcome

Discussion

This natural experiment evaluated the impact of a proposed food hub that was intended to improve the built and social food environment in an urban neighborhood considered to be a food desert. Findings reveal there were small improvements in objective measures of retail availability of healthy foods over time in the intervention neighborhood. Nevertheless, access to healthy foods in both the intervention and comparison neighborhoods remained low during the entire study timeframe with most stores reporting low availability of healthy foods. Despite these small improvements in the food retail environment, there were no statistically significant changes in diet over time among participants living in the intervention neighborhood versus a comparison neighborhood. Additionally, there were no changes in perceptions of healthy food availability or social support for healthy eating.

This study was designed to externally evaluate real world implementation of a proposed food hub. The food hub planned to address two mechanisms related to changing dietary behaviors including improvements to the: (a) built food environment to make fresh and healthy foods more accessible and affordable and (b) social food environment to promote social interactions, norms, and support related to fresh and healthy foods.[13] Ultimately, not all components of the proposed food hub were implemented. Among those implemented, the dose and reach were low. The most significant component, implementation of the large food hub infrastructure, was never developed. Accordingly, the findings of this study do not provide information about the impact of a fully implemented food hub. Rather, findings reveal a low dose food hub was ineffective at changing dietary behaviors, perceptions of healthy food availability, and social support for health eating among residents living near the food hub. Furthermore, it did not contribute to practically significant improvements to healthy food availability in the LILA neighborhood.

There is limited research focused on evaluating the impact of federal HFFI funded food retail projects.[33] We are aware of one evaluation of the effect of opening a food hub on diet. Sharpe et al. (2019) found perceptions of the food environment improved significantly, but dietary outcomes did not change.[34] Three other studies of HFFI funded grocery store interventions found no direct link between the intervention and improvements in diet among adults or children.[35–37] Similar to our findings, other studies found limited changes in healthy food availability, marketing, or affordability after implementation of a HFFI funded grocery store.[38, 39] Non-HFFI funded food hub evaluations mostly focus on the economic impact and rural development potential.[40–42] One study evaluated the ability of food hubs to increase equitable access to healthy foods revealing substantial financial risks related to achievement of this goal.[43]

Taken together, evaluations of HFFI projects and food hubs provide guidance for future food environment interventions. First, these interventions have multiple goals (i.e., expand access to healthy foods in underserved areas, create and preserve quality jobs, revitalize low-income communities) making evaluation more complex. Second, there is still a black box understanding of the mechanisms through which food environment interventions impact diet and weigh status. Third, results highlight limited effectiveness of several HFFI projects on diet. This aligns with a “fixes that fail” system archetype, which points to the need for interventions that address the root cause of a problem rather than consequences.[44]

Finally, this study offers guidance for the evaluation of food environment natural experiments.[45] These lessons may be useful for natural experiments focused on other large-scale initiatives (i.e., housing, transportation) designed to improve community health.

Although there was a proposed plan for the food hub, actual implementation diverged from this plan. Key factors driving implementation such as negotiations between the public and private partners, capital development, project management, and organizational capacity were not systematically monitored in the natural experiment. These are sensitive factors that may be difficult to monitor, but appear to be important indicators of implementation. Future natural experiments would benefit by including methods to capture decision-making processes, funding flow and management, and organizational capacity that influence implementation.

Another challenge was related to a general lack of program theory guiding the proposed food hub model. The proposed plan included five main components (Table 1) that are conceptually related to built and social food environments. Inclusion of all of these components in the proposed food hub was novel and seemed to have great potential. However, the proposed plan lacked specific details about the “active ingredients” for each component related to factors such as the specific strategy to improve outcomes, hypothesized mechanisms of change, and necessary intensity and duration required to produce an effect.[46, 47] Future natural experiment research, and HFFI funded projects more broadly, may benefit by a priori articulation of the theory of change including documentation of what is trying to be accomplished, why, and how as well as assumptions about the dose and intensity required to yield an effect.[48]

Finally, the natural experiment revealed a challenge to traditional quasi-experimental evaluation models that examine the impact of one intervention (i.e., a food hub) on specific outcomes (i.e., diet). Even though the food hub model was multicomponent, this was not the only change in the neighborhood. Other food-related activities such as emergency food assistance and healthy food incentive programs occurred concurrently yet independent in both the intervention and comparison communities. Research methods that take into account the dynamic complexity of neighborhoods may offer insights into interdependencies between multiple neighborhood-level interventions that have the potential to collectively influence population health outcomes.[49, 50]

Several limitations must be considered in interpreting these results. The incomplete implementation of the planned food hub limited our ability to examine the true impact of a food hub. Many participants did not contribute sufficient data to be included in the planned analyses, which may bias findings. The duration of follow-up may not have been adequate to determine the impact of the food hub on outcomes that take longer time horizons to change. Finally, many of the study outcomes relied on self-reported data.

Conclusion

In conclusion, we found low dose implementation of a food hub led to small improvements in availability of healthy foods but not in dietary intake. Further work is needed to reduce barriers to implementing food hubs in LILA neighborhoods. These insights will be critical to realize community health benefits from efforts designed to improve neighborhood food environments.