Growing the Field: Current Approaches to Data Collection at Farmers’ Markets

Abstract

There is limited published research about the dietary impacts of farmers’ markets. We sought to understand whether market managers collect data about markets and to examine the instruments and strategies used. Of the 359 market managers contacted across the United States, representing 543 markets, 185 managers participated in a telephone survey. A subset supplied copies of data collection tools for further analysis. Ninety-three percent of market managers collect data such as customer surveys, vendor applications, customer counts, or demographics. The potential utility of the data collected by mangers and suggestions for study of the dietary impacts of farmers markets are discussed.

INTRODUCTION

In light of Americans’ low fruit and vegetable intake,^1,2 poor overall dietary quality,³ the current obesity epidemic, and rising health care costs due to obesity⁴ and chronic diseases,⁵ efforts are needed to improve Americans’ dietary behaviors. There is increasing recognition in the public health community that access to healthy foods is an important factor to improve the dietary behaviors and health of communities.⁶ Therefore, with national initiatives like Communities Putting Prevention to Work and Community Transformation Grants, many cities’ and states’ obesity prevention efforts are focused on approaches to increase access to affordable, high-quality, fresh food, especially in low-resource communities. The development of farmers’ markets along with other fresh food retail opportunities has become an essential point of interest in these efforts for public health agencies, health advocates, and communities.^7–10 Farmers’ markets are often seen as feasible alternatives to other store formats because of their emphasis on fresh fruits and vegetables, the relatively limited physical infrastructure needed to host a market, and the potential benefits to farmers, communities, and consumers.^7,11 Additionally, residents of low-income neighborhoods, where supermarkets are scarce and the small grocery and convenience stores that do exist may sell limited fresh produce,¹² may benefit most from access to fruits and vegetables provided through farmers’ markets.

Food and nutrition assistance programs, such as the Supplemental Nutrition Assistance Program (SNAP), special Supplemental Nutrition Program for Women, Infants, and Children (WIC), Senior Farmers’ Market Nutrition Program (FMNP), and the WIC FMNP have begun to extend benefits to include farmers’ market purchases for fruits and vegetables, usually through the use of coupons or electronic benefit transfer (EBT) debit cards.^13–15 The percentage of farmers’ markets that accept SNAP benefits through the use of EBT debit cards has increased substantially over the past 5 years.¹⁶ Nonprofit organizations such as Project Bread in Massachusetts, the San Diego Hunger Coalition and Wholesome Wave, and local health departments (eg, New York City and Philadelphia) also have begun to leverage these expanded benefits through matching programs available at farmers’ markets.^17,18 Given the level of interest and efforts underway, as well as limited budgets of public health agencies, it is important that we more fully understand the impact of these markets on Americans’ dietary intake quality.

In general, the economic impacts of farmers’ markets are reasonably well documented via the US Department of Agriculture’s (USDA) Farmers’ Market Managers’ Survey.¹⁹ Additionally, many studies and reports have shown positive benefits to communities as well as increased income for farmers as a result of public markets such as farmers’ markets.^20,21 However, data examining the dietary impacts of farmers’ markets on individuals’ purchasing at the market are limited.^22,23 A recently published review of farmers’ market evaluations found that 12 farmers’ market evaluations published between January 1980 and January 2009 examined any nutrition- or weight-related outcomes.²² However, all of these evaluations were in the context of incentive programs: WIC FMNP and farmers’ markets programs for seniors, including Senior FMNP. Although many of the studies suggested that program participation was associated with improved fruit and vegetable intake and other nutrition-related outcomes, this review revealed the lack of studies unrelated to incentive programs. In addition, the review found that a wide variety of nutrition-related measures and outcomes were examined, which makes it difficult to compare results across evaluations. This has also made it difficult to develop a consensus on the degree of diet-related benefits expected from the introduction of a farmers’ market in a community.

The present study begins to fill these gaps by examining current approaches to data collection at the customer and vendor level with an eye toward informing and growing future research in the field. Broadly, we aim to inform the study of the diet-related health impacts of farmers’ markets, especially those that serve low-income individuals. Specifically, our research sought to understand (1) whether farmers’ markets routinely collect information about their markets, (2) how such data were collected, and (3) the nature of the data most commonly collected. The authors provide concrete recommendations for future health-related research approaches related to farmers’ markets.

METHODS

In order to understand the types of data collection methods market managers used at markets, a semistructured interview was initially conducted with market managers identified via the USDA Agricultural Marketing Services (AMS) directory of 6132 farmers’ markets in the United States²⁴ as of June 2010. From that list, markets accepting SNAP and/or WIC FMNP were selected for participation in the current study (n = 543). The decision was made to limit inclusion to markets that accept SNAP and/or WIC Vouchers (which usually require EBT machines) because the level of organization and administrative sophistication needed to provide EBT access at markets suggested that they would have the most capacity for collecting market data. In addition, given the public health field’s interest in narrowing health disparities, the authors wanted to focus on markets that were most likely to serve low-income consumers.

Trained research assistants called the managers of each of the 543 markets during the months of June, July, and August 2010. Many market managers were responsible for multiple markets; thus, a total of 359 managers were called, representing all 543 markets. Messages were left when possible, and e-mail correspondence requesting information was sent after the first phone message was left. Staff called each manager 3 times before terminating the attempt. Of the 359 managers called, 12 declined to participate, 149 were unable to be reached after 3 attempts, 6 no longer had markets in operation, and 7 did not qualify for participation due to lack of SNAP or WIC FMNP redemption. The final sample included 185 managers who together managed 286 markets in 34 states, giving an overall response rate for the market managers of 53%. Managers were asked to identify just one market when responding to the interview questions, however, and this was the market used for the analysis.

At the end of the interviews, managers were asked to forward any data collection tools they used at their markets for further analysis and were asked to disclose who created the data collection tools. These instruments were further analyzed for content. Results of both the market manager interview and the survey content analysis are presented.

MARKET MANAGER INTERVIEW

Market managers were provided a brief introduction to the study and asked a series of 8 open- and closed-ended questions about their approach to farmers’ market data collection. Conversations lasted anywhere from 5 to 30 minutes depending on the length of responses. Questions asked about ongoing use of and approach to data collection methods, including customer surveys, vendor applications, how instruments were collected, tracking of sales data (including EBT and WIC sales), shopper data (including customer counts and demographics), tracking of customer health or diet, and use of an interactive survey method like a dot survey, as described by Lev et al.²⁵ When respondents indicated that they collected data at their markets, they were asked who created the data collection instrument(s). Participants were also asked to send a copy of the data collection tools used such as vendor applications and customer surveys for inclusion in a survey item analysis.

DATA COLLECTION TOOLS

Of the 185 manager participants, 86 sent their data collection tools, which, when compiled, represented 124 distinct instruments. Two types of instruments represented 98% of all tools: customer surveys (n = 40) and vendor applications (n = 82). Data collection instruments were stored in a database of similar tools (ie, vendor applications were kept separate from customer surveys) and question items were entered and coded according to tool-specific question-based categories developed by the research team. A 2-level hierarchy of codes was derived, revealing at the highest level general or overarching themes, which were then broken down into a number of more specific topics.

ANALYSIS

An overall chi-squared statistic was calculated to determine differences in data collection approaches according to market size. Markets were divided into quartiles (<12 vendors, >12 and <20 vendors, >20 and <30 vendors, and >30 vendors) to examine these differences. P-values of <.05 were considered statistically significant. Descriptive statistics were used to analyze the frequency with which the identified themes were included on customer surveys and vendor applications.

RESULTS

Market Manager Interview

In total, 14% of the 185 markets included in the sample were open year-round, and average market size was 27 vendors. Ninety-three percent (n = 172) of market managers interviewed reported at least one of the following data collection strategies: customer surveys, vendor applications, customer counts, or customer demographic tracking. Among those who collect data, nearly all (96%, n = 165) used a vendor application, about two thirds (64%, n = 110) reported surveying customers, more than half (57%, n = 98) conducted customer counts, and about one third (31%, n = 53) collected customer demographic information. Among those managers who reported conducting customer surveys, two thirds (n = 73) indicated that they created their own instruments. Manager interviews indicate that instruments tended to be developed by the managers themselves or in conjunction with informal partnerships with local university students or interns. Fewer than 10% (n = 16) of market managers reported collecting any data about customer diet, dietary change, or health.

Data Collection Approaches and Market Size

Fewer than half of markets in the lowest quartile of size conducted customer surveys, whereas 75% of markets in the highest quartile of size surveyed customers (P = .01). Tracking customer counts, the collection of demographic information, and the likelihood of creating their own customer surveys were not significantly different across market size quartiles.

Customer Survey

A total of 40 customer surveys were collected among the 110 managers who reported administering these surveys. The surveys collected are used in 14 states (Arizona, California, Connecticut, Indiana, Massachusetts, Michigan, New Mexico, New York, Ohio, Oregon, Pennsylvania, South Carolina, Vermont, and Washington) and were comprised of 3 to 35 questions each (mean = 11). Eight themes were identified through a qualitative analysis of survey questions that were asked about customers’ shopping frequency, reason for shopping, market preferences, expenditures, demographics, food assistance benefit (FAB) use, transportation/travel time, and diet/health.

The results indicate that the majority of customer surveys included questions on shopping frequency (85%), reason for shopping (83%), and market preferences (80%; Figure 1). Questions related to expenditures were present on about 3 out of 4 customer surveys (73%) and demographics questions were present on 67% of surveys. Slightly fewer than half of the surveys included questions about FAB (43%) and transportation/travel time (43%). One in 5 surveys included questions asking directly about diet or health (20%). Examples of questions in the health/diet category include agree/disagree statements such as: “I eat more fruits and vegetables as a result of the farmers’ market” and “My personal consumption of fresh fruits, vegetables, and whole grains has increased due to purchases at this market.” Multiple-response questions were also identified, such as “Since coming to this market, how has your consumption of fruits and vegetables changed, if at all? (increased, decreased, unchanged)” or “Since coming to this market, do you believe that you eat more, less, or the same variety of fruits and vegetables?” Open-ended questions such as “Do you think that this market helps the community be healthier? How/Why?” were also asked. A list of the question codes and examples of subtopics for the customer surveys are provided in Table 1.

FIGURE 1.

Frequency of themes appearing on customer surveys analyzed (n = 40).

TABLE 1.

Customer Survey Question Codes and Examples of Subtopics

Question code
Shopping frequency:
Customer shopping frequency (nonspecific)
Number of customers accompanying main shopper/referral by main shopper
Number of people you are shopping for in your household?
Do you shop at other farmers’ markets?
Reason for shopping:
Reason for shopping (nonspecific)
Relationship with vendor/other customers
Referral to market/how did you hear about the market?
Customers’ gardening experience/grow food at home?
Market preferences:
Customer market preferences (general)
Price satisfaction
Customer product/variety preferences
Customer food quality preferences
Educational material/demo suggestions/feedback
Entertainment/events at the market
Items/features customers would like to see at the market (items/features presently not available)
Expenditures:
Frequently purchased items
How do you pay at the market (cash/debit/credit card/EBT/WIC/coupons)?
Demographics
FAB Program—use/frequency/satisfaction:
FAB program satisfaction
Types of benefits (SNAP, WIC, bonus coupons) use/expenditure
Bonus coupon use/frequency
EBT use/frequency
WIC use/frequency
Knowledge of acceptance of FAB
Transportation/travel time:
How long does it take (minutes) to get to the market?
How far do you travel (miles)?
Mode of transportation to the market
Diet/health:
FAB impact on diet change/healthy habits
Market impact on diet changes/healthy habits

Vendor Applications

Among the 165 managers who reported administration of a vendor application, 82 shared their instruments. The applications varied in length with a minimum of 2 questions, maximum of 32, and mean of 11. Applications represented markets from 22 states (Arizona, California, Connecticut, Illinois, Indiana, Iowa, Kentucky, Maine, Maryland, Massachusetts, Michigan, New Jersey, New Mexico, New York, Ohio, Oklahoma, Oregon, Pennsylvania, South Carolina, Vermont, Washington, and Wisconsin). A total of 11 overarching themes were identified: product information, demographics, legal documents/certifications, market fees, equipment needs, product volume and sales, market feedback, interest in activities outside the farmers’ market, EBT/WIC/bonus coupon sales, communication methods with the manager, and FAB attitudes/experiences.

As seen in Figure 2, the vast majority of surveys contained questions about vendor product information and vendor demographics (96% and 93%, respectively). Additionally, about two thirds of the vendor surveys contained questions about vendor legal documents and certifications. Questions pertaining to EBT/WIC and bonus coupon sales were present in approximately one quarter of the vendor surveys, and only 2% of surveys asked questions about the vendors’ attitudes about and experiences with FAB. A list of the question codes and examples of subtopics for the vendor applications are provided in Table 2.

FIGURE 2.

Frequency of themes appearing on vendor applications analyzed (N = 82).

TABLE 2.

Vendor Application Question Codes and Examples of Subtopics

Question code
Product information:
What the vendor grows/produces/crop availability
Primary product
Value-added products
Farm/garden; size/acre; local; nursery
Baked goods/ingredient characteristics
Prepared/processed foods
Demographics
Legal documents/certifications:
Licensing/insurance (liability)
Tax ID/SS#/Federal ID#/Inspection #
Certified organic
State Department of Agriculture/other certified/inspection (ie, food/nursery/kitchen/scale)
Market fees
Equipment needs:
Scale
Require electricity
Number of people employed/staffing/apprentices
Product volume and general sales:
Forms of accepted payment
Total percentage of product brought (what is brought to market) to market
Total percentage of leftover product
Total volume
Price of items to be sold
Market feedback:
Strategies to increase customer attendance
Relationship with (regular) customers/feedback from customers
Observations/changes with customer base
Reasons for participating in market
Customer counts
Interest in other sales/activities outside the farmers’ market:
Food/product demonstrations/sampling
Vendor offer CSA
EBT/WIC/bonus coupon sales:
Accept bonus coupons
Accept EBT/SNAP
Accept WIC
Percentage daily total sales/gross market revenue bonus coupons
Percentage daily total sales/gross market revenue EBT
Percentage daily total sales/gross market revenue WIC
Accept other forms of FAB (ie, senior)/participate in FMNP
Percentage daily sales/gross market revenue of other types of FAB
Communication methods with manager
FAB—attitudes/experiences:
Changed farming/marketing practices because accept FAB
Accepting FAB improves sales/help business
Other vendors accept FAB influence you to participate/recommend accepting FAB to others
Knowledge/education about FAB
Types of marketing/promos to FAB customers

DISCUSSION

Our findings indicate that managers of farmers’ markets that accept SNAP and WIC vouchers are collecting a detailed amount of information about farmers’ markets. Almost all (93%) of the farmers’ market managers in our sample conducted at least one type of data collection activity. The most common types of data collection activities used by the managers were vendor applications and customer surveys, reported by 96% and 64% of the sample, respectively. Of the managers conducting customer surveys, two thirds created their own instrument. Customer surveys were more frequently administered at markets in the largest quartile of size (>30 vendors). Manager interviews indicated that instruments tended to be developed by the managers themselves or in conjunction with informal partnerships with local university students or interns. One exception is the case of California, where state-monitored market certification programs mandate farmer-reporting requirements. As a result, for markets in California, instruments are not developed by the managers or with university students/interns. Fewer than 10% of market managers reported collecting any data about customer diet, dietary change, or nutrition and, of the customer surveys collected, 20% included questions related to diet or health. The most common topics included in customer surveys were shopping frequency, reason for shopping, and market preferences. Vendor applications focused on vendor product information and legal documents and certifications.

Given the often limited budgets and reliance on volunteers at farmers’ markets, our finding that dietary questions are not prominent components of farmers’ market manager data collection efforts is not surprising. Evaluation activities and the development of tools to assess health are most likely not priorities among market workers. Furthermore, market managers may not have expertise in these areas and they may not view efforts to capture the health impact of markets as part of their mission. However, because many farmers’ market managers are already collecting data via customer surveys and vendor applications, opportunities may exist for local and state health practitioners to leverage data already being collected and to expand these data collection activities.

For example, state and local health practitioners and researchers may be able to use the data collected with customer surveys and vendor applications to examine issues related to accessibility of markets, track the utility and success of interventions such as increasing food assistance benefits use at farmers’ markets, and understand how to improve the use of farmers’ markets in increasing fruit and vegetable intake and improving overall health.

For instance, 73% of the customer surveys collected information about customer expenditures, including questions like most frequently purchased items, method of payment, and use of FAB programs. Forty-three percent of customer surveys asked about transportation method and travel time to the markets. Vendor applications are widely used and provide valuable information about market sales, product mix, total sales from various FAB programs, and types of marketing/promotion aimed at FAB customers that can be leveraged in studies of market impact. The findings also suggest that it is feasible to incorporate diet- or nutrition-related questions on surveys (20% included them), although this practice has not been widely adopted. Because smaller markets are less likely to conduct customer surveys, additional effort may be needed to include them in data collection activities.

Though customer surveys and vendor applications represent a viable, practical way to access information about markets, the authors recognize that larger and more sophisticated studies are needed to understand the full impact of markets on diet. Some examples include assessing the dietary and health impact of the introduction of a farmers’ market through pre- and post-surveys of households living within a reasonable radius around the farmers’ market site and assessing the long-term change in diet of farmers’ market attendees. For instance, little is known about the effects of changes in product mix on purchasing and consumption over time; maintenance of behavior changes in seasons when markets are closed; and spillover effects of frequenting farmers’ markets to other healthy habits such as increased water consumption or decreased consumption of foods with low nutritional value. Given the current epidemic of childhood obesity, studies should also focus on understanding whether farmers’ markets impact childrens’ behaviors.

Regardless of the types of data collection done, efforts should be made to use consistent and widely accepted assessment tools, such as those available on the National Cancer Institute’s website²⁵ and on the National Collaborative on Childhood Obesity Research’s Measures Registry’s website.²⁶ The wide variety of measures collected, measurement tools, and outcomes examined when evaluating the dietary impacts of farmers’ markets makes it difficult to compare results across evaluations and develop consensus on the dietary impact of introducing a farmers market in a community. Groups already working in the field should share tools and survey questions to facilitate the generation of comparable data. Public health funders should invest in surveillance and evaluation infrastructure to create a Web-based central clearinghouse for recommended measures of health and diet appropriate for the farmers’ market context and central location where these data can be uploaded so that a national data set is available for analysis.

Limitations

This study had several strengths and limitations that should be noted. Though the sample in this analysis consisted of only those farmers’ markets in the AMS online directory that accepted SNAP and WIC FMNP (less than 10% of the total number of markets), our markets were similar to the national average for average number of vendors (27 vs 31) and percentage of markets open year-round (14% vs 13%).^13,14 The response rate (53%) for the market manager interview was low, and only a small number of tools were available for analysis compared to the number of managers interviewed. Because the study was conducted during the summer months, the busiest time for many market managers, response rates may have been lower than at other times of the year. However, to our knowledge, this is the first study to assess whether farmers’ markets routinely collect nutrition data from their customers. The information about the type of data being collected at farmers’ markets and the usefulness of such data may encourage continued and expanded collection and prompt collaborations between public health officials and market managers.

CONCLUSIONS

A limited number of studies suggest that the benefits of farmers’ markets extend beyond increased community development and income for farmers^19,20 to improved health for consumers, especially low-income individuals.²¹ Yet in the peer-reviewed literature, nutrition and diet outcomes are infrequently measured outside of farmers’ market incentive programs. In our study, we found that customer surveys done by farmers’ market managers do not commonly include questions related to diet (20% of surveys analyzed). Given efforts to promote and increases in the number of farmers’ markets across the country, it is important that we better understand the impact and potential of these markets to encourage healthier eating and better health by making better use of current data collection efforts at markets and by partnering with organizations working in this area. The timing is critical for evaluation data. Improving food environments through environmental and policy strategies is at the forefront of many public health efforts and an important focus for US states and communities, yet many states are in budget crises and with limited resources and therefore must focus efforts on those with greatest impact. A better understanding of the health impacts of farmers’ markets will aid policy makers, philanthropists, and nonprofit agencies in weighing the costs and benefits and identifying best practices.