The SNAP-Ed Evaluation Framework: demonstrating the impact of a national framework for obesity prevention in low-income populations

The SNAP-Ed Evaluation Framework offers a new suite of evaluation measures toward eliminating disparities that contribute to poor diet, physical inactivity, food insecurity and obesity.

Abstract

The United States Department of Agriculture’s Supplemental Nutrition Assistance Program Education, known as SNAP-Ed, is the country’s largest and most diverse community nutrition program. In 2017, nearly 140 SNAP-Ed implementing agencies (SIAs) and hundreds of contractors delivered nutrition education to almost 5 million people in nearly 60,000 low-resource sites. Millions more were impacted with social marketing campaigns and policy, systems, and environmental changes. This article introduces and describes the benefits of the newly developed SNAP-Ed Evaluation Framework (Framework) and companion Interpretive Guide to consistently measure SNAP-Ed outcomes across different settings. The Framework uses the social ecological model as its underlying theory and features 51 indicators across four levels: Individual, Environmental Supports, Sectors of Influence, and Population Results. Topline findings from the first-year Census to track Framework adoption found that most SIAs intended to impact indicators closer to the inner levels of influence: Individual (mean = 59% of SIAs; SD = 22%) and Environmental Settings (mean = 48%; SD = 23%). As yet, few SIAs targeted outcomes for long-term indicators (mean = 26%; SD = 15%), Sectors of Influence (mean = 20%; SD = 12%), or Population Results (mean = 30%; SD = 11%). An in-depth example of how one state is using the Framework is described. The SNAP-Ed Evaluation Framework offers a new suite of evaluation measures toward eliminating disparities that contribute to poor diet, physical inactivity, food insecurity and obesity. Practitioners will need technical assistance to implement the Framework, especially to measure longer-term, multi-sector and population results, and to maximize effectiveness in SNAP-Ed.

Implications.

Practice: Practitioners should use the new national SNAP-Ed Evaluation Framework to achieve a common set of outcomes to prevent obesity by improving healthy eating, increasing physical activity, and decreasing food insecurity.

Policy: The new, Congressionally mandated SNAP-Ed electronic reporting should use the indicators and measures in the SNAP-Ed Evaluation Framework as the foundation to help decision makers at the local, state, and national levels be assured that SNAP-Ed resources are achieving reductions in disparities in low-resource communities.

Research: To maintain the scientific integrity of the SNAP-Ed Evaluation Framework, researchers should work with practitioners to add and continually test indicators, metrics, and evaluation instruments suitable for multilevel and multi-sector community-based interventions across the nation.

INTRODUCTION

The Supplemental Nutrition Assistance Program Education (SNAP-Ed) Evaluation Framework (Framework) [1] focuses the United States Department of Agriculture’s (USDA) largest community nutrition education program on outcomes that help eliminate diet- and physical activity-related health disparities. As the nutrition arm of SNAP (formerly the Food Stamp Program), SNAP-Ed addresses healthy eating, food insecurity, physical activity, and obesity prevention by developing and using evidence-based interventions designed for a wide range of low-resource settings [2]. Its intended audience are the 90 million Americans with incomes below 185% of the Federal Poverty Level (FPL), including the subset of 40 million people with incomes below 130% of the FPL who may be eligible to participate in SNAP in any given month, especially families with children and working-age adults [3–5]. Collectively, SNAP-Ed interventions strive to layer complementary efforts in direct education, social marketing, and changes in policies, systems, and environments (PSE) upon one another to achieve large-scale, sustainable results. Many low-income and food-insecure families experience obesity and hunger simultaneously. The interaction of limited resources; poor access to healthy, affordable food; high levels of stress, anxiety, and depression; cycles of food deprivation and overeating; fewer opportunities for physical activity; greater exposure to food marketing; and limited access to healthcare in low-resource settings contributes to excess rates of obesity [6]. Low income is associated with a higher prevalence of chronic disease [7] and low diet quality [8] than is high income. One study found that food insecurity is positively associated with a variety of chronic health conditions, including diabetes, hypertension, coronary heart disease, kidney disease, arthritis, hepatitis, stroke, and cancer [9]. It also found that food insecurity status was a stronger predictor of chronic health conditions than income.

SNAP-Ed is designed as an ongoing, comprehensive solution to help change long-term, systemic conditions that negatively impact Americans whose low incomes qualify them for food assistance or similar public programs. Funded at $433 million (2019), SNAP-Ed is delivered in all 50 states, the District of Columbia, Guam, and the U.S. Virgin Islands by about 140 university, public, nonprofit, and tribal agencies under contract with SNAP State Agencies [3]. Known as SNAP-Ed Implementing Agencies (SIAs), many subcontract with other governmental, nonprofit, and business organizations to deliver local services, conduct media campaigns, or provide evaluation support. SNAP-Ed translates recommendations in the Dietary Guidelines for Americans (DGA) into behavior change strategies and interventions tailored to low-income populations.

At its initiation in the early 1980s, the SNAP-Ed goal of assisting low-resource households to eat healthfully on a limited budget was pursued solely through direct education. USDA added social marketing strategies in the late 1990s. The passage of the Healthy, Hunger Free Kids Act of 2010 expanded the SNAP-Ed mission to include physical activity, obesity prevention, and the use of evidence-based approaches, and it required the use of comprehensive public health approaches [10]. In light of this change, there was a need for a new Framework to guide SNAP-Ed agencies in planning, implementing, and evaluating comprehensive public health interventions. In response, starting in 2013, eight Western states began developing and testing the Framework to provide a consistent strategy in measuring SNAP-Ed effectiveness. In 2016, it was refined for national use by USDA’s Food and Nutrition Service and SNAP-Ed practitioners under the auspices of the National Collaborative for Childhood Obesity Research (NCCOR) which convened the SNAP-Ed Evaluation Framework Working Group of researchers, evaluators, and funders [1].

This article demonstrates how the design of the Framework focuses on key outcomes and provides a comprehensive approach to measure and track changes resulting from program efforts across the country. The Framework is flexible and provides a menu of options for measuring changes from interventions across a wide variety of regions, states, territories, Indian tribal organizations, health departments, and non-governmental organizations that implement SNAP-Ed. Its evaluation indicators draw from reputable measures such as those in Healthy People 2020 and the Food Environment Atlas [11,12]. Indicators and outcome measures are tailored to low-resource settings and conform to USDA program requirements [2]. The companion Interpretive Guide provides consistent measures and validated instruments for each indicator; promising evaluation approaches for planning, assessing, and catalyzing obesity prevention efforts; and the potential to aggregate results among all SNAP-Ed programs [1]. Online, the Interpretive Guide provides links to all outcomes, each with validated measures and evaluation tools (see: https://snapedtoolkit.org/framework/index/) [1]. The Framework complements the SNAP-Ed Strategies & Interventions Obesity Prevention Toolkit for States (Toolkit), an on-line, interactive compendium of evidence-based SNAP-Ed interventions that have been evaluated for effectiveness with various SNAP-Ed audiences (see: https://snapedtoolkit.org/) [13].

OVERVIEW OF THE SNAP-ED EVALUATION FRAMEWORK

The Framework applies the social–ecological model (SEM) to obesity prevention. The SEM is a well-established, theory-based, interdisciplinary and broadly applied paradigm that highlights how multiple factors interact to influence population behaviors at complementary spheres of influence: individual, environmental settings, sectors of influence, and social/cultural norms and values [14]. The SEM underscores the importance of identifying leverage points, intentionally layering interventions, using reinforcing strategies, and engaging multiple sectors to achieve sustainable population-level results. Because of the variety of settings, subpopulations, and specific intervention techniques that are employed, SNAP-Ed is classified as a complex intervention that, by definition, is faced with exquisite evaluation challenges [15].

The Framework makes the SEM actionable by providing specific outcome indicators for each level of influence and by capturing the progression of short-, medium-, and longer-term outcomes over time. It starts with readiness and capacity-building to initiate sustainable behavioral changes or new organizational practices. As shown in Figure 1, the complete Framework offers a choice of 51 indicators that SNAP-Ed and similar community programs can use to plan and track the success of their specific strategies and interventions. USDA requires SNAP-Ed agencies to report on a minimum of seven Priority Indicators annually, and most SIAs do more. Partnerships operating at-scale in each sphere of the SEM are necessary precursors to conducting successful interventions that can reduce disparities and support health in whole populations over time.

Fig 1.

SNAP-Ed Evaluation Framework: nutrition, physical activity, and obesity prevention indicators.

Each sphere of influence in the SEM is translated into four levels of the Framework: Individual, Environmental Settings, Sectors of Influence, and Population Results. The principal evaluation question, intervention focus, and progression of indicators used in the Framework follows.

Individual level—To what extent does SNAP-Ed programming improve and sustain participants’ healthy eating and physical activity behaviors?

Framework indicators at this level align with the individual and group nutrition education and physical activity promotion activities that are a core strategy for most SNAP-Ed programs. Short-term indicators track participants’ preliminary steps toward behavior change such as their goals and intentions about nutrition and physical activity. In the medium-term, behavior change is measured in a variety of possible areas—healthy eating, food resource management, physical activity and reduced sedentary behavior, and food safety— depending on the intervention’s focus. Most commonly, SNAP-Ed practitioners measure behavior change by administering validated, reliable surveys, such as the Visually-Enhanced Food Behavior Checklist, at the beginning and end of a multi-lesson educational series [1]. Long-term indicators assess the maintenance of reported behavior changes for a minimum of six months after program completion, in accordance with the Transtheoretical (Stages of Change) Model [16,17]. For example, in one state, improvements in food security scores persisted in households with children for one year after a 4- to 10-week SNAP-Ed intervention [18].

Environmental Settings—To what extent does SNAP-Ed programming create and sustain access and appeal of healthy eating and physical activity choices in the settings where people eat, learn, live, play, shop, and work?

The environmental settings indicators reflect the application of the widely used RE-AIM framework to the planning and evaluation of PSE change(s) at diverse SNAP-Ed settings where people eat, learn, live, play, shop, and work [1,19]. In addition to efficacy or effectiveness, RE-AIM emphasizes the importance of factors such as reach, adoption, implementation and maintenance — to the real-world impact of a public health intervention [19,20]. In the short term, Need and Readiness (Indicator ST5) identifies site needs, as well as assessing staff and organizational readiness to make changes. For example, the one way to measure ST5 is the Alliance for a Healthier Generation’s Healthy Schools Program Assessment Tool helps schools to identify areas for improvement in their nutrition and physical activity policies and practices. Medium-term indicators track schools’ adoption of PSEs such as incorporating physical activity breaks in the classroom or the further adoption of USDA’s Smart Snack standards, as well as the number of students reached by these changes. Long-term indicators measure implementation and whether elements are in place that are known to be important to positive outcomes such as training and parent/community involvement. In one state, SNAP-Ed interventions increased students’ physical activity scores in a large metropolitan area, as well as school-based physical activity opportunities (in-classroom, recess, and before school programming) in elementary schools statewide [21,22]. Another state reported process and outcome evaluation results from multilevel SNAP-Ed interventions in diverse community settings such as schools, pre-schools, worksites, catering trucks, SNAP offices, and health centers [23].

Sectors of Influence—To what extent do SNAP-Ed programs work with others in the public, non- profit and business sectors to collectively impact lifelong healthy eating and active living in low-income communities?

Indicators at this level move beyond a single program or funding stream. The complexity of factors contributing to chronic diseases, such as food insecurity and obesity, coupled with structural and systemic factors that perpetuate health inequity, require a broader frame. The single short-term indicator focuses on the quality of multi-sector partnerships or coalitions that are addressing nutrition or physical activity-related community changes. SNAP-Ed implementers are expected to partner with diverse stakeholders in sectors such as agriculture, public health, healthcare, education, community design, media, and the food industry to increase access to healthy foods and create environments more conducive to physical activity in low-income areas. Indicators at this level track specific policies and practices in those sectors that have been associated with healthier behaviors and community conditions.

Though funding for SNAP-Ed may be used only in settings or census tracts where the majority of the audience is low-income, many collaborative efforts can have a more far-reaching positive impact since their scope can be city- or county-wide, regional, or statewide. SNAP-Ed programs have the potential to contribute to these broad-scale, multi-sector changes. SNAP-Ed program partners can contribute to observed outcomes through Collective Impact approaches [24]. The data for Sectors of Influence indicators are diverse and come from both primary and secondary sources. For instance, Food Systems (Indicator LT12) tracks food policy councils, farm-to-community initiatives, and census tracts with healthier food retailers from a variety of sources including USDA’s Food Hubs Directory and Food Environment Atlas. Another example is Health Care Cost Savings (LT17), which is calculated from the reduction in prevalence of persons told by a medical professional they have high blood pressure, Type 2 diabetes or pre-diabetes, high blood cholesterol, obesity, and asthma. Potential data sources include the Federal Health Resources Services Administration (HRSA) data warehouse and other existing electronic medical registries from Federally-Qualified Health Centers [1].

Population Results—To what extent does SNAP-Ed programming help achieve recommendations in the DGA in low-income subgroups, compared with the general population?

Interventions at the individual, organizational, and sectoral levels of the Framework all contribute to population-level results. The Interpretive Guide calls for two main evaluation approaches: (a) surveys among cohorts of SNAP-Ed participants at qualified sites and (b) population-level surveillance of low-income audiences. For example, one state SNAP-Ed program incorporated a demographic screener into the state’s biennial Behavioral Risk Factor Surveillance System (BRFSS) that noted participation in SNAP, the Special Supplemental Nutrition Program for Women, Infants, and Children (WIC), and/or the National School Lunch Program. This allowed them to track and compare changes between the general population, low-income adults using USDA food assistance programs, and others with similar low incomes [1]. Another state compared dietary behaviors reported through its telephone survey of low-income moms in census tracts with high, moderate and low exposure to SNAP-Ed interventions. Higher program exposure was associated with lower reported consumption of sugar-sweetened beverages and high-fat foods and higher intake of fruits and vegetables [25].

NATIONAL DISSEMINATION OF THE FRAMEWORK IN ITS FIRST YEAR

To determine whether SIAs were using the new Framework at baseline, an interdisciplinary team conducted the 2017 Census of Intervention, Evaluation, and Reporting Activities Planned by SNAP-Ed Implementing Agencies (Census) [26]. The Census was administered electronically to all 136 known SIAs. Each was asked which of the 51 indicators in the Framework and its subcontractors intended to impact and/or evaluate in 2017. A total of 124 unique SIAs representing the work of hundreds of partners completed the Census (91% response rate). At least one SIA in all 50 states and the District of Columbia completed the Census. The following are highlights based on original Census data collected and to be published in a manuscript in progress [26].

Fifty-four percent (54%) of all SIAs were from university settings, either USDA Cooperative Extension agencies in Land Grant Universities or other universities. Another 23% were from nonprofit organizations, 15% from state or local government, particularly public health, and 6% were from Indian Tribal Organizations. Of the 28 states that ranked in any of the “top 10” for prevalence of obesity, food insecurity or low-income residents, 75 of the 85 SIAs in those states (88%) responded. No patterns of non-response that would limit generalizability were found.

While results for all 51 Framework indicators were assessed, only those for USDA’s seven Priority Indicators and others relevant to this special issue are presented in Figure 2.

Fig 2.

Census data showing intent to impact and evaluate selected SNAP-Ed Evaluation Framework indicators

The Census found wide variability in intent to impact and/or evaluate the Framework indicators in SIAs nationally. Respectively, the shading in each bar of Figure 2 displays the percent of SIAs that: intended to impact and evaluate each indicator; that intended to impact, but not evaluate or track each indicator; and that had not planned interventions to impact an indicator, or were unknown. As was expected with the newly introduced Framework, more SIAs intended to impact indicators closer to the inner spheres of influence in the SEM: Individual sphere (mean = 59%; SD = 22%) and Environmental (mean = 48%; SD = 23%), with the fewest agencies working in Sectors of Influence (mean = 20%; SD = 12%). A somewhat larger percentage of SIAs (mean = 30%; SD = 11%) intended to impact Population-level outcomes. Not shown is the drop-off in evaluation of the long-term indicators, as with Individual behaviors that persist at least 6 months and in Environmental Settings where repeated organizational assessments of PSEs show measurable change. Since the Framework was implemented nationally only in 2017, working toward and evaluating longer-term, multi-sector and population changes was likely to have been a new activity for many SIAs.

Figure 2 also indicates that at every sphere of influence, fewer SIAs planned or were able to conduct evaluation or track the specific indicators for all the topics in their annual state plan. Overall, SIAs, on average, intended to impact 19 indicators (SD = 11 indicators) and evaluate 12 indicators (SD = 7 indicators). Across all 51 indicators the mean gap was 15 percentage points between intent to impact and evaluate (range = 3%–30%). While the use of evidence-based interventions reduces the need for repeated evaluation, other barriers include resource constraints, the lack of reporting methods and systems, and the shortage of specialized technical know-how.

Real-world example: applying the Framework to the implementation of food security interventions in the health care setting

Maine SNAP-Ed is using the Framework to plan, implement, and evaluate health care clinical-community interventions as one of several statewide strategies. With its health care partners, Maine SNAP-Ed works across all levels of the Framework to serve its food-insecure residents by establishing direct education referrals, developing community resource guides, coordinating emergency food aid, integrating gleaned produce into food distribution systems, and developing on-site social marketing messages that support healthy lifestyle changes. For the Individual and Environmental levels, interventions are established in clinical sites that serve predominantly low-income individuals, such as Federally Qualified Health Centers. Use of the validated, two-question food security screening tool The Hunger Vital Sign developed by Children’s HealthWatch [27] helps to identify individuals for nutrition education referrals and to connect patients to systems and resources designed to increase access to healthy food options. For Sectors of Influence, Maine SNAP-Ed coordinates efforts to scale up and expand the reach of clinical-community linkages through multi-sector collaborations and coordination of social marketing efforts. Complementary sector-wide interventions in the community are designed to collectively impact long-term reductions in disparate health outcomes for low-income residents.

Figure 3 demonstrates how the Framework is used across intervention levels using a variety of intervention types, illustrating how its design can be applied to real-world planning of interventions and evaluation. Evaluations will be conducted at all levels of the Framework over the course of the 5-year funding cycle. Initial evaluations at the Individual level will measure goals/intentions and behavioral changes by pre- and post-testing of self-reported behavioral change following nutrition education. Longer-term changes measure the maintenance of behavior change at a minimum of 6 months post-intervention.

Fig 3.

SNAP-Ed Evaluation Framework level, Maine SNAP-Ed intervention type, and indicators impacted.

For Environmental Supports, coalition inventories will examine community capacity by measuring the strength and depth of sector-level partnerships and collaborations. Within health care settings, practice-tracking tools, patient/provider experience surveys, and interviews will be used to identify PSE changes and estimate the number of patients reached and initial outcomes. Longer-term measures will track adoption and maintenance of changes such as referral systems, patient and provider satisfaction, and patient food distribution/acquisition.

Sector-level measures will study the number of sites and systems that newly adopt changes due in whole or in part to SNAP-Ed activities with partners. Plans to examine health care costs savings are preliminary. Initially, metrics like reductions in prevalence of persons told by a medical professional they have obesity, or a related chronic disease will be used. Longer-term measures will be informed by now-emerging chronic disease cost calculators.

Finally, public health surveillance systems will be used to examine Population results, dietary behaviors, and weight. National databases will be accessed for state- and local-level food security metrics. Combined, the Framework indicators and associated evaluation approaches are designed to increase the capacity of Maine’s SNAP-Ed program to capture outcomes across all spheres of the SEM and throughout the phases of the interventions, setting the stage for meaningful translation and dissemination of results to stakeholders. Maine has successfully overcome many of the barriers identified across states surveyed in the Census.

DISCUSSION

Framework use in practitioner settings

The primary benefit of the Framework is that it is practical and designed for use in a variety of community settings. The Framework is intended to make behavioral theories, including the SEM, RE-AIM, and the Transtheoretical Model accessible to practitioners working to eliminate disparities in healthy eating, physical activity, food insecurity, and obesity. For multidisciplinary practitioners at state and local levels, the Interpretive Guide provides an evidence-based template for setting priorities and choosing measures of success when planning, implementing, and evaluating their programs. The inventory of carefully selected and validated evaluation assessments, tools, and secondary data sets is designed to help agencies select outcomes suited to their mission and resources, join with like-minded stakeholders, and collect aggregable data that demonstrate collective impact on large, complex problems that none can tackle alone. For the first time, these SNAP-Ed resources provide consistency in the types of interventions, evaluation tools, and metrics to systematically build the evidence base among different practitioners and community partners. Taken together as living documents that will be updated as the field matures, the Framework, its Interpretive Guide and the SNAP-Ed Toolkit allow practitioners to compile an evidence base of emerging, practice-tested, and research-tested tools. So far, over 100 success stories from 36 states have been compiled, and 88 evidence-based interventions from 26 states have been aligned with Framework indicators in the SNAP-Ed Toolkit [13].

A second benefit is that the Framework provides a standardized yet flexible road map for linking program planning with evaluation. It requires practitioners to think through how multiple components of a complex intervention relate to, interface with and reinforce one another, what partnerships are needed to sustain change long-term without SNAP-Ed resources, and how these components can be evaluated across the enterprise [28]. Ultimately, SNAP-E evaluation data can be aggregated within a state or among states, thereby providing a conceptual foundation for the activities needed to achieve milestones toward goals such as policy change at a jurisdictional or organizational level. For instance, one statewide SNAP-Ed agency systematically identified data from third-party sources referenced in the Framework’s Interpretive Guide for tracking key metrics associated with its multi-sector interventions [29].

Settings and Sectors-level work occurs in partnership with other organizations and stakeholders, so the evaluation of such efforts can have a beneficial impact on partnering organizations. For example, 10 SNAP-Ed affiliated coalitions comprised of cooperative extension services, local health departments, and nonprofit partners in one state were asked to evaluate the characteristics of their coalitions as a short-term outcome to measure their capacity to address community-level changes in childhood obesity and food security [30]. Even though the initiatives varied by community, participating members reported that the process of coalition evaluation assisted them in strengthening their structures and processes, such as seeking a greater diversity of membership and developing stronger norms for participation. The process of evaluating themselves in alignment with the Framework and subsequently taking action based on the results strengthened certain characteristics of the coalitions themselves. This not only supported the SNAP-Ed evaluation, but it also increased buy-in and the likelihood of progress toward their community goals [30].

FRAMEWORK USE IN RESEARCH

Despite the Framework being relatively new, a growing number of scholarly works reference the Framework (see Supplementary Table 1). Researchers have published results using the indicators and measures featured in the Framework and Interpretive Guide, suggesting the Framework’s value in assessing the effectiveness of interventions across the multiple levels of the Framework. Use and refinement of the Framework’s existing metrics and tools, as well as the dissemination of newly identified evaluative approaches, will expand the technical capacity of states to scale up effective interventions. The Framework provides a platform, via the SNAP-Ed Interventions Toolkit, to examine how best to disseminate and/or translate successful locally- or regionally-implemented interventions.

Another series of research questions derives from the use of the Framework and the field of implementation science [31]. How does the Framework help stakeholders communicate about collective impact within and beyond partnerships like those in SNAP-Ed? Although the Framework may provide a common agenda and suggest common metrics, other challenges include the extent to which such indicators are adopted beyond the SNAP-Ed funding stream, what mutually reinforcing activities are conducted, and how continuous communications assure that such efforts result in synergistic outcomes. With every state having many different funding sources aimed at similar critical outcomes for vulnerable populations and low-resource communities, what do effective backbone organizations look like? What will it take for the research community to partner with others working in low-resource venues, such as SNAP-Ed, to capitalize on the Framework as a new resource in important, less understood fields such as agriculture and food systems, community design and safety, educational policy and attainment, marketing and mass communications, community preventive services, economics, and social norms?

FRAMEWORK USE IN POLICY DECISIONS

The Framework is explicit in its operationalization of policy as occurring at multiple levels and in all sectors, not solely government. An organization makes a policy change when it sources locally or regionally grown foods in its cafeteria, when a jurisdiction sets policy by establishing a fund for walking trails and pedways, or when a grocery store chooses to promote healthy foods or offer them at lower prices [1]. The ability to articulate these changes as program outcomes using Framework indicators provides a more complex view of SNAP-Ed interventions. What remains a challenge, however, are better ways to measure, aggregate, and communicate the impact of PSEs to decision makers.

The Framework isolates complex facets of the SEM and focuses attention to community policies resulting in low-income audiences having inequitable access to environments that favor healthy lifestyles [1]. Breaking this cycle has the potential to impact health care costs, a Framework indicator at the Sectors of Influence level and discussed earlier in the Maine example [1]. For example, each food-insecure person is estimated to incur an additional $1813.17 in health care costs annually, much of which is borne by Medicaid and other public health care. When multiplied by more than 41 million Americans who are food insecure, about $77.5 billion is incurred each year [32]. Clearly, health care providers can help address “upstream” determinants of diet-related disease, obesity, and food insecurity. For instance, one SNAP-Ed program developed the Champion Provider Fellowship, a novel strategy that empowers, trains and supports health care providers to use their respected voices to influence local PSEs [33]. Champion Providers connect with their local health departments and collaborate with local leaders, community-based organizations, schools, faith-based organizations and businesses to advocate for solutions. In turn, the health clinics measure the impacts to food insecurity and obesity as tracked in electronic medical records and public health surveys.

NEXT STEPS/FUTURE OF THE FRAMEWORK

New Congressional statutes call for SNAP-Ed SIAs to accelerate their shift toward addressing a wider range of outcomes, including those that are longer-term and in the outer spheres of the SEM [34]. The Agriculture Improvement Act of 2018, commonly known as the Farm Bill, added new provisions that codify many current evaluation practices as well as aggressive new requirements that will shape ongoing work of SNAP-Ed with its partners [34]. These include electronic reporting, a clearinghouse of best practices, annual SIA reports becoming publicly available, and an annual Federal report evaluating the level of coordination among all USDA nutrition education programs. The intent stated by Congress is to standardize and improve evaluation, aggregate data, leverage results, partner with research-capable organizations, and share longitudinal data, especially for multi-year projects. These requirements offer stakeholders new opportunities for planning, implementing, and evaluating or tracking progress over time.

Further research is also needed to answer questions about the intensity, dose, and duration of intervention components needed for population impact, in accordance with the model of measuring population dose in community-based prevention interventions [35]. Because SNAP-Ed funds may be used only for service and evaluation, not research, the field would benefit greatly from research on its most popular interventions, from longitudinal studies of results, and from comparison studies with people or organizations that are ineligible for SNAP-Ed, or with those who are eligible but for whom SNAP-Ed resources are inadequate to offer services.

TRANSLATIONAL IMPLICATIONS

The Framework identifies a specific, comprehensive, and logical set of outcomes that are key to measuring health in low-resource communities and low-income populations. One of the challenges in improving the country’s health through healthier eating and increased physical activity is that “everyone has a role” [14], so where should any agency start? Any stakeholder can identify where to work based on its own mission, priorities, and resources, and choose among the indicators included in the Framework. The Framework’s inventory of evidence-based outcomes and measures serve as a focused menu for any partner that wants to contribute toward collective impact within any setting, at any level of influence, and across any sector.

Despite this flexibility, systemic barriers that SIAs reported in the Census make evaluation of long-term and population-level results in practice settings challenging. Whereas states like Maine have their own in-house evaluation and data systems, findings from the Census revealed one of the barriers to evaluating more complex multi-sector interventions is the lack of a data or reporting system. Consequently, the Framework inspired the creation of a new third-party Program Evaluation and Reporting System (PEARS) and over 26 state SNAP-Ed programs are entering uniform data [36]. Additionally, many SIAs are accessing the newly developed SNAP-Ed Engagement Network public-only portal that produces customized reports using secondary data sources for 30 of the 51 Framework indicators [37]. These user-friendly data portals make it easier for more states to track, measure, and report outcomes in accordance with the Framework. Increased utilization of common metrics and reporting of SNAP-Ed strategies and intervention will ultimately strengthen the dissemination of practice-based evidence, helping to replicate approaches for nutrition education and obesity prevention across the country.

CONCLUSIONS

The SNAP-Ed Evaluation Framework is designed to address the gap between knowledge and practice to support greater dissemination and translation that is necessary to scale up diverse evidence-based interventions designed to improve food security and health. The Framework provides a long-needed template for action that can be as diverse as the communities SNAP-Ed serves. Long-term, population-level improvements in diet, physical activity, and obesity prevention are primary goals of SNAP-Ed, and addressing these factors in the presence of food insecurity is daunting. Using an evidence-based approach, working together to coordinate efforts, and critically evaluating progress holds promise for achieving measurable, sustainable impacts in entire populations. SNAP-Ed Implementing Agencies require in-depth technical assistance, standardized data systems, and further evaluation training to fully realize and benefit from the national SNAP-Ed Evaluation Framework. Ultimately, researchers and practitioners will be able to speak with one voice when communicating the collective impact of the nation’s largest community nutrition.

Funding

This work was supported in part by USDA, Food and Nutrition Service. This funder did not have a role in writing this work or deciding to submit it for publication. This manuscript does not reflect the official views of USDA.

Compliance with Ethical Standards

Conflicts of Interest: The authors declare they have no conflicts of interest.

Human Rights: This article does not contain any studies with human participants performed by any of the authors.

Informed Consent: This study does not involve human participants and informed consent was therefore not required.

Welfare of Animals: This article does not contain any studies with animals performed by any of the authors.