Leveraging Implementation Science in the Public Health Response to COVID-19

The rapid mobilization of coronavirus disease 2019 (COVID-19)–related research (and strategies for addressing behavioral and societal implications) shows that the 17-year lag for research evidence to translate into practice or policy that benefits the public’s health is not an inevitability.^1,2 The immediate threat of the pandemic thrust public health and well-being into the spotlight.³ Scientific journals and funding agencies have adopted expedited review processes, publishers have waived open access fees, and researchers who are able to do so have mobilized to investigate, write, and share findings with broad audiences, often working collaboratively to keep pace as the situation unfolds. Collectively, these efforts have helped generate and disseminate crucial research that is accessible to practitioners.^4,5

The current situation—unprecedented acceleration of research; rare, swift passage of health-focused policies; and increased attention to science—has led to many opportunities for public health research.^4,5 As the pandemic continues, persistent public health concerns such as chronic disease risk and food insecurity (ie, household-level condition of limited or uncertain access to adequate food associated with myriad adverse health outcomes) will also continue, and inequities will be exacerbated.⁶ Already, the pandemic is illuminating glaring disparities in health outcomes among people of color, and research to address these disparities has been prioritized, resourced, and valued.^7-13 The pandemic presents a timely opportunity to study the altered course of upstream policy, environmental, and organizational efforts designed to alleviate disparities.^7,11-15 In these unprecedented times, many interesting and worthy research questions arise; however, it is critical to take full advantage of the increased efficiencies in the research process and prioritize questions that can benefit population health and promote health equity.¹⁴ Rigorous, stakeholder-driven, “consequential” natural implementation research can reveal the gaps in existing policies and systems and allow for more efficient approaches to identify and evaluate innovative strategies to fill those gaps.^14,16-18

In this commentary, we examine the crucial role of dissemination and implementation (D&I) science—the study of methods to promote adoption and integration of evidence-based research in real-world policy or practice—to improve public health post–COVID-19.¹⁸ D&I science was created for this very situation, in which scientific knowledge is greatly needed but only if it holds practical relevance for the policy, environmental, and organizational systems that advance health. To demonstrate the utility of timely D&I research, we discuss its application to rapid evaluations of federal child nutrition assistance programs deployed during the COVID-19 pandemic. Such programs have long played a role in the nutrition safety net in the United States, where 14% of households with children were food insecure in 2018, and disparities among children of color and children in rural areas are well documented.^19,20 The programs remained critical during COVID-19, when schools closed, unemployment rates rose precipitously, and food insecurity rates among children doubled.²¹

We highlight how researchers, administrators, advocates, and other partners mobilized, working quickly as a collaborative to rigorously study policy implementation in ways that have not yet been possible. We discuss 2 related concepts: (1) federal nutrition assistance programs, which D&I scientists would call “the thing,” or the evidence-based intervention, policy/program, or innovation, and (2) implementation strategies, which are strategies used in schools or districts to “do the thing”—that is, ensure that programs are adopted and implemented, reach food-insecure children, and can be equitably sustained.^22,23 We describe pre–COVID-19 implementation strategies for federal nutrition assistance programs and outline how collaborative D&I research during and after COVID-19 can help refine and/or scale these strategies to expand program impact. In addition, given that D&I guidance on strategies for dissemination, adoption, and implementation of evidence-based policies is underdeveloped,^24,25 we also provide examples of policy- and practice-relevant research questions using the RE-AIM (Reach, Effectiveness, Adoption, Implementation, Maintenance) framework.²⁶ We then highlight D&I tools that can help ensure that discoveries during this unique time are relevant in the long term and prioritize equity.^18,27

School-Based Child Nutrition Programs Pre–COVID-19: A Brief History

The US Department of Agriculture (USDA) Food and Nutrition Service administers a suite of school-based federal nutrition assistance programs (eg, the National School Lunch Program, School Breakfast Program, and Summer Meals Programs [SMP], which can be offered in non–school-based settings).^28-30 The USDA sets standards for program adoption and implementation and works with state entities that are responsible for approving and overseeing local educational authorities (hereinafter, districts) to provide nutritious meals for free or at a reduced price each school day for more than 29 million children living in low-income households.³⁰ In 2010, the Healthy, Hunger-Free Kids Act (HHFKA) allowed the USDA to strengthen these programs, which effectively reduce food insecurity and provide nutritious meals but were overdue for updates using the latest evidence-based nutrition and chronic disease prevention research.^29,31 Among other changes, HHFKA strengthened nutritional standards for foods sold on school campuses and provided implementation guidance and technical assistance to help schools meet updated standards.³²

Despite current administration efforts to roll back stronger nutrition standards,³³ research demonstrates the positive impact and successful implementation of HHFKA provisions.³⁴ However, school meal programs are underused; average daily rates of student participation are 56% for lunch programs and 21% for breakfast programs.³⁴ Several implementation strategies (ie, state or local actions taken to enhance adoption, implementation, and sustainability of evidence-based child nutrition programs), many of which align with the Expert Recommendations for Implementing Change (ERIC; a compilation of implementation strategies gathered by D&I experts),²³ appear to improve the implementation of child nutrition programs. Collaborative research with federal, tribal, state, and district agencies during the past decade has focused on identifying participation barriers and examining innovative strategies to increase participation equitably. As examples, strategies such as centralized technical assistance through state agencies, learning collaboratives, and accessible funding through USDA grants have increased quality and participation.^35,36 In addition, a major federal policy change in HHFKA was the Community Eligibility Provision, which allows eligible high-poverty schools and districts to offer free meals to all students, with no paperwork required.^37-39 This provision uses 3 ERIC strategies: making billing easier, changing records systems, and alternating fee structures. At the school level, tested strategies include marketing and increasing access by reducing time and financial barriers (eg, grab-and-go options, universal free meals).^40-44 Substantial progress has been made, but additional research could determine which strategies, or combinations of strategies, are best suited for scale up and what adaptations might be needed.³⁴

Underuse is of even greater concern for the SMP, which in 2018 reached only 1 in 7 low-income children who participated in school meals during the school year.⁴⁵ Increasing participation in the SMP is critical, because summer months (and other periods of school closure) bring increased food insecurity, reduced diet quality, and accelerated weight gain, particularly among children of color.^45,46 Despite its importance, the SMP is understudied compared with in-school programs.⁴⁷ A primary barrier to site-level SMP participation is that the requirements are similar to those in school (ie, congregate meal service, time between meals, meal limits), which limits site eligibility (because sites have to accommodate congregate meals), hours of operation, and number of meals served. Another challenge is transportation, particularly in rural areas.^45,46,48,49 Additional barriers include a lack of awareness of site locations among families and a lack of cultural inclusivity.⁵⁰ Although implementation strategies have been identified to address these barriers, few have been empirically tested for effectiveness in controlled studies. Alternative strategies have been developed to address barriers to participation, such as altering incentive/allowance structures²³ through Summer Electronic Benefits Transfer (ie, providing additional resources for families who participate in meals programs to purchase food during the summer months). Although pilot work shows this strategy reduces food insecurity and improves diet, it has not yet been implemented on a national scale.^45,47,49

Conducting implementation research at the scale required to assess key policy outcomes (eg, compliance, adoption/implementation, adaptation, and acceptability) and test, compare, or scale implementation strategies is challenging for child nutrition assistance programs.⁵¹ For federal policies to reach their full potential impact, they must be interpreted and implemented by a variety of intermediate organizations or legislative jurisdictions (ie, federal, tribal, state, district, school boards).⁵² The policies themselves are complex and not necessarily designed with extensive testing of implementation strategies in mind or consideration of inevitable local adaptation by stakeholders.¹⁷ As one example, the HHFKA-stipulated nutrition standards were authorized by Congress, went through extensive USDA rule-making processes, were rolled back by the next administration, and have been reviewed in federal court.⁵³ In addition, stakeholders at multiple levels face different implementation challenges (eg, schools/districts facing procurement issues; states implementing policy changes with few staff and out-of-date administrative data systems) and likely have competing interests (eg, cost vs children’s product preferences) that can hinder implementation. Finally, variability in contexts and outcomes reduces the scope of most implementation studies, which could have unintended consequences for health equity if such work fails to identify implementation and adaptation strategies needed to reach children most at risk for food insecurity and chronic disease.^24,52,54

School-Based Child Nutrition Programs During COVID-19: A Research Opportunity

When schools closed in response to COVID-19, the importance of policy-based solutions to address food insecurity and prevent chronic disease became evident.^15,55 The country experienced a rare situation in which the stakeholder consensus was nearly unanimous: that one of schools’ top priorities during the pandemic must be mitigating skyrocketing food insecurity by providing nutritious meals.⁵⁶ The USDA used its congressional authority to issue a series of waivers and guidance memoranda, which allowed states, districts, and non-school SMP sponsors (eg, community organizations, food banks) to offer modified meal service.⁵⁷ Although intended to support social distancing and reduce virus spread, many waivers allowed for broad use of the ERIC strategies described previously, which antihunger and school nutrition professionals recommend to improve program reach and implementation.^{29,31,38,45,49} Seemingly overnight, political and administrative agencies supported the scale up of these strategies, creating a somewhat controlled policy environment and opportunity to study implementation and local adaptation of multiple strategies with the potential to strengthen policies and/or local implementation long term.^24,52,58

Shortly after initial school closures, the leaders of Healthy Eating Research (HER), a national program of the Robert Wood Johnson Foundation, and the Nutrition and Obesity Policy Research and Evaluation Network (NOPREN), a Centers for Disease Control and Prevention–funded network of child nutrition researchers and practitioners, supported an effort to combine membership of existing working groups focused on food insecurity and school wellness to form an ad hoc COVID-19 School Nutrition Implications Working Group.⁵⁹ The working group initially focused on advocating for rapid action to support the successful delivery of essential child nutrition programs. Over time, it has expanded its scope to identify and study innovative implementation strategies with the potential for long-term policy and practice improvements. In just 6 months, this working group grew to include more than 380 members, drawing from academic institutions and collaborative relationships with more than 15 organizations and professional societies (Figure).

Figure.

Healthy Eating Research and Nutrition and Obesity Policy Research and Evaluation Network School Nutrition Implications Working Group collaborators, projects, and products, United States, 2020. Abbreviations: COVID-19, coronavirus disease 2019; NOPREN, Nutrition and Obesity Policy Research and Evaluation Network.

Collectively, this group brought together (1) strong partnerships with child nutrition administrators in many states and localities, (2) knowledge of the historical and COVID-19–era federal policy change process, and (3) multidisciplinary expertise (eg, nutrition, law, epidemiology, advocacy) in stakeholder-driven food insecurity and obesity prevention research. For more than a decade, many members have collaborated through HER/NOPREN working groups to generate policy-relevant research. Thus, the group is well positioned to conduct high-quality policy implementation research that capitalizes on existing data sources or pragmatic data collection through strong partnerships. Research stemming from this working group seeks to identify which strategies and local adaptations were used to feed children during the pandemic and identify which strategies have the potential to improve implementation and extend program reach and impact beyond the pandemic recovery.¹⁷

Since mid-March 2020, the policy landscape related to school meals has evolved on an almost daily basis; thus, the implementation strategies used by local authorities have also evolved. Initially, local authorities were tasked with quickly identifying appropriate feeding sites, deploying staff members, and developing safe, contact-free delivery methods. Their approaches inevitably changed as recommendations were updated, staff members tested positive for COVID-19, available resources changed, and budget shortfalls became a concern. Relaxed federal and state restrictions allowed for local implementation to be “flexible, tailored, and thoughtful,”⁵⁵ and several studies identified a wide range of challenges and successes nationwide.^60,61 Congress authorized USDA waivers that eliminated the requirement that a certain percentage of children in the area around the meal site be documented as low income and waived congregate feeding requirements.⁶² In addition, state and district entities launched highly visible awareness campaigns, including text updates and site locators, and provided tailored and frequent technical assistance while establishing new partnerships with local nonprofits, restaurants, and food banks. It is not surprising, then, that many implementation research questions emerged; although these questions are all interesting, not all of them are relevant for reducing inequities in food security and chronic disease prevention long term. We provide examples of the HER/NOPREN COVID-19 working group’s policy- and practice-relevant implementation research questions using RE-AIM²⁶ (Table).

Table.

The US Department of Agriculture congressionally authorized COVID-19 waivers and related RE-AIM research questions for policy and practice change, United States, 2020^a

Domain and description	Research aim
Reach: number, proportion, and representativeness of eligible students who participated in programs during COVID-19	Determine the number and proportion of children who receive meals during COVID-19 school closures. Determine representativeness among children receiving meals during COVID-19 closures of (1) children receiving free meals through the Free and Reduced-Price Meals program or the Community Eligibility Provision and (2) children who had not previously participated in the Summer Meals Program, with attention to race/ethnicity, immigration status, income, and rurality. Describe methods used by state and local authorities to recruit children/families to participate.
Effectiveness: impact of programs on participating students’ quality of life and economic outcomes, including variability across subgroups	Compare food insecurity trends between states and localities to describe the variable impact of programs. Identify the impact of other complementary programs (eg, pandemic electronic benefits transfer) on adoption, reach, and implementation of school meals programs. Describe unintended consequences for students (eg, lower nutritional quality of meals, reallocation of funds to nonmeal programs).
Adoption: number, proportion, and representativeness of local authorities who operated during COVID-19	Describe organizational characteristics (eg, personnel, readiness, leadership) and geographic distribution of local authorities who mobilized during COVID-19 compared with those who did not. Determine representativeness among local authorities who operated during COVID-19 closures compared with authorities who had previously opened during summer periods.
Implementation: local authorities’ implementation of program requirements and waiver changes, including consistency, time, costs, and adaptations/fidelity	Describe common implementation strategies (eg, grab-and-go, delivery, multiple meals) allowable by waivers and used by local authorities. Identify innovative or culturally specific implementation strategies and adaptations used by local authorities serving subpopulations with high rates of food insecurity (eg, rural areas). Describe operational costs of various implementation strategies.
Maintenance: whether student impact is sustained and whether program operations during COVID-19 become part of routine practice	Compare long-term child food insecurity trends between states and localities to describe the variable impact of programs. Describe the extent to which implementation strategies fit within the capacity of sites/sponsors during and beyond the waiver period. Describe long-term costs associated with unanticipated mobilization of sponsors/sites during COVID-19 closures.

Abbreviation: COVID, coronavirus disease 2019.

^aBased on the RE-AIM (reach, efficacy, adoption, implementation, maintenance) framework.²⁶

The strong partnership among researchers, advocacy organizations, and professional societies ensures that data can readily translate to policy recommendations and inform scale up or areas for future research on novel implementation strategies and local adaptation of those strategies to mitigate food insecurity, prevent chronic disease, and reduce health disparities, all of which have been exacerbated by the pandemic.^21,63,64 Importantly, the relationships of HER/NOPREN working group members with key stakeholders will ensure that data collection and analysis continue to give a voice to people on the front lines of schools’ response to the pandemic and economic recovery. Finally, the collaborative efforts of multidisciplinary researchers and multisectoral practitioners across the country can lend generalizability to findings and inform future federal efforts to strengthen child nutrition programs and secure tailored support for local implementers.

D&I Science in COVID-19–Related Research

Regardless of the area of public health focus, opportunities for D&I research are abundant, and the work is needed.^18,27 Our example provides a reminder of what implementation science is about: ensuring research done—during an opportune time to conduct research but also an inopportune time from which to generalize—can be translated to more equitable policies and public health practice in the future than those that exist today.

The expertise required to conduct D&I research naturally lends itself to this challenge. First, D&I research is stakeholder driven and interdisciplinary by default. At a time when research opportunities are abundant, D&I researchers are well-equipped collaborators, trained to ensure that inquiry is timely, interesting, aligned with priorities, and most relevant for future policy and practice.^65-67 As interdisciplinary teams respond to rapid-release funding opportunities, it is important to ensure that proposals reflect the needs and interests of stakeholders, identify policy gaps, and highlight concerns of decision makers. Second, D&I researchers have a wealth of foundational and operationalizable theories and frameworks, which can define implementation strategies (eg, ERIC compilation), track implementation processes, measure innovations and local adaptations, and draw attention to the factors that can best inform adoption, uptake, and implementation in the new world that exists beyond the pandemic across a variety of settings.^{22,23,51,66,68} In combination, stakeholder engagement and conceptual frameworks can ensure that evaluation methods and measures are rigorous yet pragmatic to administer amid a chaotic environment (the type of environment to which D&I researchers are most accustomed).^65,66,69

A third contribution of D&I research is that, where traditional research methods work to create experimental conditions that eliminate disruptive contextual factors, D&I methods embrace and evaluate those factors across multiple levels.⁶⁵ The uncontrollable and unique context of this pandemic is the very thing that can provide insight into the cracks in our systems and recognize potential innovations needed to repair them. Fourth, D&I researchers have a methodologic imperative to disseminate findings, often working alongside stakeholders to test various dissemination approaches and find the best or fastest way to facilitate translation of evidence into practice or policy.^24,65,66 D&I researchers are also equipped to study how the practice and policy changes that result from COVID-19 might work in the long term, drawing from a long list of trial designs to compare the effectiveness of various implementation strategies to ensure that policies and practices achieve their intended purpose.⁷⁰

Finally, the focus on equity often promoted by D&I scientists should pervade every element of research during and after the pandemic period.^54,71 D&I scientists must engage organizations working with populations that face inequities to ensure that the most relevant contextual factors are considered.⁵⁴ Recruitment and data collection methods must seek out priority populations, and dissemination strategies must elevate those narratives. In developing post–COVID-19 recommendations for policies and practices implemented during the pandemic, it will be important to consider the opportunities for cultural adaptations and tailored support and guidance for locally driven implementation strategies. Above all, we need to ask ourselves whether our research during this period will inform policies and practices that address the systemic inequities in our nation’s health system. Otherwise, it is not research worth doing.

ORCID iD

Hannah G. Lane, PhD, MPH https://orcid.org/0000-0001-8370-6647