Lessons learned and recommendations in conducting solutions-driven environmental and public health research

Abstract

Solutions-driven research is a transdisciplinary approach that incorporates diverse forms of expertise to identify solutions to stakeholder-identified environmental problems. This qualitative evaluation of early solutions-driven research projects provides transferable recommendations to improve researcher and stakeholder experiences and outcomes in transdisciplinary environmental research projects. Researchers with the U.S. Environmental Protection Agency (EPA) Office of Research and Development recently piloted a solutions-driven research approach in two parallel projects; one addressing nutrient management related to coastal waters and another studying wildland fire smoke impacts on indoor air quality. Studying the experiences of those involved with these pilots can enhance the integration of researcher and experiential expertise, improving solutions-driven research outcomes. Data collection included semi-structured interviews with 17 EPA researchers and 12 other stakeholders and reflective case narratives from the authors. We used conventional content analysis to qualitatively analyze perspectives on implementing innovative engagement and research approaches in a solutions-driven process. Findings that reflect common perspectives include the importance of continuous engagement, the challenges of differing timelines and priorities for researchers and stakeholders, and the need to define consistent markers of success across researchers and stakeholders. Key lessons to improve transdisciplinary research identified from the analysis are (1) improving clarity of roles and responsibilities; (2) planning to provide sufficient, continuous project funding over multiple years; (3) expecting research needs and plans to adapt to evolving circumstances; and (4) clearly defining the end of the project.

1. Introduction

Finding real-world solutions to environmental and public health challenges requires integrating the expertise of scientists and stakeholders. Central to bridging the gap between science and practice is stakeholder participation, as engaging stakeholders in scientific research can lead to more informed decision-making and improved environmental and public health outcomes (Newig et al., 2023; O’Mara-Eves et al., 2015). Lasting challenges in bridging the science and practice gap include creating usable science (Dilling and Lemos, 2011; Wall et al., 2017) and accessible science communication (Bray et al., 2012; Sarewitz and Pielke Jr. 2007). The development of effective science communication strategies is critical for connecting environmental research and practice and ensuring that scientific findings are effectively translated into actionable solutions (Bray et al., 2012; Enquist et al., 2017; Lemos and Morehouse, 2005). Transdisciplinary research collaborations and translational science frameworks have been key to integrating scientific research into practical solutions to complex societal challenges (Cash et al., 2003; Campbell, 2005; Collins et al., 2011).

Transdisciplinary research emphasizes collaboration and co-production of knowledge across disciplines, sectors, and stakeholders to generate actionable solutions that are informed by diverse perspectives and knowledge systems (Lang et al., 2012; Klein et al., 2001). Co-production of knowledge involves collaboration between researchers and stakeholders in the development of research questions, methods, and findings (Lemos and Morehouse, 2005). The technical expertise included in peer-reviewed scientific research comes from formal education degrees and access to expensive equipment (Lave, 2015). Scientific researchers who hold such degrees bring theoretical and methodological awareness and ideas from multiple disciplines to a collaborative project. Complementing research expertise with context-specific understanding, or local knowledge, of the social, political, and environmental dimensions of a problem can improve the identification of environmental exposures and risks beyond what can be done using formal scientific knowledge alone (Corburn, 2003; Weber et al., 2014) and greatly improve project outcomes (Bammer et al., 2020). Including the expertise of local knowledge often provides a way to incorporate historically marginalized and underrepresented voices in science and is effective in addressing complex environmental challenges such as climate change adaptation and natural resource management (Prokopy et al., 2017; Reed et al., 2013). Transdisciplinary collaboration can be challenging to implement with the diverse perspectives and priorities at play, but it can also lead to more effective, acceptable, and sustainable outcomes in environmental management (Gross et al., 2007; Wiek et al., 2012).

Translational science emerged in the medical field as a transdisciplinary approach to improving health outcomes (Austin, 2018), and the foundational principles have since been brought into ecology and environmental public health fields. For ecologists, translational ecology is a “boundary-spanning environmental science that leads to actionable research focused on maintaining or enhancing the resilience of social-ecological systems” (Brunson and Baker 2016; See also Schlesinger, 2010). Ecologists emphasize the adaptive, interdisciplinary nature of this approach, as well as the focus on promoting mutual learning and the usability of research among communities and stakeholders. The National Institute of Environmental Health Science (NIEHS) defines environmental public health as the science of conducting and translating research into action to address environmental exposures and health risks of concern to the public. NIEHS has proposed a translational research framework that “encourages the translation of environmental health research into concrete strategies that protect and improve human health” (Pettibone et al., 2018).

1.1. Translational science as “Solutions-driven research” at EPA

The U.S. Environmental Protection Agency (EPA) Office of Research and Development (ORD) has drawn from principles and practice of transdisciplinary collaboration and translational science to develop a solutions-driven research approach. The EPA originally interpreted translational science as the process of turning observations in the laboratory, field, clinic, and community into decisions, interventions, communications, or best practices that improve environmental quality and public health (NIH, 2023; Maxwell et al., 2019). Seeing translational science as the process of moving from science to solutions, the agency’s solutions-oriented translational science framework is known as “Solutions-Driven Research” (SDR) (Figure 1). As originally conceptualized, SDR incorporates the key concepts of integrated research activities, information exchange, synthesis, scientific translation, and engagement between researchers and stakeholders. SDR often involves engaging with stakeholders, such as policymakers, industry representatives, or community members, to identify their needs and concerns, and working collaboratively to develop approaches and solutions that are feasible, appropriate, meaningful, and effective (Booth et al., 2015; Pearson et al., 2005). Focusing on the co-production of knowledge and integration of stakeholders throughout the process, SDR encourages multidisciplinary teams of scientists to collaborate in understanding the socioeconomic and biophysical aspects of environmental challenges and develop integrated solutions that fit the needs of stakeholders (Eisenhauer et al., 2021; Watts, 2017). SDR uses rigorous scientific approaches to inform solutions to environmental and public health problems, and expand the range of effective alternatives, while clarifying the positive and negative impacts of different choices (McNie et al., 2016; O’Rourke and Crowley, 2013; Watts, 2017).

Figure 1.

The Solutions-Driven Research (SDR) process as practiced in the EPA Office of Research and Development. Not every project incorporating SDR concepts will include all aspects of this process. Adapted from NIH (NIH, 2023).

As the initial projects implementing SDR approaches began, the value of focusing on science integration, science translation, and using community-engaged approaches in place-based SDR projects grew in importance and became integrated into the updated SDR framework. The current EPA ORD framework describes key characteristics of the SDR approach:

• Planned stakeholder engagement throughout the research process, starting with problem formulation and informing all elements of research planning, implementation, dissemination, and evaluation.

• A focus on solutions-oriented research outputs identified in collaboration with stakeholders.

• Coordination, communication, and collaboration both among ORD researchers and between researchers and stakeholders to develop integrated research that is valuable to multiple stakeholders.

• Cooperation with stakeholders to apply research results to develop solutions that are feasible, appropriate, meaningful, and effective (US EPA, 2022a).

Rather than being a stringent methodology, SDR is a collaborative, flexible approach that focuses on an engaged process, communication, coordination, and collaboration and development of usable results that meet stakeholder needs. To further the use of SDR, ORD conducted pilot projects to test and evaluate the functionality, effectiveness, and transferability of applying these characteristics to two distinct environmental and public health challenges (described in depth in the next section). While not using the systematic SDR approach, there have also been efforts throughout ORD that employ several of the tenets of a solutions-driven process (e.g., Baldauf, 2016; Bradley et al., 2015; Hano et al., 2019). The pilot projects are unique in their intentional and systematic application of SDR methods to plan, conduct, apply, and evaluate integrated research that addresses a well-defined and unmet need of stakeholders.

Early SDR project planning documents aimed to ensure the outcomes of SDR projects are usable and transferable science. These planning documents outlined key roles on an SDR team, the phases of research from problem formulation through research application evaluation, and the description of key tasks for a successful SDR project. Primarily, this approach suggests a shift towards stakeholder-identified problems and interdisciplinary research efforts to address such problems, with a systematic approach to stakeholder engagement and research integration. Consistent engagement throughout the project is intended to ensure consensus about the goal and to settle disagreements about how to proceed. Specific roles of the Translational Navigator, Science Lead, and Communications and Engagement Lead outlined in these planning materials sought to support internal coordination and assist with stakeholder engagement (Table 1). All leaders are responsible for understanding the environmental and social context of stakeholders and understanding research needs to successfully perform their positions. Continual project team communication, including team leaders, researchers, and stakeholders, ensures that the researchers’ efforts are complementary and in line with community needs. The SDR pilots served as a living laboratory for EPA to implement the SDR framework, learn what engagement practices worked and evaluate their impacts on research direction and outputs, and learn how to improve the practice of SDR at the Agency.

Table 1.

Solutions-Driven Research Project Team Leadership

Role	Responsibilities (not exhaustive)
Translational Navigator	First point of contact for pilot project (including addressing stakeholder questions and concerns) Facilitates and works with Science Lead(s) to prepare project plan and troubleshoot implementation Helps researchers navigate research-related processes Helps to identify and obtain funding/resources Helps determine research support needs (data collection and analysis, statistical analysis, synthesis preparation, informatics, etc.) Helps overcome bureaucratic obstacles to working with stakeholders in applying and evaluating research outputs Establishes translational research network for the topic area to encourage collaboration, coordination, and communication
Science Lead	Prepares project plan(s), leads research activities, data analyses, summaries of results, etc. Actively identifies potential barriers/obstacles for sharing data/models/code across tasks, identifies any science or technical issues that may impact other tasks/products. Actively engages with researchers to understand where linkages can be improved, where schedules need to be aligned, and where specific design decisions can impact across tasks and products. Coordinates with Translational Navigator to address support needs from scientists.
Communications and Engagement Lead	Works with Translational Navigator, Science Lead, and appropriate EPA staff to develop, implement, and evaluate a stakeholder engagement plan (including EPA partner offices and regions). Coordinates strategic stakeholder outreach and engagement efforts targeting EPA, other federal agencies, state agencies, international organizations, non-governmental organizations, and industry. Develops and updates materials describing research for stakeholders based on their needs as the project evolves. Implements stakeholder engagement activities such as events, workshops, webinars, etc. Tracks outreach and engagement activities to determine what is most effective. Refines future engagement plans based on best practices and other lessons learned.

This paper evaluates how two pilot SDR projects at EPA implemented these tenets and builds on early studies from EPA on translational research (Eisenhauer et al., 2021; Maxwell et al., 2019) and evaluation of SDR (Canfield et al., 2022). This paper provides perspectives on multiple pilot SDR projects, including participant and manager perspectives from throughout the research process, and lessons learned to improve future SDR projects. This paper provides recommended practices for SDR projects moving forward at EPA that can also be applied to solutions-oriented research in other venues.

2. Case descriptions

The two initial pilot projects were designed to both conduct innovative, stakeholder-engaged social and environmental research and to provide recommendations on how to shift practices moving forward to better achieve both community and EPA goals in SDR projects. Both pilots involved a planning process that engaged stakeholders from the outset and implemented the EPA framework for SDR. These two pilots occurred simultaneously and in parallel. They are considered together to understand how and whether these initial efforts display the transferability of an SDR approach. Including multiple cases in our analysis reduces bias and allows for triangulation of findings, giving more reliable results and conclusions than looking at perspectives from a single case (Yin, 1994). The two SDR pilot projects described below include pilot studies on (1) managing nutrient levels in surface and groundwater, and (2) air quality research in smoke-impacted communities.

2.1. Nutrients

Excess nutrient loading from non-point sources presents states and communities across the United States with a multifaceted problem. In many parts of the country, onsite wastewater treatment and land management practices including fertilizer use have contributed to elevated nutrient levels in surface and groundwaters, resulting in environmental and public health impacts. Excess nutrients from human activity are an increasingly serious threat to estuaries, wetlands, and freshwater ponds globally, as they contribute to algal blooms, low dissolved oxygen, degradation of seagrass, impaired freshwater and estuarine ecosystems, and, in extreme cases, fish kills.

The Nutrients SDR Pilot: Reducing Non-point Sources of Nutrients via Non-traditional Approaches (Nutrients SDR Pilot) addressed this long-term problem by providing watershed-based solutions that can support states and communities across the nation. The research was focused in the Three Bays watershed located mostly within the town of Barnstable, on Cape Cod, Massachusetts. The primary source of nutrient loading in the watershed is septic systems used for wastewater management, which are not projected to be converted to sewers for several decades, and the conversion will not address legacy nutrient pollution. The project team worked in partnership with the Barnstable Clean Water Coalition (BCWC) to evaluate whether watershed-based solutions could be complementary to sewering plans and help to achieve nutrient reduction goals more quickly and cost-effectively. Key stakeholders include the BCWC, EPA Region 1, U.S. Geological Survey, The Nature Conservancy, the Massachusetts Alternative Septic System Test Center, the Massachusetts Department of Environmental Protection, the Massachusetts Division of Ecological Restoration, the Town of Barnstable, and the Cape Cod Commission.

ORD co-developed a social and environmental research project based on the insights from an initial stakeholder engagement and problem formulation workshop (2018) and a follow-up workshop (2019) (Twichell et al., 2019). The Nutrients SDR Pilot included a baseline estuarine benthic condition assessment (Erban et al., 2021) and pilot scale studies of promising interventions to reduce nitrogen loading, including source controls through a neighborhood scale novel innovative septic systems pilot (Rudman et al., 2023); groundwater nitrogen interception through restoring a cranberry bog system; water column controls through shellfish restoration and aquaculture (Ayvazian et al., 2021; Rose et al., 2021); and beneficial reuse of nutrient-rich dredged materials. Complementary social science research investigated water quality impacts on recreation (Furey et al., 2022) and perspectives on alternative technologies for nutrient management (Mulvaney et al., 2023; Rudman et al., 2023). The team also had an integrated communication plan detailing goals, audiences, and appropriate products for sharing results (Rudman et al., 2021) based on best practices in science communication, and specifically for nutrient communication (Canfield et al., 2021). The project team, which included ORD scientists and stakeholders, evaluated interventions to reduce nitrogen at the source, in the groundwater, and in the receiving waters of the estuary through an integrated environmental and social science lens. The results of these studies are helping our stakeholders to identify the most effective suite of solutions to their non-point source nutrient challenges, and important social considerations in installing those systems (Canfield et al., 2022; Mulvaney et al., 2023; Rudman et al., 2023). Results can be used in other coastal areas along the eastern seaboard to help solve non-point source nutrient challenges and address water quality goals.

2.2. Wildland Fire Smoke

Wildland fire smoke is a global public health issue. Communities may be exposed to smoke from wildland fires, including wildfires and prescribed or controlled burns, for days, weeks, or even months each year. This smoke exposure can lead to increased health risks. To reduce exposure, a common recommendation is to stay indoors and close all doors and windows, but smoke can infiltrate buildings, emphasizing the importance of creating cleaner indoor air spaces that effectively reduce smoke exposure.

The Wildfire Advancing Science Partnerships for Indoor Reductions of Smoke Exposures Study (Wildfire ASPIRE Study) is a multi-component research project that began in late 2018 and continually adapted to opportunities and needs as they arose. Initial problem formulation occurred in virtual and in-person meetings between EPA and stakeholders in Missoula, Montana, including the Missoula City-County Health Department. The overarching concerns identified were translated into targeted research questions focused on better understanding indoor air quality during wildfire smoke events and how to reduce indoor smoke exposures and related health risks. An initial web summit was held to understand the state-of-the-science and related public health issues (Davison et al., 2021). These discussions helped inform the designs for specific study components, including complementary field and laboratory studies to investigate indoor air quality and exposure reduction approaches during smoke events and a prize-based challenge to spur the development of innovative air cleaning solutions. Simultaneously, EPA Region 9 proposed research on a popular approach to reducing smoke exposure based on a box fan and a furnace filter known as a Do-It-Yourself (DIY) air cleaner. The similar objectives of this proposed research made it a natural fit in the broader Wildfire ASPIRE study.

The field study was initiated in Missoula during the 2019 wildfire season. EPA researchers also established a partnership with the Hoopa Valley Tribal Environmental Protection Agency in Hoopa, CA, which has an extended wildfire season. Tribal partners were interested in smoke impacts associated with residential wood burning and wintertime thermal inversions. The team monitored indoor and outdoor fine particulate (PM_2.5) concentrations in public and commercial buildings in Missoula and Hoopa to understand real-world variation in indoor air quality under typical ambient conditions and when smoke events occurred.

Laboratory studies evaluated DIY air cleaners to answer questions from EPA partners and the public about their effectiveness (Holder et al., 2022) and safety (Davis and Black, 2021). The laboratory study expanded to include a field component, ASPIRE-Health, to evaluate the ability of air cleaners in homes to improve indoor air quality and health. Pilot studies in Hoopa, CA, in collaboration with the Hoopa Valley Tribal EPA, informed plans for a larger field study in Tulare County, CA, working with the Central California Environmental Justice Network (CCEJN) and UL-Chemical Insights Research Division (US EPA, 2022b). Concurrent with the field and laboratory studies, a prize-based challenge was initiated with ten federal, state, local, and tribal organizations to encourage the development of innovative, low-cost, effective, and sustainable methods to reduce indoor PM_2.5 concentrations in homes during wildfire smoke or other high pollution events (US EPA, 2023).

As the project progressed it adapted by adding new partners and expertise in heating, ventilation, and air conditioning (HVAC). A collaboration with ASHRAE (formerly the American Society of Heating, Refrigerating and Air-Conditioning Engineers) helped develop guidance to prepare commercial and public buildings for wildfire smoke episodes, and an HVAC expert was brought in to evaluate building and HVAC characteristics in study buildings to inform our understanding of how building infrastructure may impact indoor air quality during smoke events.

Community and Tribal stakeholders plan to use the results of the Wildfire ASPIRE Study to identify and improve potential cleaner air spaces that can be used to reduce exposure during smoke events. They also plan to use the results to educate and encourage building managers and community members to prepare for smoke events and to use the most effective practices to reduce indoor smoke exposure. The lessons learned working with community and tribal partners as well as our complementary laboratory studies and related research activities will be helpful for other communities impacted by wildland fire smoke. In addition, the Hoopa Valley Tribal EPA is using the results of this research project to support grant proposals to continue its air quality work and to prepare Tribal members for future smoke events. The results from this work are informing the development of ASHRAE guidelines for commercial buildings, including schools, to reduce smoke exposure for occupants across the United States (ASHRAE, 2021; EPA, 2021; Javins et al., 2021).

2.3. Differences between pilots

While there were many shared experiences across the pilots, there were also logistical differences. The primary differences were in the physical distance between researchers and the stakeholder communities, the sizes of the research teams, and the types of stakeholders involved. Both teams executed SDR pilots that followed most of the tenets of SDR as outlined in planning documents, and participants across pilots provided similar feedback for future projects.

The ease of travel between the researchers’ labs and the communities in which they were working differed greatly between the two projects. The Nutrients SDR Pilot was focused on Cape Cod, Massachusetts, with the researchers based predominantly in Narragansett, Rhode Island (within relatively easy driving distance). Nutrients SDR Pilot researchers were able to drive and work in person with stakeholders much more often than Wildfire ASPIRE Study team members and stakeholders were able to collaborate in person. The Wildfire ASPIRE Study team was predominantly located in North Carolina, while their stakeholders and research sites were based in Missoula, Montana; Hoopa Valley, California; and Tulare County, California. The challenge of collaborating in person was exacerbated by the COVID-19 pandemic starting in early 2020, with restrictions on work travel and in-person meetings. The Wildfire ASPIRE Study faced additional challenges of nontypical use of buildings included in air monitoring experiments and increased research responsibilities for key project stakeholders due to EPA travel restrictions.

The project teams also had notably different numbers of EPA researchers consistently involved. There were more EPA researchers continually involved with the Nutrients SDR Pilot compared to the Wildfire ASPIRE Study team. The Wildfire ASPIRE Study team had a high turnover rate of scientists and a small cohort of researchers that has been continually working on the project since its start. Many of the researchers originally involved retired, changed positions, or finished postdoctoral positions, and some key positions were not backfilled when they became vacant.

A final notable difference was the kinds of stakeholders chosen for these projects. In both projects, EPA had existing relationships with individual stakeholders or organizations that ended up as key collaborators. In the Nutrients SDR Pilot, the key stakeholder was a nonprofit that was focused on the technical side of the solutions, with at least three staff primarily dedicated to the SDR goals and approach. With the Wildfire ASPIRE Study, a major challenge was the bandwidth of the key stakeholders to take on the SDR effort in addition to an already full work agenda. In this case, the key stakeholder had decision-making power in their community, which was valuable in ensuring access to necessary data and achieving planned research goals.

3. Methods

This project sought to answer three questions through qualitative social science:

1. How do researchers and stakeholders involved in SDR perceive their experience with this engaged research process?

2. What can we learn from those involved in SDR pilots about how to improve future SDR efforts?

3. How do the experiences with SDR compare across multiple pilot projects?

The research entailed a qualitative case study approach to answer the research questions, pursued through semi-structured interviews and reflective case narratives. A case study research approach allows for an in-depth and longitudinal understanding of the culture and context of the environmental challenge (Williams and Hummelbrunner, 2011). In seeking to capture research participants’ distinct experiences, this study used conventional content analysis applied through hybrid inductive-deductive coding to identify major themes and lessons (Crotty, 1998).

Semi-structured interviews were conducted with EPA researchers and external key stakeholders from the two SDR pilot projects focused on their experiences with the solutions-driven approach to engagement in research. Semi-structured interviews use a predetermined set of questions designed based on literature review and research questions and allow for additional questions as needed based on the interviewee’s responses (Patton, 2002). For the Nutrients SDR Pilot, interviews were conducted in 2020, early in the project’s implementation (Canfield et al., 2022) and in 2022, at the conclusion of the pilot stage of the project. Interviews with the Wildfire ASPIRE Study team were conducted in April 2023, when some aspects of the project were complete but additional research activities were ongoing. Predetermined questions were slightly different for EPA ORD researchers and external stakeholders to address their different roles in the projects. The questions were largely the same across the two SDR projects (see Appendix 1 for questionnaires) to allow for a comparative analysis. As there were two rounds of interviews with the Nutrients SDR Pilot participants, there were additional questions included in the end-of-project interviews to evaluate change in perspectives over time.

All interviews were recorded, transcribed, and then analyzed using conventional content analysis (Hsieh and Shannon, 2005). This widely used qualitative data analysis method involves identifying key categories, or themes, that consistently emerge from the data rather than comparing data to preconceived categories (Mayring, 2004). Interviews were coded using NVivo 12 qualitative coding software to identify shared experiences and perspectives across data (Fereday and Muir, 2006). Building on past work using this method and the in-process interviews that are also included in this analysis (Canfield et al., 2022), the initial inductive categories were identified in the earlier analysis. Using a hybrid inductive-deductive coding approach, end-of-project findings were identified through an iterative review of interview transcripts. This included thematically identifying perspectives that aligned with categories from the in-progress analysis (deductive) and adding further categories (codes) that arose through the end-of-project analysis (inductive). A total of 122 categories, or codes, were identified in the analysis of the initial interviews (Canfield et al., 2022). Additional categories were added as they emerged as common topics from the new interviews, such as a high-level theme of “change over time,” with subcategories including “personnel changes.” The final codebook includes 220 categories, with many of the high-level or primary categories having minimal content coded to them, as there were more specific codes nested within them that better capture the perspective shared (see Appendix 2). The themes and findings discussed below are based on these categories. The coding was completed by one researcher. Another researcher independently coded 15% of the interviews to ensure reliable application of the coding structure (similar to Floress et al., 2017). All codes had at least 90% agreement, and discrepancies discussed between coders revealed no meaningful differences. Consistency of findings across multiple pilot projects, complemented with reflective case narratives and reviewed project engagement materials, improves the validity of the findings (Cresswell and Miller, 2000).

With a focus on providing lessons learned, we also use reflective case narratives to include perspectives of managers on accomplishments and directly confront internal evaluation limitations. Reflective case narratives focus on evaluation and help address some of the bias involved in self-reporting accomplishments (Klein, 2008). This method explicitly invites reflexivity on one’s role in all stages of the research process and relationships with others involved in the project (Becker and Renger, 2017). The intentional critical examination of personal perspectives provides a check on identified lessons learned in this internal evaluation (Jewiss and Clark-Keefe, 2007). Data were produced through open-ended conversation with co-authors who were also involved in pilot project leadership and management on their experiences leading and supervising these projects. Notes collected through these conversations were categorized into the existing coding categories from the interview analysis. These narratives serve to supplement the perspectives of those implementing the pilot projects and inform discussion and recommendations from a managerial perspective.

All EPA researchers involved in either pilot for at least one year as of the time of the interviews were invited to participate, with 11 EPA ORD researchers from the Nutrients SDR Pilot and 6 EPA ORD researchers from the Wildfire ASPIRE Study participating (Table 2). The difference in the number of researchers from the nutrients team interviewed in 2020 and 2022 reflects one researcher’s retirement who had not been replaced at the time of the 2022 interviews. Key stakeholders who were involved in research planning and/or implementation were invited to participate, with an intentional effort to capture perspectives across different aspects of these projects. Nutrient stakeholders interviewed covered the government (local, state, and federal agencies), nonprofit, and consulting sectors. Wildfire ASPIRE Study stakeholders came from local government and Tribal environmental management.

Table 2.

Number of interviews with EPA ORD researchers and stakeholders across two solutions-driven research pilots.

Interview group and year	EPA researcher interviewees	Stakeholder interviewees
Nutrients 2020	11	9
Nutrients 2022	10	9
Wildfire 2023	6	3

While the number of interviewees may seem low, the sample includes all external key stakeholders involved in both projects and most members of the EPA research teams (further quantification of roles is left out for confidentiality). Not all researchers and stakeholders who participated in these projects were interviewed, but data saturation was achieved with both groups, which occurs when additional interviews reveal no new information or perspectives (Saunders et al., 2018), demonstrating a comprehensive review of perspectives. This is determined by reviewing the content of each subsequent interview and seeking a diversity of perspectives. When the preexisting themes are further confirmed rather than discovering additional themes, even when seeking diverse perspectives, theoretical saturation has been achieved (Dey, 1999). Thus, despite the relatively small number of participants, the perspectives and findings are a reliable portrayal of the themes and lessons learned in these SDR projects.

3.1. Addressing limitations

While internal evaluations can present limitations in the self-reporting of success (Ziegler et al., 2021), the multi-method approach allows for the inclusion of multiple perspectives and reflexivity on personal perspectives (Roux et al., 2010). The reflexivity improves the validity of the qualitative methods, as it acknowledges the cultural and experiential biases present in the analysis (Creswell and Miller, 2000). The chosen analytical approach answers the research questions based on the lived experiences of those participating in the SDR pilots as researchers, stakeholders, and managers, providing multiple perspectives on the strengths and weaknesses of the current SDR approach. Further, including both in-progress and summative perspectives allows for analysis of the functionality and transferability of adaptive research in EPA ORD (Canfield and Chatelain, forthcoming). Together, these methods allow for a more holistic review of the successes of the projects and lessons learned from researchers, stakeholders, and managers to improve future SDR projects.

4. Key perspectives and experiences of stakeholders and EPA researchers

What follows are the key findings collected in the interviews, supplemented with reflective case narratives as appropriate to explain the context more fully. Unless specifically noted, all mentions of EPA researcher perspectives reflect experiences of researchers across both pilot studies and mention of stakeholders refers to experiences of the interviewed key stakeholders across both pilots.

Through the analysis of the interviews and case narratives we identified 3 key findings which characterize participants’ perceptions of the unique features of an SDR process. The first isolates stakeholder experiences in the SDR process, identifying the positive experience of collaborating with EPA as well as the challenges of working through government bureaucracy to address localized challenges. The second key finding emphasizes how success in SDR is holistic, incorporating scientific impact and positive relationships. The final theme is the EPA researchers’ perceived differences of SDR from basic environmental research as both SDR’s stakeholder-driven nature and integration of research tasks. Together, these themes capture the consistent messages from interviews and reflective case narratives on both projects. Table 3 summarizes these key themes with quotes from participants.

Table 3.

Key perspectives and findings from SDR pilot interviews and relevant quotes. Quotes are numbered as related to each component of the definition of the key findings.

Perspective or Finding	Description	Supporting participant quotes
Stakeholders enjoyed working with the research arm of EPA and struggled with the bureaucracy of government research	1. A positive first experience working with EPA as a research partner rather than as a regulatory agency. External stakeholders found EPA researchers to be approachable and collaborative throughout the SDR pilot projects. 2. Stakeholders struggled with the incongruent paces of their action-oriented organizations and the government in getting approval to conduct experiments and disseminate results.	1a. External stakeholder: The positive experience of working with EPA employees within ORD, “puts a different light on EPA.” Stakeholders enjoyed the “non-threatening, very pleasant, very smart, very helpful, eager” energy that came with working with ORD. 2. External stakeholder: “Knowing how much more there was to the story but needing to wait for everything to get through the final analysis and approval process was a really long time… we really wanted to encourage [people] to take some steps.”
Researchers and stakeholders had holistic definitions of project success	Interviewees identified as markers of success for the project: Trust and relationship building, Stakeholders using SDR project science to inform actions, In-process and adaptive communication, Accomplishing research goals, Transferable and long-term results, and Publishing peer-reviewed research	1. EPA researcher: “I think the partnerships are key metric of success.” 2. External stakeholder: “The project will be successful if we can get some more definitive information around the different approaches on Cape Cod and be able to relay that to others” 3. EPA researcher: “Are partners feeling that they are learning and that they are satisfied with the way information is flowing?” 4. External stakeholder: “I think it’s successful to have good sound work that stands unto itself.” 5. External stakeholder: “You want to see those results contributing to the bigger conversation and being used elsewhere, helping inform decisions in other places.” 6. “My standard as a scientist is peer-reviewed papers. I feel strongly about sharing those across the field. I would say most of our stakeholders like scientific papers in that they provide the stamp of validation for some of our work.”
SDR is different from a basic environmental research approach	The SDR process is seen as differing in the emphasis on stakeholder- and theory-driven engagement to define research questions, implementation, and deliverables, and integration of research components that requires high levels of internal communication among multiple research disciplines	1a. EPA researcher: “This project has been far more intentional and far more proactive in terms of seeking folks out and seeking out a wide variety of people across different levels of governance and different sectors and just making sure we’re regularly communicating with them and seeking their input.” 1b. EPA researcher: “I think that it [helped keep us] grounded in what people actually need and want, instead of what we just think is intellectually interesting.” 2. EPA researcher: “Usually it’s head down at the lab bench, and nobody knows what you’re doing. I think with such a large project or so many different avenues and so many researchers, it was really helpful to have coordination meetings with the whole EPA team.”

4.1. Stakeholder experiences

Across both pilots, the stakeholders interviewed shared that they had an overwhelmingly positive experience working with EPA on the SDR pilot projects. Interviewees within and beyond EPA shared that they experienced respectful relationships throughout their participation in the projects and that the “complementary capacities” of diverse forms of expertise were valued in research design and implementation (Table 3). When asked what stakeholders valued most about working with the EPA, stakeholders noted the EPA was essential to executing the projects, primarily due to the scientific expertise and the funding EPA provided. Importantly, the converse of this statement was also true; EPA researchers stated stakeholder engagement throughout was essential to the success of the SDR pilots. Partners provided critical input to the field study design and essential data collection efforts and served as a trusted communicator in their communities. Along with the expertise that EPA brought to these projects, stakeholders also perceived that the prestige of having a federal agency lead experiments in their communities improved local community perceptions of the validity of the work and increased concern about the underlying environmental problems.

The primary struggles stakeholders experienced in collaborating with the EPA related to the federal agency bureaucracy involved in sharing research results publicly. Misaligned timelines were also a challenge from the perspective of researchers, who wanted to be as helpful as they could to their stakeholder partners. The different considerations and timing in designing, executing, and communicating experiments for federal scientists and community-based stakeholders was a challenge that required understanding and, at times, compromise among all involved.

4.2. Markers of success

With deep commitments to the community and environmental issues the pilots addressed came multi-pronged ideas of what makes for successful project outcomes. Trust and relationship building was named as a “metric of success.” This is the metric associated with the positive collaboration experience named among both EPA researchers and stakeholders. Positive collaborations fostered a sense of camaraderie and belonging for team members because of the deep care people had for the environment and public health they were working to protect. The most frequently named marker of success was stakeholders using the project findings to inform actions. Stakeholders explained a marker of success was “lifting the veil of assumptions people have,” pointing to the value of data-backed explanations to correct misinformation about these environmental challenges among community members. Relatedly, stakeholders emphasized producing transferable and long-term results, and researchers emphasized continual in-process communication and evaluation of approach as key markers of success. Another marker of success was accomplishing research goals; some interviewees defined this as simply executing specific research tasks (e.g., installing septic systems or taking air quality measurements), while others focused on the broader community-defined goals of the research endeavor (e.g., reduced nutrient loads or improved indoor air quality). Both experiment-related markers align with taking a pilot approach to these projects, recognizing that going through the process with focus on the stakeholders’ needs is also an experiment. A marker of success EPA researchers shared as evidence of personal research success was production of peer-reviewed research on the experiments, as that is their primary career currency. One stakeholder also noted that they look forward to having the data-backed evidence of a peer-reviewed publication but did not point to its publication as a marker of success.

4.3. The difference of solutions-driven research

Another major theme from the analysis was the difference in taking an SDR approach to research. The two major aspects of SDR that were seen as different from past EPA research efforts by EPA researchers were the systematic continuous engagement with stakeholders and increased research integration across disciplines. This approach helped researchers better understand the environmental research needs in these communities and opportunities for interdisciplinary collaboration.

From the perspective of all involved, the way that engagement was pursued in the two SDR pilot projects differed from previous projects, as it was more stakeholder-driven and followed a theory-based plan of SDR engagement (Evans and Scarborough, 2014). Many researchers noted that they had done stakeholder engagement in past projects, and at times felt frustrated that branding SDR as a new approach was giving the work a new name rather than presenting a different approach. For researchers more used to doing basic environmental research, the engagement focus was a complete change to the process; rather than going to the academic literature first, these researchers learned the importance of “listening to what [needs and expertise] the stakeholder has, and then going back to literature.” Across the board, researchers noted that management attention to stakeholder engagement and using a systematic, structured approach to engagement was particularly different from past projects on which they had worked that involved stakeholder engagement. Stakeholders from both projects shared that they appreciated the primacy of their priorities in EPA’s research design, and the overall focus on addressing an issue identified by the “boots on the ground.”

Along with engagement with external stakeholders, EPA researchers also reported more integration and communication among researchers involved. For the Nutrients SDR Pilot, this also took the form of greater integration of social science projects with the biophysical experiments. Biophysical researchers shared that working alongside social scientists expanded their considerations of experimental impacts, and this was essential when working so closely with stakeholders. One area for continued growth in these collaborations is in clarifying the role of social scientists, as some biophysical researchers continue to view social scientists as predominantly responsible for outreach rather than recognizing their role in investigating human dimensions of environmental challenges (similar to findings of Eisenhauer et al., 2021).

5. Discussion and Recommendations: Improving SDR at EPA

5.1. Recommended strategies from participants

The interviews with participants of the two pilot projects elicited lessons learned and opportunities to improve future SDR projects and community-driven research. Two overarching themes in these recommendations for effective community-driven research are the need for increased flexibility and continued communication. These themes arose from both interviews and reflective case narratives and were central to each of the recommendations below. The key recommendations were categorized under (1) adaptive research goals and priorities, (2) roles and responsibilities, (3) funding, and (4) project conclusion (Figure 3). Below, we detail the key lessons based on interviewees’ experiences in the Nutrients SDR Pilot and Wildfire ASPIRE Study. We discuss below how these recommendations align with research findings, and how they could improve SDR at EPA.

Figure 3.

Recommended practices for future SDR projects

5.1.1. Adapting to differing goals and priorities

SDR requires an adaptive approach to meet changing needs. Researchers consistently shared, “If you want to do solutions-driven research you have to have some flexibility.” As researchers collaborated with stakeholders and grew to better understand the nuances of these communities and environmental challenges, research questions and approaches had to adapt. Adapting to these nuances in all community-driven research also requires recognizing that stakeholder needs are not monolithic within a community: “Every stakeholder, we all have the same goal, per se. But we have slightly different interests and some change over time, and sometimes it can be a little bit difficult to navigate.” As the Nutrients SDR Pilot progressed, a primary social strain and conflict arose regarding the different interests and experimental priorities among some key stakeholders. In mid-project interviews, EPA researchers anticipated this conflict, noting the different priorities of the main stakeholder relative to many of the scientists and some other external key stakeholders. To navigate these varied goals, EPA researchers found themselves often returning to the defined project approach of addressing the problem identified by all stakeholders and conducting the experiments as prioritized by the main stakeholder. Thus, even when the researchers disagreed with the priorities, there was a clear project-based rationale for the decision. To maintain positive relationships with all the key stakeholders on the project, the EPA team prioritized identifying how the chosen experiments could address other stakeholders’ needs and ensured the recognition of all stakeholder contributions when sharing about the extent of collaborations. This demonstrated to all stakeholders that they were important to the overall project, even if every stakeholder interest was not directly addressed. Consistent communication with all stakeholders, and especially with those who had disagreements with other stakeholders, was central to resolving varied priorities and maintaining positive relationships throughout and beyond the duration of the project.

Along with varied external stakeholder perspectives, researchers also had to consider the perspectives and goals of management internal to EPA. This created a challenge in balancing the expectations of internal EPA management in the timing of research deliverables with meeting the needs of stakeholders. Researchers shared that the flexibility needed to do “good solutions-driven research” was “inconsistent” with the more rigid research planning process traditionally followed. Explicitly planning for a flexible project timeline while still providing reliable access to staffing and funding resources will be necessary for future SDR projects to succeed. Another way to work with the often-conflicting timelines of research planning and project progress is for researchers to collaborate with EPA management to identify project milestones that can serve as deliverables. Potential milestones include a report on problem formulation (e.g., Twichell et al., 2019), documentation of stakeholder engagement efforts each year of the project, preliminary results of each experimental solution, a mid-project report halfway through a research cycle, and biannual “state of SDR” updates that incorporate both engagement and solution experimental updates. Defining project milestones as deliverables would allow management to communicate SDR project progress while still allowing for more project flexibility than prescribing a timeline for a final report may allow.

5.1.2. Roles and responsibilities

The most consistently noted lesson learned was ensuring sufficient capacity to fill needed roles and clear delineation of responsibilities. Consistently having the right EPA research expertise available to participate was a challenge. Researchers pointed out the need to adjust who is on the research team as new expertise becomes necessary with shifting stakeholder needs and goals and shifting availability of researchers. Having the right expertise was also noted by stakeholders, as one explained, “Getting some of the experts that you guys have…involved in an earlier stage might have been the kind of thing that could be helpful…I think understanding exactly what the resource buckets are is important.” Making clear from the outset the range of expertise EPA brings to the project can improve the efficiency and efficacy of staffing choices. EPA researchers noted challenges with the inconsistent availability of researchers to commit to the project for its duration. Fundamentally, lessons learned about staffing pointed to need for clarity, consistency, and flexibility.

To address challenges interviewees shared around roles and responsibilities, we recommend detailing at the outset project-specific roles and goals of project leadership and all team members, as defined in the existing EPA SDR framework. Addressing staffing issues could be aided by a defined personnel plan from the outset that allows for both adding and releasing researchers if the current teams’ expertise no longer meets shifting research goals. Staffing issues could also be aided by sharing a dossier of the involved researchers with stakeholders so that they can understand the full scope of the team. SDR planning documents recommend an early definition of roles, responsibilities, and expectations which would help SDR participants, both within and beyond EPA, understand to whom they should report with various updates and questions.

Management

The distribution of management responsibilities was noted among EPA participants as particularly unclear. The EPA’s framework documents for SDR described the various roles for successful projects (Table 1), but researchers shared, “It’s never been clear to me who’s doing what” for managing the pilots. The complex organizational structure within EPA made it challenging for some researchers to understand the roles of different aspects of the Agency (e.g., differentiating among the roles of national program offices, communications staff, and the Translational Navigator) and the roles of different managers within ORD. This “unclear communication and direction from our project management” resulted in a perceived management vacuum regarding who was establishing and monitoring progress toward the project goals. As prescribed by early project documents, the roles of Science Lead and Communications and Engagement Lead would define goals and track progress towards the aspects of the project named in their titles, while the Translational Navigator would assist both Leads in developing a truly transdisciplinary project by addressing funding, bureaucratic, and other planning hurdles. The Nutrients SDR Pilot team distributed the responsibilities of these three leadership roles across more EPA researchers rather than strictly defining the roles to three individuals. The Wildfire ASPIRE Study more strictly had three people in these roles. Reflective case narratives emphasized that the existing management culture in ORD is one focused more on high-level goals than micromanagement of career scientists. While this looser management style allows for scientific innovation, it proved to be a hurdle in the SDR pilot projects in clarity of responsibility and evaluating the process. Having more stringent adherence to specific leadership roles and responsibilities on these projects may have allowed for follow-through on a more structured evaluation of the pilots, helped researchers feel more supported in defining goals and outputs, and aided stakeholders in more efficiently connecting with the right collaborators. A balance between the direction and clarity of structure and the needed dynamism for an adaptive project should be a consideration for SDR projects moving forward.

Communication

How communication roles were distributed throughout the full project team was also unclear. Both pilots had a communication lead for part of the project, but due to employee transfer and retirement, this did not last the full duration of the project. Having a process in place for transferring knowledge rather than leaving this role empty when employees change positions will be important in future projects. Researchers also shared challenges in understanding the distribution of communication responsibilities among researchers and communications staff across different branches of the Agency. A recommendation from researchers and stakeholder interviewees was to specify a liaison who is the initial EPA point person for stakeholders so they can more efficiently connect with researchers involved with specific parts of the project; this is the role of the Translational Navigator, who would then defer to the Communications and Stakeholder Engagement Lead to coordinate strategic engagement. The Communications and Stakeholder Engagement Lead should work closely with the Translational Navigator to improve tracking of which stakeholders EPA researchers are communicating with.

Tracking interactions with stakeholders and defining communication roles will allow for a better understanding of the extent of relationships the EPA team has in an SDR project. Tracking relationships is an important communication and engagement task as relationships exist more with individuals than institutions. One researcher noted, “I think having multiple stakeholders in each organization is critical so that you know you have continuity” if stakeholders change positions or organizations or face other life circumstances that make them less involved in the project. Both pilots faced the challenge of stakeholder capacity and turnover. Additionally, interviewees recommended conversations and documentation defining roles and responsibilities across researchers and stakeholders from the outset, which can help ensure shared expectations of behavior and deliverables. While there were early project efforts to define research goals, there was a perception that the definition of roles in the project could have been better defined in these early meetings. Early research and relationship planning in this way would ensure transparency and understanding in how different institutions approach collaborations, both formally and informally, and help with navigating differing stakeholder interests. Together, having broader organizational connections and clearly defined expectations among researchers and collaborative stakeholders can ensure a continued positive relationship and capacity to complete the project.

Having a Communications and Stakeholder Engagement Lead will also ensure stakeholders know whom to contact about recommendations and perspectives on effective communication outputs. The clearer delineation of project leadership roles could improve collaboration with stakeholders in identifying final practical deliverables that will meet their needs. Interviews revealed that researchers and stakeholders held different perspectives on the usefulness of some deliverables intended to be for “general information,” emphasizing the need for skilled science communications staff. One interviewee noted, “not all researchers are good communicators,” in reference to skills in preparing written summaries and other communication products for communities involved in or impacted by the SDR projects. Communicating with stakeholders and translating complex science requires additional training and distinct expertise from what scientific researchers use to share results in peer-reviewed literature (Dilling and Lemos, 2011; Fischhoff and Scheufele, 2013). Developing approaches where researchers could be trained for media interactions on a given set of topics or questions could help ensure quicker and more useful communication among EPA researchers, stakeholders, and media.

5.1.3. Funding

Researchers repeatedly identified the need for a different funding approach for future SDR projects compared to the pilots. Stable funding is essential to these projects. Interviewees from the main stakeholder organization on the Nutrients SDR Pilot repeatedly noted that “we couldn’t have done this [project] without the funding from EPA.” These pilot projects were not included in the prospective planning process used for other multiyear projects within the Agency, and their funding was structured differently. From the perspectives of researchers, the approach used in these pilots proved to be unreliable. Researchers also noted that while including SDR projects in the 4-year prospective planning would improve funding security, additional flexibility will be needed for the adaptive nature of community-engaged approaches. As one researcher explained, “The timeline of SDR doesn’t always match the timeline of our research planning. You need to make sure that you provide funding for it because if you start doing this kind of thing and you engage with people […] and then it has to stop, what message does that send to your stakeholders?” Without secure access to funding, researchers faced challenges in funding experiments added to address new community-identified needs. Funding needs to be available for the time it takes to both build and maintain relationships in these projects as well as to conduct experiments that may be iteratively defined and improved upon to find a transferable solution. This challenge parallels the need to extend grant timelines to allow for relationship-building in collaborative research in academia rather than just supporting the execution of experiments (e.g., Davenport et al., 2021; NSF, 2023). Given that EPA, like all federal agencies, is funded through a variable annual appropriation, there will always be uncertainty in funding, which makes it challenging to ensure funding over the entire period of an SDR project. Budget planning over longer time horizons can make sure that the longer-term funding needs for SDR projects are given appropriate consideration. A key need for future projects is to make sure that funding expectations are appropriately managed in engagements with stakeholders.

An additional challenge in funding SDR at EPA is the ability to compensate people participating in these projects. Based on rules of how federal research funding can be used, public participants in EPA projects are not allowed to be directly compensated for the time they spend on these projects. As one researcher noted, “If we can’t pay people, we can’t expect them to give a lot of time.” With an increasing focus at EPA on meeting the needs of historically and continually disadvantaged communities and working towards environmental justice, this raises equity concerns (Cvitanovic et al., 2019; Karcher et al., 2022). Working through contractors, the Wildfire ASPIRE Study has been able to compensate a community stakeholder and their participants, but there were limitations as to how the funding could be used. Moving forward, it will be important to find an approved way to compensate community participants.

5.1.4. Project conclusion

A final key lesson is the importance of having a plan from the beginning for how to define the end of the project, with flexibility to evolve as needed. Concluding an SDR project can be a challenge, so clear expectations from the outset can provide a defined limit to the investigation of new research questions that arise throughout the project (Woo, 2021). Developing a plan that defines the project bounds can help maintain relationships and opportunities to participate in future related projects while allowing for researchers to exit the project stage since, as one researcher put it, “We have to say we’re done at some point.” While flexibility is important in SDR, clearly identifying the bounds of the research project demonstrates respect for the time of all those involved with the project and can help ensure the transfer of key knowledge to the stakeholders.

Without identifying an endpoint or exit strategy for researchers, it is difficult to assess whether the solutions provided helped to solve the original problem and whether sufficient capacity was built for stakeholders to maintain the solutions. A researcher explained, “I think there likely will be lots of bits that can be harvested longer term. I think recognizing the challenges of how you build relationships and then move onto another project and problem there’ll be challenges for ORD to think about.” An agreed-upon project endpoint is psychologically important, allowing stakeholders to expect the end of ongoing engagement with EPA. Without a pre-defined end to the project, such a shift could be startling and impact stakeholder reflections on the positive collaboration during a project. After implementing solutions, there is still work to do at EPA in providing a longitudinal evaluation of whether the solutions continue to work after the research is complete (Peng et al., 2020). Longitudinal evaluation allows for continuing the relationships built in early stages of the project while allowing them to evolve to be lighter touch with more community-based management of solutions. Committing to a project conclusion that plans for changing relationships and evaluation can allow for working towards the long-term goals stakeholder interviewees identified, including testing the transferability of solutions in other communities, and EPA goals of iteratively improving SDR.

5.2. Expanding SDR at EPA

In evaluating these pilot projects, we also sought to understand the potential role of SDR in future research at EPA. These projects were the first to implement the complete theoretical and conceptual frameworks of SDR as defined by EPA, and thus provided a new opportunity to understand strengths and shortcomings of the existing framework and ideals of SDR. The practice of SDR has led to revised tenets that, in addition to applying to broader regional or national studies, can accommodate a more place-based approach than originally conceptualized when adapting translational research practice from biomedical to environmental science. The place-based approach provides more localized problem definition, partners, and solution implementation with the goal of providing solutions that transfer to communities facing similar challenges. SDR at a nationwide scale would involve more government and national-scale organization partners, focused on providing more generalized solutions to challenges that communities can mold to their unique needs. Just as these pilots were a learning opportunity, continued improvement of SDR will require continued evaluation of the approach.

Using a consistent evaluation framework throughout SDR projects will aid in appropriate comparison of projects and effectiveness of the SDR approach. A rigorous evaluation approach that focuses both on internal improvement and research outputs is also a key tool for documenting transferability (Torres and Preskill, 2001). Researchers explained, “What worked here might not work someplace else,” inviting development of “guiding principles,” rather than an “overly structured approach.” This will allow for working with community stakeholders in ways that are appropriate for their structure and culture and identifying a research team with the right skill set. Relatedly, researchers suggested “more cross-project exchange” for researchers working on parallel SDR projects concurrently to problem solve and share updates. Sharing more lessons learned while the projects are in process can provide additional input to help overcome roadblocks and support progress through the adaptive project process. Future projects could supplement the qualitative evaluation of SDR through quantitative evaluation against collaboratively chosen metrics and objectives. Collecting meaningful data for such an evaluation would require buy-in from all relevant project participants, while also providing useful information for cross-project comparisons.

5.2.1. EPA researcher general SDR expansion considerations

EPA researchers were supportive of the general approach of SDR, and they provided experienced-based feedback on considerations in expanding the use of SDR at EPA. Researchers shared that “it cannot be done on every project” due to the time, money, and human resources needed to do SDR well, and “we risk disappointing anybody who’s waiting for that solution.” While the overall approach may need to adapt slightly, researchers did have recommendations on training to provide early on in future projects to ensure researchers have adequate skills for the approach. Recommendations included background on relevant environmental and health policy that the SDR project outcomes may interact with, background on how social science research is conducted if relevant, and the processes and evidence-backed practices for engaging with external stakeholders throughout the project.

Along with resources limiting how much SDR should be expanded, researchers also shared the need for continued basic laboratory research that does not invite a stakeholder-driven approach. While it may not be appropriate to include stakeholders in basic lab research, investigators can design the research such that results can inform the next stage of solution development. In the current EPA research cycle, we are modeling this approach of embedding some of the solutions-driven steps in our projects based on lessons learned to date (US EPA, 2022a). Including SDR concepts of an integrated approach and usable results in EPA research can support its goals with a lower resource requirement than complete adherence to the SDR framework.

5.2.2. Choosing appropriate future SDR projects and stakeholders

In choosing projects to implement the full SDR approach, it is essential to consider not only what EPA is offering to the stakeholders, but also what they are asking the stakeholders to give. The high resource requirements of SDR add the need to select problems that are “ready for solutions” and community or other decision-making stakeholders with capacity. For example, while in one community the SDR collaboration may fit well into existing efforts at environmental problem-solving, in another community it may be necessary to compensate stakeholders for the additional staff time and effort these projects add to their job. As one interviewee explained, “Not every community with a challenge that aligns with EPA skillsets has the bandwidth to participate in an SDR project. It is demanding of the key stakeholders as well, and often this collaboration is on top of an already full plate.” This is especially true when the community in which a project is based faces environmental injustice or other marginalization, as the goal is to help solve problems, not be extractive. As EPA works to build more complete models of co-production and incorporation of local knowledge for future projects, it is essential to be mindful of the time cost and other dynamics of collaborating with stakeholders with marginalized identities. To use SDR most efficiently to tackle environmental problems, researchers recommended “strategically picking a community or a region or an organization to work with [where results] will be translatable to others in similar situations,” or identifying a problem that can be studied at “multiple locations across the country, where there are similar problems and we could…learn from the different geographies.”

It is essential to accurately structure and define the problem in solutions-driven research to provide solutions that address the stakeholders’ true needs. As SDR seeks to address complex problems with unclear solutions, it is essential to spend significant time formulating the problem to be solved, and to discuss the interdependencies and layers that make the problem, and solution, so complex (Dunn, 2018). A key factor in the successful identification of the primary problem for study in these pilot projects was the in-depth problem formulation process. The Nutrients SDR Pilot held a two-day problem formulation workshop that brought together stakeholders from across the region to identify research needs and then specify the next steps (Twichell et al., 2019). The Wildfire ASPIRE Study also held multiple problem formulation sessions with their stakeholders to ensure accurate identification of the public health challenge. While the formal problem formulation sessions were held at the start of the pilot projects, revising problem and solution definitions occurred continually throughout the projects. In problem structuring and throughout the project, consistent engagement and having a key stakeholder served to improve a shared vision and consensus. This approach of high-investment problem structuring should be applied in future projects, anticipating iterative refining of the problem and solution based on improved information and continued experimentation (Dunn, 2018).

Rather than applying the immersive, intensive SDR framework wholly to the EPA research portfolio, the approach needs to be thoughtfully applied to problems that are community priorities, and where stakeholders have interest and capacity to participate in identifying solutions. For other elements of the ORD research portfolio, the broad principles of solutions-driven research should be incorporated, while recognizing that the level of stakeholder engagement needs to be scoped to reflect available resources. Just as EPA researchers learned and adapted to stakeholder needs throughout these pilots, EPA has been able to evolve and has positioned itself to evolve further in the future. Integrating some SDR tenets into all aspects of EPA ORD research is a step towards further realizing the Agency’s mission of “protecting human health and the environment.” Stakeholders were excited by the idea of additional collaboration opportunities with EPA researchers and continuing to work with the Agency on research rather than regulatory concerns.

6. Conclusion

Solutions-driven research (SDR) allows EPA to better understand specific problems faced by stakeholders, develop integrated research plans to produce solution-relevant outputs more effectively, and ensure scientific information produced by EPA can directly inform the development of solutions. SDR pilot projects allowed ORD to test initial ideals for this approach at the Agency and improve the theoretical and practical frameworks to strive for in SDR. Further, the pilot approach utilized social science research to document successes and opportunities in applying SDR at a government agency.

The experiences of EPA researchers and key stakeholders involved in solutions-driven research projects provide valuable information on the strengths and opportunities of this transdisciplinary approach to researchers more broadly tackling environmental challenges. The consistency of perspectives collected on project successes and lessons learned across two parallel projects provides evidence of the transferability of this approach and areas to focus on for improvement. Both teams had positive working relationships with their community stakeholders throughout the pilots and succeeded at implementing innovative experimental solutions to complex environmental problems. Based on both engaged research literature (Besley et al., 2019) and researcher experiences in these pilots, following a strategic approach to engagement should be continued in future iterations of SDR (Canfield et al., 2022; Canfield and Chatelain, forthcoming) and in community-driven environmental projects. Participants’ experiences demonstrate that SDR is a valuable tool for building partnerships between the government and local communities, as well as for integrating experiential knowledge into environmental research.

Moving forward, lessons learned and recommendations in this paper will improve transdisciplinary environmental research projects both within and beyond EPA. The high-level lesson is the need for a flexible research plan that defines the overarching project goal and endpoint; specifies researcher, communication, manager, and stakeholder roles; and identifies the commitment of resources for project funding that can adapt to the needs of the project. Another major lesson is the need to acknowledge the different pace of SDR. It is both faster and slower than basic environmental research, requiring extensive time in relationship building before experimentation and continuous communication of research results. This approach requires adjusting stakeholder, manager, and Agency timeline expectations for deliverables on in-progress research and final results. The perspectives and lessons of this analysis can help to inform institutional (government or academic) expectations of outputs and metrics of success in transdisciplinary research.

Highlights.

• Solutions-driven research is a transdisciplinary approach to environmental research

• Interviews and reflective case narratives evaluate participant perspectives

• Successful collaboration demands notable time to build and maintain relationships

• Clearly defined roles, funding, & project conclusion improve solutions-driven work

• Expanding solutions-driven research demands considering capacity and resolvability