Scaling what works: a collaborative capacity-building program tailored to optimize resources and scale evidence-based interventions for cancer prevention and control

Abstract

Purpose

This study aimed to evaluate the Scaling What Works (SWW) program, which was developed to support the scale-up of projects that implement evidence-based cancer prevention and control interventions (CPC EBIs) across award recipients (hereafter, “awardees”) of the National Comprehensive Cancer Control Program (NCCCP). SWW projects included (1) Project ECHO and patient navigation, (2) Addressing risk factors for adult cancers during childhood, and (3) Increasing receipt of ovarian cancer care from gynecologic oncologists. In this evaluation, we sought to understand how tailored training and technical assistance (TA) can facilitate the implementation of CPC EBIs in NCCCP settings.

Methods

We used a mixed-methods approach, comprising a needs assessment with focus groups and information sessions. Subsequent training and TA included a symposium and four virtual learning collaborative sessions. The evaluation included pre- and post-surveys to assess changes in participants’ confidence and satisfaction alongside qualitative feedback to refine the program.

Results

The SWW program reached 89 individuals across 40 NCCCP awardees. Participants reported significant improvements in their confidence to implement, adapt, and sustain EBIs, with increases observed in all domains assessed. Feedback from the symposium and learning collaborative sessions indicated high satisfaction and notable progress in addressing barriers related to EBI implementation.

Conclusion

The SWW program effectively addressed key challenges in scaling CPC interventions by providing tailored, interactive training and TA. Further engagement and continued capacity-building support are recommended to enhance the long-term impact of SWW across the NCCCP. The findings underscore the importance of investing in tailored, large-scale capacity-building to support the scale-up of evidence-based CPC approaches.

Introduction

In the United States, preliminary registry data estimate that 1.97 million people were diagnosed with cancer in 2023 [1], and where an estimated 18.1 million cancer survivors are also living [2]. Established in 1998, the Centers for Disease Control and Prevention’s (CDC’s) National Comprehensive Cancer Control Program (NCCCP) provides funding and public health guidance to 66 award recipients (hereafter, “awardees”) including all 50 states, the District of Columbia, eight U.S. territories and U.S.-affiliated Pacific Island jurisdictions, and seven organizations serving American Indian/Alaska Native populations. NCCCP awardees help build and support local cancer coalitions with partners representing multisectoral organizations working toward implementation of jurisdiction-wide cancer plans. These coalitions work together to develop and implement cancer plans aimed at reducing risks for developing cancer, increasing access to cancer screening and treatment, and enhancing quality of life among cancer survivors in the jurisdictional area [3].

Over the years, the CDC’s Division of Cancer Prevention and Control (hereafter, CDC) has piloted projects to implement evidence-based interventions (EBIs) related to cancer survivorship and risk reduction with select NCCCP awardees. We define EBIs as cancer control programs that have been rigorously evaluated and proven effective by one or more previous research studies (National Cancer Institute 2024), such as the Give Teens Vaccine intervention [12] that promotes HPV vaccination to reduce cancer risks among adolescents. However, sustaining these practices and scaling them to additional NCCCP awardees has been challenging. Research indicates that practitioners need additional support to build the capacity required for implementing new interventions due to the complexities and competing demands of their work environments [3]. Recently, CDC identified three successful pilot projects they wanted to scale to additional NCCCP awardees. The projects were designed to implement one or more EBIs. The projects included 1) Using Project ECHO (Extension for Community Healthcare Outcomes) and Patient Navigation to Improve the Health and Wellness of Cancer Survivors in Rural Communities (hereafter, ECHO + Nav) [4], 2) Addressing Risk Factors for Adult Cancers during Childhood (hereafter, ARF) [5], and 3) Increasing Receipt of Ovarian Cancer Care from a Gynecologic Oncologist (hereafter, Gyn-Onc) (collectively referred to as “the projects”) [6, 7]. To support scale-up efforts, CDC funded the Cancer Prevention and Control Research Network (CPCRN) Coordinating Center [8] to develop the Scaling What Works (SWW) program, designed to provide targeted training and technical assistance (TA) to the broader NCCCP, especially awardees who did not participated in pilot projects, in order to expand the reach of these successful projects.

The SWW program scaled up three selected successful pilot projects within NCCCP. The first project—Project ECHO + Nav—pairs Project ECHO and patient navigation to improve cancer survivorship in rural areas [4]. ECHO + Nav demonstrated success across four NCCCP awardees in improving cancer survivorship care and in training 16 patient navigators who helped connect 164 rural cancer survivors with needed resources [4]. In the second project—ARF—three NCCCP awardees selected evidence-based primary prevention activities to reduce the risk of cervical, liver, and skin cancer later in life among children. Examples of EBIs that were implemented as part of the ARF project include one that improved provider intentions to support HPV vaccination and another that integrated sun safety programs into state parks [5]. The final SWW project—Gyn-Onc—focused on increasing the receipt of ovarian cancer care from gynecologic oncologists and was successfully piloted with three NCCCP awardees (Rim et al. 2022; Stewart et al. 2021). One NCCCP awardee developed an Ovarian Cancer Checklist to enhance providers’ knowledge, awareness, and intent to refer patients to gynecologic oncologists [11].

The CPCRN, the longest-standing research network of the CDC’s Prevention Research Centers, has over 20 years of experience conducting community-based participatory research and capacity-building activities [4]. The network focuses on empowering the cancer prevention and control workforce and reducing cancer burden for all. During the CPCRN’s fifth funding cycle (2019–2024), membership included investigators from eight funded collaborating centers, along with affiliate members and members and alumni of the CPCRN Scholars program [5]. The University of North Carolina, which serves as the Coordinating Center, facilitates cross-center collaboration among these members. SWW is a collaborative process between the CPCRN, CDC and NCCCP to harness existing CPCRN resources, including over two decades of experience providing evaluation support, training, and capacity-building TA for community partner EBI implementation [6]. The co-designed SWW program purposefully addressed NCCCP needs and preferences regarding training topics, modes of delivery, locations, timelines, and tailoring the program to the projects. The purpose of this paper is to present an overview of the newly initiated SWW program and evaluation results from the inaugural year, highlighting best practices for building capacity to scale evidence-based cancer prevention and control interventions.

Methods

Approach to SWW program design

A key aim of SWW was to help additional NCCCP awardees build upon these successful pilot projects. Instead of starting from scratch, interested awardees could use the proven models as a foundational framework, ensuring efficiency, reliability, and effectiveness. However, the goal was not to replicate these projects exactly but to thoughtfully adapt them to new contexts, ensuring they remained relevant and responsive to the specific needs of each area population to which an NCCCP awardee administers. This adaptation process allowed for flexibility in selecting which elements to implement and how. Additionally, awardees were able to choose different EBIs, provided they targeted the aims of one of the three projects. For instance, awardees that selected the ARF project with a focus on cervical cancer risk reduction could opt to implement the Give Teens Vaccine [7] or the Making Effective Vaccine Recommendations intervention [8]. SWW program supported the awardee selection of projects and related EBIs.

Scaling what works team

The SWW project was led by CPCRN investigators from UNC (UNC team) and supported by a network-wide group, effectively leveraging the network’s expertise in capacity-building (hereafter, “CPCRN SWW Group”) [5, 9]. The CPCRN SWW Group members included capacity-building experts, health services researchers, implementation scientists, health care providers, and CDC staff. The CDC members identified NCCCP leaders who had successfully implemented one or more of the three pilot projects to join the SWW team as project mentors. The NCCCP awardee that the mentor was affiliated with received $5,000 for participation in SWW; this funding was provided by CPCRN and was in addition to their overall NCCCP award. The SWW team developed, delivered, and evaluated the SWW program from September 30, 2023, through September 29, 2024. The UNC Institutional Review Board (IRB) determined that the project did not qualify as human subject research under federal regulations and therefore did not require IRB approval (IRB # 23–2794).

SWW program phases

The SWW program included two phases: 1) a needs assessment and 2) delivery of training & TA (Fig. 1). In Phase One, the SWW team completed the needs assessment through focus groups and information sessions conducted with and in partnership with the NCCCP. Phase Two consisted of a three-day in-person symposium to deliver training and TA, followed by a four-month-long virtual learning collaborative, hosted monthly.

Fig. 1.

Overview of the scaling what works (SWW) program

SWW program recruitment

The SWW team invited leaders from all 66 NCCCP awardees to participate in both phases of the SWW program. For the information sessions and focus groups comprising the needs assessment (phase one), the SWW team created invitations to be sent through an established CDC-led NCCCP listserv. For the symposium and learning collaborative to deliver training and TA (phase two), two members of the SWW team attended the Biannual CDC-led NCCCP Business Meeting to invite attendees and to share an overview of the SWW program; a registration link was provided during the presentation. This in-person invitation was followed by an email to meeting attendees. Finally, CDC staff encouraged participation in all SWW components during their regular progress meetings with NCCCP awardees. To maximize participation, mass symposium invitations were emailed to the NCCCP listserv and individual invitations were sent to all participants of the focus groups and information sessions. Up to three representatives from each NCCCP awardee were eligible to register for the SWW symposium and learning collaborative. During the registration process, participants signed a collaboration agreement, which committed them to participation in the symposium and the learning collaborative and to progress toward implementing at least one SWW project. Awardees were reimbursed for travel expenses and were provided an additional $3,175.00 to support their commitment to SWW, based on funds available.

Phase one—needs assessment

Focus groups

With the aim of identifying NCCCP barriers and facilitators to implementing the three projects, as well as their current capacity needs, and interview. The SWW team developed a guide based on the Program Sustainability Assessment Tool (PSAT) [10] to structure focus group discussions. During the focus groups, virtual polls were administered to gather additional insights into participant NCCCP roles and training and TA preferences (e.g., in-person vs. virtual, meeting frequency). Focus groups were organized by SWW project interest (i.e., ECHO + NAV, ARF, Gyn-Onc), with each session averaging 60 min. Sessions were recorded and participants’ verbal consent was provided on record. Notetakers summarized responses by PSAT domain, and experienced qualitative researchers conducted a rapid turn-around analysis [11] to quickly obtain actionable data on awardees’ needs. They then compiled the findings into a matrix to identify common themes and inform the next steps.

Information sessions

In response to needs identified in the focus groups, the SWW team conducted information sessions to simultaneously address participants’ requests for more SWW project information and collect additional data to inform the needs assessment. Separate sessions were held for each SWW project. During each session, a SWW project mentor presented a project overview and shared their implementation experience, followed by question and answer (Q&A) discussions and a virtual poll to assess further TA and training needs. Each information session lasted 60 min and was recorded with the participants’ verbal consent. Analysis strategies of information sessions included both descriptive statistical analysis of virtual polls, and rapid turn-around analysis [11] of open-ended questions and facilitator notes. Again, the SWW team followed the PSAT structure [10] to identify needs for symposium and learning collaborative planning.

Phase two—training and technical assistance

The focus of Phase Two was to provide training and TA via a learning collaborative format, drawing upon preexisting models for collective capacity building. Specifically, the SWW team adopted an “individualized, hands-on approach to building an entity’s capacity for quality implementation of innovations” modeled by Lamont and colleagues [12]. The “learning collaborative” format was used as a structured approach to bring together groups of professionals or organizations with a common goal to share knowledge, solve problems, and improve practices collectively [13].

Training curriculum

At the core of the SWW symposium and learning collaborative structure is the Putting Public Health Evidence in Action [6] (PPHEIA) curriculum. This curriculum is an interactive training designed to support public health professionals, health educators, and community program planners in the use of EBIs. Briefly, this training guides participants through the steps of EBI implementation over six 30-min modules of didactic content followed by interactive group activities focused on application to specific projects and contexts. The curriculum incorporates examples from real-world projects and research studies, making it highly adaptable to diverse audiences and health topics. The SWW team adapted this curriculum to address the specific needs of NCCCP sites implementing one of the three SWW projects, including adding an additional module address scaling and sustainability [14, 15].

Symposium

The purpose of the symposium was to provide interactive training sessions on the EBI implementation process, structured by the PPHEIA modules and tailored to SWW project and specific NCCCP contexts and needs. Taking place from May 6th to 8th, 2024, in Orlando, Florida, the symposium fostered networking and resource sharing among NCCCP participants, project mentors, and CDC staff.

Prior to the symposium, attendees completed a preparatory pre-work assignment that also provided background information on each awardee’s context and current program progress. As part of the pre-work, participants were asked to identify the project that they were considering (and EBI, if applicable), describe their objective, and answer questions on the following topics: 1) their experience with their selected project, 2) the context in which the project will be implemented (e.g., population, setting), and 3) existing barriers and facilitators to implementing their project. The pre-work also included an 18-item implementation readiness assessment used in the PPHEIA curriculum [6] and the PSAT [10] (to prepare for a discussion during the sustainability session). The SWW team hosted optional office hours leading up to the symposium to assist with the pre-work assignment and answer questions.

The symposium agenda included 1) an overview of the three SWW projects and available resources for each, presented by CDC; 2) PPHEIA training sessions focused on adapting, implementing, evaluating, sustaining, and scaling SWW projects; 3) expert Q&A panels featuring mentors from the pilot sites and CDC staff; and 4) informal networking opportunities. Each PPHEIA training session included didactic content that covered an overview of each implementation process with examples related to SWW projects, followed by interactive activities where NCCCP participants applied the training content to their SWW project. The activities were conducted in project-specific breakout rooms and facilitated by CPCRN members, CDC staff, and SWW mentors.

Through pre–post surveys, the SWW team evaluated the effectiveness of the PPHEIA training sessions on attendees’ confidence in performing session learning objectives. Survey items included the learning objectives for each training session; respondents rated their confidence from not confident at all (“1”) to very confident (“5”). We employed a paired t-test using STATA software [16] to assess pre–post changes in confidence in all attendees, by SWW project groups, and by geographical locations (e.g., states vs. U.S.-affiliated Pacific Islands).

Using the open and closed ended questions on the post-symposium survey, we evaluated attendees’ satisfaction with the symposium. The SWW team used descriptive statistics (frequency and percentage) to analyze the quantitative data. Responses to open-ended questions were categorized to identify symposium strengths and areas for improvement. The SWW team applied the findings to address ongoing needs and to develop the content of the follow-up learning collaborative sessions.

Learning collaborative

Following the symposium, the SWW team facilitated four virtual monthly learning collaborative sessions with NCCCP participants. Prior to each session, we asked participants to complete assignments that were designed to encourage their continued progress and prepare them for session activities. We asked for updates on progress, implementation plans, evaluation plans, and assignments, respectively, at each of the four learning collaborative sessions.

Each learning collaborative session began with a 10-min review of the PPHEIA training focus (i.e., adaptation, implementation, evaluation, and scaling and sustaining) [6]. Most of the following 40-min session included breakout room discussions grouped by project (i.e., ECHO + Nav, ARF, Gyn-Onc). During the breakout discussions, the session facilitator invited each participant to share a project update and discuss their assignment (i.e., implementation plan, evaluation plan, or PSAT [10]). SWW team members—including CPCRN implementation scientists, CDC staff, and pilot project mentors—answered questions and facilitated participants’ discussion around resource sharing and overcoming barriers to support the continued progress of each project. Sessions were concluded by inviting the breakout group members to return to the main room for a discussion of next steps and to complete a 4–5 question survey to evaluate the session.

As part of each learning collaborative session, participants completed surveys to assess: 1) their satisfaction with the learning collaborative sessions; 2) the influence of the sessions on their ability to adapt, implement, evaluate, and sustain their SWW projects; and 3) the progress made on their SWW project. Participants were invited to provide open-ended feedback on the session. Descriptive statistics (frequency and percentage) in Stata [16] were applied to each survey item. The SWW team reviewed the qualitative responses to inform how they may tailor the structure and content of the subsequent learning collaborative sessions.

Results

Overall, 89 individuals from 40 different NCCCP awardees participated in at least one phase of the SWW program (i.e., information session, focus group, symposium, or learning collaborative). These awardees included 31 states, two organizations serving American Indian populations, and seven U.S. territories and U.S.-affiliated Pacific Island jurisdictions. These awardees were geographically dispersed across 3,000 miles and 12 time zones—with a difference of up to 16 h (GMT-4 to GMT + 12)—depicted in Fig. 2. Participant engagement in all SWW activities including completion of the three learning collaborative assignments is depicted in Fig. 3.

Fig. 2.

Map of Participation in Scaling What Works (SWW) by National Comprehensive Cancer Control Program (NCCCP) Award Recipients (Awardees), 2023–2024. NCCCP: National Comprehensive Cancer Control Program; EEZ: Exclusive Economic Zones; SWW: Scaling What Works; USAPI: US-Affiliated Pacific Island Jurisdiction. This map is made using the geospatial data from the following sources: Pacific Data Hub. (2019). Pacific Island Countries and Territories Exclusive Economic Zones. [GeoJSON]. Pacific Data Hub, The Pacific Community. Retrieved from https://pacificdata.org/data/dataset/pacific-island-countries-and-territories-exclusive-economic-zones, United States. Census Bureau. (2018). Cartographic Boundary Files. US State {5 m}. [Shapefile]. https://www.census.gov/geographies/mapping-files/time-series/geo/carto-boundary-file.html. United States. Census Bureau. (2018). Cartographic Boundary Files. American Indian/Alaska Native Areas/Hawaiian Home Lands {5 m}. [Shapefile]. https://www.census.gov/geographies/mapping-files/time-series/geo/carto-boundary-file.html, United States. Department of State. Office of the Geographer. (2013). Detailed World Polygons (LSIB), Oceania, Malaysia, Antarctica, 2013. [GeoJSON] Humanitarian Information Unit (U.S.), Stanford University Libraries—EarthWorks. Retrieved from https://earthworks.stanford.edu/catalog/stanford-dt465jv7171

Fig. 3 .

Individual participant engagement in Scaling What Works (SWW) activities through attendance and assignment completion, by NCCCP Award Recipient, and organized by SWW project*. *Attendance data for each session do not represent unique individuals, as the same person may have attended more than one activity. **SWW participants that did not participate in the Symposium or Learning Collaboratives were not required to select a SWW project. NCCCP = National Comprehensive Cancer Control Program, SWW = Scaling What Works; LC1-LC4 = Learning Collaboratives 1–4

Needs assessment

Sixteen individuals from eight different NCCCP awardees participated in the focus groups, and 40 individuals from 24 different NCCCP awardees attended information sessions. Most participants who attended the focus groups were in the early stages of planning one of the SWW projects and had knowledge gaps and resource limitations during implementation. The most commonly reported challenges during project planning and implementation were related to not having enough personnel, difficulty engaging in and maintaining partnerships, lack of funding, and difficulty obtaining information and resources about each of the SWW projects and related EBIs. All attendees expressed the need for more funding and support in multiple areas to overcome these barriers. Regarding the information sessions, most poll respondents (85%) agreed that the information sessions improved their understanding of the SWW projects. However, their confidence in applying this knowledge remained limited, particularly in project planning, sustainability, and applying for funding. Attendees requested more examples of implementing the SWW projects, resources, funding, guidance from CDC, and connection with ECHO experts. Needs assessment participation by NCCCP awardees, including pilot program mentors, facilitated the co-design of the SWW symposium and learning collaborative to ensure that the training and TA was tailored to each program’s needs. Findings from the needs assessment were applied to develop the symposium and learning collaborative as detailed in Fig. 4.

Fig. 4.

Application of needs assessment findings to develop the Scaling What Works (SWW) program. ECHO = Extension for Community Healthcare Outcomes

Symposium

Ultimately, 51 attendees from 23 NCCCP sites, which included fifteen states, one organization serving American Indian populations, and seven U.S. territories and U.S.-affiliated Pacific Island jurisdictions, participated. Of these attendees, 23 individuals (11 sites) focused on the ARF project, 26 individuals (11 sites) on ECHO + Nav, and 2 individuals (1 site) on Gyn-Onc. A total of 25 attendees (49%) completed both pre- and post-symposium knowledge assessments. They reported statistically significant increases in their confidence in achieving all session learning objectives: project overview (p = 0.0000), adaptation (p = 0.0000), implementation (p = 0.0000), evaluation (p = 0.0001), and sustainability (p = 0.0001) (Table 1). We also observed significant increases in their confidence in identifying resources, such as tools and guidance, for adapting (p = 0.0002), implementing (p = 0.0002), evaluating (p = 0.0094), and sustaining (p = 0.0001) SWW projects.

Table 1.

Changes in mean of the confidence in achieving the session learning objectives before and after the Scaling What Works (SWW) symposium

Symposium Sessions	Total (N = 25)	Geographical Location		SWW Projects
	Total	US Continental (n = 18)	USAPI (n = 7)	ECHO + Nav (n = 14)	Gyn-Onc (n = 3)	ARF (n = 8)
Section 1. Scaling What Works Project Overview
Understand the rationale behind each of the three Scaling What Works projects	+ 0.84***	+ 0.78**	+ 1.00*	+ 0.71*	+ 1.00	+ 1.00**
Identify resources to help implement the Scaling What Works projects	+ 0.64**	+ 0.45	+ 1.14*	+ 0.36	+ 0.67	+ 1.13**
Identify mentors for Scaling What Works projects	+ 1.12***	+ 1.06***	+ 1.29**	+ 1.00**	+ 1.00	+ 1.37**
Total	+ 2.6***	+ 2.29**	+ 3.43**	+ 2.07**	+ 2.67	+ 3.50**
Section 2. Adapting Scaling What Works Projects to your Programs and Settings
Describe basic principles for selecting EBIs that fit your objectives, setting, and population	+ 0.64***	+ 0.50**	+ 1.00	+ 0.64**	+ 0.66	+ 0.63
Define adaptation, fidelity, and core elements	+ 0.88***	+ 0.78**	+ 1.14*	+ 0.86**	+ 0.66	+ 1.00*
Describe the process and steps for adaptation	+ 0.88***	+ 0.84***	+ 1.00	+ 0.86**	+ 0.66	+ 1.00
Discuss which changes can be made without affecting the effectiveness vs. which cannot	+ 1.08***	+ 1.05***	+ 1.15*	+ 1.14***	+ 0.66	+ 1.12*
Total	+ 3.48***	+ 3.17***	+ 4.29	+ 3.50***	+ 2.64	+ 3.75
Section 3. Implementing Scaling What Works Projects
Describe six steps to successful program implementation	+ 1.20***	+ 1.00***	+ 1.71*	+ 0.86**	+ 1.34	+ 1.75
Assess a setting’s readiness to implement an EBI	+ 0.80***	+ 0.67**	+ 1.14	+ 0.93**	+ 0.33	+ 0.75
Conduct Plan Do Study Act Cycles	+ 1.20***	+ 1.22***	+ 1.14	+ 1.00***	+ 1.67*	+ 1.37*
Develop an implementation plan	+ 1.08***	+ 1.11***	+ 1.00	+ 1.00***	+ 1.67*	+ 1.00
Total	+ 4.28***	+ 4.00***	+ 4.99*	+ 3.79***	+ 5.01*	+ 4.87*
Section 4. Evaluating Evidence-based Interventions
Define different types of evaluation	+ 1.04***	+ 0.78***	+ 1.71*	+ 1.00***	+ 0.66	+ 1.25*
Explain the difference between process and outcome evaluation questions	+ 0.96***	+ 0.83***	+ 1.29	+ 0.86***	+ 1.33	+ 1.00
Identify measurable outcomes related to aims, objectives, and activities	+ 0.88***	+ 0.72***	+ 1.28	+ 0.78**	+ 1.00	+ 1.00
Collect reliable and valid performance data	+ 0.80**	+ 0.56*	+ 1.43*	+ 0.64*	+ 0.66	+ 1.13
Total	+ 3.68***	+ 2.89***	+ 5.71*	+ 3.28***	+ 3.65	+ 4.38
Section 5. Sustaining and Scaling Evidence-based Interventions
Define sustainability	+ 0.68***	+ 0.61**	+ 0.86*	+ 0.57*	+ 1.00	+ 0.75*
Define scale-up	+ 1.08***	+ 0.89***	+ 1.58*	+ 0.79**	+ 1.34	+ 1.50*
Conduct a sustainability assessment to identify factors that impact sustainability	+ 1.08***	+ 1.06***	+ 1.14	+ 0.93**	+ 1.33	+ 1.25*
Select strategies or frameworks to plan for sustainability	+ 0.88***	+ 0.83**	+ 1.00	+ 0.78**	+ 1.00	+ 1.00
Apply criteria to monitor and evaluate sustainability	+ 0.84**	+ 0.89***	+ 0.72	+ 0.64**	+ 1.67*	+ 0.87
Total	+ 4.56***	+ 4.27***	+ 5.30	+ 3.72**	+ 6.33	+ 5.38
Section 6. Confident in ability to find materials and resources for the following
Adapting Scaling What Works Projects to your Programs and Settings	+ 1.00***	+ 0.95***	+ 1.15	+ 0.93**	+ 1.00	+ 1.12
Implementing Scaling What Works Projects	+ 1.02***	+ 0.94***	+ 1.21	+ 1.02***	+ 0.66	+ 1.12
Evaluate Evidence-based Interventions	+ 0.72**	+ 0.50	+ 1.29	+ 0.71*	+ 0.67	+ 0.75
Sustaining Scaling What Works Projects	+ 1.16***	+ 1.06***	+ 1.43	+ 1.29**	+ 1.34	+ 0.88

Scale (range 1–5) with 1 = not confident at all to 5 = very confident; * < .05, ** < .01, *** < .001 (Bold text indicates <.05)

USAPI United States Affiliated Pacific Islands, ECHO + Nav Using Project ECHO (Extension for Community Healthcare Outcomes) and Patient Navigation to Improve the Health and Wellness of Cancer Survivors in Rural Communities, Gyn-Onc Increasing Receipt of Ovarian Cancer Care from a Gynecologic Oncologist, ARF Addressing Risk Factors for Adult Cancers during Childhood

Differences regarding improvements in confidence in achieving learning objectives emerged across attendee groups, including by SWW program and by geographical locations (Table 1). There were significant improvements in all objectives for attendees focused on ECHO + NAV (n = 14), except for one objective under SWW project overview. However, significant improvements were observed in three objectives concerning project implementation and sustainability for attendees focused on Gyn-Onc (n = 3). Attendees focused on ARF (n = 8) reported significant increases in their confidence in achieving about half of the session objectives. When grouped by geographical locations, most attendees from the continental US (n = 18) reported statistically significant improvements in almost all objectives; in contrast, this significance applied to about half of the objectives among attendees from territories or jurisdictions outside the continental US (n = 7). In addition, significant improvements in confidence to identify resources were only observed among attendees focused on ECHO + NAV and those from continental US.

Thirty-nine participants (77%) completed the post-symposium satisfaction survey. Overall, the satisfaction mean scores ranged from 4.4 to 4.7 across the symposium Sects. (1 = fully dissatisfied – 5 = fully satisfied). Most participants (92%) agreed that the training was appropriate for their level of experience (1 = too basic, 2 = about right, 3 = too advanced). Most (74%) felt the balance between lecture and group interaction was just right (1 = too much lecture, 2 = too much group interaction, 3 = about right). One participant shared, “The format with large group, breakout, and then reporting out was great. Going step by step and immediately applying what we learned to our projects was very helpful.” Most participants provided positive qualitative feedback on the symposium’s content and structure. For example, one participant reported, “The breakout sessions were phenomenal. I’m excited to bring back what I learned to adapt and implement in my state.” Similar feedback was provided across respondents focused on different projects and from different locations. One participant requested even more time to connect with mentors, “I wish we had more time to talk with other programs and states that have successfully implemented our projects.” Another participant requested that more of the learning take place as part of the pre-work assignment, saying, “There was too much theory—some of that could’ve been pre-work.” Participants from awardees in U.S. territories and U.S.-affiliated Pacific Island jurisdictions felt that it was a long distance for them to travel, and jet lag made it difficult to completely engage with certain components of the symposium; one respondent would have liked a more central location than Orlando, FL.

Learning collaborative

All but one of the NCCCP awardees that attended the symposium participated in at least one of the four learning collaborative sessions. This awardee cited time and scheduling constraints for not participating. Attendance in each of the four learning collaboratives and the status of assignments for each is detailed in Fig. 3.

Across the four learning collaborative sessions, participants demonstrated high satisfaction and increased confidence, and made notable progress in adapting, implementing, evaluating, and sustaining their projects (Table 2). The percentage of the attendees who were satisfied with the session ranged from 91 to 100% across the four learning collaborative sessions, with participants appreciating breakout discussions, resource sharing, and feedback on their work plans. Across the sessions, about 82% to 96% of respondents reported at least moderate confidence in applying session content. In the post-surveys, most participants demonstrated progress in various topics of the learning collaborative. When asked how much they have progressed in their focused SWW project (1 = not at all, 2 = very little, 3 = somewhat, 4 = to a great extent), the percentage of respondents reporting “to a great extent” increased from 6 to 37% by the final session. Meanwhile, those who reported “somewhat” decreased from 71 to 53%. Additionally, 8% initially indicated “very little” progress in the first session, but this figure changed to 11% by the end of the collaborative.

Table 2.

Learning collaborative session survey results

Learning Collaborative (LC)	> Moderate confidence in applying the session content	Extent to which progress in the focused Scaling What Works project was achieved			Mostly or completely satisfied with the session
		To a great extent	Somewhat	Very Little
LC#1 Adaptation (n = 22)	18 (81.8%)	1 (5.9%)*	12 (70.6%)*	3 (17.6%)*	21 (95.5%)
LC#2 Implementation (n = 20)	Not tested	6 (30%)	10 (50%)	3 (15%)	19 (95%)
LC#3 Evaluation (n = 22)	21 (95.5%)	5 (22.7%)	13 (59.1%)	4 (18.2%)	20 (90.9%)
LC#4 Sustaining (n = 19)	16 (84.2%)	7 (36.8%)	10 (52.6%)	2 (10.5%)	19 (100%)

*n = 17

LC Learning Collaborative

Generally, positive feedback regarding building a supportive community of scientists, mentors, and NCCCP participants through learning collaborative sessions was reported. One participant indicated that they “appreciated the focused discussion on practical strategies… The insights shared by fellow participants were valuable, and I found the collaborative atmosphere conducive to generating actionable ideas.” Another said that the learning collaborative helped them in “looking at our processes… [to] strengthen our working relationship with other partners who we have not connected with.” Ongoing needs related to data collection and measurement for program evaluation, funding, and finding and maintaining adequate staff and non-traditional partnerships were identified.

Discussion

The SWW program is one of the first to demonstrate the impact of using tailored training and TA to support the scale-up of EBIs across diverse NCCCP awardees with jurisdiction-wide reach [17]. The SWW program bridges the critical gaps in scale-up research by engaging a multidisciplinary team to support national public health networks in order to move EBIs from local pilots to regional or national levels [17]. By integrating insights from a needs assessment with the expertise of the CPCRN and pilot site mentors, the SWW program delivered tailored TA to address key barriers, including resource limitations, knowledge gaps, and funding constraints. Participating awardees reported high satisfaction with the SWW program, with significant improvements in their confidence in adapting, implementing, evaluating, and sustaining their selected projects.

An important finding of SWW is the widespread need for tailored capacity-building support among NCCCP awardees. The needs assessment highlighted common challenges to scaling EBIs, such as limited resources, staffing constraints, and opportunities to improve knowledge related to adaptation, implementation, and sustainability. However, these challenges appear in unique ways for each awardee, underscoring the need for modifying capacity-building approaches/strategies to address the specific contexts and needs of individual awardees [18]. The needs assessment findings align with prior research emphasizing the complexities of applying EBIs in varied, real-world public health settings, where practitioners face competing demands and limited resources [19]. The SWW program effectively addressed these challenges through a structured and interactive training and TA approach, combining an in-person symposium (with didactic and practical components) with virtual learning collaborative sessions. These platforms provided foundational training and ongoing support, equipping participants to overcome implementation obstacles and adapt interventions to their unique contexts. The SWW mentors and facilitators worked to cultivate a safe space, free of judgement, for attendees to feel empowered to both contribute their ideas and be receptive to new ideas from colleagues, enhancing their relationships and increasing their collaboration with other NCCCP awardees. The SWW program facilitated the sharing of resources, including toolkits and best practices, saving valuable time amidst staffing constraints.

NCCCP engagement with the SWW program was notable, with more than half of the 66 sites (61%) participating in at least one aspect of the project (i.e., information sessions, symposium, learning collaboratives). Facilitators for engagement included funds provided for travel and project implementation and the signed collaboration agreement. Following the second learning collaborative, participants were reminded of their signed collaboration agreement and the requirement to participate in SWW activities to receive funding. This led to increased attendance, aligning with findings from a prior evaluation of learning collaboratives [9].

Strengths of the SWW model included its emphasis on co-design, peer mentorship, and actionable learning. By incorporating input from NCCCP awardees and pilot project mentors, the training and TA content was both relevant and practical. The use of the PPHEIA curriculum further facilitated the development of skills and confidence in participants. Additionally, the program fostered resource sharing and collaborative problem-solving through valuable networking opportunities, which were particularly beneficial given the geographic dispersion of participants doing similar work. Future iterations of SWW could focus on providing continued training tailored to specific EBIs, tailoring TA to address varied programmatic contexts, partnering with additional mentors from a larger range of sites with different programmatic contexts and levels of resources; future aims can also include enhancing sustainability and facilitating a bidirectional relationship of continued collaboration across NCCCP awardees in conjunction with linkages between NCCCP awardees and experts in SWW project content areas within and beyond the CPCRN.

This report is not without limitations. First, the extent to which survey respondents (49% of SWW participants represent the experiences of survey non-respondents (51% of SWW participants) is limited. Future iterations of SWW could identify additional methods to reach more participants. Further, results may have limited application to the NCCCP awardees which did not participate in SWW. For example, this program only reached one of the seven NCCCP awardees serving American Indian populations. Future efforts could aim to increase engagement from additional awardees to ensure a broader representation of the NCCCP.

Conclusion

The SWW program demonstrates the importance of investing in capacity-building infrastructure, including tailored training, TA, and mentorship. Our findings can help inform policymakers and funding agencies about the potential benefits of such capacity-building investments to support the long-term success and sustainability of EBIs in cancer prevention and control. Overall, the SWW program models the potential of a training and TA model to address persistent challenges in scaling EBIs within a geographically diverse, nationally reaching network. While the SWW program involves collaborations between government agencies and national networks (e.g., CDC, CPCRN, and NCCCP), it provides a generalizable model for integrating diverse expertise into training and TA to support scale-up. The structure of the SWW program could be replicated on a smaller scale and does not necessitate collaboration between large national networks. For example, our team is currently exploring a similar model in which we provide similar training and TA to a smaller state-wide network of free and charitable clinics to support their cancer prevention and control efforts. However, the success of the SWW program demonstrates the value of funding opportunities that support the national infrastructure for cancer prevention and control. By fostering collaboration and building capacity, the SWW program has developed a foundation that can impact future advancements in cancer prevention and control research and practice.

Author contributions

Conceptualization: Rachel Hirschey, Mary Wangen, Jingle Xu, Amy Tran, Elizabeth A. Rohan, Renée M. Ferrari, Prajakta Adsul, Rebecca J. Lee, Mary Puckett, Shixiu Ricardo, Alyssa LaMonica, Nikki Hayes, Amy Tran, Alison T Brenner, and Stephanie B. Wheeler; Methodology: Rachel Hirschey, Mary Wangen, Jingle Xu, Amy Tran, Elizabeth A. Rohan, Meghan C. O’Leary, Renée M. Ferrari, Rebecca J. Lee, Lisa P. Spees, Amy Tran, Alison T Brenner, and Stephanie B. Wheeler; Investigation: Rachel Hirschey, Mary Wangen, Jingle Xu, Amy Tran, Elizabeth A. Rohan, Renée M. Ferrari, Meghan C. O’Leary, Rebecca J. Lee, Julie Kranick, Nathaniel Woodard, Jaron King, Malesa Pereira, and Rogelio Robles-Morales; Formal analysis: Rachel Hirschey, Mary Wangen, Jingle Xu, Amy Tran, Julie Kranick, Lisa P. Spees, Jaron King, and Malesa Pereira; Writing—original draft preparation: Rachel Hirschey, Mary Wangen, Jingle Xu, Elizabeth A. Rohan, and Malesa Pereira; Writing—review and editing: Rachel Hirschey, Mary Wangen, Jingle Xu, Amy Tran, Elizabeth A. Rohan, Meghan C. O’Leary, Cam Escoffery, Renée M. Ferrari, Prajakta Adsul, Julie Kranick, Rebecca J. Lee, Lisa P. Spees, Amy Thompson, Cari Herington, Kelly Smith, Katie Jones, Mary Puckett, Shixiu Ricardo, Alyssa LaMonica, Nikki Hayes, Nathaniel Woodard, Jaron King, Malesa Pereira, Rogelio Robles-Morales, Alison T Brenner, and Stephanie B. Wheeler; Funding acquisition: Rachel Hirschey, Alison T Brenner, Stephanie B. Wheeler, Mary Wangen, and Rebecca J. Lee; and Supervision: Cam Escoffery, Prajakta Adsul, Alison T Brenner, Stephanie B. Wheeler, Amy Thompson, Kelly Smith, Katie Jones, Cari Herington, Elizabeth A. Rohan, Mary Puckett, Shixiu Ricardo, Alyssa LaMonica, and Nikki Hayes.

Funding

The Scaling What Works program reported in this manuscript is supported by the Centers for Disease Control and Prevention of U.S. Department of Health and Human Services (HHS) as part of a financial assistance award with 100 percent funded by CDC/HHS (Cooperative Agreement Number U48 DP006400). The contents are those of the author(s) and do not necessarily represent the official views of, nor an endorsement, by CDC/HHS, or the U.S. Government.

Data availability

The evaluation datasets generated during the Scaling What Works program are available to the public upon request.

Declarations

Conflict of interest

The authors declare no competing interests.

Ethical approval

The University of North Carolina at Chapel Hill Institutional Review Board (IRB) determined that the project does not qualify as human subject research under federal regulations and did not require IRB oversite (IRB # 23–2794).

Consent to participate

Not applicable.

Consent for publication

Not applicable.