Model for regional collaboration: Successful strategy to implement a pediatric early warning system in 36 pediatric oncology centers in Latin America

Abstract

Background

Pediatric early warning systems (PEWS) aid in the early identification of deterioration in hospitalized children with cancer; however, they are under‐used in resource‐limited settings. The authors use the knowledge‐to‐action framework to describe the implementation strategy for Proyecto Escala de Valoracion de Alerta Temprana (EVAT), a multicenter quality‐improvement collaborative, to scale‐up PEWS in pediatric oncology centers in Latin America.

Methods

Proyecto EVAT mentored participating centers through an adaptable implementation strategy to: (1) monitor clinical deterioration in children with cancer, (2) contextually adapt PEWS, (3) assess barriers to using PEWS, (4) pilot and implement PEWS, (5) monitor the use of PEWS, (6) evaluate outcomes, and (7) sustain PEWS. The implementation outcomes assessed included the quality of PEWS use, the time required for implementation, and global program impact.

Results

From April 2017 to October 2021, 36 diverse Proyecto EVAT hospitals from 13 countries in Latin America collectively managing more than 4100 annual new pediatric cancer diagnoses successfully implemented PEWS. The time to complete all program phases varied among centers, averaging 7 months (range, 3–13 months) from PEWS pilot to implementation completion. All centers ultimately implemented PEWS and maintained high‐quality PEWS use for up to 18 months after implementation. Across the 36 centers, more than 11,100 clinicians were trained in PEWS, and more than 41,000 pediatric hospital admissions had PEWS used in their care.

Conclusions

Evidence‐based interventions like PEWS can be successfully scaled‐up regionally basis using a systematic approach that includes a collaborative network, an adaptable implementation strategy, and regional mentorship. Lessons learned can guide future programs to promote the widespread adoption of effective interventions and reduce global disparities in childhood cancer outcomes.

Lay summary

• Pediatric early warning systems (PEWS) are clinical tools used to identify deterioration in hospitalized children with cancer; however, implementation challenges limit their use in resource‐limited settings.

• Proyecto EVAT is a multicenter quality‐improvement collaborative to implement PEWS in 36 pediatric oncology centers in Latin America.

• This is the first multicenter, multinational study reporting a successful implementation strategy (Proyecto EVAT) to regionally scale‐up PEWS.

• The lessons learned from Proyecto EVAT can inform future programs to promote the adoption of clinical interventions to globally improve childhood cancer outcomes.

Précis

Proyecto EVAT is a multicenter, multinational quality‐improvement collaborative that supported successful regional implementation and scale‐up of an evidence‐based intervention—a pediatric early warning system—at 36 pediatric oncology hospitals in Latin America. The knowledge‐to‐action framework was used to explain the project’s implementation strategy and to assess implementation outcomes, including the quality, the time required, and the global impact.

INTRODUCTION

The global burden of pediatric cancer is disproportionately shifted to low‐income and middle‐income countries (LMICs), which bear >90% of childhood cancer cases, ¹ with a dismal survival rate of 20%. ² To reduce these disparities, the World Health Organization Global Initiative for Childhood Cancer ³ and other initiatives ⁴ recently emphasized the need to improve access to and outcomes of childhood cancer treatment globally. However, hospitals in low‐resource settings frequently lack the infrastructure and staffing needed to deliver appropriate supportive care during cancer treatment, ⁵ , ⁶ , ⁷ , ⁸ resulting in high rates of preventable deaths. ⁹ , ¹⁰ In Latin America, children with cancer experience frequent clinical deterioration events (CDEs), with mortality of 30%. ¹¹ The coronavirus disease 2019 (COVID‐19) pandemic has further challenged childhood cancer care, ¹² , ¹³ , ¹⁴ disproportionately affecting hospitals in LMICs and worsening existing disparities. ¹⁵ , ¹⁶ There is an urgent need for effective, low‐cost, and feasible supportive care interventions, including strategies to promote early identification and timely management of clinical deterioration, to improve equity in global childhood cancer survival.

Pediatric early warning systems (PEWS) are nursing‐administered bedside acuity scoring tools associated with escalation algorithms ¹⁷ , ¹⁸ that facilitate the early identification of clinical deterioration in hospitalized children. Whereas data from high‐resource settings report conflicting impacts of PEWS on patient outcomes, ¹⁹ the implementation of PEWS in resource‐limited hospitals has been shown to reduce CDEs, optimize intensive care unit (ICU) use, ²⁰ improve family ²¹ and interdisciplinary communication, ²² reduce negative provider emotions, ²³ increase perceived hospital quality of care, ²⁴ and produce significant cost savings. ²⁵ PEWS have been validated to identify deterioration and facilitate patient triage in both high‐resource ²⁶ , ²⁷ and resource‐limited settings. ²⁸ , ²⁹

Despite global consensus that PEWS are needed to improve pediatric cancer care, ³⁰ , ³¹ these interventions are not widely used in resource‐limited hospitals, in part because of challenges with implementation. ³² , ³³ , ³⁴ To address this practice gap, in 2017, St Jude Children’s Research Hospital (St Jude) partnered with pediatric oncology centers in Latin America to initiate Proyecto Escala de Valoracion de Alerta Temprana (Proyecto EVAT), a multicenter collaborative to improve the outcomes for children with cancer who experience deterioration through the implementation of PEWS. ⁴ , ³⁵ Proyecto EVAT has supported PEWS implementation in pediatric oncology hospitals of various resource levels, ³⁶ with preliminary results from individual centers showing improvement in patient outcomes. ³² , ³⁷ , ³⁸ , ³⁹ , ⁴⁰

The experience of Proyecto EVAT represents a successful strategy for regional adoption and scale‐up of an evidence‐based practice (PEWS) in real‐world settings. The knowledge‐to‐action (KTA) framework ⁴¹ is a commonly used implementation science model for how to translate research knowledge into active use to improve patient outcomes, including knowledge creation and the action cycle or knowledge application. The KTA action cycle has seven phases: (1) identify the problem/knowledge to address the problem; (2) adapt knowledge to local context; (3) assess barriers to knowledge use; (4) select, tailor, and implement interventions; (5) monitor knowledge use; (6) evaluate outcomes; and (7) sustain knowledge use. In the current study, we used the KTA action cycle to describe the Proyecto EVAT implementation strategy to support the scale‐up of PEWS in Latin America and report relevant implementation outcomes.

MATERIALS AND METHODS

Ethical approval

This study was approved by the Institutional Review Board of St Jude as quality improvement (nonhuman subjects research). Additional approvals were obtained by participating centers as needed.

Proyecto EVAT

EVAT is a Spanish‐language PEWS that has been validated to predict the need for unplanned ICU transfer in hospitalized children with cancer. ²⁸ Proyecto EVAT is a quality‐improvement collaborative formed by St Jude in partnership with regional stakeholders in Latin America to improve the outcomes of children with cancer who experience deterioration. ³⁵ In 2017, 16 hospitals joined the collaborative, ¹¹ with 10–15 additional centers enrolling annually, for a total of 73 centers as of October 2021. Hospitals in Latin America that care for children with cancer are recruited to Proyecto EVAT through collaboration with the St Jude Global Alliance ⁴ or by learning about the program from others. Participating centers self‐identify as resource‐limited because of multiple challenges, including inadequate nursing and physician staff, limited equipment and physical space, and patients with low socioeconomic, educational, and nutritional indicators. ⁴² , ⁴³ , ⁴⁴ , ⁴⁵ Hospitals apply to an annual cohort, obtain institutional approval to participate, and are assigned a regional mentor training center (the EVAT Center of Excellence [CoE]). Initially, four pediatric oncology centers that implemented Proyecto EVAT before 2017 served as CoEs; subsequently, five additional centers completed implementation and became training centers (see Figure S1). Proyecto EVAT is led by St Jude in partnership with the EVAT Steering Committee (EVAT SC), composed of 27 nurse and physician PEWS experts from 10 hospitals in eight countries in Latin America.

Each center joining Proyecto EVAT assembles a local PEWS implementation leadership team, including at minimum a pediatric oncology nurse, a pediatric oncology ward physician, and an intensivist, adjusting the size according to local needs. This team is mentored through the phases of PEWS implementation by St Jude and regional experts through bimonthly group and on‐demand, 1:1 virtual meetings, with all activities occurring in Spanish.

We use the KTA action cycle (Figure 1) to describe the strategy used by Proyecto EVAT to support PEWS implementation in collaborating centers.

FIGURE 1.

The Proyecto EVAT implementation strategy. Modified KTA action cycle describing the Proyecto EVAT PEWS implementation strategy. CDE indicates clinical deterioration event; KTA, knowledge‐to‐action; PEWS, pediatric warning systems; Proyecto EVAT, the Early Warning Assessment Scale Project; SWOT, Strengths, Weaknesses, Opportunities, Threats assessment.

Identify the problem

Children with cancer who develop critical illness are at high risk of mortality; however, most resource‐limited hospitals do not track deterioration or outcomes in these patients. To address this issue, each center is guided to implement a uniform, de‐identified, prospective quality‐improvement registry of CDEs in pediatric oncology patients, defined as a hospitalized patient who requires unplanned ICU transfer, receives an ICU‐level intervention on the ward, or experiences a nonpalliative death (for a summary of clinical and implementation outcomes collected by Proyecto EVAT hospitals, see Table S1). Centers also report monthly pediatric oncology patient volume, defined as ward admissions and in‐patient days. Centers collect at least 6 months of prospective CDE data to describe the baseline frequency and outcomes of deterioration in their setting (the problem). These data are used to explain the need for PEWS to clinical staff and leadership and to describe common challenges in caring for this patient population in their setting. ¹¹

Adapt knowledge to local context

Implementation teams are educated on the PEWS protocol as follows: the PEWS score (range, 0–11) is calculated by using the PEWS scoring tool (see Figure S2) with every set of routine vital signs. The PEWS action algorithm (see Figure S3) then guides the medical team response, with yellow scores (PEWS ≥ 3) requiring increased monitoring and medical assessment and red scores (PEWS ≥ 5) requiring ICU consultation. ²⁰ To maintain effectiveness, fidelity with no changes is recommended to the validated components of the PEWS tool or how it is used in patient care. ¹⁷ Centers, however, are encouraged to adapt other elements of PEWS to their setting, including adjusting the wording of the PEWS tool to match local medical Spanish and the details of the PEWS algorithm to better fit with available resources and processes for care escalation in hospitalized children. ³⁴ Centers are taught a standard strategy to train staff on PEWS use; however, they also adapt these core materials using education techniques, such as visual platforms (e.g., videos, documentaries) and hands‐on learning (e.g., interactive digital games, patient simulations) based on local needs. Frequently, centers also adapt their internal processes, including nursing documentation and frequency of vital sign assessments, to facilitate implementation and monitoring of PEWS. ³⁴

Assess barriers to knowledge use

Upon joining Proyecto EVAT, centers complete a situational analysis of their pediatric oncology and critical care service, including a Strengths, Weaknesses, Opportunities, Threats (see Table S1, SWOT) assessment and anticipated challenges when implementing PEWS. In addition, centers are advised to evaluate their existing resources, including any gaps in human and material resources needed to implement PEWS (e.g., clinical staffing, office supplies, vital sign equipment). This evaluation is supplemented by a formal stakeholder analysis describing key stakeholders influential in PEWS implementation and their relationship (supportive or in opposition) to the project. These assessments are completed with expert mentorship on common barriers experienced during PEWS implementation, such as resource limitations and staff resistance, ³⁴ effective strategies to address common challenges, and formal Spanish‐language training in quality‐improvement methods and stakeholder engagement. Through this process, local implementation teams are guided to systematically identify and develop strategies to address local barriers before PEWS implementation.

Select, tailor, and implement interventions

The implementation teams receive a Spanish‐language, two‐part, train‐the‐trainer course, in which St Jude and regional experts from the EVAT CoEs teach PEWS implementation strategies, including engaging clinical staff and leadership, teaching them how to use PEWS and follow the algorithm, piloting, scaling unit wide (implementation), and measuring the quality of PEWS use (see Table S1). Implementation teams then leverage their situational analyses, adaptations of the PEWS program, and training to plan local PEWS implementation, including completing a formal pilot plan, anticipating and addressing potential implementation challenges, and receiving additional technical advice, as needed. This process typically includes early engagement of additional internal (e.g., hospital directors, quality‐improvement leaders) and external (e.g., foundations, experts from other centers) stakeholders to address identified local barriers to PEWS implementation and use.

The implementation leaders then train clinical staff, including all nurses and physicians caring for hospitalized children with cancer, in how to use PEWS and follow the action algorithm. They then conduct a PEWS pilot, make changes to their strategy based on feedback, and implement PEWS throughout their pediatric oncology unit(s). Local leads continue to attend regular Spanish‐language bimonthly virtual meetings to discuss implementation challenges, share success stories, and receive mentorship from other Proyecto EVAT centers and the CoEs.

Monitor knowledge use

From the start of the PEWS pilot, local leaders track the quality of PEWS use, including three types of PEWS errors (see Table S1). Correct PEWS use is defined by three types of errors: (1) omissions (documented vital signs without using PEWS), (2) errors in PEWS scoring, and (3) PEWS algorithm nonadherence, with high‐quality PEWS use defined as <15% in all three types of PEWS use errors. As a balancing measure, all red PEWS (scores ≥5), subsequent medical responses, and patient outcomes are also recorded. Summary data are sent monthly to the study coordinating center at St Jude, which aids centers to analyze, interpret, and conduct ongoing process improvement to increase the quality of PEWS use. Examples include staff retraining, presenting quality data, and group discussion of difficult cases. After the PEWS pilot, centers conduct a formal assessment and evaluation led by regional experts from the CoEs. Implementation teams then address identified challenges and implement PEWS throughout their pediatric oncology unit(s). Implementation completion is defined as sufficient high quality of PEWS use (<15% errors) for at least 2 consecutive months (see Table S1).

Evaluate outcomes

Centers evaluate the impact of PEWS in several ways, including clinical and implementation outcomes. Clinical outcomes are assessed using the prospective CDE registry, with CDE frequency and mortality described using pre/post analyses. Implementation outcomes include the time required to move through project phases, staff satisfaction, and global Proyecto EVAT impact (see Table S1). Staff satisfaction with PEWS is assessed using anonymous satisfaction surveys (see Figure S4), with feedback incorporated into ongoing improvement activities. The global impact of Proyecto EVAT is reported annually by each center, including the number of clinical staff trained in PEWS; patient admissions since PEWS implementation; presentations about PEWS at local, national, and international conferences; and any special recognitions or awards.

Sustain knowledge use

After implementing PEWS, the centers focus on PEWS sustainment. During this phase, the centers continue collaborating with Proyecto EVAT through monthly virtual sustainability meetings or become a CoE. Centers are mentored to plan for sustainability using the Project Sustainability Assessment Tool framework, ⁴⁶ including repeating a stakeholder analysis and describing the impact of PEWS implementation at their center. Planning for sustainability includes developing processes to institutionalize PEWS as part of ongoing hospital quality improvement, formalizing PEWS training for new clinical staff, and ongoing PEWS quality measurement. During this time, centers continue to send monthly data on implementation and clinical outcomes to the study coordinating center for 18 months after implementation.

Adjustment to the COVID‐19 pandemic

Before 2020, Proyecto EVAT leaders from the CoE conducted in‐person, train‐the‐trainer courses with local implementation leaders (one training at the new center and one at the CoE) and an in‐person, postpilot evaluation visit. Starting in March 2020, travel was not possible because of the COVID‐19 pandemic. To continue programmatic activities, the St Jude team revised, updated, and adapted the in‐person training program to virtual, including Spanish‐language synchronous teaching sessions, interactive workshops using web‐based virtual collaboration tools such as Mural ⁴⁷ and Kahoot, ⁴⁸ and observed PEWS use simulation, with iterative improvements based on feedback. Virtual training was supplemented by bimonthly virtual meetings and individual 1:1 mentorship.

RESULTS

Since the start of Proyecto EVAT in April 2017, 73 Latin American pediatric oncology centers joined the collaborative in one of five annual cohorts (2017–2021). As of October 2021, 36 centers from 13 countries successfully implemented PEWS, with 13 completing implementation after the start of the COVID‐19 pandemic (March 2020). Of the remaining Proyecto EVAT centers, four implemented PEWS before April 2017 (original Proyecto EVAT CoEs), 19 are in the planning phase, and 14 are in the early PEWS implementation phase.

The 36 centers with successful PEWS implementation have diverse hospital organizations, resources, and patient volumes, jointly managing over 4100 new pediatric cancer diagnoses annually (Table 1). Although all centers successfully implemented PEWS, they required variable time to move through the program phases (Figure 2). Most centers continued with their original cohort; however, six centers (17%) experienced a range of challenges, including public health (i.e., pandemics), political (government changes), and resource (national financial crisis) challenges, that required deferring participation to a future cohort, in which they ultimately successfully implemented PEWS. Centers required an average of 8 months (range, 5–14 months) to complete initial activities, including identifying the problem, adapting the PEWS tools, and assessing and addressing barriers (phase 1), 4 months (range, 2–10 months) to receive training and plan PEWS implementation (phase 2a), and 7 months (range, 3–13 months) to pilot PEWS and meet criteria for implementation completion (phase 2b). By October 2021, 24 centers completed the 18 months of postimplementation evaluation (phase 3), with the remaining centers still collecting these data.

TABLE 1.

Characteristics of Participating Centers

Characteristic	No. of centers (%)
Country, by World Bank income level
LMICs
Bolivia	1 (2.8)
El Salvador	1 (2.8)
Haiti	1 (2.8)
Nicaragua	1 (2.8)
UMICs
Argentina	2 (5.6)
Brazil	1 (2.8)
Colombia	1 (2.8)
Costa Rica	1 (2.8)
Dominican Republic	2 (5.6)
Ecuador	3 (8.3)
Mexico	17 (47.2)
Panama	2 (5.6)
Peru	3 (8.3)
Hospital type
Pediatric multidisciplinary	16 (44.4)
General, adult and pediatric	11 (30.6)
Oncology, adult and pediatric	6 (16.7)
Women's and children’s	3 (8.3)
Financing
Public	26 (72.2)
Public/private	6 (16.7)
Private	4 (11.1)
Separate PHO unit
Yes	4 (11.1)
No	32 (88.9)
Type of ICU
Pediatric	31 (86.6)
Adult	3 (8.3)
None	2 (5.6)
Ward nursing ratio: Nurse/no. of ward patients
1 nurse/3–4 patients	10 (27.8)
1 nurse/5–6 patients	13 (36.1)
1 nurse/7–8 patients	7 (19.4)
1 nurse/>8 patients/maximum 18 patients	6 (16.7)
Annual no. of new PHO diagnoses: Median [range]	85 [19–800]

Abbreviations: ICU, intensive care unit; LMICs, low‐middle income countries; PHO, pediatric hematology/oncology; UMICs, upper‐middle income countries.

FIGURE 2.

The Proyecto EVAT PEWS implementation phases. This graph describes the time required for each collaborating center to move through the PEWS implementation phases, with the x‐axis representing time and the y‐axis indicating the 36 Proyecto EVAT centers. Centers that completed PEWS implementation after March 2020 (during the COVID pandemic) are marked with the blue COVID symbol. The phases described are as follows: phase 1 (red), time from the start of prospective tracking of clinical deterioration events to completing all necessary adaptation to implement PEWS; phase 2a (orange), time from the start of PEWS training to the start of the PEWS pilot; phase 2b (yellow), pilot start to implementation completion; and phase 3 (green), implementation completion to October 2021 (maximum, 18 months of postimplementation data collection). COVID indicates coronavirus disease; KTA, knowledge‐to‐action; PEWS, pediatric warning systems; Proyecto EVAT, the Early Warning Assessment Scale Project.

Although centers had variable PEWS use error rates during the pilot, all centers, including the 13 that completed PEWS implementation during the COVID‐19 pandemic, ultimately achieved the goal of <15% in all three types of PEWS errors and maintained high‐quality PEWS use after (for PEWS errors by implementation month, see Figure 3). Some centers experienced intermittent increases in errors after implementation but addressed these quickly and improved PEWS use over time (Figure 3).

FIGURE 3.

The quality of PEWS use over time. These graphs describe the results of monthly monitoring of three types of PEWS errors used to assess the quality of PEWS use at all centers: (A) omissions (documented vital signs without using PEWS), (B) PEWS score calculation errors, and (C) PEWS algorithm nonadherence (not following the PEWS algorithm correctly for high scores). At each center, data for PEWS errors were collected from the start of the PEWS pilot through October 2021 (or until 18 months after implementation). PEWS errors were calculated two or three times each week through a review of nursing vital signs and PEWS documentation for all hospitalized patients by the local PEWS implementation leaders and were aggregated monthly. In each graph, the x‐axis is the implementation month or the month since the start of the PEWS pilot at each center, and the y‐axis is the percentage errors measured that month. Dots represent data for each of 36 Proyecto EVAT centers; blue dots indicate the months before the COVID‐19 pandemic (before March 2020), and red dots indicate the months after the start of the COVID‐19 pandemic (after March 2020). The solid red line represents the median percentage of errors across the 36 centers during each implementation month. The black dotted line represents the goal threshold (GOAL) used to define high‐quality PEWS use (<15% errors in each error type). Centers with monthly error results above this threshold (>15% errors) more than 6 months after the start of the PEWS pilot are marked with their center number (Center ID Code). COVID‐19 indicates coronavirus disease 2019; PEWS, pediatric warning systems; Proyecto EVAT, the Early Warning Assessment Scale Project.

As a balancing measure, all red PEWS (scores ≥5) and resulting clinical responses were monitored (Figure 4) at all centers. The frequency of red scores varied significantly across centers, ranging from 0 to 17 per month (median, 3.2 per month) or from 0.38 to 45.63 per 1000 inpatient days (median, 9.03 per 1000 inpatient days; Figure 4A), and correlated with the number of CDEs at each center (Figure 4B).

FIGURE 4.

The frequency of red PEWS scores (≥5). Graphs describe the total number of documented red PEWS at Proyecto EVAT centers from the start of the PEWS pilot through October 2021 (or until 18 months after implementation), with 10–30 months of data per center. Red PEWS scores, defined as scores ≥5, were documented by local PEWS implementation leadership teams through a prospective quality‐improvement registry from the start of the PEWS pilot. Monthly numbers of in‐patient hospital days and clinical deterioration events, defined as an unplanned ICU transfer, the use of ICU interventions on the wards, or nonpalliative ward death, were also documented by all centers for the same period. One center was not able to share patient‐level data because of national regulations but collected these data locally for quality improvement (resulting in n = 35 centers for some measures, as labeled in the illustration). Each dot represents data from one center with: (A) the average number of red PEWS per month (red) and the rate of red PEWS normalized to 1000 in‐patient days at each center (blue), with the black line representing the median among all centers; and (B) a comparison of the total number of red PEWS (x‐axis) and CDEs (y‐axis) at each center during the same period, with a solid red line indicating regression and the dotted red lines indicating the 95% confidence interval. CDEs indicates clinical deterioration events; ICU, intensive care unit; PEWS, pediatric early warning systems; Proyecto EVAT, the Early Warning Assessment Scale Project.

The global impact of Proyecto EVAT was described by the number of clinical staff trained in PEWS, patient admissions with PEWS used in their care, and presentations about PEWS by implementation at local, national, and international conferences (Table 2). As of October 2021, over 11,100 physicians and nurses have been trained in PEWS, and over 41,000 pediatric hospital admissions have benefited from the use of PEWS in their care. PEWS leaders at collaborating centers presented their experience at over 127 local, national, and international conferences, including two conference awards for best abstract. ⁴⁹ Other notable successes of Proyecto EVAT include integration of PEWS into the nursing educational curriculum of the Universidad de Bolivar (Ecuador) and the Universidad Austral (Argentina), ⁵⁰ integration with the medical student and pediatric resident curriculum in the Universidad Autonoma de Nuevo Leon (Mexico), ⁵¹ receipt of national Ministry of Health quality‐improvement awards recognizing the PEWS program in Mexico, Peru, and El Salvador, ⁵² , ⁵³ and a plan to include PEWS in the Ministry of Health nursing recommendations for the management of children with cancer in Peru.

TABLE 2.

Global impact of the Early Warning Assessment Scale Project

Characteristic	Total no. of centers	Range per center
No. of clinical staff (physicians and nurses) trained in PEWS	11,110	46–1463 Trained staff (median, 187)
No. patient admissions since PEWS pilot	41,901	287–3487 Patient admissions (median, 1020)
Educational presentations about PEWS outside of center	55	0–6 (median, 1)
Oral presentations at national or international conferences	45	National, 36; international, 9
Abstract/poster presentations at national or international conferences	27	National, 6; international, 21

Abbreviation: PEWS, Pediatric Early Warning System.

DISCUSSION

We describe the implementation strategy used by Proyecto EVAT, an international quality‐improvement collaborative of pediatric oncology centers, to improve patient outcomes through regional scale‐up of PEWS, representing a real‐world example of KTA translation. ⁴¹ As of October 2021, Proyecto EVAT supported PEWS implementation in 36 diverse hospitals, including 13 that completed implementation during the COVID‐19 pandemic, with all centers maintaining high‐quality PEWS use for up to 18 months after implementation. Five Proyecto EVAT centers joined the original four CoEs to mentor new centers, and multiple centers expanded PEWS beyond pediatric oncology to benefit other hospitalized patients. This regional dedication and enthusiasm facilitated Proyecto EVAT scale‐up to 73 hospitals, with 33 pediatric oncology centers joining since 2020 and plans to enroll 10–15 centers annually. This work demonstrates the feasibility of scaling up an evidence‐based practice across hospitals of various organization and resources, despite the challenges of geography, language, and the COVID‐19 pandemic.

Our experience offers several learning points that can be leveraged in future efforts to implement evidence‐based interventions in resource‐limited settings. Proyecto EVAT included multiple elements previously shown to be effective for quality‐improvement collaboratives in resource‐limited settings, including combining collaborative activities with additional interventions (such as training in quality improvement and clinical skills); proactively addressing contextual factors, such as culture and resource availability; titration of external support and mentorship intensity to local needs; and promoting multidisciplinary teamwork. ⁵⁴ , ⁵⁵ , ⁵⁶ This experience also supports findings from the Lancet Commission describing effective strategies to improve health care provider practices in resource‐limited settings by combining training, supervision, and group problem solving. ⁵⁷ In Proyecto EVAT, these strategies systematically identified barriers to PEWS implementation and proposed effective solutions to overcome them, facilitating implementation. ³⁴

Several elements of Proyecto EVAT, however, represent novel approaches to overcome implementation challenges unique to resource‐limited settings. The hospitals participating in Proyecto EVAT reflect the diversity of centers managing childhood cancer in Latin America, with a range patient volumes, hospital types (general hospitals to subspecialized oncology centers), and resources (equipment, staffing). Prior work has demonstrated that barriers to PEWS implementation are similar across these centers ³⁴ ; however, more research is needed to explore how center characteristics affect the time required for implementation of interventions like PEWS. In this study, all 36 centers ultimately successfully implemented PEWS; however, they required variable time and support during the implementation process. This variability included flexibility to pause or slow down Proyecto EVAT participation during times of political instability, major changes in hospital leadership, local health emergencies (e.g., dengue outbreak), or the COVID‐19 pandemic, with a plan to restart activities at a future time. Proyecto EVAT’s strong regional leadership through the EVAT SC and CoE provided essential contextual understanding of factors affecting the hospitals and project teams, with training conducted in the local language (Spanish) and advice tailored to each center’s unique resource limitations. Frequent communication with local leaders, repeated situational analysis, and mentorship from the St Jude and regional experts allowed for flexibility in the implementation process and ultimate success of the program across centers of different structures and resource levels. Proyecto EVAT highlights the feasibility of implementing evidence‐based interventions like PEWS in hospitals of various resources through adaptable, regionally informed implementation strategies.

In addition to the regional expertise provided by the EVAT SC and the CoEs, the program’s success was driven by motivated, engaged local leadership team members. For many, Proyecto EVAT was their first experience with a quality improvement or working in a multidisciplinary team, and the mentorship provided from St Jude and the EVAT CoEs was integral to empowering these teams to complete all programmatic activities. This success is evident in the broad impact and scale of Proyecto EVAT, including the large number of clinical staff trained and patient admissions with PEWS use in their care, dissemination of lessons learned by team members at international conferences, integration of PEWS into professional nursing and medical curriculum, and receipt of national quality‐improvement awards. We hope that this empowerment of local leaders to implement local change and share their accomplishments globally will facilitate the implementation of future improvement initiatives at participating hospitals.

An unexpected challenge faced by Proyecto EVAT was the need to rapidly adjust in‐person training to a virtual format in March 2020 because of the COVID‐19 pandemic. The pandemic significantly disrupted pediatric cancer care in the region ¹³ , ¹⁵ and affected clinical staff in multiple direct and indirect ways. ¹⁴ , ¹⁶ The adaptation of Proyecto EVAT to the pandemic was possible because of a concerted effort by the St Jude team and the EVAT SC. The virtual training plan was strengthened by integration of unique tools to support virtual engagement, flexibility of scheduling to accommodate clinical responsibilities of local site leads, and frequent communication between teams. The virtual format also offered unexpected benefits, including the ability to involve more staff than during in‐person sessions, the participation of experts from multiple CoEs offering diverse experience, an opportunity to use quality‐improvement methods to iteratively improve training materials, and decreased programmatic costs. As the ongoing pandemic continues to restrict international travel, lessons learned from our experience can benefit future programs that use virtual facilitation. Future work is needed to establish best practices for virtual engagement in implementation science ⁵⁸ and to explore the pandemic’s impact on ongoing quality‐improvement initiatives like PEWS.

Limitations

Our work has several limitations. Although participating Proyecto EVAT centers varied in resources and organization, they were all hospitals managing children with cancer in Latin America and focused on the implementation of a single intervention (PEWS). This potentially limits the generalizability of our experience to the implementation of other evidence‐based interventions in other regions and clinical settings, particularly among hospitals in low‐income countries, which were not well represented in Proyecto EVAT. Our team is currently adapting the Proyecto EVAT experience to support PEWS implementation in other regions, including English‐speaking centers in Africa and India and Portuguese‐speaking centers in Brazil. Future work should explore how to best adapt the Proyecto EVAT implementation strategy to different regions, languages, and contexts. In addition, the current work describes the implementation strategy used by Proyecto EVAT and thus reports implementation outcomes relevant to evaluate the fidelity and success of the PEWS implementation process. ⁵⁹ Future work is needed to evaluate the clinical impact of PEWS on patient outcomes, resource utilization, and hospital cost of care, and to identify factors that promote the sustainability of PEWS use and continued impact on patient outcomes at these centers over time. ⁶⁰

Conclusions

We describe the first regional experience using an adaptable implementation strategy to scale‐up an evidence‐based intervention, PEWS, across 36 diverse resource‐limited pediatric oncology centers in Latin America. This experience can guide future programs to promote widespread adoption of interventions to improve global outcomes and reduce disparities in children with cancer.

AUTHOR CONTRIBUTIONS

Asya Agulnik and Carlos Rodriguez‐Galindo developed the idea. Hilmarie Muniz‐Talavera, Angela K. Carrillo, Dora Judith Soberanis Vasquez, Carlos Acuña Aguirre, Yvania Alfonso, Shillel Y. Álvarez Arellano, Deiby Argüello Vargas, Rosario Batista, Erika E. Blasco Arriaga, Mayra Chávez Rios, María E. Cuencio Rodríguez, Ever A. Fing Soto, Wendy Gómez‐García, Rafael H. Guillén Villatoro, Maria S. Juárez Tobias, Norma A. López Facundo, Ruth A. Martínez Soria, Kenia Miller, Scheybi Miralda, Roxana Morales, Natalia Negroe Ocampo, Alejandra Osuna, Claudia Pascual Morales, Clara K. Pérez Fermin, Carlos M. Pérez Alvarado, Estuardo Pineda, Carlos Andrés Portilla, Ligia E. Rios López, Jocelyn Rivera, Arely S. Sagaón Olivares, Mélida C. Saguay Tacuri, Beatriz T. Salas Mendoza, Ivel Solano Picado, Verónica Soto Chávez, Isidoro Tejocote Romero, Daniel Tatay, Juliana Teixeira Costa, Erika Villanueva, and Marielba Villegas Pacheco collected the data. Asya Agulnik, Alejandra Gonzalez Ruiz, Adolfo Cárdenas, Marcela Garza, Tania Conde, Monika L. Metzger, Paola Friedrich, and Carlos Rodriguez‐Galindo provided supervision. Asya Agulnik, Alejandra Gonzalez Ruiz, Hilmarie Muniz‐Talavera, Angela K. Carrillo, Adolfo Cárdenas, Maria F. Puerto‐Torres, Marcela Garza, and Tania Conde conducted the data analyses. Asya Agulnik, Alejandra Gonzalez Ruiz, Hilmarie Muniz‐Talavera, Adolfo Cárdenas, and Maria F. Puerto‐Torres wrote the initial draft and prepared the figures and tables. All authors contributed to the interpretation of the findings, the editing of the article, and the approval of the final submitted version.

CONFLICTS OF INTERESTS

The authors made no disclosures.

FUNDING INFORMATION

American Lebanese‐Syrian Associated Charities; the Conquer Cancer Foundation Global Oncology Young Investigator Award to Asya Agulnik. The funders were not involved in the design or conduct of the study; the collection, management, analysis, or interpretation of the data; the preparation, review, or approval of the article; or the decision to submit the article for publication.

Supporting information

Appendix S1. Supporting Information