Documenting adaptations to an evidence-based intervention in 58 resource-variable pediatric oncology hospitals across implementation phases

Abstract

Background

Adaptation of evidence-based interventions (EBIs) often occurs when implemented in new local contexts and settings. It is unclear, however, during which phase of implementation adaptations are most frequently made and how these changes may impact the fidelity, effectiveness, and sustainability of the EBI. Pediatric Early Warning Systems (PEWS) are EBIs for early identification of deterioration in hospitalized children with cancer. This study evaluates adaptations of PEWS made among resource-variable pediatric oncology hospitals in Latin America implementing and sustaining PEWS.

Methods

We conducted a cross-sectional survey among pediatric oncology centers participating in Proyecto Escala de Valoración de Alerta Temprana (EVAT), a collaborative to implement PEWS. Adaptations to PEWS were assessed via 3 multiple choice and 1 free text question administered as part of a larger study of PEWS sustainability. Descriptive statistics quantitatively described what, when, and why adaptations were made. Qualitative analysis of free text responses applied the Framework for Reporting Adaptations and Modifications Expanded (FRAME) to describe respondent perspectives on PEWS adaptations.

Results

We analyzed 2,094 responses from 58 pediatric oncology centers across 19 countries in Latin America. Participants were predominantly female (82.5%), consisting of nurses (57.4%) and physicians (38.2%) who were PEWS implementation leaders (22.1%) or clinical staff (69.1%). Respondents described multiple PEWS adaptations across all implementation phases, with most occurring during the planning and piloting of EBIs. Adaptations included changes to PEWS content (algorithm, scoring tool, terminology, and use frequency) and context (personnel delivering or population). Respondents felt adaptations streamlined monitoring, enhanced effectiveness, improved workflow, increased comprehension, and addressed local resource limitations. Qualitative analysis indicated that most adaptations were categorized as fidelity consistent and planned; fidelity inconsistent adaptations were unplanned responses to unanticipated challenges.

Conclusion

Adaptations made to PEWS across implementation phases demonstrate how EBIs are adapted to fit dynamic, real-world clinical settings. This research advances implementation science by highlighting EBI adaptation as a potential strategy to promote widespread implementation and sustainability in hospitals of all resource levels.

Supplementary Information

The online version contains supplementary material available at 10.1186/s43058-024-00664-y.

Contributions to the literature.

• This study offers insights into the adaptation process of Pediatric Early Warning Systems (PEWS) that occurs across implementation stages and highlights the rarity of fidelity-inconsistent adaptations, primarily made during the sustainability phase.

• It highlights the effective use of Framework for Reporting Adaptations and Modifications Expanded (FRAME) to systematically document adaptations to evidence-based interventions (EBIs), aiding in the optimization of implementation delivery.

• This research provides valuable guidance for the global scale-up of EBIs, emphasizing the importance of adaptations to improve implementation success across a diverse range of hospitals of varying resource levels.

• By exploring adaptations in real-world clinical settings with varying resources, this paper contributes to key gaps in the implementation science literature, informing the understanding of how adaptations can be leveraged to promote successful and widespread implementation and sustainability.

Background

A pivotal goal of dissemination and implementation science is to advance understanding of how to enhance the implementation and uptake of evidence-based interventions (EBIs) across diverse real-world settings [1, 2]. Numerous studies underscore the need to better understand adaptations of EBIs and their impact, defined as the process of modifying intervention design or delivery to heighten alignment and effectiveness within specific settings through adoption, implementation, and long-term sustainability [3–9]. Moreover, fidelity—an essential aspect measuring the extent to which an intervention program is implemented as originally intended —is acknowledged as a critical consideration that must be balanced and understood alongside adaptation in understanding the reach, delivery, and effectiveness of EBIs [10–15]; for example, adaptations to EBIs can be considered ‘fidelity consistent’ or ‘fidelity inconsistent’ in their approach and alignment with the original core components, intended functions, or underlying theoretical premise of the EBI. Despite growing interest in implementation science in the process, nature, and outcomes of adaptations, there is limited research on a large-scale in how adaptations are made throughout the implementation process, their relationship to fidelity and their impact on the sustainability of an intervention. The majority of research related to adaptation and fidelity has been carried out in High-Income Countries (HICs), with a particular focus on the implementation of EBIs in the United States [16–20]. Since many EBIs are carried out in Low-and Middle-Income Countries, balancing adaptation and fidelity may be even more critical to ease EBI adoption and implementation given that many EBIs are developed in HICs [2, 21].

Until recently, researchers and practitioners have lacked comprehensive guidance on capturing intervention adaptations. However, the Framework for Reporting Adaptations and Modifications-Expanded (FRAME) is a framework utilized to systematically document and analyze adaptations to EBIs, led by Wiltsey Stirman and colleagues [18, 22–24]. This framework provides a structured approach to document adaptations made to EBIs throughout the implementation process [24]. However, to date, little work has examined some of the outcomes and consequences of such adaptations. Numerous adaptations to different EBIs have been well-documented, utilizing these comprehensive analytical frameworks [16–20], and guide critical elements about adaptations that should be measured and considered in relationship to the outcomes they produce.

Pediatric Early Warning Systems (PEWS) are EBIs common in high-resource in-patient settings with increasing implementation in low-resource in-patient settings. PEWS consists of a scoring tool and action algorithm that aid early identification and management of clinical deterioration in hospitalized children [25–27]. Implementation of PEWS in the care of high-risk patients, like children with cancer, results in multilevel advantages including reduced mortality [28], improved interdisciplinary communication [29], enhanced familial communication [30], provider empowerment [31], and reduced cost [32]. However, implementing PEWS within resource-variable hospitals presents unique challenges, requiring adaptation of PEWS to align with the contextual constraints of these environments [33, 34]. Various components of PEWS can be adapted while maintaining fidelity. These adaptations may include increasing its frequency of use (more often than every 8 hours); modifying language to accommodate dialect, definition, and terminology differences; providing clarifications and specifications to its algorithm and scoring tool; and making specific adjustments to the algorithm to account for the available resources in hospitals. It is unknown which components of PEWS are commonly adapted, when these adaptations are made, the reasons for adaptation in these settings, and the impact of these adaptations. This study aims to address this knowledge gap and characterize adaptations made to PEWS during different phases of implementation among Latin American pediatric oncology centers with varying levels of resources.

Methods

This mixed-methods study included Spanish- and Portuguese-speaking pediatric cancer centers involved in Proyecto Escala de Valoración de Alerta Temprana (EVAT). Proyecto EVAT is a quality improvement collaborative to support the implementation of PEWS at pediatric oncology centers in Latin America [26, 30, 35]. Proyecto EVAT centers are mentored in cohorts to progress through phases of PEWS implementation including planning, pilot, implementation, and sustainability. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) cross-section reporting guidelines and the Consolidated Criteria for Reporting Qualitative Research (COREQ) Checklist were utilized for reporting of quantitative and qualitative findings, respectively (Checklists can be found in Additional Files). This study was approved by the St. Jude Children’s Hospital Institutional Review Board (IRB).

Participants

This study is a component of a larger study known as INSPIRE of PEWS sustainability among 58 Proyecto EVAT centers [36]. All current Proyecto EVAT centers were purposefully recruited to participate in this study. Local PEWS implementation team leaders were briefed about this study and asked to create a list of eligible staff to participate. Eligible participants were all clinical staff using PEWS, including members of the PEWS implementation leadership team, nurses, and physicians at each center. Potential participants were contacted via email by the research team with a link to an anonymous electronic survey. Participants were given three to four weeks to complete the survey and received weekly reminders. While individual incentives were not provided, each participating center received a report summarizing responses specific to their institution [37].

Survey instrument and administration

Participants at included centers completed an anonymous electronic survey based on the Clinical Sustainability Assessment Tool (CSAT), which assesses sustainability capacity across seven domains [37, 38]. Additionally, the survey included six demographic questions on participants’ profession, role in the PEWS implementation, years of professional experience, gender, age, and the hospital at which they worked (See Additional File 1 in supplemental materials for demographic questions). To assess the nature and content of adaptations made to PEWS, participants responded to three multiple-choice questions and one free-text question (Table 1). The survey was administered via Qualtrics in Spanish and Portuguese; it required approximately 10–15 min to complete [39]. Surveys from all eligible participants were collected between June 15, 2021, and March 26, 2023. Each center completed the survey at different stages of their implementation process, leading to multiple time points recorded for each site. Some centers took the survey more than once during this period, depending on their progress in the implementation process. Survey data were supplemented with center characteristics which were collected from the study leads in January 2022.

Table 1.

Adaptation questions on the Clinical Sustainability Assessment Tool (CSAT) Survey

1. Has your hospital adapted PEWS in any of the following ways (choose all that apply)? a. Changes to when you use PEWS with patients b. Changes in how often you use PEWS with patients c. Changes in the patient population you use PEWS (for example, general pediatrics) d. Changes to the wording (language) of the PEWS scoring tool e. Changes to the content (details) of the PEWS scoring tool f. Changes to the PEWS algorithm g. Other, please explain h. No, we have not made any changes to PEWS

2. Which factors were the main reason(s) you adapted PEWS (choose all that apply)? a. Lack of necessary material resources (i.e. vital sign equipment) b. Lack of time or competing demands on time c. Lack of hospital leadership support for PEWS d. Lack of coordination with physician or other frontline staff e. To make PEWS more understandable and easier to use f. To better integrate with workflow g. For PEWS to be more effective h. So that it is easier to monitor and evaluate PEWS

3. When were these adaptations to PEWS made (choose all that apply)? a. During the planning phase (pre-implementation) b. During the PEWS pilot c. During the PEWS implementation (after the pilot, before successful implementation) d. During the PEWS sustainability phase

4. In a few sentences, please tell us how PEWS has been adapted or changed in your hospital. (Free Text Response)

Questions 1–3 where choose all that apply questions with multiple responses

Quantitative analyses

Quantitative analyses of the 3 multiple-choice questions described the characteristics of the PEWS adaptations. Guided by FRAME, frequencies of adaptation type (what was adapted), reasoning (why adaptations were made), and timeline (when during the process) were summarized using descriptive statistics. Chi-squared tests were used to analyze differences in whether adaptations were made based on participant demographics. Additional chi-squared tests were used to determine differences in types of adaptations reported by participant characteristics. The Holms correction was used to adjust p-values for multiple chi-squared testing due to its balance in controlling family-wise error rate and reducing Type II errors [40]. Reliability of the types of adaptations reported by respondents was assessed with Krippendorff’s alpha coefficients due to its flexibility in accounting for multiple raters and multivalued attributes [41]. Analyses were conducted using R version 4.3.1 [42].

Qualitative analyses

Qualitative analyses of free-text responses were conducted using a framework analysis guided by FRAME [24]. A codebook was developed a priori using the proposed FRAME 2023 codebook [43] and supplemented with codes developed from an iterative review of free-text responses (Fig. 1), including eight facets; 1) when and how adaptations were made; 2) if adaptations were planned/proactive or unplanned/reactive; 3) who decided the modification should be made; 4) what is modified; 5) the level of delivery at which the modification is made; 6) type and nature of modification made; 7) the relationship to fidelity of a modification made; and 8) reason or goal for modification made.). During this review, unused or unmentioned components of FRAME were removed from the codebook (see Additional File 2 for final codebook). Two authors (ACQS and AS) coded the free-text responses using the 2022 edition of MAXQDA software, achieving a kappa of 0.9 to 0.99. To ensure consistency of coding, disagreements between coders were resolved with the help of a third author (AA). Thematic content analysis focused on characteristics of adaptation, examining their connection to PEWS fidelity, potential moderating factors, and the resulting outcomes of adaptation, with comparative analysis across free-text responses. Fidelity was defined as the degree to which PEWS was used in patient care according to its original design and validation studies, maintaining its core elements and functions [26, 36, 44, 45]. We categorized adaptations as fidelity-consistent, fidelity-inconsistent, or fidelity-unknown based on these foundational elements, as outlined by the PEWS experts on our team (Fig. 2). We developed and followed a precise definition of adaptation types to determine these categories. In cases of discrepancies or uncertainties, we consulted a team of PEWS experts, including physicians and nurses with expertise in PEWS implementation, to ensure accurate categorization. Adaptations to the intervention were categorized in alignment with FRAME and included adaptations made for different levels of implementation, including for the patient, clinician, clinic/unit, or hospital. Subsequently, we analyzed relationships between adaptation characteristics, possible moderators, and mediators influencing adaptation. This included reasons and goals (objectives) behind adaptations, whether adaptations were planned or unplanned, their relationship to fidelity, and understanding how these factors relate to the outcomes and impacts [46].

Fig. 1.

Modified FRAME to assess adaptations in PEWS delivery. Illustrates the modified FRAME framework for assessing adaptations to PEWS delivery, incorporating the Framework for Reporting Adaptations and Modifications-Expanded (FRAME). The elements are divided into three sections including adaptation characteristics, adaptation process and adaptation impact. This framework is adapted from Baumann A, Cabassa LJ & Stirman SW (2017) and Stirman SW, Miller CJ, Toder K & Calloway A (2013)

Fig. 2.

Fidelity of adaptations made to PEWS. Demonstrates how adaptations made to PEWS relate to fidelity. We categorize these adaptations using the FRAME framework into three groups: Fidelity Consistent, Fidelity Unknown, and Fidelity Inconsistent modifications. Each category of fidelity is defined with various examples provided. Additionally, we consider modifying factors like planning and reasoning/goals within both Fidelity Consistent and Fidelity Inconsistent categories. FRAME: Framework for Reporting Adaptations and Modifications Expanded

Data synthesis

We employed a complementary mixed methods analysis approach to assess adaptations made to PEWS throughout implementation stages. To achieve this, we used findings from quantitative results to guide qualitative analyses ultimately using qualitative data to extend and deepen quantitative findings. Utilizing FRAME, our qualitative analysis delved deeper into adaptation characteristics, notably distinguishing between content and contextual adaptations. Furthermore, we categorized quantitative findings on why adaptations were made into their goal (objective) and the reasons why they were made, in accordance with FRAME. Where applicable, we synthesized quantitative and qualitative findings into tables. During analysis, we identified additional qualitative themes; these were added to synthesized tables.

Results

The study included 2094 responses across 58 pediatric oncology centers in 19 countries (Fig. 3). Of the 2094 responses, 1909 (91%) had free text responses. On average, 68.2% of eligible participants responded per center (range 8.8% to 100% across centers). Participant and center characteristics are provided in Table 2. Participants were predominately female (83%) and nurses (58%) or physicians (38%). The majority were clinical staff using PEWS (69%) or PEWS implementation leaders (22%). Primary areas of work included pediatric/oncology wards (82%). Centers were mostly located in upper middle-income countries (81%) and were primarily a mix of general hospitals, pediatric multidisciplinary hospitals, and Oncology (Adult and Pediatric) Hospitals (36%, 38%, and 15.5% respectively). A majority were publicly funded (67%) and teaching hospitals (81%). Centers treated an average of 119 new pediatric oncology diagnoses a year (range 5–800) and an average 1:6 nurse-to-patient ratio on the oncology wards (range 3–12). The average time since centers completed PEWS implementation was 32 months (range 1 to 96 months) at the time of data collection. Additionally, 51 centers (88%) had multiple data collection timepoints, with each center collecting data from one to three times at different phases of PEWS adoption, implementation, and sustainability. The average spacing of each data collection phase was 8 months (range 1 month to 15 months) depending on the implementation timeline of each center.

Fig. 3.

Participating Proyecto EVAT Centers. Participating centers (n = 58) map depicting 19 Proyecto EVAT collaborating pediatric oncology centers participating in the pilot of the Spanish and Portuguese-language CSAT with center characteristics. CSAT, Clinical Sustainability Assessment Tool; EVAT, Escala de Valoración de Alerta Temprana; PEWS, Pediatric Early Warning Systems

Table 2.

Participant and center characteristics

Participant Characteristics	n	%	Center Characteristics
Gender			Country Income Level (n, %)
Female	1,728	82.5	Lower Middle Income	6	10.3
Male	363	17.3	Upper Middle Income	47	81.0
Other	3	0.1	High Income	3	5.2
Profession			Hospital Type (n, %)
Nurse	1,203	57.4	Pediatric Multidisciplinary	22	37.9
Physician	799	38.2	General	21	36.2
Healthcare Administration	66	3.2	Oncology (Adult and Pediatric)	9	15.5
Data Manager/Research	14	0.7	Women and Children’s	4	6.9
Other	12	0.6	Pediatric Oncology	1	1.7
Primary Area of Work			Funding Type (n, %)
Pediatric/Oncology Floor	1,706	81.5	Public	39	67.2
Intensive Care Unit	218	10.4	Private	5	8.6
Non-clinical Work	101	4.8	Mixed (Public and Private)	6	10.3
Emergency Department	36	1.7	Teaching Hospital (n, %)
Other	33	1.6	Yes	47	81.0
Role in PEWS			No	3	5.2
Clinical Staff	1,447	69.1	Timepoints per center (n, %)
Implementation Leader	462	22.1	One timepoint	7	12.1
Data Manager	56	2.7	Two timepoints	45	77.6
Hospital Administrator	49	2.3	Three timepoints	6	10.3
Other	80	3.8
Length of Employment at Hospital			Observations per timepoint (mean, range)	140	12 – 245
Less than 1 year	91	4.3	Annual New Diagnoses (mean, range)	119	5 – 800
1–5 years	764	36.5
6–10 years	551	26.3	Nurse-to-Patient Ratio (mean, range)	1:6	1:3 – 1:12
11–15 years	299	14.3
16–20 years	177	8.5	Months since PEWS Implementation
More than 20 years	212	10.1	Completion (mean, range)	32	1 – 96
Total	2,094	100	Total (n, %)	58	100

Nature of adaptations

Of respondents, 20.2% (n = 422) reported no adaptations to PEWS (Table 3). The most commonly reported adaptation was changes in frequency of PEWS use (n = 1207, 57.6%). Approximately 20% of respondents reported adaptations to various components of the PEWS scoring tool, including its content (n = 483, 23.1%), algorithm (n = 454, 21.7%), and language (n = 399, 19.1%). Other reported adaptations included changes to the patient population in which PEWS is used (n = 496, 23.7%).

Table 3.

Characteristics of adaptations made to PEWS

How PEWS was adapted	n	%	Sample Quote
Content Adaptations
No changes made	422	20.2	“It has remained unchanged, as it has worked excellently for the benefit of our children” (Clinical staff, Mexico)
			“No modifications have been made in the last 6 months” (PEWS leader, Argentina)
Frequency of PEWS use	1207	57.6	“Before the pandemic, PEWS was done with each vital sign assessment, since the pandemic it is twice a day” (Clinical staff, Peru)
			“Reduced frequency due to limited staffing and consequently reduced time” (Clinical staff, Dominican Republic)
Content of the PEWS Scoring Tool	483	23.1	“Changes in oxygen usage with respect to the altitude (in relation to geographical location and sea level)” (PEWS leader, Ecuador)
			“Blood pressure was removed, as we were given higher scores for the differentials” (Clinical staff, Argentina)
Algorithm	454	21.7	“A new color has been added to the response algorithm for patients with terminal conditions to limit the activation of PEWS in these cases” (PEWS leader, Costa Rica)
			“The flow of the algorithm was adapted to the institution’s resources.” (Hospital Administrator, Mexico)
Language	399	19.1	“Some terms were reconstructed, to be more understandable for all personnel, since it is multidisciplinary.” (Clinical staff, Mexico)
Other	54	2.6	n.a
Contextual Adaptations
Patient Population in which PEWS is used in	496	23.7	“We now apply it [PEWS] to all hospitalized pediatric patients” (Clinical staff, Mexico)
			“It was extended to other departments- emergency, surgery, and internal medicine” (Clinical staff, Mexico)
Personnel implementing PEWS*	n.a	n.a	“We have adapted the frequency with which it is used in, the areas where it is needed, and the personnel who carry it out” (Clinical staff, Mexico)

Note: This was a choose all that apply questions with multiple responses. The percentages will not add up to 100%

^*Theme identified from qualitative analysis only

Qualitative analysis revealed a similar nature of PEWS adaptation, highlighting both content adaptations made to the procedures, materials, or delivery of PEWS, and contextual adaptations, meaning changes made to the ways PEWS treatment was delivered to better align with the existing setting. Content adaptations included adjustments in frequency of use, altering PEWS scoring tool, algorithm, and language. Multiple centers made content adaptations to the PEWS algorithm: “We continue with the same tool, it was just adapted the algorithm that we already had defined as a rapid response within the institution, where the emergency department is responsible for assessing the needs of hospitalized patients” (Hospital Administrator, Mexico). Content adaptations to the PEWS scoring tool included adjustments in oxygen levels based on geographical location (i.e. high altitude/mountainous locations) and specific changes intended to enhance patient outcomes: “In some patients with chronic illness situations, the score on the scale was lowered to 1. For example, this occurred in cases where patients were left with neurological sequelae following a tumor of the central nervous system” (PEWS Leader, Argentina). Participants described frequent language adaptations using local medical terminology to help staff understand the PEWS scoring tool and algorithm: “Some terminologies and parameters have been slightly adapted to make the scale easier to interpret for the staff who will use it (PEWS Leader, Guatemala).

Contextual adaptations encompassed shifts in patient populations for whom PEWS was applied, extending beyond pediatric oncology wards to emergency department, surgery, and general pediatrics: “It is used in all pediatric patients admitted to the general pediatric ward, regardless of the pathology” (Clinical Staff, Argentina). Additionally, participants noted different personnel utilizing PEWS: “There have been no changes to the scoring tool itself; the changes have been in the personnel administering the scoring tool” (Clinical Staff, Mexico) (Additional quotes supporting qualitative themes can be found in Additional File 3).

Timing of adaptations

Adaptations were reported across all program phases (see Table 4), with the highest frequency observed prior to implementation during the planning (n = 659, 31.5%) and pilot (n = 908, 43.4%) phases. Qualitative results describe different types of adjustments at different stages: “For now, we are in the pilot phase, and previously, the scales were adapted for better understandable language, as well as the nursing sheet primarily” (Hospital Administration, Mexico). Changes made during the implementation and sustainability phase were typically minimal, with many respondents reporting no changes during these phases.

Table 4.

Timing of Adaptations made to PEWS across implementation phases

Phase	n	%	Sample Quote
Planning	659	31.5	"At the beginning, before the implementation, there were changes in the algorithm and the PEWS scoring tool to leave them more clear and easier to navigate" (PEWS Leader, Colombia)
Pilot	908	43.4	“In our hospital, PEWS is in the pilot phase, the nursing flow sheet has been modified and PEWS has practically been accepted without modifications due to being validated, the only changes that were made were the visible characteristics of the skin (pale, rosy-cheeked, marbled)” (Clinical Staff, Mexico)
Implementation	653	31.2	“After the pilot, small changes were carried out for the tool and algorithm, keeping in mind the opinions of the operational staff” (PEWS leader, Mexico)
			“During the last 6 months, we have not made any changes as we’re already in a successful implementation phase” (PEWS leader, Colombia)
Sustainability	575	27.5	“We did not make any changes to PEWS after the implementation” (Clinical Staff, Colombia)
			“It has been adapted in an excellent manner with a successful level of sustainability and with favorable results for the patients” (Clinical staff, Panama)
Missing	7	0.3	n.a

Note: This was a choose all that apply questions with multiple responses. The percentages will not add up to 100%

Differences in reported adaptations across participants and hospitals

When comparing differences in whether adaptations were made to PEWS by respondent demographics, likelihood of reporting any adaptations were not different by profession, area of work, or role in PEWS implementation (Table 5). However, there were significant differences by time working at hospital (X²(5) = 37.73, p < 0.001). Staff working at the hospital for less than 1 year were least likely to report that adaptations were made (42%) and those working at the hospital for 11–15 years were most likely to report adaptations (86%) (p < 0.001).

Table 5.

Reporting of Adaptations by Respondent Demographics

	Adaptations Made (%)
Participant Demographics	Yes	No	X2	df	p-value
Profession			4.41	4	0.35
Nurse	81.1	18.9
Physician	78.6	21.4
Healthcare Administration	77.3	22.7
Data Manager/Research	64.3	35.7
Other	75	25
Primary Area of Work			4.12	4	0.39
Pediatric/Oncology Floor	80.4	19.6
Intensive Care Unit	78.9	21.1
Non-clinical Work	75.2	24.8
Emergency Department	83.3	16.7
Other	69.7	30.3
Role to PEWS			6.90	4	0.14
PEWS Implementation  Leader	81.8	18.2
Clinical Staff	78.5	21.5
Hospital Administrator	81.6	18.4
Data Manager	87.5	12.5
Other	86.3	13.8
Length of Work at Hospital			37.73	5	< .001
Less than 1 year *	58.2	41.8
1–5 years	79.8	20.2
6–10 years	77.9	22.1
11–15 years *	85.9	14.1
16–20 years	84.7	15.3
More than 20 years	81.6	18.4

* Indicates significance

When comparing differences in types of adaptations by respondent characteristics, there were significant differences by profession and role in PEWS implementation (Additional File 4). There were significant differences by profession in reporting changes to the wording of the PEWS scoring tool and reporting other adaptations made. Nurses were less likely to report changes in wording of the PEWS scoring tool and physicians were more likely to report these changes (X²(2) = 16.87, p = 0.002). Staff from other professions were most likely to report other adaptations made (X²(2) = 14.55, p = 0.004). Similarly, there were significant differences by PEWS implementation role in reporting changes to various components of the PEWS scoring tool, including wording, content, algorithm, and other adaptations made. Clinical staff were less likely to report changes in wording of the PEWS scoring tool and implementation leaders were more likely to report these changes (X²(2) = 58.9, p < 0.001). Clinical staff were less likely to report changes in content of the PEWS scoring tool and implementation leaders were more likely to report these changes (X²(2) = 27.0, p < 0.001). Clinical staff were less likely to report changes to the PEWS algorithm and implementation leaders were more likely to report these changes (X²(2) = 38.01, p < 0.001). Staff with other roles in PEWS implementation were most likely to report other adaptations made (X²(2) = 36.35, p < 0.001).

Reliability of adaptation reporting within centers and timepoints was low, with Krippendorff’s alpha ranging from -0.102 < α < 0.483 (average α = 0.119), showing little to no agreement between respondents within the same center and timepoint (Values greater than 0.8 indicate high reliability) [47]. When broken down by respondent characteristics within individual centers and timepoints, reliability remained low across all groups for profession (nurses average α = 0.132, physicians average α = 0.145) and role in PEWS implementation (implementation leader average α = 0.2, clinical staff average α = 0.11). Data from selected centers were used to demonstrate examples of these differences across respondents by profession and role (Additional File 5). Similarly, qualitative analysis identified differences in perspectives across provider roles.

Why adaptations were made

Common goals or objectives for adaptations were identified, including those aimed at facilitating PEWS monitoring and evaluation (n = 919, 43.9%), enhancing effectiveness (n = 879, 42%), integrating workflow more seamlessly (n = 830, 39.6%), and improving the comprehensibility and ease of use of PEWS (n = 702, 33.5%) (refer to Table 1). The least frequently cited reason for adaptations was the absence of support from hospital leadership (n = 92, 4.4%). Despite the majority of participating centers facing various resource constraints, issues such as lack of human resources (including time) and material resources were less commonly reported as reasons influencing adaptations (16.6% and 8.9%, respectively).

Table 6.

Why Adaptations were made

	n	%	Sample Quote
Goal (objective) of adaptation made
Easier to Monitor and Evaluate	919	43.9	"In reality, it was adapted to make the measurement [of PEWS] easier" (Clinical staff, Mexico)
Improve Effectiveness	879	42	"Algorithm adaptations were made to enhance effectiveness. Guidelines were added on when to assign points and when not to" (Clinical staff, Brazil)
Better Workflow Integration	830	39.6	"The frequency of vital signs monitoring was adjusted to the routine schedules" (PEWS leader, El Salvador)
Better Comprehension and Easier use	702	33.5	"We have changed technical words to improve comprehension for the nursing personnel" (PEWS leader, Dominican Republic)
Address Cultural Factors*	n.a	n.a	“Adaptation to the language to be more local” (PEWS Leader, Costa Rica)
Reason why adaptation was made
Lack of Team Coordination & Time	348	16.6	"We reduced the frequency due to the lack of personnel and time to carry it out" (Data Manager, Colombia)
Lack of Necessary Material Resources	186	8.9	“The continued monitoring has been something complicated due to the lack of electronic monitors, since we have to check other patients” (Clinical Staff, Mexico)
Lack of Hospital Leadership Support	92	4.4	"We are missing leadership in the oncology nursing area and support by our oncology nurse supervisor" (PEWS Leader, Chile)
Other	151	7.2	n.a
COVID*	n.a	n.a	“According to the availability of personnel, the use of beds during the Covid period, and the resources” (PEWS Leader, Colombia)

Note: This was a choose all that apply questions with multiple responses. The percentages will not add up to 100%

^*Theme identified from qualitative analysis only

Qualitative analysis provided additional explanation that generally supported quantitative findings. Adaptations aimed to integrate PEWS into clinical workflows primarily focused on adjusting clinical routines to allow sufficient time for PEWS use and accommodate for personnel in training: “According to the specific characteristics of hospital work teams, PEWS was adapted to the large influx of health personnel in training” (`PEWS leader, Mexico). Responses highlighted the need to address cultural factors, specifically language, to increase staff comprehension. Frequently, respondents mentioned changes made to PEWS to accommodate for variations in the Spanish language dialects, considering significant linguistic diversity across regions: “It was adapted to ‘Costa Rican’ Spanish for a better understanding of the terms” (Clinical staff, Costa Rica).

Contrary to the quantitative results, where resource limitations were not as prominent, qualitative findings shed light on the challenges posed by limited resources. The most frequent resource limitation described was insufficient human resources, including limited personnel required for PEWS use: “Our issue is the lack of staff; we have very few nurses for so many patients; therefore, PEWS is not fulfilled completely. More nurses are needed” (Clinical Staff, Honduras); or limited access to pediatric specialists: “The algorithm was adapted regarding when to call or involve the critical care physician, considering that in the ward, the patient is evaluated most of the time by a floor intensivist” (PEWS Leader, Paraguay). Respondents also commonly reported insufficient material resources as a reason for PEWS adaptation: “It [PEWS] was adapted to work with minimal vital sign measurement materials, as they are frequently lost when borrowed by internal medicine interns or residents, who often fail to return or damage them” (Clinical staff, Dominican Republic).

In contrast to quantitative results, free text responses mentioned leadership frequently, suggesting difficulty with implementation due to lack of leadership: “Currently, the lack of a leader in the Nursing body results in the algorithm of PEWS and processes being affected” (Clinical Staff, Chile). Finally, COVID was frequently mentioned in free-text responses as a reason for PEWS adaptation. Adaptations in response to the COVID pandemic involved rapid adjustments to a range of human and material resources limitations: “During the pandemic, there were changes in the frequency of applying the scale due to the lack of personnel and changes in work schedules. Vital signs monitoring, which used to occur every 8 h, was adapted to every 12 h, with additional adjustments based on the patient’s progression.” (Clinical Staff, Argentina) (Additional quotes supporting qualitative themes can be found in Additional File 6).

Adaptation level of delivery

In qualitative analysis participants identified adaptations made for the patient, clinician, clinic/unit, and hospital (Table 1). These findings, not covered in the quantitative questions, are solely supported by qualitative evidence. Changes made for patients included adjustments based on factors like the patients’ age or their baseline health status: “The frequency depends on the patient’s condition and at what stage of recovery or medical alterations it is needed” (Clinical staff, Colombia). Adaptations for the clinician and the clinic/unit level involved modifying the frequency of tasks based on staff availability and workflow: “Assessments and vital signs monitoring were adapted according to the routine frequency of these tasks carried out by nursing (PEWS Leader, Colombia). On the hospital level, adaptations were frequently related to resource limitations: “Some parameters were modified, adapting them according to the hospital’s reality” (Data Manager, Peru). Additionally, participants mentioned language adaptations made to align with the hospital’s terminology, ensuring clarity for all staff members: “We adapted the vocabulary to better align with what is used in the hospital” (PEWS Leader, Argentina).

Table 7.

Level of Delivery of Adaptation made (for whom was the adaptation made?)

Domain	Sub-domain	Example Quote
Clinician	Availability and Resource Limitations	“Adaptations were made according to the availability of medical personnel to conduct the assessments” (Clinical Staff, Peru)
		“PEWS is adapted to provide follow-up according to the user's needs and in accordance with the resources available in the healthcare facility in order to achieve better results” (Leader, El Salvador)
Patient	Patient Age	“Now with PEWS, we assign scores based on the child's age. And that makes a difference” (Clinical Staff, Peru)
		“The change was aimed at the age ranges for the children” (Clinical Staff, Peru)
	Patient Baseline	“As it has been used, it was adapted according to the baseline health states of the patients” (Clinical Staff, Mexico)
		“In Guatemala, the assessment of heart rate is adapted because our population generally exhibits bradycardia. It is assumed to be due to the population's inherent characteristics, so it is adapted to symptomatic and asymptomatic bradycardia. Additionally, the evaluation of oxygen saturation related to the use of oxygen is performed with post-operative patients” (Nurse or Nursing Assistant/Technician, Guatemala)
Clinic/ Unit Level	Changes in Frequency	“The frequency of taking vital signs was changed because it is easier for the nurses” (Clinical Staff, Colombia)
	Staff Comprehension	“It has been adapted to a more understandable language for all personnel” (PEWS Leader, Honduras)
		“Modifying some terms to make them understandable to all personnel” (Clinical Staff, Mexico)
Hospital	Healthcare institution actuality	“Minor changes due to being a regional hospital; we do not have a PICU (Pediatric Intensive Care Unit)” (PEWS Leader, Honduras)
	Hospital Terminology	“We have adapted PEWS to our hospital terminology” (PEWS Leader, Costa Rica)

Adaptation relationship to fidelity

Qualitative analysis revealed that most adaptations were fidelity consistent, aligning with the original intent of PEWS and its application in patient care (Table 6). Fidelity-consistent adaptations were typically planned and made early in the implementation process (planning or pilot phase). These adaptations involved language changes, increasing the frequency of PEWS implementation, integrating PEWS into medical systems and records, and introducing constructive changes to the PEWS algorithm. For example, participants described fidelity-consistent adaptations made during the pilot phase: “So far in the pilot and training, some language details in the scoring tool and algorithm were clarified” (Clinical Staff, Mexico).

Table 8.

Fidelity of adaptation made

Domain	Sub-domain	Example Quote
Fidelity Consistent Adaptation	Increase in frequency of PEWS	“Turns are taken every shift in green, every 4 h in yellow, and every hour in red.” (Clinical Staff, Ecuador)
		"Changes in the PEWS frequency (green—once per shift, maximum every 8 h, yellow—every 3 h, red—every 1 h)” (PEWS Leader, Spain)
	Integration into Medical System/ Records	"It was integrated into the computer system so that it reminds the staff of its completion." (Clinical Staff, Colombia)
		"PEWS is being implemented in the electronic medical record since the Pediatric Hemato-Oncology ward no longer works with paper." (PEWS Leader, Spain)
	Constructive Changes to PEWS Algorithm	“Also, certain information about the names of the rooms to which the deteriorating patient is transferred and the necessary data to ensure staff comprehension and facilitate workflow has been added to the algorithm” (PEWS Leader, Mexico)
		“The algorithm has been updated to involve on-call doctors and registered nurses who act as supervisors during the evenings and nights, addressing issues and responding to yellow and red calls in the ward where PEWS is implemented” (PEWS Leader, Honduras)
Fidelity Inconsistent Adaptation	COVID pandemic	“During the pandemic, PEWS was only performed at each shift change, meaning twice a day” (Clinical staff, Argentina)
		“Evaluation of PEWS is conducted during each shift of work. Previously, there were 3 shifts in 24 h. Now, there are 2 shifts of 12 h each due to staff shortages and the pandemic” (Clinical Staff, Peru)
	Lack of Participation in PEWS use	“It was challenging to implement, but it was achieved. However, there are now doctors who do not consider the nurse's assessment when using the PEWS scale” (Clinical staff, Peru)
		“Bad because the medical staff does not participate” (Clinical staff, El Salvador)
	Changes to PEWS Scoring Tool	“The score at which we start rating a patient as yellow and red was reduced” (Clinical staff, Colombia)
		“It has been adapted as blood pressure was included, as it was an early symptom in the patient on several occasions” (Clinical staff, Colombia)

Fidelity inconsistent adaptations were less common and involved changes that altered the fundamental nature of PEWS, potentially affecting its effectiveness. These adaptations typically occurred after implementation or during the sustainability phase and were unplanned/reactive responses to unforeseen shifts in clinical capacity, such as unexpected changes in human and material resources due to the COVID pandemic. Respondents specifically mentioned that frequency of PEWS use was often reduced during the acute phase of the pandemic. Similarly, decreased frequency or lack of PEWS use was occasionally reported due to a lack of material resources: “We have had good follow-up; sometimes the nursing staff, due to a lack of resources and medical equipment, does not perform PEWS” (PEWS leader, Dominican Republic).

Adaptation outcomes

Although not explicitly asked, in free text responses, several participants described their perceptions of the impact of adaptations on outcomes. Participants often noted ease of adaptations and improved fit with their local context: “In our hospital, PEWS has seamlessly adapted to our needs and resources without any issues” (Mexico, PEWS leader), often due to team familiarity with PEWS: “It [PEWS] has adapted easily in the hospital because all team members are familiar with PEWS” (Clinical Staff, Haiti). However, some challenges in adaptation were noted, particularly in the case of staff turnover. The influx of new staff members due to turnover may result in a lack of familiarity with existing protocols and procedures. This can lead to delays or inconsistencies in implementing adaptations, ultimately hindering their smooth integration within centers: “It has adapted with some difficulty because there is a constant turnover of substitute staff, who do not receive prior training” (Clinical staff, Mexico). Additionally, some respondents noted how PEWS adaptations impacted outcomes, with several reporting adaptation of PEWS facilitated their implementation and use in the hospital: “In the hemato-oncology ward, it has been adapted effectively and is a useful tool that helps us act promptly” (Clinical Staff, Panama).

Discussion

This study describes adaptations made to PEWS across implementation phases in a large number of resource-variable Latin American pediatric oncology centers. Our study is one of the first studies to document adaptations made to an EBI implemented at a large number of sites, across multiple implementation phases, from the perspective of multiple individuals responsible for implementing and using the intervention, and primarily situated in LMICs. We found that adaptations were widespread throughout all phases, with the majority occurring before implementation, including adjustments to PEWS content and use. The goals of these adaptations aimed to improve fit with the context (i.e., pediatric oncology centers in LMICS) where it was being implemented, in particular to facilitate PEWS monitoring and evaluation, improve effectiveness, enhance workflow integration, and use, and increase comprehension of PEWS. In general, the reasons for these adaptations were driven by the need to align with local human and material resources and navigate unanticipated challenges such as the COVID pandemic. Lastly, while we did observe differences among individual responses at a single center, we did not observe significant differences among hospitals. While we suggest that more research is needed to support this finding, it does indicate that there may be a limited number of adaptations routinely made to EBIs.

In the ongoing debate of whether adaptations are in contrast to or complementary to EBI fidelity, our results demonstrated notably, most adaptations were fidelity consistent, indicating alignment with core components of PEWS. In addition, our results indicate that adaptations were made to enhance and ease adoption and implementation of PEWS which suggests that adaptations were complementary to fidelity rather than in contrast as it is sometimes characterized. Fidelity consistent adaptations were proactively made most commonly in the early implementation phases, while fidelity inconsistent adaptations were predominantly reactive and occurred due to unanticipated circumstances such as rapid changes in resources, like COVID, in the sustainability phase. Our findings regarding the relationship between adaptations and fidelity align with existing literature on the importance of anticipating and planning adaptations [6, 14, 48]. We suggest that guidance could be efficiently provided especially since our results suggest that implementers are likely to only make a limited number of adaptations. Furthermore, this study provides insight into adaptations made in the sustainability phase of intervention implementation and that suggests that these may be of more concern in attempting to maintain the effectiveness of interventions. Guidance on adaptations in response to fluctuating institutional capacity as an important sustainability strategy in resource-variable settings would be a valuable resource as research moves from a focus on adoption and implementation to sustainment [49].

Our findings emphasize the importance of measurement when identifying adaptations specifically to seek input from multiple sources regarding intervention adaptation to obtain a more inclusive and accurate assessment of how interventions are adapted in diverse real-world healthcare settings [50]. Reported adaptations varied across respondents, with newer staff less likely to report adaptations to PEWS, possibly due to lack of knowledge of changes made before their tenure at the institution. Various staff involved in PEWS, including healthcare providers, administrators, and other key personnel, also varied in the adaptations reported, helping to ensure a more comprehensive and nuanced reporting of PEWS adaptations. We occasionally observed differences in our quantitative and qualitative results indicating the value of mixed-method research. For example, respondents did not relate resource constraints to adaptations in the quantitative responses, yet it was an emergent theme in the qualitative results. Providing open-ended questions allowed respondents to explain adaptations in a manner that aligned with their own conceptualization of adaptations that otherwise would have remained hidden. Additionally, while our study benefits from a large sample size and diverse perspectives from staff with varying levels of experience, the variability in reported adaptations within facilities highlights the complexity of capturing a comprehensive view of these changes. The differences in responses, even within the same center or time point, reflect the range of individual experiences, roles, and perceptions regarding the adaptations made. This suggests that the understanding of adaptations is likely a composite of multiple viewpoints rather than a single, uniform account. This variability emphasizes the importance of sampling a broad range of participants with diverse roles to obtain a more nuanced and accurate understanding of the adaptations. Our approach values these diverse perspectives, which are essential for capturing the multifaceted nature of adaptations and enriching our understanding of the implementation process.

Our methodology adds further real-world evidence for how to leverage FRAME as a foundation to systematically analyze adaptations to diverse EBIs [23, 24, 43, 51]. Although FRAME was helpful to guide analysis, we made multiple modifications to FRAME constructs and eliminated some components to more accurately reflect PEWS adaptations described in our data. One component of FRAME which was particularly difficult to apply for adaptations made to PEWS was the concept of “Level of Delivery” due to the varying levels in healthcare in implementation of PEWS. Determining the appropriate level for these adaptations required considering scope, impact, and resources at each level, highlighting the difficulty of applying this component effectively within the data collected for this study. Yet, our results demonstrate that adaptations are made to benefit different levels of the system in which they are implemented, including patients, clinicians, and the hospital at large. We suggest researchers using FRAME similarly adjust the framework to align with their distinct intervention and study questions.

This study has several limitations. The issue of recall bias is particularly relevant when retrospectively reporting adaptations instead of prospective tracking in real-time, potentially leading to over or underreporting by respondents [52]. Participants at centers sustaining PEWS were asked to reflect on adaptations made earlier in the implementation process, and it’s possible that some changes were forgotten or not known by current staff working at these centers. Another limitation is related to the structure of adaptation multiple-choice questions, which were answered only once per participant. This design does not allow individual types of adaptation to be assigned a reason or stage when they were made. While this approach is efficient for participants, it limits our ability to capture detailed information about the motivations or stages of individual adaptations. However, this quantitative data was supplemented by qualitative analysis of free-text responses where participants addressed these relationships, allowing for a more detailed understanding of motivation behind and timing of adaptations in these settings. Furthermore, our complementary mixed methods approach, combining quantitative data with detailed qualitative analysis, enhances our research and facilitates a deeper analysis of the correlation between adaptations made to PEWS and personnel with different backgrounds. Lastly, we did not explicitly examine how adaptations varied across centers of different characteristics nor the relationship between adaptations made and their potential impact on PEWS implementation, sustainment, and ultimate effectiveness with patients. Future research should focus on prospective tracking of adaptations and exploration over time of how adaptations impact the implementation and sustainability and benefit of EBIs in resource-variable settings. Additionally, while this study focused on adaptations made to PEWS use in patient care, to promote scale-up of this effective intervention future work should examine adaptations to strategies that promote global PEWS adoption and sustainability.

Conclusion

This study describes characteristics of adaptations to an EBI, PEWS, across implementation phases within resource-variable hospitals, underscoring the importance of adaptation for effective implementation. Furthermore, it advances our understanding of adaptation dynamics in implementation science, offering valuable insights for optimizing EBIs in diverse clinical settings. Future work must systematically examine strategies to guide and facilitate adaptations to promote implementation and sustainability of EBIs across hospitals of varying resource levels on a global scale.

Abbreviations

EBIs

Evidence-Based Interventions

PEWS

Pediatric Early Warning Systems

EVAT

Escala de Valoración de Alerta Temprana

FRAME

Framework for Reporting Adaptations and Modifications Expanded

Authors’ contributions

ACQS: Substantial contributions to the conception, design, analysis, and interpretation of data; and substantial contribution toward writing. SI: Substantial contributions to the conception, design, analysis, and interpretation of data; and substantial contribution toward writing. AS: Substantial contributions to the conception, design, analysis, and interpretation of data; and substantial contribution toward writing. SM: Substantial contributions to the conception and interpretation of data; and critical review for important intellectual content. MFPT: Substantial contributions to the conception and interpretation of data; and critical review for important intellectual content. AC: Substantial contributions to the conception and interpretation of data; and critical review for important intellectual content. KP: Substantial contributions to the conception and interpretation of data; and critical review for important intellectual content. YAC: Substantial contribution to the acquisition of data, and critical review for important intellectual content. SYAA: Substantial contribution to the acquisition of data, and critical review for important intellectual content. DAV: Substantial contribution to the acquisition of data, and critical review for important intellectual content. GICB: Substantial contribution to the acquisition of data, and critical review for important intellectual content. RDC: Substantial contribution to the acquisition of data, and critical review for important intellectual content. MCDB: Substantial contribution to the acquisition of data, and critical review for important intellectual content. JET: Substantial contribution to the acquisition of data, and critical review for important intellectual content. EF: Substantial contribution to the acquisition of data, and critical review for important intellectual content. ZG: Substantial contribution to the acquisition of data, and critical review for important intellectual content. CJHG: Substantial contribution to the acquisition of data, and critical review for important intellectual content. YVJA: Substantial contribution to the acquisition of data, and critical review for important intellectual content. MSJT: Substantial contribution to the acquisition of data, and critical review for important intellectual content. LLMF: Substantial contribution to the acquisition of data, and critical review for important intellectual content. NALF: Substantial contribution to the acquisition of data, and critical review for important intellectual content. JMMT: Substantial contribution to the acquisition of data, and critical review for important intellectual content. STMM: Substantial contribution to the acquisition of data, and critical review for important intellectual content. EM: Substantial contribution to the acquisition of data, and critical review for important intellectual content. ZNC: Substantial contribution to the acquisition of data, and critical review for important intellectual content. CAPF: Substantial contribution to the acquisition of data, and critical review for important intellectual content. MS: Substantial contribution to the acquisition of data, and critical review for important intellectual content. MSM: Substantial contribution to the acquisition of data, and critical review for important intellectual content. MXSL: Substantial contribution to the acquisition of data, and critical review for important intellectual content. VSG: Substantial contribution to the acquisition of data, and critical review for important intellectual content. AV: Substantial contribution to the acquisition of data, and critical review for important intellectual content. DMVC: Substantial contribution to the acquisition of data, and critical review for important intellectual content. BJC: Substantial contributions to the conception, design, analysis, and interpretation of data; and substantial contribution toward writing. RCS: Substantial contributions to the conception and interpretation of data; and critical review for important intellectual content. DG: Substantial contributions to the conception and interpretation of data; and critical review for important intellectual content. CA: Substantial contributions to the conception and interpretation of data; and critical review for important intellectual content. DAL: Substantial contributions to the conception and interpretation of data; and critical review for important intellectual content. VM: Substantial contributions to the conception, design, analysis, and interpretation of data; and substantial contribution toward writing. AA: Substantial contributions to the conception, acquisition, design, analysis, and interpretation of data; and substantial contribution toward writing. All authors read and approved the final manuscript and agreed to be accountable for all aspects of the work.

Funding

This study is funded by the NCI (R37CA276215-01) and the Implementation Science Collaborative between St. Jude Children’s Research Hospital and Washington University in St. Louis. ACQS was funded through the St. Jude Children’s Research Hospital Pediatric Oncology Education Program via grant R25CA23944 from the National Cancer Institute. This work was supported in part through pilot funds awarded to Dr. McKay through grant #P50CA24431 awarded to Brownson and Colditz at Washington University in St. Louis. None of these funders were involved in the study’s design, conduct, data collection, analysis, interpretation, manuscript preparation, review, approval, or submission for publication.

Data availability

Alejandra Catalina Quesada-Stoner, Amela Siječić, Sayeda Islam, and Asya Agulnik had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. The de-identified datasets analyzed during the current study are available from the corresponding author on reasonable request. Correspondence to: asya.agulnik@stjude.org.

Declarations

Ethics approval and consent to participate

The parent study was approved by the St. Jude Children’s Hospital Institutional Review Board (IRB) as minimum risk. Additional approvals were obtained by participating centers as needed. As an exempt study, written participant consent was waived; verbal consent was provided at the start of each interview.

Consent for publication

Not applicable.

Competing Interests

The authors declare that they have no competing interests.