Skip to main content
PMC home page
HHS Author Manuscripts logoLink to HHS Author Manuscripts
. Author manuscript; available in PMC: 2024 Jan 1.
Published in final edited form as: J Nurs Scholarsh. 2022 Jul 5;55(1):388–400. doi: 10.1111/jnu.12798

Establishing Nursing-Sensitive Quality Indicators for Pediatric Oncology: An International Mixed Methods Delphi Study

Courtney E Sullivan 1,5, Sara W Day 2, Nataliya Ivankova 3, Adelais Markaki 1, Patricia A Patrician 1, Wendy Landier 1,4
PMCID: PMC9946155  NIHMSID: NIHMS1844853  PMID: 35790072

Abstract

Introduction:

Nursing sensitive indicators (NSIs) measure factors influencing nursing care quality and patient outcomes. Established NSIs reflect general and select specialty nursing practices. However, a core set of NSIs for international pediatric oncology nursing practice does not currently exist. Without valid and reliable quality indicators, the impact of nursing care on children and adolescents with cancer cannot be effectively measured and improved. The purpose of this study was to develop a preliminary core set of NSIs for international pediatric oncology nursing that would be important, actionable, and feasible to measure across varied resource settings and countries.

Design/Methods:

A multiphase sequential mixed methods research design, intersected with a classical Delphi method, was utilized. Through purposive snowball sampling, 122 expert pediatric oncology nurses from 43 countries participated. Round One: Panelists identified 5 potential NSIs and constructs. Open-ended responses were coded and categorized through descriptive content analysis and integrated into the next round. Round Two: Panelists selected their top 10 NSIs and constructs and ranked them by importance to patient care quality. Mean importance scores were calculated through reverse-scoring; the top 10 NSIs and constructs were integrated into the next round. Round Three: Panelists ranked the top 10 NSIs and constructs by order of importance for this particular population, then rated each NSI/Construct for actionability and feasibility of measurement by Likert-scale. Rounds Two and Three were analyzed using descriptive statistics. Mixed methods meta-inferences were derived from the integration of Rounds One and Three findings.

Results:

Eighty-five (70%) panelists from 38 countries completed all Delphi survey rounds. The preliminary core set of NSIs and constructs identified by the expert panel, and ranked in order of importance, were: safe chemotherapy administration and handling, infection prevention/control, pediatric oncology nursing orientation program, early warning score system/recognition of patient deterioration, chemotherapy/biotherapy education/course, pain assessment/management, symptom assessment/management, patient and family education, palliative/end of life care, and continuing nursing education/competency. All NSIs and constructs were rated as actionable; all but palliative/end of life care were rated as feasible to measure. Each of the 10 NSIs and constructs were nominated in Round One by at least one expert panelist from low- and middle-income and high-income countries, and at least one panelist from the Americas.

Conclusion:

Preliminary core NSIs and constructs provide insight into common attributes of international pediatric oncology nursing practice that are important, actionable, and feasible for quality measurement.

Clinical Relevance:

NSIs have the potential to drive quality improvement, guide comparison with other institutions, promote knowledge-sharing, and advance pediatric oncology nursing outcomes around the world. These NSIs and constructs may also be relevant to other pediatric and adult oncology settings.

Keywords: international health/global health, quality improvement/quality of care/quality of services, child health/pediatrics/child welfare, survey methodology/data collection, nursing practice, oncology/cancer

INTRODUCTION

As one of the leading causes of childhood death, cancer is increasingly becoming a global child health priority (World Health Organization [WHO], 2021). Each year, it is expected that 429,000 new cases of cancer arise in children and adolescents (ages 0 to 19 years), worldwide (Lam et al., 2019). In many low- and middle-income countries (LMIC), where 89% of children with cancer reside, five-year survival rates average 30%. This contrasts to more than 80% average survival rates in high-income countries (HIC) (Lam et al., 2019). To address this disparity, the World Health Organization (WHO) launched in 2018 the Global Initiative for Childhood Cancer with the goal of increasing global survival rates to 60% and reducing suffering by 2030 (WHO, 2021).

In most countries around the world, nurses comprise the largest group of health professionals (WHO, 2020), and the impact of nursing care on patient outcomes is well-established (Aiken et al., 2011; Cheung et al., 2008). Nursing-sensitive patient outcomes (NSPOs) are influenced by interventions based on nurses’ scope of practice and nursing care processes. NSPOs impact patients’ symptoms, functional status, psychological distress, safety and/or healthcare costs (Given & Sherwood, 2005). Nursing-sensitive indicators (NSIs) are defined as proxy measures of nursing care quality and performance that reflect structures, processes and outcomes directly or indirectly influenced by nursing care, but for which nursing may not be exclusively responsible (National Quality Forum [NQF], 2004). NSIs have been linked with patient, nurse and organizational outcomes in HIC, and are relevant across variable resource settings (Cheung et al., 2008).

Current NSIs reflect general and select specialty nursing care and have largely been derived from HIC. To date, an internationally standardized set of NSIs, reflecting attributes of pediatric oncology nursing care, has not been established. Without valid and reliable quality measures, the influence of nursing care on patient outcomes cannot be effectively demonstrated. To enhance childhood cancer outcomes in LMIC and HIC, quality indicators reflective of pediatric oncology nursing care are needed. The purpose of this mixed methods Delphi study was to i) identify potential NSIs for international pediatric oncology nursing practice based on an open-ended electronically administered survey of an international panel of pediatric oncology nurse experts; ii) rank-order potential NSIs for international pediatric oncology nursing practice by overall importance, as determined by an international panel of pediatric oncology nurse experts; iii) establish an important, actionable and feasible preliminary core set of NSIs for international pediatric oncology nursing practice; and iv) understand how the preliminary core set of NSIs selected for international pediatric oncology nursing practice reflect the open-ended survey responses of nurse experts from both LMIC and HIC and by World Health Organization (WHO) region (Sullivan, 2021).

METHODS

Conceptual Framework

Two complementary conceptual frameworks underpinned this study. The first one, Donabedian’s Quality of Care Framework (Donabedian, 1988), is commonly used to classify nursing-sensitive and health care quality indicators (Montalvo, 2007) across three domains: structure, process, and outcomes (S-P-O). Structural attributes of health organizations are defined as the physical and organizational components of the setting in which care is delivered. Processes of care include the services, interventions and treatments provided to patients; and outcomes are the results of those services, interventions and treatments. All three domains are interlinked, with structures of care influencing processes, which in turn affect outcomes (Montalvo, 2007).

The second guiding framework in this study was the Compassionate Collaborative Care (CCC) Model and Framework (Pfaff & Markaki, 2017). Compassionate care (care delivery with compassion and empathy) and interprofessional collaborative care (interprofessional communication and teamwork), are recognized as key enablers to achieving quality health care. The CCC Model builds upon the Donabedian framework through added dimensions of individual, interprofessional, and organizational contextual factors within the structures, processes, and outcomes of care (Pfaff & Markaki, 2017).

Design

This study used a multiphase sequential mixed methods research (MMR) design, intersected with a classical Delphi method. Delphi methodology is an iterative consensus process to achieve expert consensus on a topic where little evidence or agreement exists (Keeney et al., 2011). It consists of multiple survey rounds and is commonly used to develop quality indicators. A classical approach has an open-ended qualitative first round (Keeney et al., 2011), which facilitated the reflection of nurses’ priorities in both LMIC and HIC.

Delphi studies have been considered an advanced application of MMR, where the Delphi-approach is intersected or meaningfully combined with a multiphase sequential MMR design (Plano Clark & Ivankova, 2016). MMR combines quantitative and qualitative research methods to more comprehensively answer research questions, leveraging the strengths and offsetting the weakness of studies utilizing either quantitative or qualitative methods alone (Plano Clark & Ivankova, 2016). The MMR design logic provides a guiding framework for qualitative and quantitative study phase prioritization, sequential timing and integration of study findings, enhancing the Delphi rigor in design, conduct, and analysis (Plano Clark & Ivankova, 2016).

Sample and Setting

Expert panel members were recruited using both purposive and snowball sampling methodologies. Expert panel inclusion criteria consisted of a nurse with five or more years’ experience in pediatric oncology, formal or informal leadership experience, and the ability to read English. The minimum of five years’ experience for the attainment of expertise in a specialty field was based on available literature (Conway, 1998).

A minimum sample size of 24 to 48 expert panelists representing all six WHO regions (Africa, the Americas, South-East Asia, Europe, Eastern Mediterranean, and Western Pacific) (WHO, 2022), retained through all three Delphi rounds, was targeted. To account for non-participation in the study and attrition across Delphi rounds, a minimum of 48 to 96 eligible expert panelists were targeted for recruitment, allowing for a 50% response rate. The minimum sample size was chosen to allow for at least four to eight expert panel members representing each of the WHO regions. Among the WHO regions, a goal was set for balanced representation between HIC and LMIC based on the four World Bank Country Income Classification categories and dichotomous WHO groupings. HIC have a gross national income per capita of $12, 375 or greater; and LMIC are below this cut-point (The World Bank, 2020; WHO, 2022).

Recruitment and Study Information

Expert panel members were recruited through key contacts of the following organizations that support global pediatric oncology nursing: St. Jude Global (St. Jude Children’s Research Hospital’s international collaborative); Global Hematology-Oncology Pediatric Excellence (HOPE) (Texas Children’s Hospital initiative in Sub-Saharan Africa); the University of Alabama at Birmingham WHO Collaborating Center for International Nursing (part of a collegiate network in the Americas), the WHO Global Initiative for Childhood Cancer; the Association of Pediatric Hematology/Oncology Nurses (APHON) Global Outreach Committee; and the International Society of Pediatric Oncology (SIOP) Board of Directors (which includes continental presidents), Nursing Steering Group, and Global Health Nursing Working Group.

An introductory email was sent to key contacts known to the primary investigator inviting them to self-nominate or to nominate others who met study eligibility criteria. A link to an electronic eligibility screening survey and Delphi questionnaire was included in the email. Follow-up emails were sent to participants four times over the four weeks of open study enrollment. Participants were informed that participation was voluntary and that they could withdraw from the study at any time. Participants were provided a study information sheet and were notified that by clicking the link to the survey, they consented to participate.

Study Procedures

Surveys were managed through the secure web-based electronic survey system, REDCap (Research Electronic Data Capture) (Harris et al., 2009). Expert panel members were asked to complete each Delphi round within a two-week time period. To assure adequate representation, each Delphi round remained open until an adequate number of responses from each subgroup (i.e., LMIC/HIC; WHO region) was attained. Study reminders were sent to expert panelists who did not complete the survey at one week, and at one day, prior to the two-week time-point end. Additional weekly reminders were sent to expert panelists who did not complete the survey after the two-week time-point, until adequate expert panel representation was attained. Follow-up emails were sent from a separate study email address, following the distribution of survey invitation emails asking expert panelists to check their Spam folders for study-related and reminder emails. For expert panel members who did not complete all three Delphi survey rounds, responses from the rounds in which they participated were included in the analysis.

Those participants who completed all Delphi survey rounds were given the option to be entered into a drawing for the chance to win their choice of 1) an annual Association of Pediatric Hematology/Oncology Nurses (APHON) membership, or 2) an International Society of Pediatric Oncology (SIOP) membership. Five incentive awards were drawn. Participants were also provided the option to include their name in the acknowledgement section of a publication.

Data Collection and Analysis

Three Delphi survey rounds (sequential MMR phases) were conducted with expert panel members. Expert panelists’ qualitative responses in the Delphi Round One survey (MMR Phase 1 – Aim 1) were analyzed and used to generate quantitative items in the Round Two survey. Round Two (MMR Phase 2 – Aim 2) quantitative responses were analyzed and integrated as items in the quantitative Round Three survey. Round Three survey (MMR Phase 3 – Aim 3) responses were analyzed and used to derive study findings. Quantitative study findings were then compared to qualitative Delphi Round One responses by country income classification (CIC) group and by WHO region to derive MMR study meta-inferences (MMR Aim 4). See Figure 1 for the procedural diagram.

FIGURE 1.

FIGURE 1

Open in a new tab

Procedural diagram

Note. MMR = mixed methods research; NSIs = nursing-sensitive indicators; QUAL = qualitative; QUAN = quantitative. Blue box = data collection in each Delphi round/MMR phase. Yellow oval = data analysis performed after each round/phase and to derive study meta-inferences. Green oval = data integration between Delphi rounds/MMR phases and study findings.

Delphi Round One – Specific Aim 1 (QUAL-quan)

Round One consisted of an electronic survey requesting demographic information and potential NSIs. Expert panel members were asked to list five potential NSIs for inclusion in a core set of NSIs to measure the quality of pediatric oncology nursing practice, internationally.

Although the aim of the study was to identify specific NSIs, both NSIs and broader constructs (hereafter, referred to as NSIs and constructs) were nominated by expert panel members. Descriptive content analysis was utilized to code and categorize responses. If clarification was needed, panel members were contacted by email. Through peer-checking, two study team members independently coded and categorized NSIs and constructs, and final categories were established through iterative consensus. Descriptive statistics were calculated to quantify responses by frequency overall, by LMIC and HIC groups, and by WHO Region. Round One NSIs and constructs were integrated as items in the quantitative Delphi Round Two survey.

Delphi Round Two – Specific Aim 2 (QUAN)

To address the quantitative Research Aim 2, in the Delphi Round Two survey, NSIs and constructs nominated in Round One were displayed in a table for panelists to view. Expert panelists were asked to select and rank the 10 potential NSIs and constructs identified in Round One that they believed to be most important, and then rank them by order of importance (1=most important to 10=least important) for inclusion in a preliminary core set.

Responses were analyzed according to a reverse-order point system, with points assigned to each rank-order position. Ten points were awarded to the first rank-order position, and one point was awarded to the tenth rank-order position. Mean-rank order scores were calculated for each NSI and construct by summing their respective importance scores, and dividing by the total number of expert panel members in Round Two. The 10 NSIs and constructs with the highest mean-rank scores were integrated as items in the quantitative Delphi Round Three survey.

Delphi Round Three – Specific Aim 3 (QUAN)

To address the quantitative Research Aim 3, in the Delphi Round Three survey, expert panelists viewed the 10 highest mean-ranked NSIs and constructs from Delphi Round Two. Next, they re-ranked them by order of importance (1=most important to 10=least important) for inclusion in the preliminary core set. Responses were again scored according to the reverse-order point system used in the Delphi Round Two analysis, and descriptive statistics (of mean-rank scores) were calculated.

Next, expert panel members were asked to rate each NSI and construct for 1) “actionability” (i.e., ability to provide decision-makers with clear direction for improvement of international pediatric oncology nursing practice), and 2) feasibility (i.e., possibility of measurement within a pediatric oncology nursing practice setting). Each construct was rated for both actionability and feasibility on a 5-point Likert-scale (with 1=strongly disagree and 5=strongly agree). NSIs and constructs with a mean-score of 4 or higher for actionability or feasibility were determined as actionable or feasible, respectively. All 10 NSIs and constructs were included in the preliminary core set and were described according to each NSI or construct’s importance, actionability and feasibility.

MMR Study Meta-Inferences – Specific Aim 4 (QUAN-qual)

To address the MMR AIM 4, the final quantitative core set of NSIs and constructs from Delphi survey Round Three was compared to the coded open-ended survey responses submitted by nurses in LMIC and HIC groups, and by each WHO Region in Round One. Common NSIs and constructs were identified, and their frequencies were calculated. Study meta-inferences were drawn from integrating quantitative and qualitative responses.

RESULTS

Of the more than 283 individuals invited to participate in the study, 122 complete responses were received for Round One. Twenty-seven partial responses were received but were not usable and therefore, were excluded from the analysis. The estimated overall response rate, based on the invitations sent via the purposive sampling method, was 122/283 (43.1%). One panelist completed the Round One survey twice; the most recent response from this panelist was included in the analysis.

In Delphi Round One, 122 expert panel members participated, with 91 (74.6%) continuing to Round Two and 85 (69.7%) continuing to Round Three, resulting in an overall attrition rate of 30.3% across the three rounds. Minimum targets for participation by CIC and WHO Region were met in each Delphi Round. See Table 1 and Figure 2 for expert panel member characteristics by WHO Region, CIC, and country in Delphi Rounds One, Two, and Three. Expert panel member characteristics are displayed in Supplemental Table S1, Table S2, and Table S3.

TABLE 1.

Expert Panel Member Characteristics by WHO Region and Country Income Classification in Delphi Rounds One, Two, and Three

Round 1 Round 2 Round 3
Characteristic n (%) n (%) n (%)
WHO region
 Africa 24 (19.7) 16 (17.6) 15 (17.6)
 Region of the Americas 32 (26.2) 24 (26.4) 23 (27.1)
 South East Asia 12 (9.8) 9 (9.9) 8 (9.4)
 European 22 (18.0) 16 (17.6) 16 (18.8)
 Eastern Mediterranean 13 (10.7) 10 (10.9) 9 (10.6)
 Western Pacific 19 (15.6) 16 (17.6) 14 (16.5)
Country income classification
 LMIC 67 (54.9) 48 (52.7) 44 (51.8)
 HIC 55 (45.1) 43 (47.3) 41 (48.2)
Total 122 (100.0) 91 (100.0) 85 (100.0)
Open in a new tab

Note. WHO=World Health Organization; LMIC = low- and middle-income countries; HIC = high-income countries.

Figure 2.

Figure 2.

Open in a new tab

Expert Panel Member Characteristics by Country in Delphi Rounds One, Two, and Three

Note. 122 expert panelists from 43 countries participated in Round One, 91 panelists from 38 countries in Round Two, and 85 panelists from 38 countries in Round Three. WHO=World Health Organization

Delphi Round One Findings

Qualitative.

In the first Delphi round, 122 expert panel members from 43 countries each identified five NSIs and constructs for inclusion in a preliminary core set. Thirty-two unique NSIs and constructs were generated from coded and categorized responses. These NSIs and constructs reflected 11 broader themes: chemotherapy safety, education, infection, nursing practice environment, nursing assessment, oncologic emergencies, patient safety, patient/family satisfaction, medication safety, staffing, and vascular access device. Each of the 32 NSIs and constructs was nominated in open-ended Round One items by at least one expert panel member from LMIC. All NSIs and constructs except identifying patients correctly were nominated by at least one expert panelist from HIC. Nomination of NSIs by expert panel members from each of the six WHO Regions, varied by region. No region nominated all 32 NSIs and constructs. See Supplemental Table S4 for Round One NSIs and constructs by CIC and WHO Region.

Quantitative.

Nurse to patient ratio was most frequently nominated by expert panelists from both LMICs (n=25; 37%) and HICs (n=24; 44%). The NSIs and constructs most frequently nominated by each of the six regions included: quality of the nursing practice environment (Africa [n=12; 50%]); safe chemotherapy administration and handling (Americas [n=19; 59%); pediatric oncology specialist certification (Eastern Mediterranean [n=8; 62%]); error/incident reporting (South-East Asia [n=8; 67%); and nurse to patient ratio (European [n=12; 55%] and Western Pacific [n=8; 42%]).

Delphi Round Two Findings

In the second Delphi round, 91 expert panel members representing 38 countries participated. The top 10 NSIs and constructs by mean-importance scores were identified as: safe chemotherapy administration and handling, infection prevention/control, pediatric oncology nursing orientation program, continuing nursing education/competency, early warning score system/recognition of patient deterioration, palliative/end of life care, patient and family education, pain assessment/management, chemotherapy/biotherapy education/course, and symptom assessment/management. The NSIs and constructs ranked highest by the expert panel members from LMIC, HIC, and all six WHO Regions were safe chemotherapy administration and handling and infection prevention and control.

Delphi Round Three Findings

In the third Delphi round, 85 expert panel members from 38 countries participated. The preliminary core set of NSIs and constructs for international pediatric oncology nursing practice, by order of mean-importance, include: safe chemotherapy administration and handling, infection prevention/control, pediatric oncology nursing orientation program, early warning score system/recognition of patient deterioration, chemotherapy/biotherapy education/course, pain assessment/management, symptom assessment/management, patient and family education, palliative/end of life care, and continuing nursing education/competency. All core NSIs and constructs were rated as actionable, and all were rated as feasible to measure except palliative/end of life care. See Table 2.

TABLE 2.

Round Three: Top 10 NSIs and Constructs by Mean Importance, Feasibility and Actionability

NSIs and Constructs Importance Feasibility Actionability
Rank Order Sum Total Mean-Rank Score Mean Mode Median IQR (Q1-Q3) Mean Mode Median IQR (Q1-Q3)
Safe chemotherapy administration and handling 1 701 8.2 4.4 5 5 1 (4–5) 4.6 5 5 0 (5–5)
Infection prevention / control 2 592 7.0 4.4 5 5 1 (4–5) 4.6 5 5 1 (4–5)
Pediatric oncology nursing orientation program 3 539 6.3 4.2 5 5 1 (4–5) 4.3 5 5 1 (4–5)
Early warning score system / recognition of patient deterioration 4 526 6.2 4.4 5 5 1 (4–5) 4.5 5 5 1 (4–5)
Chemotherapy/biotherapy education / course 5 445 5.2 4.3 5 5 1 (4–5) 4.3 5 5 1 (4–5)
Pain assessment / management 6 442 5.2 4.5 4 5 1 (4–5) 4.5 5 5 1 (4–5)
Symptom assessment / management 7 433 5.1 4.3 5 5 1 (4–5) 4.5 5 5 1 (4–5)
Patient and family education 8 356 4.2 4.1 4 4 1 (4–5) 4.3 5 4 1 (4–5)
Palliative / end of life care 9 326 3.8 3.9 4 4 1 (4–5) 4.3 5 4 1 (4–5)
Continuing nursing education / competency 10 315 3.7 4.1 5 4 1 (4–5) 4.3 5 5 1 (4–5)
Open in a new tab

Note. N = 85 expert panel members. IQR = Interquartile Range; Q1 = Quartile 1 (25th percentile); Q3 = Quartile 3 (75th percentile).

Mixed Methods Integration and Meta-Inferences

When comparing the quantitatively-derived core set of NSIs and constructs (Round Three) with the qualitatively-derived NSIs and constructs (Round One), results varied by CIC and WHO Region. Overall, each of the 10 NSIs and constructs included in the preliminary core set were nominated by at least one expert panelist in Round One open-ended responses from LMIC, HIC, and from the Region of the Americas. The number of core NSIs and constructs that were nominated in Round One, by at least one expert panelist from the remaining five regions, ranged from five in South-East Asia to nine in the Western Pacific and European regions. Of the 10 core NSIs and constructs, three were identified by at least one expert from all six regions in Round One: continuing nursing education/competency, pain assessment/management, and patient and family education. See Table S5 for core NSIs and constructs by WHO Region and CIC in Round One.

DISCUSSION

Safe Chemotherapy Administration and Handling

Safe chemotherapy administration and handling ranked highest in the preliminary core set of NSIs and constructs by expert panel members in both LMIC and HIC. Chemotherapy is a main treatment modality in pediatric cancers and is primarily administered (and at times prepared) by nurses. As a hazardous drug, exposure to chemotherapy has been linked to miscarriage, infertility, and cancer in health workers. Key components of safe-handling have been described as a) well-trained personnel, b) adequate personal protective equipment, and c) evidence-based policies and procedures to guide care (National Institute for Occupational Safety and Health [NIOSH], 2016; Pan American Health Organization [PAHO], 2013).

Although, standards and guidelines exist for safe chemotherapy administration and handling (NIOSH, 2016; PAHO, 2013), standardized NSIs are lacking. Chemotherapy-related quality indicators have been developed in international adult and pediatric oncology studies that may be relevant to nursing. Such indicators reflect structures, processes, and outcomes such as: chemotherapy administration competency, closed system transfer devices; administration delays, documented start and stop times/scheduled delivery, guideline-adherent processes, personal protective equipment use; and drug dose errors (Bradley et al., 2013a; Looper et al., 2016; Teichman et al., 2017). Potential indicators nominated by expert panelists in Round One open-ended responses include: patient identification; safe medication administration; adverse event documentation; and extravasation rates (Jackson-Rose et al., 2017; Looper et al., 2016). Future research is needed to define standard NSIs for safe chemotherapy administration and handling.

Infection Prevention and Control

Infection is one of the leading causes of toxic death in children and adolescents with cancer, and toxic death is tenfold higher in LMICs than HICs (Mukkada et al., 2018). IPC quality indicators are well established in several countries and across nursing foci (Bradley et al., 2013a; Solutions for Patient Safety [SPS], 2021; Wilson et al., 2012). Examples include CLABSIs, CAUTIs ventilator associated pneumonia, pressure injuries, surgical site infections, C. Difficile and antimicrobial stewardship, and hand hygiene (NQF, 2004; SPS, 2021). To specifically address infectious considerations for childhood cancer, pediatric hematology/oncology units in the United States formed a CLABSI initiative through the Children’s Hospitals’ Solutions for Patient Safety collaborative (SPS, 2021).

Several infection-related NSIs and constructs were nominated by expert panel members in the initial round of this Delphi-study: infection prevention and control, CLABSI, hospital acquired infections (non-CLABSI), hand hygiene, time to antibiotics in febrile neutropenia (an oncologic emergency), vascular access care and education, and peripheral intravenous extravasation/infiltration/phlebitis rates. It may be that expert panel members had difficulty narrowing their selection of specific infection-related NSIs, and opted for the larger umbrella of infection prevention and control to encompass a broader range of potential NSIs. Future studies are needed to further define the construct, infection prevention and control, to inform specific NSIs for inclusion in a final core set.

Pediatric Oncology Nursing Orientation

In many HIC, subspecialty orientation programs are standard in health care delivery settings. However, in LMIC, structured subspecialty orientation programs are often limited or nonexistent; instead, nurses receive informal on-the-job training (Day et al., 2015; Morrissey et al., 2019). Unfortunately, in both HIC and LMIC nursing students often receive little to no pediatric oncology education in their undergraduate pediatric curricula.

The SIOP Global Health Nursing Working Group has published six baseline nursing standards for practice settings in LMIC, which are also relevant to HIC (Day et al., 2015). One of the six SIOP Baseline Nursing Standards recommends that all newly hired nurses participate in a structured pediatric oncology nursing orientation program prior to independently caring for children and adolescents with cancer. The standard recommends orientation programs consist of a minimum of two weeks of didactic followed by three or more weeks of precepted clinical experience, and that the curriculum is organized by learning objectives and is competency-based. Examples of specific topics for inclusion in didactic content are pediatric cancers, oncologic emergencies, infection prevention/control, chemotherapy administration and safe handling, and palliative and end of life care (Day et al., 2015). The endorsement of pediatric oncology nursing orientation for inclusion in a core set of NSIs and constructs by expert panel members in this study reinforces the significance of orientation to quality pediatric oncology nursing practice internationally. Furthermore, the SIOP baseline standard may inspire future NSI development for orientation programs.

Early Warning Systems

Early warning systems have been utilized within health care delivery settings to facilitate the early detection of patient deterioration and timely escalation of care. Early warning systems are evidence-based tools for nursing assessment and monitoring of patients’ risk for clinical deterioration. Pediatric early warning systems (PEWS) have been validated in the pediatric oncology patient population in both a HIC (United States) and a LMIC (Guatemala) (Agulnik et al., 2016; Agulnik et al., 2017). PEWS has been identified as a quality indicator and research priority for onco-critical care internationally (Arias et al., 2020; Wong et al., 2017), and as a nursing-sensitive indicator in pediatric cardiology (Connor et al., 2016). Furthermore, in a multicenter PEWS study with 16 pediatric oncology units in Latin America, nurse to patient ratio was associated with event-related mortality (Agulnik et al., 2021). Thus, PEWS may serve as a promising process indicator (mediator variable) linking nurse staffing (structure) and event-related mortality (outcome).

Chemotherapy and Biotherapy Education

As previously noted, chemotherapy/biotherapy nursing education is a key component of safe chemotherapy/biotherapy administration and handling. It has been identified as a pediatric oncology quality indicator in both HIC and LMIC (Bradley et al., 2013b; Wong et al., 2017), and is recommended as a topic for new-hire orientation and continuing education (Day et al., 2015). The APHON chemotherapy/biotherapy certificate course is a standard for pediatric oncology nurses in North America, and was recently expanded internationally with Spanish and Portuguese languages (Sullivan et al., 2019). This comprehensive course serves as an exemplar for pediatric chemotherapy/biotherapy certificate-based education.

Expert panel members’ selection of two chemotherapy-related NSIs and constructs (education and safe handling) emphasizes the importance of chemotherapy-related quality measurement. Furthermore, chemotherapy/biotherapy education was prioritized in addition to new-hire orientation and continuing education, even though it is often a topic in both. Therefore, the need for distinct pediatric chemotherapy/biotherapy nursing education is clearly warranted for pediatric oncology nurses.

Pain Assessment and Management

Pain is a commonly reported symptom by children and adolescents with cancer (Rodgers et al., 2019). Symptoms may be disease- (e.g., medullary overcrowding), treatment- (e.g., mucositis/stomatitis), or procedural-related (e.g., lumbar puncture; vascular access). Pain assessment and management is a well-established NSI in adults and pediatrics (Montalvo, 2007; Start et al., 2018). It has also been selected as a pediatric oncology quality indicator for HIC and LMIC (Teichman et al., 2017; Wong et al., 2017). Although pain is one of several common symptoms in pediatric oncology, both pain and symptom assessment/management were included in the preliminary core set by expert panel members; highlighting pain as a priority symptom for pediatric oncology nursing assessment/management internationally.

Symptom Assessment and Management

Symptom assessment/management is a growing focus of research in pediatric oncology and is a research priority of the Children’s Oncology Group (COG) Nursing Discipline (Landier et al., 2013). Children and adolescents with cancer experience multiple, interrelated co-occurring disease- and treatment-related symptoms, referred to as “symptom clusters”. Symptoms vary by intensity and levels of distress across treatment phases and may impede treatment and quality of life. Commonly experienced symptoms include fatigue, pain, nausea and vomiting, changes in appetite, sleep disturbances, anxiety and depressive symptoms (Rodgers et al., 2019).

Although screening instruments have been developed (Tomlinson et al., 2014), a gold standard symptom assessment instrument in pediatric cancer does not currently exist (Hockenberry & Landier, 2019). Pediatric clinical nursing assessment tools are well-established for select symptoms such as pain (i.e., Faces Pain Scale) and nausea (i.e., Baxter Animated Retching Faces [BARF] Scale) (Baxter et al., 2011). NSIs for symptom management have been prioritized for adult oncology settings in the United States and England (Armes et al., 2014; Fessele et al., 2014). These measures focus heavily on assessment, intervention, and post-treatment education. Examples of symptom-related measures include: fatigue, pain, nausea and vomiting, psychosocial distress, and sleep-wake disturbances. Adult oncology NSIs, pediatric nursing assessment tools, and the research being conducted by the COG Nursing Discipline and others may align with future development of clinically relevant NSIs for international pediatric oncology nursing practice.

Patient and Family Education

Patient and family education has been identified as a priority across varied resource settings and countries. In addition to illness-related distress, the COG Nursing Discipline selected patient and family education for their research agenda (Landier et al., 2013). They recommended three key areas of focus: 1) diagnosis/treatment, 2) psychosocial coping, and 3) care of the child. To address the variability in education provided to newly diagnosed patients/families, the COG Nursing Discipline developed a checklist with specific topics for teaching upon initial discharge, within the first month after diagnosis, and topics for discussion before the completion of therapy (Rodgers et al., 2019). Similarly in Peru, the national nursing committee for the WHO Global Initiative for Childhood Cancer prioritized the following education topics: diagnosis, treatment, sequential effects of chemotherapy, and home care. Adapting to the COVID-19 pandemic in 2020, the nursing committee published an education booklet for parents with pandemic-related recommendations (Liliana Vasquez, personal communication, 12, 2020).

Given the cultural, resource, language, and common cancer-type variations within and between countries internationally, patient and family education is largely context dependent. However, there are likely core topics relevant to all pediatric cancers, their treatment, and patient and family-centered care across settings.

Palliative and End of Life Care

Recognizing palliative care as an essential human right and ethical responsibility of health systems globally for children and adolescents with cancer, the WHO Global Initiative for Childhood Cancer included the reduction of suffering in its overall goals (World Health Organization [WHO], 2021). The WHO (2018) defines palliative care as the:

prevention and relief of suffering of adult and pediatric patients and their families facing the problems associated with life-threatening illness. These problems include the physical, psychological, social and spiritual suffering of patients, and psychological, social and spiritual suffering of family members (p.5).

Palliative and end of life care is an integral part of pediatric oncology nursing. Nurses deliver holistic patient-family centered care, spanning the care continuum and goals of cure, life-prolongation, or comfort. In this study, expert panel members rated palliative/end of life care as important and actionable, but lower for feasibility of measurement. This finding aligns with several studies, citing palliative and end of life care as a multifaceted and difficult concept to measure (Bradley et al., 2013a; Seow et al., 2009; Teichman et al., 2017). Palliative and end of life quality concepts have been described according to the CCC Framework as 1) patient and family-centered care, 2) communication, 3) shared-decision making and goal-setting, and 4) patient/family-health professional-organizational development and satisfaction (Pfaff & Markaki, 2017).

The majority of palliative care quality indicators have been developed for adult care (Seow et al., 2009), with only two of over 200 National Quality Forum endorsed measures specific to pediatric pain assessment/management in the PICU setting (Johnston et al., 2020; NQF, 2004). To date, standard indicators have not been developed in pediatrics or pediatric oncology, internationally. To address this gap, the conceptual exploration and development of potential pediatric palliative care indicators (Downing et al., 2018) and pediatric oncology palliative care indicators are presently underway (Johnston et al., 2020; Nagoya et al., 2020; M. McNeil, personal communication, December, 2021). These potential indicators may hold promise for international pediatric oncology nursing practice and ultimately help reduce suffering in children.

Continuing Education and Competency

Pediatric oncology is a rapidly changing field within the broader context of dynamic health care delivery systems. As with any specialty, ongoing professional development is required for nurses to keep pace with the latest evidence and practice changes relevant to pediatric oncology. Similar to orientation programs, subspecialty continuing education opportunities for nurses in LMIC are often limited compared to those in HIC (Day et al., 2015). The third SIOP Baseline Nursing Standard recommends a minimum of 10 hours of continuing education each year for pediatric oncology nurses (Day et al., 2015). In a survey of 101 hospitals in 54 countries, 49.5% met and 50.5% did not meet the continuing education standard. In LMIC, less than 50% of nurses met the standard compared to 60% in HIC. Furthermore, it was identified that nurses in the WHO Region of Africa were six times less likely than nurses in other regions to have 10 hours of continuing education each year (Morrissey et al., 2019; Sullivan et al., 2020). Nomination of this Baseline Nursing Standard as a core NSI and construct by expert panel members affirms continuing education as a priority for international pediatric oncology nursing.

MMR Meta-Inferences and Reintroduction of Unselected NSIs and Constructs

All ten NSIs and constructs included in the preliminary core set were reflective of expert panel members’ open-ended responses in the initial Delphi round across country income classifications (LMIC and HIC). Representativeness of core NSIs and constructs in the initial Delphi round varied across the six WHO regions. Although not included in the core set, nurse to patient ratio was one of the top two most frequently nominated NSIs and constructs in the initial round by experts in Africa, the Americas, Europe and Western Pacific. Quality of the nursing practice environment, pediatric oncology specialist education/certification and error/incident reporting were also frequently nominated in the initial round by expert panelists in several regions, but were not reflected in the preliminary core set. Future studies may benefit by reintroducing NSIs and constructs from the initial Delphi round for further exploration.

Limitations

There were several limitations in this study. Although the aim of this study was to identify NSIs, expert panel members identified both NSIs and broader constructs for inclusion in the preliminary core set. Additionally, in this study we did not identify specific definitions and calculations for measurement with the goals of reducing participant burden and finalizing the core set in future studies.

Although recruitment targets were met as a purposive snowball sample, the expert panel was not representative of all pediatric cancer practice settings around the world. Furthermore, the study was delivered in English, and therefore study findings may not be generalizable to non-English speaking nurses and settings. Discrepancies were found in the responses of seven expert panelists who had less than five years of experience working in pediatric oncology nursing in the demographic questionnaire in the initial Delphi round, despite having affirmed that they did have at least five years of experience during eligibility screening. Similarly, two expert panelists reported having less than five years total nursing experience in the demographic questionnaire. It was not possible to determine which responses were correct, and therefore all seven expert panel member responses were included in the study.

Another challenge was that the study was conducted in the midst of the global novel coronavirus pandemic from September 1, 2020 to December 23, 2020, a time in which hospitals internationally were responding to this emergency. Therefore, the pandemic may have had an impact on recruitment and retention throughout the study. An IRB amendment was obtained during study recruitment to allow for “Spam Screening” emails to be sent by the study PI from a non-study email, due to the possibility that study communications could inadvertently be sent to computer spam folders. It is unknown whether all intended email communications were received. Thus, the purposive survey response rate (43.1%) may be underestimated.

Significance and Future Research

This is the first study to identify priority areas for quality measurement in pediatric oncology nursing practice across World Bank Country Income Classifications, and across each of the six WHO regions. Future studies are planned to finalize a core set of NSIs with nurse- and additional stakeholder-input. The final core set of NSIs has the potential to drive quality improvement, guide comparison with other institutions, promote knowledge-sharing, and advance pediatric oncology nursing outcomes around the world. Core NSIs may also have health policy implications and be adaptable for Ministry of Health reporting. Finally, these NSIs contribute to the growing science of multidisciplinary quality measurement in pediatric oncology, and may be relevant to other pediatric and adult oncology settings.

CONCLUSION

The preliminary core set of NSIs and constructs identified in this study reflect topics that expert nurses from varied resource settings and countries identified as relevant, actionable and feasible to measure. These findings provide insight into common attributes of pediatric oncology nursing practice internationally that are relevant for quality measurement and comparison. Overall, the expert panel identified 10 NSIs and constructs for inclusion in the preliminary core set of NSIs. Findings from this study will inform future studies to finalize a core set of NSIs for international pediatric oncology nursing practice.

Supplementary Material

Table S1

TABLE S1: Expert Panel Member Characteristics by Country, WHO Region, and CIC in Delphi Rounds One, Two, and Three

Table S2

TABLE S2: Delphi Round One Expert Panel Member Characteristics

Table S3

TABLE S3: Delphi Round One Characteristics of Hospitals in which the Expert Panelists Practiced

Table S5

TABLE S4: Round One NSIs and Constructs by Frequency & Proportion: WHO Regions and CIC

Table S4

TABLE S5: Representation of Core NSIs and Constructs in Round One Responses by WHO Region and Country Income Classification

CLINICAL RESOURCES.

APHON Chemotherapy and Biotherapy Program: https://aphon.org/ped-chemo-bio/pediatric-chemotherapy-biotherapy-provider-program

SIOP Baseline Nursing Standards: https://siop-online.org/baseline-nursing-standards

WHO Global Initiative for Childhood Cancer: https://www.who.int/publications/m/item/global-initiative-for-childhood-cancer

Acknowledgements:

The primary investigator of this study was funded by: 1) the Oncology Nursing Foundation Doctoral Scholarship, and 2) the American Cancer Society Doctoral Degree Scholarship in Cancer Nursing. This study would not have been possible without the contribution of the following nurse expert panelists: Abdullah Odat; Aeltsje Brinksma; Aisha Nedege; Allenidekania; Ana Loseo; Ananda Fernandes; Andhrias Maria Pushpam; Andries Gontshwanetse; Anna Negre Loscertales; Anum Salman Mistru; Bharti Veer; Chosha Mulungu; Christina Baggott; Colleen Nixon; Curtis Wong; Débora Rebollo de Campos; Dimah Albakour; Dorian René Navarro Díaz; Eleanor Baker; Elianeth A. Kiteni; Elna Hamilton Larsen; Enyo Asi Bosumprah; Farhat Naz; Frieda Clinton; Glenn Mbah Afungchwi; Gloria Ceballo Batista; Helen Morris; Ida Bremer Ophorst; Iman Ali Al Dhouyani; Joan Nakabiri; José García; Joy Mariz F. Dumayas; Julia M Challinor; Julia Downing; Julie M. Buser; Katherene P. Batuhan; Kenneth Henry F. Goyena; Kgomotso Pearl Semetsa; Kristian Juusola; Laura Mei Lian Tan; Lauri Linder; Lim Yan Yin; Linda Z. Abramovitz; Linda Sanderson; Liz Sniderman; Lomtunzi Chidziwa; Lorena Segovia Weber; Manlokiya Barnabas Raymond; Margareta af Sandeberg; Marjorie S. Kjellin; Marta Canesi; Mengxue He; Micaela Angela D. Tagud; Mohammad R. Alqudimat; Mohammed Alqaddi; Muhammad Ali Jadoon; Nastasia Esther Pereira; Noor Siti Noviani Indah Sari; Pramod Kumar Nagar; Rachael Kunkel; Ratidzo Mhiti; Reena S. Nair; Rehana Elahi; Rhahim Bank; Richard Ramos; Rima Saad; Rosie Mebelle Tongco; Ruhul Islam Chowdhury; Sabina Ncororo Ituana, Sandeep Elizabeth; Sarah Kaczor; Sherry Johnson; Silvana Espinoza Manjarrez; Simon Dean; Tendai Lionel Chisamba; Viridiana Mata Vela; and those who chose to remain anonymous.

Footnotes

Disclosure of Conflicts of Interest: The authors have no conflicts of interest to disclose.

REFERENCES

  1. Agulnik A, Cárdenas A, Carrillo AK, Bulsara P, Garza M, Alfonso Carreras Y, Alvarado M, Calderón P, Díaz R, de León C, Del Real C, Huitz T, Martínez A, Miralda S, Montalvo E, Negrín O, Osuna A, Perez Fermin CK, Pineda E, ... Rodriguez-Galindo. (2021). Clinical and organizational risk factors for mortality during deterioration events among pediatric oncology patients in Latin America: A multicenter prospective cohort. Cancer, 127(10), 1668–1678. 10.1002/cncr.33411 [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Agulnik A, Forbes PW, Stenquist N, Rodriguez-Galindo C, & Kleinman M (2016). Validation of a pediatric early warning score in hospitalized pediatric oncology and hematopoietic stem cell transplant patients. Pediatric Critical Care Medicine, 17(4), e146–e153. 10.1097/PCC.0000000000000662 [DOI] [PubMed] [Google Scholar]
  3. Agulnik A, Méndez Aceituno A, Mora Robles LN, Forbes PW, Soberanis Vasquez DJ, Mack R, Antillon‐Klussmann F, Kleinman M, & Rodriguez‐Galindo C (2017). Validation of a pediatric early warning system for hospitalized pediatric oncology patients in a resource‐limited setting. Cancer, 123(24), 4903–4913. 10.1002/cncr.30951 [DOI] [PubMed] [Google Scholar]
  4. Aiken LH, Cimiotti JP, Sloane DM, Smith HL, Flynn L, & Neff DF (2011). Effects of nurse staffing and nurse education on patient deaths in hospitals with different nurse work environments. Medical Care, 49(12), 1047–1053. 10.1097/MLR.0b013e3182330b6e [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Arias AV, Garza M, Murthy S, Cardenas A, Diaz F, Montalvo E, Nielsen KR, Kortz T, Sharara‐Chami R, Friedrich P, McArthur J, & Agulnik A (2020). Quality and capacity indicators for hospitalized pediatric oncology patients with critical illness: A modified Delphi consensus. Cancer Medicine, 9(19), 6984–6995. 10.1002/cam4.3351 [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Armes J, Wagland R, Finnegan-John J, Richardson A, Corner J, & Griffiths P (2014). Development and testing of the patient-reported chemotherapy indicators of symptoms and experience: Patient-reported outcome and process indicators sensitive to the quality of nursing care in ambulatory chemotherapy settings. Cancer Nursing, 37(3), E52–E60. 10.1097/NCC.0b013e3182980420 [DOI] [PubMed] [Google Scholar]
  7. Baxter AL, Watcha MF, Baxter WV, Leong T, & Wyatt MM (2011). Development and validation of a pictorial nausea rating scale for children. Pediatrics, 127(6), e1542–e1549. 10.1542/peds.2010-1410 [DOI] [PubMed] [Google Scholar]
  8. Bradley NM, Robinson PD, Greenberg ML, Barr RD, Klassen AF, Chan YL, & Greenberg CM (2013a). Measuring the quality of a childhood cancer care delivery system: Assessing stakeholder agreement. Value in Health, 16(4), 639–646. 10.1016/j.jval.2013.02.016 [DOI] [PubMed] [Google Scholar]
  9. Bradley NM, Robinson PD, Greenberg ML, Barr RD, Klassen AF, Chan YL, & Greenberg CM (2013b). Measuring the quality of a childhood cancer care delivery system: Quality indicator development. Value in Health, 16(4), 647–654. 10.1016/j.jval.2013.03.1627 [DOI] [PubMed] [Google Scholar]
  10. Cheung RB, Aiken LH, Clarke SP, & Sloane DM (2008). Nursing care and patient outcomes: International evidence. Enfermería Clínica, 18(1), 35–40. 10.1016/s1130-8621(08)70691-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Connor JA, Larson C, Baird J, & Hickey PA (2016). Use of a pediatric cardiovascular nursing consortium for development and evaluation of quality measures: The C4-MNP Experience. Journal of Pediatric Nursing, 31(5), 471–477. 10.1016/j.pedn.2016.04.010 [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Conway JE (1998). Evolution of the species “expert nurse”. An examination of the practical knowledge held by expert nurses. Journal of Clinical Nursing, 7(1), 75–82. 10.1046/j.1365-2702.1998.00127.x [DOI] [PubMed] [Google Scholar]
  13. Day S, Challinor J, Hollis R, Abramovitz L, Hanaratri Y, & Punjwani R (2015). Paediatric oncology nursing care in low- and middle-income countries: A need for baseline standards. Cancer Control, 2015, 111–116. [Google Scholar]
  14. Donabedian A (1988). Quality Assessment and Assurance: Unity of Purpose, Diversity of Means. Inquiry, 25(1), 173–192. http://www.jstor.org/stable/29771941 [PubMed] [Google Scholar]
  15. Downing J, Namisango E, & Harding R (2018). Outcome measurement in paediatric palliative care: Lessons from the past and future developments. Annals of Palliative Medicine, 7(Suppl 3), S151–S163. https://apm.amegroups.com/article/view/19946. https://doi.org/10.21037/apm.2018.04.02 [DOI] [PubMed] [Google Scholar]
  16. Fessele K, Yendro S, & Mallory G (2014). Setting the bar: Developing quality measures and education programs to define evidence-based, patient-centered, high-quality care. Clinical Journal of Oncology Nursing, 18, Suppl., 7–11. 10.1188/14.CJON.S2.7-11 [DOI] [PubMed] [Google Scholar]
  17. Given BA, & Sherwood PR (2005). Nursing sensitive patient outcomes—A white paper. Oncology Nursing Forum, 32(4), 773–784. 10.1188/05.ONF.773-784 [DOI] [PubMed] [Google Scholar]
  18. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, & Conde JG (2009). Research Electronic Data Capture (REDCap)—A metadata-driven methodology and workflow process for providing translational research informatics support. Journal of Biomedical Informatics, 42(2), 377–381. 10.1016/j.jbi.2008.08.010 [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Hockenberry M, & Landier W (2019). Symptom assessment during childhood cancer treatment. Journal of Pediatric Oncology Nursing, 36(4), 242–243. 10.1177/1043454219852611 [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Jackson-Rose J, Del Monte J, Groman A, Dial LS, Atwell L, Graham J, O’Neil Semler R, O’Sullivan M, Truini-Pittman L, Cunningham TA, Roman-Fischetti L, Costantinou E, Rimkus C, Banavage AJ, Dietz B, Colussi CJ, Catania K, Wasko M, Schreffler KA, West C, . . . Rice RD. (2017). Chemotherapy extravasation: Establishing a national benchmark for incidence among cancer centers. Clinical Journal of Oncology Nursing, 21(4), 438–445. 10.1188/17.CJON.438-445 [DOI] [PubMed] [Google Scholar]
  21. Johnston EE, Molina J, Martinez I, Dionne‐Odom JN, Currie ER, Crowl T, Butterworth L, Chamberlain LJ, Bhatia S, & Rosenberg AR (2020). Bereaved parents’ views on end‐of‐life care for children with cancer: Quality marker implications. Cancer, 126(14), 3352–3359. 10.1002/cncr.32935 [DOI] [PubMed] [Google Scholar]
  22. Keeney S, Hasson F, & McKenna H (2011). The Delphi technique in nursing and health research Wiley-Blackwell. [Google Scholar]
  23. Lam CG, Howard SC, Bouffet E, & Pritchard-Jones K (2019). Science and health for all children with cancer. Science, 363(6432), 1182–1186. 10.1126/science.aaw4892 [DOI] [PubMed] [Google Scholar]
  24. Landier W, Leonard M, & Ruccione KS (2013). Children’s Oncology Group’s 2013 blueprint for research: Nursing discipline. Pediatric Blood and Cancer, 60(6), 1031–1036. 10.1002/pbc.24415 [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Looper K, Winchester K, Robinson D, Price A, Langley R, Martin G, Jones S, Holloway J, Rosenberg S, & Flake S (2016). Best practices for chemotherapy administration in pediatric oncology: Quality and safety process improvements (2015). Journal of Pediatric Oncology Nursing, 33(3), 165–172. 10.1177/1043454215610490 [DOI] [PubMed] [Google Scholar]
  26. Montalvo I (2007). The National Database of Nursing Quality Indicators(TM)(NDNQI®). Online Journal of Issues in Nursing, 12(3), 2. [DOI] [PubMed] [Google Scholar]
  27. Morrissey L, Lurvey M, Sullivan C, Challinor J, Forbes PW, Abramovitz L, Afungchwi GM, Hollis R, & Day S (2019). Disparities in the delivery of pediatric oncology nursing care by country income classification: International survey results. Pediatric Blood and Cancer, 66(6), e27663. 10.1002/pbc.27663 [DOI] [PubMed] [Google Scholar]
  28. Mukkada S, Smith CK, Aguilar D, Sykes A, Tang L, Dolendo M, & Caniza MA (2018). Evaluation of a fever‐management algorithm in a pediatric cancer center in a low‐resource setting. Pediatric Blood and Cancer, 65(2), e26790. 10.1002/pbc.26790 [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Nagoya Y, Miyashita M, Irie W, Yotani N, & Shiwaku H (2020). Development of a proxy quality-of-life rating scale for the end-of-life care of pediatric cancer patients evaluated from a nurse’s perspective. Journal of Palliative Medicine, 23(1), 82–89. 10.1089/jpm.2018.0598 [DOI] [PubMed] [Google Scholar]
  30. National Institute for Occupational Safety and Health [NIOSH]. (2016). NIOSH list of antineoplastic and other hazardous drugs in healthcare settings 2016. By Connor TH, MacKenzie BA, DeBord DG, Trout DB, O’Callaghan JP. Cincinnati, OH: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Institute for Occupational Safety and Health, DHHS (NIOSH) Publication Number 2016–161 (Supersedes 2014–138). [Google Scholar]
  31. National Quality Forum [NQF]. (2004). National voluntary consensus standards for nursing-sensitive care: An initial performance measure set Retrieved March 7 fromhttps://www.qualityforum.org/Publications/2004/10/National_Voluntary_Consensus_Standards_for_Nursing-Sensitive_Care__An_Initial_Performance_Measure_Set.aspx
  32. Pan American Health Organization [PAHO]. (2013). Safe handling of hazardous chemotherapy drugs in limited-resource settings Retrieved March 7 from https://www.paho.org/hq/dmdocuments/2014/safe-handling-chemotherapy-drugs.pdf
  33. Pfaff K, & Markaki A (2017). Compassionate collaborative care: An integrative review of quality indicators in end-of-life care. BMC Palliative Care, 16(1), 65. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5709969/pdf/12904_2017_Article_246.pdf. https://doi.org/10.1186/s12904-017-0246-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Plano Clark V, & Ivankova NV (2016). Mixed methods research: A guide to the field (Vol. 3) Sage Publications. [Google Scholar]
  35. Rodgers CC, Hooke MC, Taylor OA, Koerner KM, Mitby PA, Moore IM, Scheurer ME, Hockenberry MJ, & Pan W (2019). Childhood cancer symptom cluster: Leukemia and health-related quality of life. Oncology Nursing Forum, 46(2), 228–237. 10.1188/19.ONF.228-237 [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Seow H, Snyder CF, Mularski RA, Shugarman LR, Kutner JS, Lorenz KA, Wu AW, & Dy SM (2009). A framework for assessing quality indicators for cancer care at the end of life. Journal of Pain and Symptom Management, 38(6), 903–912. 10.1016/j.jpainsymman.2009.04.024 [DOI] [PubMed] [Google Scholar]
  37. Solutions for Patient Safety [SPS]. (2021). How we work Retrieved March 6 from https://www.solutionsforpatientsafety.org/about-us/how-we-work/
  38. Start R, Matlock AM, Brown D, Aronow H, & Soban L (2018). Realizing momentum and synergy: Benchmarking meaningful ambulatory care nurse-sensitive indicators. Nursing Economics, 36(5), 246–251. [Google Scholar]
  39. Sullivan CE (2021). Developing a core set of nursing-sensitive indicators for international pediatric oncology nursing practice (Publication No. 28414564) [Doctoral dissertation, The University of Alabama at Birmingham]. ProQuest Dissertations and Theses Global [Google Scholar]
  40. Sullivan CE, Morrissey L, Day SW, Chen Y, Shirey M, & Landier W (2020). Predictors of hospitals’ nonachievement of baseline nursing standards for pediatric oncology. Cancer Nursing, 43(4), E197–E206. 10.1097/NCC.0000000000000688 [DOI] [PubMed] [Google Scholar]
  41. Sullivan C, Segovia Weber L, Viveros Lamas P, Zhao X, Lu Z, Navarro Diaz D, & Challinor J (2019). Analysis of Latin American pilot series: Association of pediatric hematology/oncology nurses Spanish chemotherapy/biotherapy course. [Abstract]. Pediatric Blood and Cancer, 66, S96–S97. [Google Scholar]
  42. Teichman J, Punnett A, & Gupta S (2017). Development of quality metrics to evaluate pediatric hematologic oncology care in the outpatient setting. Journal of Pediatric Hematology/Oncology, 39(2), 90–96. 10.1097/MPH.0000000000000656 [DOI] [PubMed] [Google Scholar]
  43. The World Bank. (2020). World Bank country and lending groups Retrieved June 15 from https://datahelpdesk.worldbank.org/knowledgebase/articles/906519-world-bank-country-and-lending-groups
  44. Tomlinson D, Dupuis LL, Gibson P, Johnston DL, Portwine C, Baggott C, Zupanec S, Watson J, Spiegler B, Kuczynski S, Macartney G, & Sung L (2014). Initial development of the Symptom Screening in Pediatrics Tool (SSPedi). Supportive Care in Cancer, 22(1), 71–75. 10.1007/s00520-013-1945-x [DOI] [PubMed] [Google Scholar]
  45. Wilson S, Hauck Y, Bremner A, & Finn J (2012). Quality nursing care in Australian paediatric hospitals: A Delphi approach to identifying indicators. Journal of Clinical Nursing, 21(11–12), 1594–1605. 10.1111/j.1365-2702.2011.04004.x [DOI] [PubMed] [Google Scholar]
  46. Wong C, Billett A, Friedrich P, Lehmann L, Albanti I, Morrissey L, Houlahan K, & Rodriguez-Galindo C (2017). Developing a tiered-system of quality of care indicators for pediatric oncology resource-limited settings. [Abstract]. Pediatric Blood and Cancer, 64, S433–S434. 10.1002/pbc.26772 [DOI] [Google Scholar]
  47. World Health Organization. [WHO]. (2020, April 6). State of the world’s nursing 2020: investing in education, jobs and leadership https://www.who.int/publications/i/item/9789240003279
  48. World Health Organization [WHO]. (2021, December 13). Childhood cancer Retrieved March 7, 2021 from https://www.who.int/news-room/fact-sheets/detail/cancer-in-children
  49. World Health Organization [WHO]. (2022). Country groupings Retrieved April 27, 2022 from https://www.who.int/observatories/global-observatory-on-health-research-and-development/classifications-and-standards/country-groupings

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Table S1

TABLE S1: Expert Panel Member Characteristics by Country, WHO Region, and CIC in Delphi Rounds One, Two, and Three

NIHMS1844853-supplement-Table_S1.docx (37.4KB, docx)
Table S2

TABLE S2: Delphi Round One Expert Panel Member Characteristics

NIHMS1844853-supplement-Table_S2.docx (34.3KB, docx)
Table S3

TABLE S3: Delphi Round One Characteristics of Hospitals in which the Expert Panelists Practiced

NIHMS1844853-supplement-Table_S3.docx (33.9KB, docx)
Table S5

TABLE S4: Round One NSIs and Constructs by Frequency & Proportion: WHO Regions and CIC

NIHMS1844853-supplement-Table_S5.docx (33.6KB, docx)
Table S4

TABLE S5: Representation of Core NSIs and Constructs in Round One Responses by WHO Region and Country Income Classification

NIHMS1844853-supplement-Table_S4.docx (38.5KB, docx)

RESOURCES