Priorities for paediatric critical care research: a modified Delphi study by the Australian and New Zealand Intensive Care Society Paediatric Study Group

Abstract

Objective: Most interventions in paediatric critical care lack high grade evidence. We aimed to identify the key research priorities and key clinical outcome measures pertinent to research in paediatric intensive care patients.

Design: Modified three-stage Delphi study combining staged online surveys, followed by a face-to-face discussion and final voting.

Setting: Paediatric intensive care units in Australia and New Zealand.

Participants: Medical and nursing staff working in intensive care.

Main outcome measurements: Self-reported priorities for research.

Results: 193 respondents provided a total of 267 research questions and 234 outcomes. In Stage 3, the top 56 research questions and 50 outcomes were discussed face to face, which allowed the identification of the top 20 research questions with the Hanlon prioritisation score and the top 20 outcomes. Topics centred on the use of intravenous fluids (restrictive v liberal fluids, use of fluid resuscitation bolus, early inotrope use, type of intravenous fluid, and assessment of fluid responsiveness), and patient- and family-centred outcomes (health-related quality of life, liberation) emerged as priorities. While mortality, length of stay, and organ support/organ dysfunction were considered important and the most feasible outcomes, long term quality of life and neurodevelopmental measures were rated highly in terms of their importance.

Conclusions: Using a modified Delphi method, this study provides guidance towards prioritisation of research topics in paediatric critical care in Australia and New Zealand, and identifies study outcomes of key relevance to clinicians and experts in the field.

More than 12 000 children require admission to intensive care in Australia and New Zealand every year for potentially life-threatening conditions.¹ However, there is scarce evidence for most of the interventions and practices, and the number of trials in paediatric critical care internationally remains low.2, 3 Observational studies demonstrate substantial variability of care even for common interventions and diseases.⁴ In order to overcome obstacles to paediatric critical care research, such as heterogeneity and low prevalence of many diseases, collaboration across research networks and prioritisation of topics and outcomes are required.5, 6, 7

Several research prioritisation studies have been published in related disciplines; for example, in neonatal intensive care nursing, paediatric emergency medicine, or paediatric onco-critical care.8, 9, 10 However, there is a lack of contemporary research prioritisation studies generated by multidisciplinary clinicians and researchers in paediatric critical care and a corresponding lack of studies identifying and prioritising the key research outcomes.

The Australian and New Zealand Intensive Care Society (ANZICS) Paediatric Study Group (PSG) is a collaborative body of multidisciplinary paediatric intensive care unit (PICU) clinicians that aims to promote, design and conduct multicentre research in critically ill children in Australia and New Zealand.

This study aimed to define the key priorities for research in paediatric critical care pertinent to Australia and New Zealand through a modified Delphi process.

Methodology

We conducted a three-stage modified Delphi survey of research priorities and clinical outcome priorities with senior medical and nursing PICU staff working in Australia and New Zealand (Supporting Information, supplemental methods). The study was conducted between 1 September 2019 and 30 April 2020. The surveys for each stage were administered via an online REDCap database,¹¹ and consent was obtained at the start of the survey. The study was approved by the Human Research and Ethics Committee (Children’s Health Queensland LNR/19/QCHQ/54762).

Stage 1: collection of priorities for research and research outcomes

Medical and nursing staff working in PICUs across Australia and New Zealand who met the inclusion criteria were invited to participate in the Stage 1 online survey.10, 12 Respondents were requested to state the five most important research questions and the five most important outcomes for research, each in free text. Reminders were sent every 2 weeks for 2 months.

All unedited responses were distributed to an expert panel (GB, SR, LJS, DL, SE, MF, JM) for review and refinement (Supporting Information, supplemental tables 1–3). Duplicate questions were excluded. Each reviewer was asked to independently assess each of the proposed research questions with regard to the following criteria:

• •Is the question amenable to multicentre research? As the ANZICS PSG is a multicentre research network, questions of relevance to just a single centre were excluded.
• •Is there sufficient current existing evidence to answer the question? A question was excluded if the expert panel agreed that existing evidence could sufficiently answer the question.

Through this process, the number of items was reduced and suitable questions were formatted using the population, intervention, comparison and outcome (PICO) model. The proposed clinical outcomes were reviewed using a similar approach by the same expert panel.

Stage 2: ranking of research priorities and outcomes

The Stage 2 survey consisted of the refined full list of research questions and clinical outcomes generated by Stage 1, with ordering by random sequence. Respondents were requested to score the importance of each research question and clinical outcome on a seven-point Likert scale. The responses for research questions and outcomes were then ranked by their mean score. Only the items that remained in the top 50% for all respondents, nursing respondents or medical respondents progressed to Stage 3.

Stage 3: Hanlon prioritisation process

Stage 3 was conducted restricted to the ANZICS PSG committee, including one medical and one nursing representative for each PICU in Australia and New Zealand. The Hanlon prioritisation process measures prevalence, seriousness and feasibility of a research question. During a full-day face-to-face meeting with videoconference option, all of the top 50% of questions and outcomes were discussed individually, organised by random order according to themes (Supporting Information, supplemental tables 2 and 3). The discussion, facilitated by an international expert in paediatric intensive care research (MP), specifically focused on: i) prevalence of the condition; ii) seriousness of the condition; iii) feasibility of the research; and, iv) likelihood to participate in a trial addressing the research question. The participants were then asked to score each PICO question under these four domains; outcomes were scored for seriousness and feasibility only.

Statistical analyses

Respondent characteristics are represented as number (percentage). The Delphi score was calculated as the mean score across all respondents in the Stage 2 survey. The Hanlon score for research questions was the product of mean scores for the three domains (prevalence [P], seriousness [S] and feasibility [F]; Hanlon score = P x S x F). For outcomes, the Hanlon score was the product of the mean score for importance and feasibility.

Results

The overview of the Delphi process and the characteristics of respondents are presented in Figure 1 and Table 1 respectively. A total of 193 PICU staff (33.7%) responded to the Stage 1 survey, including 172 doctors and nurses out of 572 eligible PICU staff in Australia and New Zealand (response rate, 30.1%), of which 134 (77.9%) responded to the Stage 2 survey. Based on the Delphi scores, the list was then reduced to 56 research questions and 50 outcomes to include the top 50% of priorities chosen by the whole group, uniquely by doctors and uniquely by nurses.

Figure 1.

Flow chart showing participation at the various stages of the prioritisation study

Table 1.

Characteristics of respondents for each stage of the prioritisation study

Variable	Category	Stage 1	Stage 2	Stage 3
Total number of respondents		193	134	24
Country of practice	Australia	153 (79%)	111 (83%)	23 (96%)
	New Zealand	28 (15%)	22 (16%)	1 (4%)
	No response	12 (6%)	1 (1%)	0
Role	Doctor	65 (34%)	50 (37%)	16 (67%)
	Nurse	113 (59%)	83 (62%)	6 (25%)
	Other	2 (1%)	0	2 (8%)
	No response	13 (7%)	1 (1%)	0
Years in PICU	< 2 years	12 (6%)	7 (5%)	1 (4%)
	2-5 years	33 (17%)	21 (16%)	2 (8%)
	> 5 years	135 (70%)	105 (78%)	21 (88%)
	No response	13 (7%)	1 (1%)	0
Training	Completed	135 (70%)	106 (79%)	22 (92%)
	Ongoing	45 (23%)	27 (20%)	2 (8%)
	No response	13 (7%)	1 (1%)	0
Involved in research	Yes	80 (41%)	65 (49%)	23 (96%)
	No	100 (52%)	68 (51%)	1 (4%)
	No response	13 (7%)	1 (1%)	0

PICU = paediatric intensive care unit.

Twenty-four ANZICS PSG Committee members (92.3%) completed the Stage 3 of the Hanlon prioritisation process.

Prioritised research questions

The top research priority was to investigate whether a restrictive fluid strategy improves outcomes in critically ill children (Table 2) (Supporting Information, supplemental table 4 and figure 1). “What matters to the families” was ranked second, followed by health-related quality of life. Key themes were observed in the top 20 research priorities, including fluid therapy (along with fluid resuscitation and early inotropes in sepsis), family experience, care of families and staff (moral distress and burnout in staff), restrictive oxygen therapy, and domains related to the LIBERATION quality improvement initiative.¹³

Table 2.

Top 20 research questions^⁎ identified in the prioritisation study

Rank	Research question	Hanlon score†	Delphi score; mean (SD)	Participation score; mean (SD)
1	Does a restrictive fluid strategy improve outcomes in critically ill children? P: critically ill children requiring fluid resuscitation and maintenance; I: restrictive fluid strategy; C: liberal strategy; O: duration of organ dysfunction, PICU LOS, mortality	505.0	5.0 (1.1)	7.8 (1.5)
2	What matters to families when their child is in PICU? P: critically ill children and families; I/C: na; O: survey of what matters to critically ill children and what matters to families/family experience	494.4	5.4 (1.1)	7.8 (1.9)
3	What is the health-related quality of life for children following PICU discharge? P: critically ill children admitted to PICU; I/C: na; O: health-related quality of life assessment at baseline (parent/patient interviews) and at 6 months (telephone interviews) to determine health-related quality of life following paediatric critical illness	435.0	5.5 (1.1)	7.9 (1.2)
4	Does early inotrope use improve outcomes in paediatric sepsis? P: critically ill children with septic shock; I: early inotrope infusion; C: standard (Surviving Sepsis Campaign) treatment algorithm (40–60 mL/ kg); O: organ failure/ventilation/ICU stay/mortality	427.0	5.4 (1.2)	7.5 (1.8)
5	Is a conservative versus liberal oxygenation target safe? P: children in ICU; I: conservative oxygen target; C: liberal oxygen target; O: mortality/ organ dysfunction/ adverse events	425.7	4.6 (1.3)	7.1 (2.0)
6	Do fluid boluses increase mortality in paediatric sepsis? P: patients with sepsis; I: restrictive fluid bolus strategy; C: standard/liberal fluid bolus strategy; O: duration of organ dysfunction mortality, PICU LOS	409.5	5.3 (1.1)	7.9 (1.7)
7	Does the use of delayed consent within the acute intensive care setting reduce parental stress? P: families of critically ill children considered for interventional research; I: delayed (deferred, consent to continue) consent; C: prospective consent; O: family stress level during admission	389.3	4.6 (1.5)	6.7 (2.7)
8	Does early psychological intervention improve family wellbeing during/ after a PICU admission? P: families of critically ill children; I: early psychological support intervention; C: standard family care; O: family wellbeing and stress during PICU and post discharge	363.0	5.2 (1.3)	6.9 (1.9)
9	What factors predict adverse events and long term outcomes following PICU admission? P: critically ill children admitted to PICU; I/C: na; O: longitudinal follow-up after PICU discharge to determine long term functional status, neurodevelopmental outcomes, predictors of adverse outcomes	354.3	5.2 (1.4)	7.1 (2.3)
10	Does early mobilisation improve outcomes in critically ill children? P: critically ill children in PICU; I: early mobilisation; C: standard care; O: ICU/hospital stay, ventilation time, incidence of delirium	354.0	5.3 (1.2)	6.5 (1.9)
11	Can we use ANZPIC data to create metrics for the benchmarking of quality and safety in PICU? P: ANZPIC registry data on critically ill children admitted to PICU; I/C: na; O: development of predictive models for the likelihood of various metrics (duration of mechanical ventilation, cardiac arrest, acute kidney injury)	348.6	5.1 (1.2)	6.3 (2.8)
12	What strategies will improve nursing staff workplace satisfaction and retention? P: PICU nursing staff; I/C: na; O: survey of workplace practices to determine factors that contribute to poor satisfaction and retention of nursing staff	344.1	5.5 (1.2)	6.0 (2.5)
13	What are the long term neurological outcomes of ECMO survivors? P: ECMO survivors; I/C: na; O: longitudinal neurodevelopmental follow-up	338.5	5.2 (1.3)	7.1 (2.3)
14	What are the prevalence and factors contributing to moral distress, burnout and PTSD among PICU staff? P: PICU staff; I: survey; C: na; O: prevalence and factors contributing to moral distress, burn-out and PTSD	327.7	5.2 (1.4)	6.0 2.3)
15	Do standardised antibiotic algorithms improve antibiotic usage/resistance and outcomes for patients with suspected infection? P: critically ill children with suspected infection; I: algorithm to guide decisions to start and stop antibiotics; C: standard care; O: reduction in antibiotic usage, reduction of microbial resistance, safety, survival, PICU LOS, costs	313.6	5.0 (1.1)	6.0 (2.3)
16	Do balanced crystalloid solutions reduce the incidence of metabolic acidosis and renal dysfunction in critically ill children? P: critically ill children requiring maintenance fluid; I: balanced crystalloid solutions; C: saline; O: incidence of metabolic acidosis and renal dysfunction	298.6	4.8 (1.1)	6.4 (2.2)
17	Do pain assessment tools/algorithms reduce exposure to sedative agents? P: critically ill children requiring mechanical ventilation; I: different tools/ algorithms for pain assessment; C: standard care pain management; O: cumulative exposure to analgesic agents, duration of ventilation, duration of PICU stay	277.8	4.9 (1.2)	5.4 (1.9)
18	Can haemodynamic or echocardiographic parameters improve effectiveness of fluid resuscitation in critically ill children? P: critically ill children requiring fluid resuscitation; I: fluid administration guided by haemodynamic or echocardiographic parameters; C: current practice; O: mortality, ventilation time, inotrope use, cumulative fluid balance	268.0	5.0 (1.1)	5.9 (1.4)
19	Is POC testing for suspected infection safe and effective? P: critically ill children with suspected infection; I: POC testing for infection; C: current practice; O: sensitivity/specificity of POC testing, time to appropriate antibiotic therapy	265.1	4.4 (1.2)	6.2 (2.1)
20	Does a delirium bundle reduce the incidence of delirium in critically ill children? P: critically ill children; I: delirium prevention bundle; C: standard care; O: incidence of delirium	261.9	5.0 (1.2)	5.7 (1.9)

ANZPIC = Australian and New Zealand Paediatric Intensive Care; ICU = intensive care unit; LOS = length of stay; na = not applicable; PICU = paediatric intensive care unit; POC = point-of-care; PTSD = post traumatic stress disorder; SD = standard deviation.

^⁎Using the population, intervention, control and outcome (PICO) format.

^†Hanlon score = importance x feasibility.

Prioritised outcomes

The outcomes receiving the highest priority included mortality, PICU and hospital length of stay, and duration of ventilation (Table 3) (Supporting Information, supplemental table 5). Survival ≥ 12 months with good neurodevelopment and quality of life assessed > 12 months both after PICU discharge were among the highest ranked outcomes on the Delphi score, while they were in the top 20 priority list of outcomes according to the Hanlon score (Supporting Information, supplemental figure 2).

Table 3.

Top 20 outcomes identified in the prioritisation study

Rank	Outcome	Hanlon score⁎	Delphi score; mean (SD)
1	Mortality	81.6	5.3 (1.1)
2	Mortality in PICU	80.9	5.4 (1.1)
3	PICU length of stay	75.6	4.9(1.1)
4	Duration of mechanical ventilation	74.1	5.1 (1.0)
5	Hospital length of stay	73.8	4.9(1.1)
6	Long term mortality ≥ 12 months after PICU discharge	71.1	5.5 (1.1)
7	Ventilator-free days	71.0	4.9 (1.1)
8	Multi-organ dysfunction	66.3	5.1 (1.0)
9	Central line-associated bloodstream infection	65.3	5.2 (1.2)
10	Survival free of organ dysfunction (days alive and free of organ dysfunction)	63.9	5.0 (1.2)
11	Failed extubation	63.8	4.9 (1.2)
12	Duration of organ support	63.6	4.9 (1.2)
13	Treatment with ECMO	62.9	4.9 (1.3)
14	Quality of life assessed ≥ 12 months after PICU discharge	61.2	5.4 (1.1)t
15	Survival free of ECMO	59.6	4.9 (1.3)
16	Quality of life assessed at time of discharge from hospital	58.7	4.8 (1.1)
17	Survival > 12 months after PICU discharge with good neurodevelopment	58.3	5.5 (1.3)t
18	PICU readmission	58.1	5.0 (1.1)
19	Dependence on medical technology at discharge	55.8	4.7 (1.4)
20	Organ donation	55.5	5.2 (1.3)†

ECMO = extracorporeal membrane oxygenation; PICU = paediatric intensive care unit; SD = standard deviation.

^⁎Hanlon score = importance x feasibility.

^†Denotes outcomes which were ranked in the top ten in the Delphi score which were not in the top ten as per the Hanlon score.

Discussion

This ANZICS PSG prioritisation study explored the research priorities in Australia and New Zealand and allowed to prioritise key topics for future research based on their importance and feasibility. Currently, most children requiring PICU admission are exposed to treatments and practices for which there is no or only scarce evidence, many of which have substantial, sometimes life-threatening, side effects.¹⁴ This lack of evidence needs to be considered in light of the vulnerability of patients in terms of severity of disease, brain development, and dependence on parents or caregivers, and contrasts sharply, for example, with the level of evidence available for chemotherapy regimens in paediatric cancer patients. Given limited resources, determining priorities and feasibility for PICU research in defined settings is essential.

Our prioritisation exercise identified opinions of a binational experienced medical and nursing cohort. The top priorities for research themes included fluid therapy, family experience, health-related quality of life following PICU discharge, and family and staff wellbeing. The top-ranked outcomes were mortality, PICU and hospital length of stay, and duration of mechanical ventilation. Interestingly, there was a disparity between the Hanlon and Delphi scores for some of the research questions, likely reflecting conflicting assessment regarding importance versus feasibility, especially for survival versus quality of life at 12 months after discharge.¹⁵

The top priorities for research focused on very common therapies or interventions such as fluids, antibiotics and oxygen. The focus on highly pragmatic trial questions has important implications for future trial designs.¹⁴ Specifically, options to facilitate enrolment of patients into studies at the time of PICU admission, standardised data collection across studies, and extraction of patient, treatment and outcome data from registries and electronic health records need to be maximised. Interestingly, none of the top priorities related to personalised medicine, artificial intelligence, or omics.¹⁶ The Hanlon process does not consider innovation and instead weighs prevalence, seriousness and feasibility, which is expected to result in a more conservative pragmatic prioritisation. Therefore, future research planning should carefully consider targeted support for high innovation approaches. Furthermore, the perceived importance of research is likely strongly influenced by daily practices and current controversies. Of note, we finished the survey shortly before the coronavirus disease 2019 (COVID-19) pandemic affected Australia.¹⁷ Topics related to pandemic and disaster preparedness, personal protective equipment, resource allocation, or telemedicine were not considered to be relevant by our study respondents.

In contrast to previously published prioritisation studies, we deliberately included both medical and nursing staff in all stages of the modified Delphi process. While we observed some differences in ranking of studies in Stage 2, there was broad overlap in the majority of prioritised topics by nurses and doctors. The relevance of patient-centred outcomes was broadly acknowledged by both medical and nursing respondents. The importance of staff wellbeing, patient delirium and pain management, and family-centred care were favoured by nursing respondents.8, 12, 18

The highest priority research question in our study was whether restrictive versus liberal fluid strategy lead to improved patient outcomes or not. Several further questions ranked in the top 20 priorities dealt with fluid therapy, such as early inotropes in sepsis, type of fluid, fluid boluses, and assessment of fluid responsiveness.¹⁹ Almost a decade after the landmark FEAST (Fluid Expansion as Supportive Therapy) study,²⁰ and following a large number of observational studies demonstrating increased mortality associated with fluid overload in critically ill children,²¹ fluid therapy remains in the spotlight.10, 22, 23

Another theme that emerged was the focus on outcomes beyond intensive care admission pertaining to quality of life and neurodevelopment. In contrast to the neonatal intensive care unit setting, where long term follow-up has been considered best practice for extremely low and very low birthweight infants for almost two decades, structured follow-up programs in the PICU setting in Australia and New Zealand are largely lacking. As a result, we observed a major gap between the perceived importance of long term outcome measures and the feasibility of performing such follow-up. Correspondingly, traditional ICU outcomes such as mortality, length of stay, or duration of support were ranked top in the Hanlon prioritisation process despite being considered less important than long term outcomes measured in the Delphi process.²⁴

Our study had a number of limitations. First, several biases are well known to affect survey-based studies, such as respondent selection and survey fatigue. While we attempted to capture the entire senior medical and nursing PICU workforce, the response rate for Stage 1 was only 30%. The response rates for Stages 2 and 3 were substantially higher (77% and 92% respectively). Second, a major limitation of the prioritisation study was that only clinicians were eligible to participate. The relevance of consumer perspectives in PICU research and the need for active engagement of consumer groups at the study design stage is increasingly recognised.25, 26 There is a need to reconcile research priorities between clinical staff, researchers, patients and families.27, 28, 29 Finally, the priorities identified through this ANZICS PSG prioritisation study are specific to the epidemiology, resources and workforce of PICU staff in Australia and New Zealand,³⁰ and may not reflect wider priorities for global paediatric intensive care.

Conclusions

The ANZICS PSG prioritisation study enabled the identification of top research topics and outcomes relevant to the design of future research in Australia and New Zealand. Overall, common interventions such as fluid administration ranked highest, indicating a need for pragmatic trials. Future studies need to consider approaches to capturing long term outcomes.

Competing interests

No relevant disclosures.