Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Dec 1.
Published in final edited form as: Crit Care Med. 2020 Dec;48(12):1819–1828. doi: 10.1097/CCM.0000000000004660

A Core Outcome Set for Pediatric Critical Care

Ericka L Fink 1, Aline B Maddux 2, Neethi Pinto 3, Samuel Sorenson 4, Daniel Notterman 5, J Michael Dean 4, Joseph A Carcillo 1, Robert A Berg 3, Athena Zuppa 3, Murray M Pollack 6, Kathleen L Meert 7, Mark W Hall 8, Anil Sapru 9, Patrick S McQuillen 10, Peter M Mourani 2, David Wessel 6, Deborah Amey 11, Andrew Argent 12, Werther Brunow de Carvalho 13, Warwick Butt 14, Karen Choong 15, Martha AQ Curley 27, Maria del Pilar Arias Lopez 16, Demet Demirkol 17, Ruth Grosskreuz 2, Amy J Houtrow 18, Hennie Knoester 19, Jan Hau Lee 20, Debbie Long 21, Joseph C Manning 22, Brenda Morrow 23, Jhuma Sankar 24, Beth S Slomine 25, McKenna Smith 4, Lenora M Olson 4, R Scott Watson 26; the POST-PICU Investigators of the Pediatric Acute Lung Injury and Sepsis Investigators (PALISI) Network and the Eunice Kennedy Shriver National Institute of Child Health and Human Development Collaborative Pediatric Critical Care Research Network (CPCCRN)
PMCID: PMC7785252  NIHMSID: NIHMS1626027  PMID: 33048905

Abstract

Objective

More children are surviving critical illness but are at risk of residual or new health conditions. An evidence-informed and stakeholder-recommended core outcome set is lacking for pediatric critical care outcomes. Our objective was to create a multinational, multi-stakeholder-recommended Pediatric Critical Care Core Outcome Set for inclusion in clinical and research programs.

Design

A 2-round modified Delphi electronic survey was conducted with 333 invited research, clinical, and family/advocate stakeholders. Stakeholders completing the first round were invited to participate in the second. Outcomes scoring > 69% “critical” and < 15% “not important” advanced to round 2 with write-in outcomes considered. The Steering Committee held a virtual consensus conference to determine the final components.

Setting

Multinational survey.

Patients

Stakeholder participants from 6 continents representing clinicians, researchers, and family/advocates.

Main Results

Overall response rates were 75% and 82% for each round. Participants voted on 7 Global Domains and 45 Specific Outcomes in Round 1, and 6 Global Domains and 30 Specific Outcomes in Round 2. Using Overall (3 stakeholder groups combined) results, consensus was defined as outcomes scoring > 90% “critical” and < 15% “not important” and were included in the final PICU COS: 4 Global domains (Cognitive, Emotional, Physical and Overall Health) and 4 Specific outcomes (Child Health-Related Quality of Life, Pain, Survival, and Communication). Families (n=21) suggested additional critically important outcomes that did not meet consensus, which were included in the PICU COS - Extended.

Conclusions

The PICU Core Outcome Set and PICU COS-Extended are multi-stakeholder-recommended resources for clinical and research programs that seek to improve outcomes for children with critical illness and their families.

Keywords: Child, Critical Care, Outcome Assessment, Post Intensive Care Syndrome, Family

Introduction

Approximately 480,000 children and young adults < 20 years old are admitted to pediatric intensive care units (PICUs) annually at a cost of $8 billion in the United States alone 1,2. Mortality has decreased to 2–4% in high-resource settings3. However, child and family survivorship and recovery are frequently affected by ongoing and/or new impairments in physical, emotional, cognitive, and/or social health functioning, termed Post-Intensive Care Syndrome-pediatrics 4.

The vast majority of pediatric critical care research studies employ short-term (in-hospital) physiologic or mortality outcomes, with few studies assessing outcomes post-hospital discharge 5–9. Researchers report that key monetary and resource barriers to carrying out high quality trials include assessment of outcomes post-hospital discharge 10. Furthermore, heterogeneity in outcome measures and time points selected in studies prohibits systematic review and meta-analysis 11.

The status quo for PICU outcomes is shifting towards inclusion of outcomes prioritized by providers, patients, and families 12–14 rather than solely by investigators. Core Outcome Sets (COS), defined as “a patient outcome, health-related condition, or aspects of health that relevant stakeholders agree are essential to assess in all clinical research studies evaluating outcomes”, have been developed and implemented successfully for other critically ill populations, but not for pediatric critical care 15,16. Additionally, use of a COS allows for increased ability to compare outcomes across studies and populations and decreases the potential for reporting bias17.

Our objective was to develop a multi-stakeholder-informed PICU-COS. The product of this effort is a minimum set of outcome domains that should be incorporated into clinical and research programs to evaluate outcomes of critically ill children and families.

Materials and Methods

Study Design

We incorporated recommendations for methodology and quality standards for design and reporting for COS 18–20. This manuscript reports on the Delphi consensus process yielding the final PICU COS. Additional description of the study protocol 21 and preliminary aims were published 6,22,23. This project was approved by the University of Utah Institutional Review Board.

Modified, international Delphi consensus – process overview

Delphi content.

Investigators created a list of unique Global Domains and Specific Outcomes from a scoping review, qualitative study, and other relevant sources 13,14,16,24. Domains and Outcomes were paired with lay definitions that were reviewed by the Collaborative Pediatric Critical Care Research Network (CPCCRN)’s Family Network Collaborative, composed of 1–2 family volunteers from each of the 7 centers. The Steering Committee approved the Domains, Outcomes, and lay definitions. The CPCCRN’s Data Coordinating Center prepared the Delphi software.

Stakeholders and Steering Committee members.

The Steering Committee was recruited by the primary investigators. Candidates were invited with consideration for expertise in PICU Outcomes and diversity in background, region, and gender. All were fluent in English.

The Steering Committee invited 333 participants from three stakeholder groups – Research, Clinical, and Family - via an introductory email. After consenting, stakeholders were asked to respond based on their own perspective, except for those who represented an organization. Members of the Steering Committee were included as stakeholders as they represented leaders in pediatric critical care outcomes and families of children with critical illness.

Modified Delphi Consensus Methods.

We planned for a minimum of two Delphi rounds to reach consensus. Panel members scored components using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Scale, which consists of a 9-point scale: “not important for inclusion” (scores 1–3), “important but not critical for inclusion” (scores 4–6), “critical for inclusion” (scores 7–9), and “unable to score” (score 10)25. Global Domains and their related Specific Outcomes were randomized into four different orders and randomly assigned to panel members each round. Stakeholders were given approximately 3 weeks to complete each round. Non-respondents received a weekly personalized email, telephone call, or text reminder. Stakeholder response rates were calculated as the number of respondents who completed each round as a proportion of those for whom an email invitation was sent. Those who participated in Round 1 were invited for Round 2.

In Round 1, we recorded demographic information and assigned each consented participant a unique identifier. Respondents could propose novel outcomes. A priori criteria for an outcome domain from Round 1 to be included in Round 2 required >70% of responses rating ≥7 AND <15% of response rating <3. The Steering Committee used consensus to confirm the outcomes panel for Round 2. The Steering Committee reviewed new outcome suggestions from Round 1 to ensure they represented a new contribution for inclusion in Round 2

During Round 2 voting, respondents were provided aggregate responses from the first round for all stakeholders and by stakeholder group, their own response from Round 1, and new outcomes from Round 1. The Steering Committee used consensus to confirm the final COS components taking into account number of domains and importance based on scores by stakeholder group, ultimately as those in Round 2 with >90% of responses rating ≥7 AND <15% of response rating <3.

Analysis and reporting.

Each outcome’s score was analyzed based on the total number of respondents who answered the question. We report measures of central tendency, score distribution, and score changes by round of PICU COS Global Domains and Specific Outcomes as well as those considered for inclusion.

Results

Steering committee and stakeholders

The Steering Committee consisted of 23 members, including at least two representatives from each of the six continents and a heterogeneous group of clinical and research experts as well as a member from the family stakeholder group (Supplementary Table 1). The Steering Committee recommended diverse stakeholder group members (n=333 total): Research (n=59), Clinical (n=226), and Family (n=48) (Supplementary Table 2). Table 1 describes stakeholder characteristics for those who responded to both rounds. Notably, there were more female respondents for each stakeholder group. Most family respondents were located in North America (81.0%). All stakeholders reported at least some post-secondary education.

Table 1.

Demographics of stakeholders who responded to both PICU Core Outcome Set Delphi rounds.

Stakeholder
Clinician (N = 150) Family (N = 21) Research (N = 35) Overall (N = 206)
Age, (years), n (%)
 20–44 67 (44.7) 12 (57.1) 13 (37.1) 92 (44.7)
 45–64 72 (48.0) 7 (33.3) 16 (45.7) 95 (46.1)
 65+ 8 (5.3) 1 (4.8) 4 (11.4) 13 (6.3)
 Not reported 3 (2.0) 1 (4.8) 2 (5.7) 6 (2.9)
Gender
 Female 91 (60.7) 15 (71.4) 20 (57.1) 126 (61.2)
 Not reported 1 (0.7) 1 (4.8) 1 (2.9) 3 (1.5)
Region, n (%)
 Africa 14 (9.3) 2 (9.5) 4 (11.4) 20 (9.7)
 Asia 15 (10.0) 0 (0.0) 3 (8.6) 18 (8.7)
 Australia and New Zealand 12 (8.0) 0 (0.0) 4 (11.4) 16 (7.8)
 Europe 21 (14.0) 1 (4.8) 6 (17.1) 28 (13.6)
 North America 67 (44.7) 17 (81.0) 16 (45.7) 100 (48.5)
 South America 17 (11.3) 0 (0.0) 2 (5.7) 19 (9.2)
 Not reported 4 (2.7) 1 (4.8) 0 (0.0) 5 (2.4)
Education, n (%)
 Some tertiary (college or vocational) education/training 1 (0.7) 5 (23.8) 0 (0.0) 6 (2.9)
 Four-year tertiary (college) degree/bachelor’s degree or vocational license/certificate 9 (6.0) 8 (38.1) 4 (11.4) 21 (10.2)
 Some postgraduate or professional schooling 6 (4.0) 2 (9.5) 0 (0.0) 8 (3.9)
 Postgraduate or professional degree; including master’s; doctorate; medical or law degree 134 (89.3) 6 (28.6) 31 (88.6) 171 (83.0)
Years of Experience as Clinician: median [interquartile range] 17.0 [9.0–25.0] - [---] 19.0 [9.0–30.0] 17.5 [9.0–25.0]
Open in a new tab

Delphi Round 1

The response rate for Round 1 was 251/333 (75.0%) overall and 180/226 (80.0%) for clinicians, 38/59 (63.3%) for researchers, and 33/48 (68.8%) for family stakeholders. The Round 1 survey included 7 Global Domains and 45 Specific Outcomes (Figure 1). No outcomes met criteria for “not important for inclusion” among any stakeholder group (Supplemental Table 3). There was good general agreement in Global Domain scores among the groups (Supplemental Table 4). Family stakeholder group scores were generally higher than other groups for Specific Outcomes, especially for domains related to family function. Six Global Domains (all except Health Care Utilization) and 22 Specific Outcomes met the a priori cutoff of 70% of responses rating ≥7 AND <15% of response rating <3 for inclusion in Round 2. Within Global Domains, 4 of 6 Specific Outcomes from cognitive function, 7 of 10 overall health, 4 of 5 physical function, 2 of 5 emotional health, 3 of 9 family function, 2 of 6 health care utilization, and no social function met this threshold.

Figure 1.

Figure 1.

Open in a new tab

PICU Core Outcome Set: Modified Delphi Survey Flow Diagram

The Steering Committee elected to include in Round 2 an additional 4 Specific Outcomes that had an overall score from Round 1 approaching the inclusion threshold (69–69.9%) and were strongly regarded as “critical” by the Family stakeholder group (Sleep, Parent/ Legal Guardian Quality of Life, Child Participation, Hospital/Intensive Care Unit Readmission).

Participants submitted 61 write-ins from Round 1, resulting in 5 new outcomes voted on during Round 2. These included the division of Parent/Legal Guardian Overall Health into 4 Specific Outcomes (Emotional, Physical, Social, and Overall Function) as well as the addition of a new outcome (New Medical Conditions or Diseases). The remaining write-in responses were either outcome instruments or not new, unique outcomes and were not included. Definitions for some outcomes were modified to clarify criteria (Supplemental Table 5).

Delphi Round 2

The Round 2 survey had a response rate of 206/251 (82.1%): 150/180 (83.3%) for clinicians, 35/38 (92.1%) for researchers, and 21/33 (63.6%) for family/advocacy stakeholder groups. The Round 2 survey included 6 Global Domains and 30 Specific Outcomes. No outcomes met criteria for “not important for inclusion” among any stakeholder group (Supplemental Table 6). Some participants made relatively minor changes to their scores between rounds (Figures 2a and 2b). The Steering Committee approved four Global Domains (Overall, Cognitive, Physical, and Emotional Function) in addition to 4 Specific Outcomes (Child Quality of Life, Survival, Pain, and Communication) that are ultimately included in the final COS (Table 2). Four of 7 Family Function Specific Outcomes met a priori criteria for inclusion, but none met the adjusted criteria (Supplemental Table 7, Supplemental Figures 1a and 1b). Further, hospital and ICU readmission were among the lowest-scoring outcomes in Round 2. The lowest scoring Specific Outcomes were some of the newly added family outcomes which had widely disparate scoring among Family, Clinician and Research stakeholders.

Figures 2a and 2b.

Figures 2a and 2b.

Figures 2a and 2b.

Open in a new tab

Differences in scores from Round 1 to round 2 by PICU Core Outcome Set stakeholder group.

Table 2.

PICU Core Outcome Set and PICU COS – Extended

PICU Core Outcome Set
Cognitive Function Emotional Function Overall Health Physical Function
Child Communication Child Survival
Child Health Related Quality of Life
Child Pain
PICU COS – Extended
Family Function Overall Health Emotional Function Physical Function
Family Quality of Life New Medical Conditions or Diseases Mood and Feelings Organ Function
Family Relationships Overall Development Post-Traumatic Stress Disorder: Symptoms and Growth Physical Mobility
Parent/Guardian Emotional Function Sleep Sensory Functions
Parent/Guardian Overall Health Trajectory of Recovery Medical Frailty
Open in a new tab

PICU, Pediatric Intensive Care Unit; COS, Core Outcome Set

Scoring disparities among stakeholder groups were discussed by the Steering Committee via webinar. Ultimately, to recognize Family Stakeholder priorities, the Steering Committee recommended creation of a PICU COS – Extended tool (Table 2, Supplemental Figures 1a and 1b). The PICU COS – Extended includes 14 Specific Outcomes that met the “critical for inclusion” threshold by > 90% Family Stakeholders from the Global Domains Overall Health, and Family, Emotional, and Physical Function.

Discussion

We developed a multinational, multi-stakeholder and evidence informed COS for clinical and research use in pediatric critical illness. Two rounds of a modified Delphi survey led to consensus. A supplemental COS – Extended was also created by the Steering Committee to recognize outcomes important to family members that did not meet consensus.

No guidelines exist for the follow-up of pediatric critical care patients as they do for neonates and infants with congenital heart disease 3,12,26 or adults surviving critical illness16. Furthermore, pediatric critical care research programs infrequently include post-discharge outcomes 5,7,10. Finally, the status quo for outcome assessment for clinical care and for research has not yet evolved to include input from stakeholders such as patients and families 27,28.

Development of a multi-stakeholder-informed COS to guide clinicians and researchers in choosing post-hospital discharge outcomes in addition to other outcomes vital to clinical and research aims will increase program value and facilitate evidence robustness15,29. We followed international guidelines for the development of COS including a mixed-methods approach to the generation of outcomes and a multinational and multi-stakeholder Steering Committee and Delphi respondent panel 6,16,20,22.

The final PICU COS features the Global Outcome Domains of Cognitive, Emotional, and Physical Function and Overall Health. In addition, Specific Outcomes under Cognitive Function (Child Communication), and Overall Health (Child Survival, Health-Related Quality of Life, and Pain) were also included. All three stakeholder groups scored these outcomes as critically important, with some differences in most highly valued outcomes by group, including Global Emotional Function and Communication by Families, and Survival by Researchers and Families. Lasting emotional health effects in children and families affected by pediatric critical illness can be substantial, requiring monitoring and treatment 30. Survival and pain are frequently assessed within the hospital epoch but not post-hospital discharge despite reports of late pediatric deaths and ongoing pain symptoms reports in adults 9,31,32. Health-related quality of life, a subjective outcome incorporating a proxy/patient’s perception of the interplay of multiple outcome domains, was the highest rated Specific Outcome and is one of the more commonly reported post-discharge outcomes in pediatric critical illness 8,33. Cognitive function post-discharge has been overwhelmingly assessed using measures of intelligence, memory, attention, and/or executive function; reports on child communication function are lacking 34.

Patient and family stakeholders, despite placing generally greater importance on outcomes compared to the other two stakeholder groups, clearly show value preferences for certain outcomes compared to clinicians and researchers. Family functioning is impacted by pediatric critical illness and can also influence the trajectory of recovery and long-term functional outcomes 35,36. Further, the scope and depth of post-hospital discharge problems with sleep and physical function, and post-traumatic stress in both children and parents may go unrecognized by many stakeholders outside of the Family group. Hence, we recognized the need for the Extended outcome set to promote the outcomes valued by families. Additional goals for inclusion of these additional outcomes are to stimulate and support awareness, education, and research across the inpatient-outpatient spectrum of stakeholders. Post-PICU follow-up clinics are beginning to service this need but more systematic investment is needed 37,38.

Implementation of a COS would be facilitated by recommendations for specific instruments for each outcome in a Core Outcome Measurement Set 17,39. Thus, the next task for our Steering Committee is to recommend feasible (e.g., low cost, widely available, minimal administration time), reliable, age-appropriate and validated measurement instruments along with recommendations for timing of assessments post-hospital discharge to evaluate outcomes. Some outcomes in the COS may not have validated instruments available that meet these criteria and there may be overlap in some of the content of outcomes in the COS. These recommendations will require frequent reassessment as new information and outcome measures become available.

Finally, the last aim of the PICU COS program is to strategize for broad dissemination and implementation of the PICU COS. In addition to creation of a Steering Committee and Delphi stakeholders with diverse membership, we registered this program on the Core Outcome Measures in Effectiveness Trials’ Initiative (COMET) website (http://www.comet-initiative.org/Studies/Details/1131). Further, we will publish the PICU COS and PICU COS – Extended on the CPCCRN website (https://www.cpccrn.org/network-projects/) and Pediatric Acute Lung Injury and Sepsis Investigators (PALISI) POST-PICU Investigator sub-group website (https://www.palisi.org/subgroups, in development). We plan to submit abstracts and manuscripts for each program aim and secondary analyses to disseminate academically. We will provide approved fact sheets and infographics to all stakeholders for efficient dissemination to patient advocates and academic groups. We will develop an efficient process to monitor future use of the PICU COS in research proposals, grants, quality improvement initiatives, and publications. We will also lead a social media campaign to disseminate the COS after publication.

Limitations

Although we worked to recruit an equitable number of family stakeholders, this group had the smallest representation in the Delphi. Stakeholders had to be fluent to participate in the Delphi. Thus, we may have missed input from families and other stakeholders with importantly different backgrounds and experiences. Similarly, the number and breadth of geographical distribution of representatives of research funding agencies, payors, and hospital administrators was relatively small; overall, stakeholders from North America were over-represented. The PICU COS was created to serve all children with critical illness. However, we recognize that children admitted to PICUs have a diverse range of ages; ethnicities; hospital admission condition, severity, and comorbidities; social determinants of health; family structures; geographical locations; quality of healthcare resources and access to care; each of which may require an additional personalized approach and evaluation of utility to outcomes selection.

We encourage re-evaluation of the PICU COS content every 5–10 years to improve upon methodology and evolution of best practices for post-discharge outcomes in this rapidly developing field. Finally, we recommend considering the inclusion of patients and families in other aspects of the clinical/research process in addition to outcomes choices.

Conclusions

The PICU COS and PICU COS-Extended are multi-stakeholder-approved resources for clinical and research programs that seek to more systematically study and improve outcomes for children with critical illness and their families.

Supplementary Material

Supplemental Data File (.doc, .tif, pdf, etc.)_1
Supplemental Data File (.doc, .tif, pdf, etc.)_2
Supplemental Data File (.doc, .tif, pdf, etc.)_3
Supplemental Data File (.doc, .tif, pdf, etc.)_4
Supplemental Data File (.doc, .tif, pdf, etc.)_5
Supplemental Data File (.doc, .tif, pdf, etc.)_6
Supplemental Data File (.doc, .tif, pdf, etc.)_7
Supplemental Data File (.doc, .tif, pdf, etc.)_8

Supplemental Figures 1a and 1b. Delphi Survey Round 2 results by stakeholder group. Results presented as median (interquartile range) with mean identified by open diamonds and outliers identified by open circles.

Supplemental Data File (.doc, .tif, pdf, etc.)_9

Acknowledgements.

We acknowledge Dale Needham, MD, PhD (Johns Hopkins University, USA), Paula Williamson, PhD (University of Liverpool, UK), and Gabriel Frizzo Ramos, MD (University of São Paulo, Brazil), Tammara L. Jenkins, MSN, RN, PCNS-BC, FCCM (Program Officer, Pediatric Trauma and Critical Illness Branch) and Robert Tamburro, MD, MSc (Medical Officer, Pediatric Trauma and Critical Illness Branch) from the Eunice Kennedy Shriver National Institute of Child Health and Human Development; PALISI- POST-PICU Investigators (All USA unless specified): Samer Abu-Sultaneh, MD (Indiana University School of Medicine​), Manzilat Akande, MD, MPH (The Children’s Hospital of Oklahoma), Alicia M. Alcamo, MD, MPH (Children’s Hospital of Philadelphia), Stefanie G. Ames, MD MS (David Geffen School of Medicine at University of California- Los Angeles), Reinis Balmaks, MD, PhD (Riga Stradins University), Anoopindar Bhalla, MD, MSCI (University of Southern California, Children’s Hospital Los Angeles), Katherine Biagas, MD, FCCM, FAAP (Renaissance School of Medicine), Meredith Bone, MD, MSCI (Northwestern University Feinberg School of Medicine), LeeAnn M. Christie, MSN, RN (Dell Children’s Medical Center), Karen Choong, MB, FRCP(C), MSc (McMaster University, Canada), Jonna Clark, MD, MA (University of Washington School of Medicine), Mike Cronin, MD, MPH (University of Washington School of Medicine), Sabrina Derrington, MD, MA, FAAP (Northwestern University Feinberg School of Medicine), Leslie A. Dervan, MD, MS (University of Washington School of Medicine), Jane (Lin) Di Gennaro, MD, MS (University of Washington School of Medicine), Idris Evans, MD, MSc (UPMC Children’s Hospital of Pittsburgh), Reid WD Farris, MD, MS (University of Washington School of Medicine), Andy Geneslaw, MD (Columbia University Irving Medical Center), Denise M. Goodman MD MS, FCCM (Northwestern University Feinberg School of Medicine), Robert J. Graham, MD (Boston Children’s Hospital), Madhura Hallman, MD, MPH (University of Alabama at Birmingham), Mary E. Hartman, MD, MPH (Washington University in St. Louis), John C. Lin, MD (Washington University), Julia A. Heneghan, MD (University of Minnesota Masonic Children’s Hospital), Jay M. Hunter, DNP, APRN, CPNP-AC, CCRN, CPN (University of California San Francisco), Kevin Hummel, MD (University of Utah), J. Dean Jarvis, BSN, MBA, RN, CCRP (Children’s Hospital at Dartmouth-Hitchcock), Jessica M. Jarvis, PhD (UPMC Children’s Hospital of Pittsburgh); Elizabeth Y. Killien, MD, MPH (University of Washington School of Medicine), Sapna R. Kudchadkar, MD, PhD (Johns Hopkins University), Mara Leimanis, PhD (Michigan State University), Simon Li, MD, MPH (New York Medical College), Laura Loftis (Baylor College of Medicine), Michele Loi, MD (University of Colorado), Debbie Long RN PhD (Queensland Children’s Hospital), Peter M. Luckett, MD (University of Texas Southwestern Medical Center), Catherine Madurski, MD (Nemours Children’s Health System), Maureen A. Madden MSN, RN, CPNP-AC, CCRN, FCCM (Rutgers Robert Wood Johnson Medical School), Joseph C. Manning, PhD, RN (Nottingham Children’s Hospital), Mary McNally, BSRT, RRT (Children’s Hospital at Dartmouth-Hitchcock), Julie Menzies, PhD, MSc, B(Nurs), RNC (Birmingham Children’s Hospital), Kelly Michelson, MD, MPH, FCCM, FAAP (Northwestern University Feinberg), Lindy Moake, APRN, MSN, PCCNP (Children’s Medical Center), Haifa Mtaweh, MD (University of Toronto, Canada), Sarah Murphy, MD (Harvard Medical School), Jennifer A Muszynski, MD, MPH (Nationwide Children’s Hospital), Vinay Nadkarni, MD, MS (Children’s Hospital of Philadelphia), Sholeen Nett, MD, PhD (Children’s Hospital at Dartmouth-Hitchcock), Katie R. Nielsen, MD, MPH (University of Washington), AM Iqbal O’Meara, MD (Virginia Commonwealth University), Elizabeth Pace, MD (Case Western Reserve University), Andrew Prout, MD, MPH (University at Buffalo), Monique Radman, MD, MAS (University of Washington), Madiha Raees, MD (UPMC Children’s Hospital of Pittsburgh), Brian Rissmiller, MD (Baylor College of Medicine), Joan S Roberts, MD (University of Washington), Shilpa Shah, DO (Children’s Hospital Los Angeles), Steve L. Shein, MD (Case Western Reserve University), Marcy Singleton, ARNP (Dartmouth-Hitchcock Medical Center), Katherine N. Slain, DO (Case Western Reserve University), Kristen Smith, MD, MS (University of Michigan), Alexis Topjian, MD, MSCE (Children’s Hospital of Philadelphia), Chani Traube, MD (Weill Cornell Medical Center), Danielle Van Damme, DNP, CPNP-AC (University of Louisville), Sara K VandenBranden, MD (Rush University Medical Center), Ben White, MD, MA (University of Utah), Jane E. Whitney, MD MSCE (Harvard Medical School), Alan G. Woodruff, MD (Brenner Children’s Hospital), Mekela Whyte-Nesfield, MD (Children’s National Hospital), Lauren Yagiela, MD, MS (Children’s Hospital of Michigan), and Jerry Zimmerman, MD, PhD, FCCM (University of Washington); and CPCCRN members Andrew Yates, MD (Nationwide Children’s Hospital), John Berger III, MD (Children’s National Hospital); Michael Morowitz (UPMC Children’s Hospital of Pittsburgh); Michael Bell (Children’s National Hospital); Randall Burd, MD, PhD (Children’s National Hospital); Sabrina Heidemann, MD (Children’s Hospital of Michigan); Todd Carpenter, MD (Children’s Hospital Colorado).

Availability of data and material. The dataset supporting the conclusions of this article will be made available in the Eunice Kennedy Shriver National Institute of Child Health and Human Development Data and Specimen Hub (DASH) repository, https://dash.nichd.nih.gov/.

Conflicts of Interest and Source of Funding:

Drs. Fink, Maddux, Notterman, Dean, Carcillo, Berg, Zuppa, Pollack, Meert, Hall, Sapru, McQuillen, Mourani, Maddux, Watson, and Wessel and Mr. Sorenson have grant support from the National Institutes of Health.

Dr. Manning receives support from the National Institute for Health Research.

The remaining authors report no conflicts.

Funding for this project was provided by the Eunice Kennedy Shriver National Institute of Child Health and Human Development Collaborative Pediatric Critical Care Research Network (CPCCRN). It was approved by the CPCCRN Steering Committee and funded by grant numbers U01-HD049934, UG1-HD083171, UG1-HD050096, UG1-HD063108, UG1-HD049983, UG1-HD049981, UG1-HD083170, and UG-HD083166. Additional support from NICHD K23HD096018 (Maddux) and the Francis Family Foundation (Maddux).

Footnotes

Copyright form disclosure: Drs. Fink, Dean, Berg, Pollack, Meert, Hall, Sapru, Mourani, Wessel, and Olson’s institutions received funding from the National Institutes of Health (NIH). Drs. Fink, Maddux, Sorenson, Dean, Carcillo, Berg, Zuppa, Pollack, Meert, Hall, Sapru, McQuillen, Mourani, Wessel, Demirkol, and Olson received support for article research from the NIH. Drs. Maddux, Sorenson, Carcillo, Zuppa, and McQuillen’s institutions received funding from the NIH/NICHD. Dr. Maddux’s institution received funding from the Francis Family Foundation. Dr. Hall received funding from La Jolla Pharmaceuticals. Dr. Argent received funding for transport, accommodation and conference registration to a number of local and international congresses as invited speaker. Dr. Choong’s institution received funding from AFP Innovation Fund (for PICU Liber8 study) and she received funding from McMaster University. Dr. Lee received funding from KK Women’s and Children’s Hospital. Dr. Morrow received funding from University of Cape Town, Imperial College Press (royalties), and Critical Care Society of Southern Africa. Dr. Slomine received funding from law firms and rehabilitation hospitals. The remaining authors have disclosed that they do not have any potential conflicts of interest.

References

  • 1.Garber N, Watson RS, Linde‐Zwirble WT. The size and scope of intensive care for children in the US. Critical Care Medicine 2003;31((Suppl)):A78. [Google Scholar]
  • 2.Randolph AG, Gonzales CA, Cortellini L, Yeh TS. Growth of pediatric intensive care units in the United States from 1995 to 2001. J Pediatr. 2004;144(6):792–798. [DOI] [PubMed] [Google Scholar]
  • 3.Pollack MM, Holubkov R, Funai T, et al. Pediatric intensive care outcomes: development of new morbidities during pediatric critical care. Pediatr Crit Care Med. 2014;15(9):821–827. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Manning JC, Pinto NP, Rennick JE, Colville G, Curley MAQ. Conceptualizing Post Intensive Care Syndrome in Children-The PICS-p Framework. Pediatr Crit Care Med. 2018;19(4):298–300. [DOI] [PubMed] [Google Scholar]
  • 5.Fink EL, Tasker RC. Brain-related outcome measures in trials recruiting critically-ill children. Curr Opin Pediatr. 2019;31(6):775–782. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Maddux AB, Pinto N, Fink EL, et al. A Scoping Review: Domains and Instruments Used to Evaluate Morbidities after Pediatric Critical Illness [abstract]. Pediatric Critical Care Medicine. 2020;(in press). [Google Scholar]
  • 7.Duffett M, Choong K, Hartling L, Menon K, Thabane L, Cook DJ. Randomized controlled trials in pediatric critical care: a scoping review. Crit Care. 2013;17(5):R256. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Taylor A, Butt W, Ciardulli M. The functional outcome and quality of life of children after admission to an intensive care unit. Intensive Care Med. 2003;29(5):795–800. [DOI] [PubMed] [Google Scholar]
  • 9.Pinto NP, Rhinesmith EW, Kim TY, Ladner PH, Pollack MM. Long-Term Function After Pediatric Critical Illness: Results From the Survivor Outcomes Study. Pediatr Crit Care Med. 2017;18(3):e122–e130. [DOI] [PubMed] [Google Scholar]
  • 10.Duffett M, Choong K, Foster J, et al. High-Quality Randomized Controlled Trials in Pediatric Critical Care: A Survey of Barriers and Facilitators. Pediatr Crit Care Med. 2017;18(5):405–413. [DOI] [PubMed] [Google Scholar]
  • 11.Ong C, Lee JH, Leow MK, Puthucheary ZA. Functional Outcomes and Physical Impairments in Pediatric Critical Care Survivors: A Scoping Review. Pediatr Crit Care Med. 2016;17(5):e247–259. [DOI] [PubMed] [Google Scholar]
  • 12.Marino BS, Lipkin PH, Newburger JW, et al. Neurodevelopmental outcomes in children with congenital heart disease: evaluation and management: a scientific statement from the American Heart Association. Circulation. 2012;126(9):1143–1172. [DOI] [PubMed] [Google Scholar]
  • 13.Merritt C, Menon K, Agus MSD, et al. Beyond Survival: Pediatric Critical Care Interventional Trial Outcome Measure Preferences of Families and Healthcare Professionals. Pediatric critical care medicine : a journal of the Society of Critical Care Medicine and the World Federation of Pediatric Intensive and Critical Care Societies. 2018;19(2):e105–e111. [DOI] [PubMed] [Google Scholar]
  • 14.Pasek TA, Burns C, Treble-Barna A, et al. Important Outcomes for Parents of Critically Ill Children. Critical care nurse. 2019;39(3):74–79. [DOI] [PubMed] [Google Scholar]
  • 15.Williamson PR, Altman DG, Blazeby JM, et al. Developing core outcome sets for clinical trials: issues to consider. Trials. 2012;13:132. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Turnbull AE, Sepulveda KA, Dinglas VD, Chessare CM, Bingham CO 3rd, Needham DM. Core Domains for Clinical Research in Acute Respiratory Failure Survivors: An International Modified Delphi Consensus Study. Crit Care Med. 2017;45(6):1001–1010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Needham DM, Sepulveda KA, Dinglas VD, et al. Core Outcome Measures for Clinical Research in Acute Respiratory Failure Survivors. An International Modified Delphi Consensus Study. American journal of respiratory and critical care medicine. 2017;196(9):1122–1130. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Kirkham JJ, Davis K, Altman DG, et al. Core Outcome Set-STAndards for Development: The COS-STAD recommendations. PLoS Med. 2017;14(11):e1002447. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Kirkham JJ, Gorst S, Altman DG, et al. COS-STAR: a reporting guideline for studies developing core outcome sets (protocol). Trials. 2015;16:373. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Williamson PR, Altman DG, Bagley H, et al. The COMET Handbook: version 1.0. Trials. 2017;18(Suppl 3):280. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Fink EL, Jarvis JM, Maddux AB, et al. Development of a Core outcome set for Pediatric critical care outcomes research. Contemp Clin Trials. 2020:105968. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Olson L, Zickmund S, Galyean P, et al. PICU Outcomes: A Qualitative Study of Teen and Family Priorities (abstract) Pediatric Critical Care Medicine. 2020;(in press). [Google Scholar]
  • 23.Diamond IR, Grant RC, Feldman BM, et al. Defining consensus: a systematic review recommends methodologic criteria for reporting of Delphi studies. J Clin Epidemiol. 2014;67(4):401–409. [DOI] [PubMed] [Google Scholar]
  • 24.Gershon RC, Rothrock N, Hanrahan R, Bass M, Cella D. The use of PROMIS and assessment center to deliver patient-reported outcome measures in clinical research. J Appl Meas. 2010;11(3):304–314. [PMC free article] [PubMed] [Google Scholar]
  • 25.Guyatt GH, Oxman AD, Vist GE, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336(7650):924–926. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Levels of neonatal care. Pediatrics. 2012;130(3):587–597. [DOI] [PubMed] [Google Scholar]
  • 27.Patient- and family-centered care and the pediatrician’s role. Pediatrics. 2012;129(2):394–404. [DOI] [PubMed] [Google Scholar]
  • 28.Snyder CF, Jensen RE, Segal JB, Wu AW. Patient-reported outcomes (PROs): putting the patient perspective in patient-centered outcomes research. Med Care. 2013;51(8 Suppl 3):S73–79. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Clarke M, Williamson PR. Core outcome sets and systematic reviews. Syst Rev. 2016;5:11. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Colville G, Pierce C. Patterns of post-traumatic stress symptoms in families after paediatric intensive care. Intensive Care Med. 2012;38(9):1523–1531. [DOI] [PubMed] [Google Scholar]
  • 31.Wiens MO, Pawluk S, Kissoon N, et al. Pediatric post-discharge mortality in resource poor countries: a systematic review. PLoS One. 2013;8(6):e66698. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Choi J, Hoffman LA, Schulz R, et al. Self-reported physical symptoms in intensive care unit (ICU) survivors: pilot exploration over four months post-ICU discharge. J Pain Symptom Manage. 2014;47(2):257–270. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Watson RS, Asaro LA, Hutchins L, et al. Risk Factors for Functional Decline and Impaired Quality of Life after Pediatric Respiratory Failure. Am J Respir Crit Care Med. 2019;200(7):900–909. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Kachmar AG, Irving SY, Connolly CA, Curley MAQ. A Systematic Review of Risk Factors Associated With Cognitive Impairment After Pediatric Critical Illness. Pediatr Crit Care Med. 2018;19(3):e164–e171. [DOI] [PubMed] [Google Scholar]
  • 35.Meert K, Slomine BS, Christensen JR, et al. Burden of caregiving after a child’s in-hospital cardiac arrest. Resuscitation. 2018;127:44–50. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Anderson VA, Catroppa C, Dudgeon P, Morse SA, Haritou F, Rosenfeld JV. Understanding predictors of functional recovery and outcome 30 months following early childhood head injury. Neuropsychology. 2006;20(1):42–57. [DOI] [PubMed] [Google Scholar]
  • 37.Williams CN, Hartman ME, Guilliams KP, et al. Postintensive Care Syndrome in Pediatric Critical Care Survivors: Therapeutic Options to Improve Outcomes After Acquired Brain Injury. Curr Treat Options Neurol. 2019;21(10):49. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Williams CN, Hartman ME, McEvoy CT, et al. Sleep-Wake Disturbances After Acquired Brain Injury in Children Surviving Critical Care. Pediatr Neurol. 2019. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Prinsen CA, Vohra S, Rose MR, et al. Core Outcome Measures in Effectiveness Trials (COMET) initiative: protocol for an international Delphi study to achieve consensus on how to select outcome measurement instruments for outcomes included in a ‘core outcome set’. Trials. 2014;15:247. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplemental Data File (.doc, .tif, pdf, etc.)_1
NIHMS1626027-supplement-Supplemental_Data_File___doc___tif__pdf__etc___1.docx (16.1KB, docx)
Supplemental Data File (.doc, .tif, pdf, etc.)_2
NIHMS1626027-supplement-Supplemental_Data_File___doc___tif__pdf__etc___2.docx (23.7KB, docx)
Supplemental Data File (.doc, .tif, pdf, etc.)_3
NIHMS1626027-supplement-Supplemental_Data_File___doc___tif__pdf__etc___3.docx (22.4KB, docx)
Supplemental Data File (.doc, .tif, pdf, etc.)_4
NIHMS1626027-supplement-Supplemental_Data_File___doc___tif__pdf__etc___4.docx (19.1KB, docx)
Supplemental Data File (.doc, .tif, pdf, etc.)_5
NIHMS1626027-supplement-Supplemental_Data_File___doc___tif__pdf__etc___5.docx (21.3KB, docx)
Supplemental Data File (.doc, .tif, pdf, etc.)_6
NIHMS1626027-supplement-Supplemental_Data_File___doc___tif__pdf__etc___6.docx (20.5KB, docx)
Supplemental Data File (.doc, .tif, pdf, etc.)_7
NIHMS1626027-supplement-Supplemental_Data_File___doc___tif__pdf__etc___7.docx (14.1KB, docx)
Supplemental Data File (.doc, .tif, pdf, etc.)_8

Supplemental Figures 1a and 1b. Delphi Survey Round 2 results by stakeholder group. Results presented as median (interquartile range) with mean identified by open diamonds and outliers identified by open circles.

NIHMS1626027-supplement-Supplemental_Data_File___doc___tif__pdf__etc___8.tif (121.1KB, tif)
Supplemental Data File (.doc, .tif, pdf, etc.)_9
NIHMS1626027-supplement-Supplemental_Data_File___doc___tif__pdf__etc___9.tif (97.5KB, tif)

RESOURCES